One of the great things about a Linux system is that you can use the terminal to install and delete programs. Can you do that with PS without using something like Chocolatey?
You can also download and install Windows installer setups from a shell, and also uninstall them e.g by name (more elegantly in PS than in cmd). Because win apps are usually binary this is a more common way than the "make install" equivalent on Linux (Where I don't believe there exists a distro independent binary installer format?)
You can't install apps like "packages" without a package manager like choclatey, but that I assume is the same in Linux, you need to install a package manager like apt or yum if the distro didn't come with one.
Still pretty meat that you can uninstall a program from PS. Might need to give it a closer look, any suggestions on where to learn it from?
The big difference between something like Chocalatey and rpm/deb Linux package ecosystems is more one of scale (how many packages exist in the ecosystem; how many players are expected/required to buy into the ecosystem).
An interesting twist to this is that PowerShell in Windows 10 'recently' added a meta-package manager "OneGet" that was designed to use a single set of commands to learn to work with a wide collection of package ecosystems (including, but not limited to, Chocolatey, NuGet, MSI installers, Docker packages, whatever other sort of provider someone wants to build).