It sounds great but there are simple use cases where the "portable" app isn't enough. For example, if you want multiple users to be able to use the program and have their own settings, you need something to be saved to the user folder. Or, if you want any basic interaction with the system (run on startup, run from a browser address, etc), you need to start messing with the registry.

So in theory apps could be distributed portable .exes but in practice Windows doesn't any ways of interacting with the rest of the system that are that nice.