The 401k works in two ways actually. The way you mentioned. Plus, because employees have no choice of bank or investments in their 401k so they can be screwed over on fees more easily.

Stock markets were invented originally by the Dutch East India company to raise funds from the general public. This allowed them to raise the capital to expand faster than any other firm in the world, at a time when the alternatives were to convince royalty to grant you money, turn to high-interest small loans via moneylenders, or raise from your own stakeholders (e.g. guild membership).

At its height, the value of the Dutch East Company dwarfed the value of the top 40 global companies today, combined, FAANG included. Ever since, companies have sought to issue stock to the public (IPO).

Originally, the reason to buy a stock was because the company offered dividends to their shareholders. Over time, people started trading their shares of the stock (hence stream of future expected dividends) to others for cash now.

The 'stock market' became a meetingplace for those transactions. Just like other marketplaces (e.g. AirBnB), the stock market makes it easy for buyers to discover sellers and vice versa. The ability to trade assets for cash is 'liquidity' and stock markets have proven to be highly liquid, which is one of their most important qualities. Over time, these meetingplaces became regulated by governments to prevent schenanigans (e.g. Great Depression).

Originally, companies didn't care that much about their stock price after they initially raised the funds from the public. The price of stock was directly tied how much in dividends you would expect from the future, which is a proxy for the health of the company, but that's about it.

Over time, investing into these stocks became its own profession, and investing professionals developed a lot of metrics to assess that health of the company. Methods include looking through the company's balance sheets, assessing their strategy, comparing them to competitors, etc. Think of jargon like EBITDA, Free Cash Flow to Equity, Debt Leverage - there's over 20,000 pages of it throughout the CFA exam series.

Jack Welch is credited with shifting the focus of companies towards increasing their stock price by gaming the investors' metrics. Improving these metrics will have immediate effects on investor perception, much moreso than gradual adjustments to dividends. Employees and managers, who owned a bit of the company stock, would try to do all they could to amp up the price so they could sell their own shares at a high price.

Incidentally, this is the same angle for most tech stocks / VC. Tech companies generally don't focus on paying steady dividends, but rather on growing and raising the stock price to the point where you can sell it to someone else at a higher price. Hence, many tech companies are fine with being unprofitable for a long amount of time, even during an IPO, as long as the valuation of the company (# shares x price) rises via growth.

Re: Republicans, you are actually 100% correct in your suspicion, at least w/r/t the USA. Reagan changed the tax code in the 80s, around the time Welch was in his heyday, to incentivize citizens to invest in the economy. Now over half of American adults are invested in the stock market, which is among the highest participation rate of any country. In other countries, such as China, the stock market doesn't have nearly as high of a % participation from the public.

Reagan's policy is regressive. Only the people with well-paying jobs with 401ks typically invest, so the poor fall farther behind. Furthermore, to take full advantage of the investment tax incentives requires knowledge, which is typically passed down via family rather than taught in schools. This is a substantial driver of inequality.

Here's a short blog/article I wrote about the history of shareholder rights from then till today https://www.yourstake.org/academic-impact/

Yes of course it is a ploy, it has 2 effects: It is maddening employees by binding them to stock market (preventing strikes for example), to make them accept pay cuts more easily. It increases stocks because there is a lot more money on the market

401k vs pension plays into the republican talking points on a number of fronts.

Its "personal responsibility" vs "socialism" because it shifts the burden and risk from companies and the state (in the form of pension guarantee/insurance) to the individual. Which repeatedly has been shown to be a disaster on quite a number of fronts, not the least of which poor returns or a economic disaster like 2008 completely screws people who expect to retire at a certain time. This turns around and has national economic consequences because so many people instead of retiring in 2008 simply kept working, which meant that those most in need of new jobs had an even harder time finding them.

Its also problematic because younger people who are paid less are more in need of front-loading their retirement so that they can benefit from a longer growth cycle are the least likely to be able to afford it.

Then there are all the issues with the way 401ks are managed vs even IRA's which create an entire industry where the sole purpose seems to be to reduce returns below market rates. Most people would be better off simply buying vanguard total market indexes from a discount broker vs all the fees and overhead brought on by all the middlemen "adding" value by reducing the returns.

Pretty much the entirely of most pension programs problems are caused by companies/organizations that simply ignore long term average returns and under-fund the pensions. Until the 1980/1990's the mandatory formulas for pensions and what they were allowed to invest in were quite strict, but then all that was gradually loosened until now we have a bunch of cases where the pension made bad investments (CalPERS), made wildly exaggerated statements of expected returns, or massively underestimated liabilities (GM) and find themselves in situations where they can't cover current liabilities.