Also, air resistance is much more at the surface than higher up.
So by starting your ascent at a high altitude you need much less fuel both because you need less fuel, and because there is less air resistance (friction).
Space elevators still need to use the same energy to lift something, but they use an external power source so do not need to lift their own fuel, and they lift more slowly and greatly reduce the effect of air resistance.
So no, a space elevator won't get you to "outer space", but it creates a stepping stone which greatly reduces the cost of getting there.