"Like Photosynth, View3 will let you walk into and explore 3D images of photos, but will let you use as little as one photo to get started." This sounds very interesting. Maybe this is related work?

http://www.cs.cmu.edu/~dhoiem/projects/popup/

http://www.ri.cmu.edu/pubs/pub_5164.html

http://www.robots.ox.ac.uk/~mukta/cvpr06/1123_Prasad_M.pdf