Basically, no. You'd have to use references and do research to find out what the likely color of every object in the scene is.

If you know what type of black and white film was used and how it was developed, you might be able to eliminate some colors that would have appeared much darker or much lighter. If you had several black and white pictures from the same occasion shot with different film, you might be able to start to triangulate the possible colors of different objects based on how they appear differently in the different images.

But I really don't think that's how most of these are done, it's probably more about creativity and getting something that looks realistic.