Item — HN Reader

You might be right that these "multimodal" transformers, by integrating additional data from non-text sources, would be more capable than GPT-3. But I don't think that invalidates Gary Marcus's point.

The word "model" is another of those words that Minsky called "suitcase words"--- they can be used to mean many things. I don't think Marcus is saying that that LLMs have "no model", just that they don't have a model of the type that a symbolic system could have. He gives many examples of deductions that a symbolic AI system can easily do, which GPT-3 is simply incapable of.

1 comments

To hint at what I fundamentally mean by model I'd be interested to see a symbolic model of vision. E.g. take 5125123 numbers and give them names and then follow some rules to arrive at "cat" or "dog". Image recognition, I think, is demonstrably not symbolic. Likewise most transformations from the real world to model state are not symbolic. Within model state, symbolism may have uses but Church-Turing claims that it isn't necessary.

It seems clear to me that if the CLIP-like part of Imagen or Stable Diffusion can take an image made of pixels and yield "cat" and similarly take "dog" and produce a 3D neural radiance field that we can light just like any other 3D model and recognize as a dog then there must be an accurate and useful model of both how vision works and what dogs look like and the English relationship between those two things inside the machine.

I also wish Gary Marcus was replying to Google's Minerva paper instead of GPT-3. The ability to answer multi-step symbolic problems is basically here. https://ai.googleblog.com/2022/06/minerva-solving-quantitati...

I'd also note that decades of attempts at automated theorem proving with symbolic systems haven't yielded similarly impressive results. Now we have deep learning models helping with proof search.