Item — HN Reader

I’m not an expert on this, so I’m not familiar with what RDF graphs are, but I feel like everything you’re describing happens textually, and used as context? That is, it’s not at all ”learning” the way it’s learning during training, but by writing things down to refer to them later? As you say - ”ask it to summarize the insights for later use” - this is fundamentally different from the types of ”insights” it can have during training. So, it can take notes about your code and refer back to them, but it only has meaningful ”knowledge” about code it came across in training.

To me as a layman, this feels like a clear explanation of how these tools break down, why they start going in circles when you reach a certain complexity, why they make a mess of unusual requirements, and why they have such an incredible nuanced grasp of complex ideas that are widely publicized, while being unable to draw basic conclusions about specific constraints in your project.

1 comments

To me it feels very much like a brain: my brain often lacks knowledge, but i can use external documents to augment it. My brain also has limitations in what it can remember, I hardly remember anything I learned in high school or university on science, chemistry, math, so I need to write things down to bring back knowledge later.

Text and words are the concepts we use to transfer knowledge in schools, across generations, etc. we describe concepts in words, so other people can learn these concepts.

Without words and text we would be like animals unable to express and think about concepts

The point isn’t that writing and reading aren’t useful. The point is that they’re different from forming new neurological connections as you familiarize yourself with a problem. LLMs, as far as I know, can’t do that when you use them.

Does that really matter if the result is the same, they have a brain, they have additional instructions, and with these they can achieve specified outcomes. Would be interesting to see how far we can shrink the brains to get desired outcomes with the right instructions.

It matters if the result is not the same. The article argues that this is an important aspect of what a human developer does that current AI cannot. And I agree. As I said, I find the idea very convincing as a general explanation for when and why current LLMs stop making progress on a task and start going in circles.