Item — HN Reader

With clean I mean the opposite of how I currently do it, which is by asking the model to compact the whole thread into a prompt which will act as context for next model.

My way of prompting this varies and every time I receive the blob of output, I can’t fell how well it managed to capture the necessary details. This way feels lika a dirty way to transfer knowledge from one conversation to another.

You said there is? What’s the options?

1 comments

My largest issue is that when I'm looking for this I'm already dangerously close to autocompaction. And what I really want is a prompt which manages to preserve the most important parts of the chat log. And my opinion of important will not be the same as Claude, so we'll need to iterate on what that handoff really is.

I think what we really need is project/thread-scale continual learning. The problem is that the important parts of the conversation to you are the novel bits you just did, rather than all the context building the agent did to get to the point where it could do the novel bits (and even then, without really understanding the bigger picture).

If you snapshotted at 90% max context you could pretty reliably start iteratively trim that down, I think? I personally try to save the logs so agents can slice and dice them with sed/awk/jq/whatever when they need to look stuff up, because I’d rather pay the penalty on read (when it’s motivated by something) than in write(where you don’t really know what if anything will be needed), and they can figure out what they need on their own.

What I’d rather have is some way to bake history into the actual model weights (the same way it can recite certain literature or historical/factual stuff without context), with like multi-lora / “experts” that get trained out of band. But this is contrary to the “one fat model” approach to scaling and doesn’t work with closed labs’ business/IP models