How would this multiple passes work though? Unless the model actually talks about what it does, I am not sure how it would have this ability. The next word prediction mechanism is just always going to do it one shot. Your prompt paints a context that might keep it more on the rails, but it won't do multiple passes.
back
2 comments
> Unless the model actually talks about what it does
That's how the chain-of-thought approach works. You can make the model do it inline, or you can run the loop yourself, possibly summarising the progress as you go. (Although with prompt caching it's not that important anymore) You can encourage it to print out assumptions/steps/ideas as it goes.
Your prompt paints a context that might keep it more on the rails, but it won't do multiple passes.
This is probably the truth behind the black magic I’m imagining. You could have it explicitly spit out this process, in which case you would see it’s first rough draft, followed by a “My first paragraph is probably wrong”, followed by a third paragraph where it attempts to fix the first paragraph. There is no outside RAG in this process.
The mumbo jumbo part of all this is that I’ve told it to “hide” this process from the user where it doesn’t explicitly output anything but its final answer, and the accuracy has been just as good (for my use case at least).
:Shrugs:
Yeah that’s not how next token prediction works. To actually do multiple passes you’d need to do that yourself, making multiple calls and feeding the responses back to the model.
Why? The very nature of next token prediction means it's entirely capable of having that. It's not multiple passes, it's just one pass. You making multiple calls is just inserting fixed tokens then asking it to carry on completing.
making multiple calls and feeding the responses back to the model.
By asking it to reconsider half its generated response, aren’t I essentially asking it to formulate the second half of its response from the first half internally? I’m bypassing the manual process of feeding in the extra prompt.
We are constantly having to tell the LLM close, but no cigar, iterate again, more or less.
Isn't this in part what o1-preview is doing?