Whoa, interesting. What did you mean by the 'notebook tab'? Also, if I understood correctly, your workflow is to basically just transiently cohabitate in the "dream-state" of the LLM, starting and stopping at arbitrary points. That's actually pretty cool! Your premise is correct that the conversational interface is really just putting the dream-state on rails; the model itself doesn't actually "know how to talk".
back
1 comments
I’m using a local frontend: https://github.com/oobabooga/text-generation-webui
With local 8x7b: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G... (Q4_K_M)
Requires 35gb+ of regular RAM, RTX card is optional but speeds it up a little. Can’t tell if cloud llms ever exposed this “notebook” mode, I’m in a sanctioned country.
Notebook is basically a raw interface to an llm. No system prompt, no bubbles. You give it a starter text and hit generate, it generates until you hit stop or some system limit triggers. You then hit generate again.
your workflow is to basically just transiently cohabitate in the "dream-state" of the LLM, starting and stopping at arbitrary points
I basically act like voices in a mind of an llm which induce thoughts and direct it without it even noticing, for the most part.
Sometimes, a dialog is still more preferable. E.g. I can think in a person-help-required mode and then it feels more natural. But I still dialog it there in a notebook tab, where I can adjust both my and its replies at any point, as if I was at the same time one level above. Chat is just a text with A:… B:… lines.
I’m not ND btw, found it by experimenting with writing stories.