Item — HN Reader

Yes, you fully understood me! Thank you for further explaining what I'm getting at.

I find myself able to 'co-navigate' a question or idea with the LLM, better than most other people around me, as defined by quality of model output. But it's just now dawning on me that this might be explained by more than me having greater general knowledge, or having a certain personality.

Somehow, I have an intuition for what words or phrases the LLM might have a tendency to overfit on, so I avoid those in my prompts. I likewise know what words to specifically use or emphasize so that I can trigger the LLM to tap into a certain knowledge-pool based on those.

When I see my friends engage with LLMs, I witness the lower quality output they get because they are not deploying the same intuitions in their prompts, and it's hard to teach them.

1 comments

May be interested in https://news.ycombinator.com/item?id=40766684

Whoa, interesting. What did you mean by the 'notebook tab'? Also, if I understood correctly, your workflow is to basically just transiently cohabitate in the "dream-state" of the LLM, starting and stopping at arbitrary points. That's actually pretty cool! Your premise is correct that the conversational interface is really just putting the dream-state on rails; the model itself doesn't actually "know how to talk".

I’m using a local frontend: https://github.com/oobabooga/text-generation-webui

With local 8x7b: https://huggingface.co/TheBloke/Mixtral-8x7B-Instruct-v0.1-G... (Q4_K_M)

Requires 35gb+ of regular RAM, RTX card is optional but speeds it up a little. Can’t tell if cloud llms ever exposed this “notebook” mode, I’m in a sanctioned country.

Notebook is basically a raw interface to an llm. No system prompt, no bubbles. You give it a starter text and hit generate, it generates until you hit stop or some system limit triggers. You then hit generate again.

your workflow is to basically just transiently cohabitate in the "dream-state" of the LLM, starting and stopping at arbitrary points

I basically act like voices in a mind of an llm which induce thoughts and direct it without it even noticing, for the most part.

Sometimes, a dialog is still more preferable. E.g. I can think in a person-help-required mode and then it feels more natural. But I still dialog it there in a notebook tab, where I can adjust both my and its replies at any point, as if I was at the same time one level above. Chat is just a text with A:… B:… lines.

I’m not ND btw, found it by experimenting with writing stories.