Item — HN Reader

I was curious to understand how Copilot implements its harness, and also how I was exhausting my quota so quickly. End up going down a rabbit hole of intercepting its network traffic with mitmproxy.

A few interesting things I found along the way:

- watched model/capability discovery and routing happen in real time - looked at what gets injected into context and sent with ghost completions - found that recent edits can pull in context from files other than the one you're currently editing (including infamous .env) - found the SQLite session store behind Chronicle, including previous prompts/responses - watched the model query that history through tool calls

I then went through the VS Code source to reconcile some of what I was seeing on the wire with the actual implementation.

Overall some interesting lessons around how their harness is implemented.

1 comments

How do you actually cleanly solve that .env issue?

Anything cross platform and coding agent agnostic?

I suppose that .env file should be removed, but then things aren’t easy: no native multiplatform secret manager, or the std lib of the language doesn’t offer an API over the native secret store, etc.

Or a "secret injection proxy" for some cases could work I guess.

Infisical, or Bitwarden Secret Manager? Those two look like perfectly reasonable if the llm is just careless (but still, nothing prevents the LLM from intentionally cat'ing /proc/self/environ or from running /usr/bin/env or set or similar)

https://varlock.dev (free, open source) can pull secrets from many places, and has a credential broker (proxy) to inject placeholders, then replace with real secrets at the network boundary. There are a few other tools like this, but ours seems to be the most flexible so far.

Do you feel that something like https://secretspec.dev/ addresses these points?

Data retention clauses?

I dont see how you can ever really trust an LLM anyway to follow instructions.