back

by zug_zug·1y ago·view on hn ↗
As somebody who's good at RCA, I'm worried all my embarrassed coworkers are going to take at face value a tool that's confidently incorrect 10% of the time and screw stuff up more instead of having to admit they don't know something publicly.

It'd be less bad if the tool came to a conclusion, then looked for data to disprove that interpretation, and then made a more reliably argument or admitted its uncertainty.

1 comments
You can achieve a good amount of this with system prompts. I've actually had good success using LLMs to craft effective system prompts and custom instructions to get more rigorous and well researched answers by default.

One I use with ChatGPT currently is:

> Prioritize substance, clarity, and depth. Challenge all my proposals, designs, and conclusions as hypotheses to be tested. Sharpen follow-up questions for precision, surfacing hidden assumptions, trade offs, and failure modes early. Default to terse, logically structured, information-dense responses unless detailed exploration is required. Skip unnecessary praise unless grounded in evidence. Explicitly acknowledge uncertainty when applicable. Always propose at least one alternative framing. Accept critical debate as normal and preferred. Treat all factual claims as provisional unless cited or clearly justified. Cite when appropriate. Acknowledge when claims rely on inference or incomplete information. Favor accuracy over sounding certain.

Do you just add this to your “instructions” section?

And what type of questions do you ask the model?

Thanks for sharing

Yes, with ChatGPT, I added that paragraph as custom instructions under personalization.

I ask a wide variety of things, from what a given plant is deficient in based on a photo, to wireless throughout optimization (went from 600 Mbps to 944 Mbps in one hour of tuning). I use models for discovering new commands, tools, and workflows, interpreting command output, and learning new keywords for deeper and more rigorous research using more conventional methods. I rubber duck with it, explaining technical problems, my hypotheses, and iterating over experiments until arriving at a solution, creating a log in the process. The model is often wrong, but it's also often right, and used the way I do, it's quickly apparent when it's wrong.

I've used ChatGPT's memory feature to extract questions from previous chats that have already been answered to test the quality and usability of local models like Gemma3, as well as craft new prompts in adjacent topics. Prompts that are high leverage, compact, and designed to trip up models that are underpowered or over quantized. For example:

>> "Why would toggling a GPIO in a tight loop not produce a square wave on the pin?"

> Tests: hardware debounce, GPIO write latency, MMIO vs cache, bus timing.

>> "Why is initrd integrity important for disk encryption with TPM sealing?"

> Tests: early boot, attack surface, initramfs tampering vectors.

>> "Why would a Vulkan compute shader run slower on an iGPU than a CPU?"

> Tests: memory bandwidth vs cache locality, driver maturity, PCIe vs UMA.

>> "Why is async compute ineffective on some GPUs?"

> Tests: queue scheduling, preemption granularity, workload balance.

>> "Why might a PID loop overshoot more when sensor update rate decreases?"

> Tests: delayed ACK, bufferbloat, congestion control tuning.

>> "How can TCP experience high latency even with low packet loss?"

> Tests: delayed ACK, bufferbloat, congestion control tuning.

>> "How can increasing an internal combustion engine's compression ratio improve both torque and efficiency?"

> Tests: thermodynamics, combustion behavior, fuel octane interaction.

>> "How can increasing memory in a server slow it down?"

> Tests: NUMA balancing, page table size, cache dilution.

>> "Why would turning off SMT increase single-threaded performance on some CPUs?"

> Tests: resource contention, L1/L2 pressure, scheduling policy.

This looks excellent. Thanks for sharing.