Item — HN Reader

If you can do this repeatedly, say 15 times, and record the numbers each time (say just put them in a spreadsheet), you can actually pretty easily run a "statistical test" to prove your theory at 95% confidence (which is what research journals often do).

You obviously don't need to justify your observations to anybody else, but if you did find a conclusive result it would make a good blog post.

[P.S. chat gpt could help with the statistical test part]

1 comments

I do find chatgpt is helpful in laying out the formulas you should you in the context that they should be used in. Something I personally know I have no business doing.

... At the same time, I also am aware of how hilariously dumb chat GPT can be in deep technical contexts. I've taken to saying that when it comes to a technical topic, chatGPT will confidently tell you the wrong thing to do 50% of the time, but that's fine because it will give you the terms and context you can use to audit its solution yourself. Even if you don't understand the answers you can easily have chat gpt explain the gaps, again, 50%, but giving real information contextualizes the conversation better. I would expect this to improve accuracy, and my personal experience bares this out.

There was a brief window of time where the strongest chess players were chimeras, humans who assessed AI suggestions. Quickly even they found themselves outperformed by engines.

I suspect this is happening here as well. And I also suspect its going to take a good deal longer.