Item — HN Reader

While I could generally agree, in this specific instance if the AI were “thinking” correctly it should have found the mistake. I admit it was a difficult problem though (solving it required creativity).

To be more precise, the prompt actually pointed to where there could be issues, and the issue, which was exactly of the kind that was pointed at, was not found.

1 comments

There are a lot of factors in "should have found" which my recommendation improves.

If you told it to write unit test coverage, you would have covered more of the codebase. That reduces the size of context necessary for the next mistake finding investigation - it'll see it's already covered a lot of the paths. Then you say "Go look for issues" (or whatever you asked it to do) and it'll be able to think more deeply about what's left over.

What specific model were you using, at what effort? How big was your context window?