Item — HN Reader

Good.

They should make it easier, to detect slop so we can ignore it quickly.

I hope Pangram makes an API or an extension to analyze a page to detect slop on a page and then closes the tab immediately.

Nobody should be wasting time on garbage LLM output in code, text, image and videos.

1 comments

Panagram is a scam.

It's not. Pangram is quite accurate. Not being perfect doesn't make it a scam.

(This is the part where you provide extensive extraordinary evidence to your claim)

No, they are the ones making claims, especially their CEO saying things like a 1/10000 false positive rate. Their own testing showed a 2% rate, which is insanely high when you talk about the number of papers students turn in. Worse their testing methodology compared it with pre-llm documents and not post llm documents that were human written (much harder and more expensive to verify), by treating language as static.

You're saying because it has some false positives that Pangram is 100% a scam?

Is their research also a scam too?

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

https://www.pangram.com/blog/pangram-4-technical

If so, what is the best one out there other than Pangram then?

Scam might be too strong a word but it certainly has far higher false positive rates than they are claiming, and their output is at best misleadingly presented: https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...

This is on Pangram 3 which is very very old now and the founder responded below

https://freddiedeboer.substack.com/p/i-wouldnt-say-pangram-i...

What about on Pangram 4?

https://www.pangram.com/blog/pangram-4-technical

Pangram is subjectively very useful and I personally subscribe, but the burden of proof is on them. The product is very much "trust me bro" and I fear that if they ever try to improve recall both their precision and reputation will tank.

Then what is the best way to know that something is AI generated slop then?

Talking to the person who gave it to you, in my experience.

In my own testing, Pangram is excellent at detecting the default output styles of LLMs.

If you tell the LLM to change its output style, so it’s not full of “load-bearing spaced em dashes that aren’t X, they aren’t Y. they’re Z.” constructions (which humans are pretty good at detecting on their own), the false negative rate soars.

The question you're asking has nothing to do with who has the burden of proof when it comes to claims about Pangram, but I'll answer it anyway.

Today, the best way is probably Pangram. Tomorrow, it might not be, especially if they try to push their recall up.

You might have to make peace with the fact that there may not always be a tool that does what you want.

So Pangram is the best one right now, that all I need to know, and I can safely assume that the Claude AI marks will make it even stronger.

Thanks!

> But the burden of proof is on them...

I mean is this enough proof?

https://www.pangram.com/blog/pangram-4-technical

https://pangram-public.s3.us-east-1.amazonaws.com/pdf/pangra...

Or is this marketing, a public stunt or not real research?

I think this is enough for me to know they are actually improving their AI slop detector.