Item — HN Reader

I should have read that, it’s actually quite reasonable and I don’t really understand the objections in TFA having read it.

> One of my fundamental problem with this is that no two synonyms carry the exact same meaning. “He leaped at the chance” and “He jumped at the opportunity” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter.

Doesn’t make sense at all in light of the actual approach, they’re just choosing a different RNG. It’s not like they’re corrupting it by flipping words.

Should add I don’t support the watermarking and requiring it is idiotic.

2 comments

I don't understand Gruber's points either, I wonder if there is some fundamental technical misunderstanding. Does he think that the logits should be sampled from in a "pure" manner without introducing any other bias? Does he know that there's already a sampling temperature, and that most providers have probably moved on to sampling strategies other than top-k? Does he know that the word choices have already been altered irreversibly during RLHF which is how you get the obvious Claudism like "load bearing" and "seams"?

Perhaps it would be useful to publish examples of samples with/without watermark. I'd suspect that the variability from simply sampling repeated times would dwarf any semantic differences you'd detect with the watermark.

I think Anthropic should have put all the info into one blog post. Splitting it up is really confusing people.