I don't see any ethical connection between adding canary tokens to your output to catch people breaking your accepted ToS through ongoing distillation and stealing your PII off of your machine. How are legitimate users in US or China possibly harmed by Anthropic silently changing the apostrophe in Today's or the date separator from - to /?
back
3 comments
It's clandestine behavior, we all here assume it's trying to signal whether a user is in China, but this breaks an implicit trust.
How do we know this isn't the work of a rogue developer at anthropic and that they are not subtly switching visually identical characters in other contexts to ship your ssh keys or whatever.
We don't. After the trust that the software doesn't do things it doesn't disclose has been broken you can assume it operates like malware.
> I don't see any ethical connection
Trust isn’t about ethics, it is about individual judgement of how much risk some actor poses to your own interests. Trust is context-dependent
There can be unethical actors whom you have good reason to trust, and highly ethical actors whom you have good reason to distrust
You are talking about "stealing" from people who already used stolen texts to train their models.