Item — HN Reader

I don't see either of those as reasons not to train on it. Training isn't zero sum. You don't need to take something out to train ethics, and whether you explicitly train it or not, models will take ethical stances. Your second point actually argues in favor of training because that is still an ethical stance.

I think a lot of people confuse ethics training with "don't be evil"

2 comments

When you put it like that, it's actually probably worse to train it on a subset of ethics curated by the people doing the training, aligning with their world view and interests. If the model learned ethics just from raw training it would at least be statistically in line with it's training corpus.

Manually training on "approved" ethical stances sounds a lot like censorship.

I mean, sure, its censorship to the extent that you can apply the term to an institution that is also the creator of the censored work, and that's exactly the point and the whole meaning of “alignment”.

I would eat my shorts if we could get alignment in a HN thread on what the top 25 "don't be evil" required ethical items for a AI lab would even look like.

The first 10 maybe easy, but we'd see so much disagreement even without money nothing would get done.