Item — HN Reader

While non-BMP Unicode has only become popular somewhat recently due to emoji, it's been part of the relevant official specs for a quarter century.

I know you were kinda joking by using scare quotes for "recently", but let me fill that in for people who might be less familiar with the history. Unicode broke out of the 16-bit limit as of version 2.0, released in July 1996, so yeah, a quarter century. The first RFC for UTF-8 (RFC 2044, Oct 1996) explicitly supports UCS-4. As far as I know, there has never been a time where UTF-8 restricted to the BMP (ie 3 bytes) has been in widespread use. In other words, I am questioning your assertion that 3-byte UTF-8 has ever been considered "long enough," except perhaps in sloppy implementations made without much care for following specs.

2 comments

Indeed. It somewhere between sad and amusing for those of us who care about i18n that encoding issues in various software that we'd been pointing out for the better half of the last 20+ years suddenly became no big deal to fix in the last ~5 years as 4-byte Unicode Emojis got widely deployed for users whose languages would otherwise fit in US-ASCII or Latin-1.

It's also a lesson for future generations. If you want an encoding specification to be fully implemented make sure that doing so is synonymous with the ability to display something like dog poop emoji. Clearly "this allows millions of people to read their native language" was too boring of a reason to care.

Now if we could just come up with an application of accessibility APIs that appeals to the average non-disabled user, maybe we could indirectly advance the state of accessibility as well.

> except perhaps in sloppy implementations made without much care for following specs.

Describes MySQL perfectly ;)