back
1 comments
Python "cheats" by converting everything to Unicode internally. It seems like a simple solution, but it's not a solution since not everything can be converted safely back to the original encoding.
Python 3 doesn't use an internal encoding, it uses unicode. Python 2 can use unicode strings if you specify them. I'm not familiar enough with ruby to comapare the two, but I don't see a problem with how python handles it. You sometimes need to know about your string encoding, which is just a fact of modern programming.

[EDIT] I think I missed my point slightly. Python 3 doesn't change the encoding of strings, it decodes them to unicode. You can encode the string back to the original encoding without loss.

If a CP932 '\' is interpreted as '¥', but is exported as CP932 '¥', there's data loss. Unless Python 3 keeps the original data around when it converts to Unicode (probably UTF-8 or UTF-16), there will be data loss in those cases. It's unavoidable.
UTF-\d is not unicode. Unicode isn't an encoding, it's the decoded representation of a character encoding.

    [edit] clarified example
    >>> s = u'\xa5' # shiftjis decoding of \
    >>> print s.encode('shiftjis')
    \
    >>> print s.encode('utf-8')
    ¥