The tl;dr doesn't quite make sense to me. To me the core difference is that a Box<str> takes one less word on the stack, because by virtue of the str being immutable it doesn't need to track the capacity of the allocation as distinct from the length. This is analogous to Box<[u8]> vs Vec<u8> (and in fact those are the same data types except for the guarantee of valid UTF-8).
back
2 comments
C++ programmer here: which one guarantees valid utf8, and why would a primitive container make guarantees about the values it's storing?
The guarantee exists to speed up UTF-8 processing, so that it can safely assume working with whole codepoints/sequences (without extra out of bounds checks for every byte) and to ensure you can always losslessly roundtrip every string to and from other Unicode encodings without introducing any special notion of a broken character. There's also a security angle in this: text-processing algorithms may have different strategies for recovering from broken UTF-8, which could be exploited to fool parsers (e.g. if a 4-byte UTF-8 sequence has only 3 bytes matching, do you advance by 3 or 4 bytes?).
Having the "valid UTF-8" state being part of the type system means it needs to be checked only once when the instance is created (which can be compile-time for constants), and doesn't have to be re-checked later, even if the string is mutated. Unlike a generic bag of bytes, the pubic interface on string won't allow making it invalid UTF-8.
"str" and "String" guarantee UTF-8. To make a String from an array of bytes, call
pub fn from_utf8(vec: Vec<u8, Global>) -> Result<String, FromUtf8Error>
pub fn from_utf8_lossy(v: &[u8]) -> Cow<'_, str>
Vec<u8> and array slices such as &[u8] are primitive containers - they can store any sequence of u8 values. String is more like an object with access methods.
> why would a primitive container make guarantees about the values it's storing
If you know you have valid UTF-8, you can safely skip bounds checks when decoding a codepoint that spans multiple bytes.
Everything labelled as "string" is a valid UTF-8 string in Rust, and to my knowledge this decision was made very early in the history of Rust (before 0.1). Many "modern" languages (including modern enough C++) have a distinction between Unicode strings and byte strings however they are called and Rust just followed the suit.
OsStr and OsString aren't necessarily UTF-8, the data for those are "in the operating system’s preferred representation".
str/String and CStr/CString are defined as UTF-8, though.
> in the operating system’s preferred representation
This is unfortunately misleading and a common misconception about OsStr. The documentation now explains:
> OsStr losslessly represents a borrowed reference to a platform string. However, this representation is not necessarily in a form native to the platform
What this means is that valid sequences of Unicode scalars are encoded as utf8 in OsStr on both Windows and Linux. The difference between OsStr and str is that the former can round-trip with the native encoding; that means that for Windows, there's a special way it encodes unpaired surrogates (wtf8) and on Linux it's actually just an arbitrary byte sequence.
This choice of representation means that on every platform: you can always borrow an OsStr from a str (as_ref) and you can sometimes borrow a str from an OsStr (to_str). This cross-borrowing wouldn't be possible if OsStr were UCS-2 on Windows.
Note that if you don't want this guarantee, you can use [u8] and Vec<u8> (and Box<[u8]>). This does not support all string methods unfortunately, but there's work to do that.
One notable difference is that ToOwned for &str gives you a String, whereas ToOwned for &[u8] gives you a [u8] by cloning the slice you have.
In fact all four standard library types that are ToOwned without invoking Clone are more or less strings (str, CStr, OsStr, Path)
What? No: https://doc.rust-lang.org/1.61.0/src/alloc/slice.rs.html#854.... ToOwned for [T] gives you Vec<T>.
Huh. TIL
Actually now that I think harder about this, WTF did I think was really going on here before. If we clone the slice to make an array, where does the array go? We don't know how big that array is, so we can't put it on the stack.
Yeah, that was crazy. Thanks for pointing it out.
Perspective on ToOwned: It's just Clone with an extension to a number of DST types that can't be Clone themselves. They are dynamically sized types, hence no surprise that they are string-like.