▲ 146 points
back
131 comments
He's not writing the C++ standard library from scratch. He's writing his own library, in a different namespace, with some similar functionality. It's easy to write a non-standard library that only satisfies your limited subset of needs. People do it all the time. It's not special.
The ABI stability boast is based on having no legacy to support. It will work fine as long as everything is shipped only as code and the whole world needs to be rebuilt from scratch every time and even then, you never change any code ever even for a major bugfix. That's hardly practical in the real world where one tiny misstep on the ABI front can result in billion-dollar multinationals threatening suit (ask me how I know). It's a facile claim.
The C++ standard library hasn't been known as "the STL" for almost 30 years, ever since part of the STL was modified and adopted into the C++ standard library. Most of the features he's providing implementations for were never a part of the STL (file I/O, strings, hash maps, UTF-8).
I maintain an implementation of the C++ standard library for a living. It's a full-time job. It's a huge library (note to the committee: please stop) and it's really easy to mess something up. But if you want to write your own library that doesn't do what the standard library does or meet any of its requirements and implementation constraints or serve its real-world purpose, go right ahead. Just don't claim you're writing your own C++ standard library. You're not.
This is an unnecessarily combative comment.
The author doesn't claim to be implementing the C++ standard library. They clearly say they are implementing a C++ standard library.
It's obvious from the context that they mean a hobby-scale set of basic datatype and algorithm libraries. It would take an uncharatible reading to not realize that they mean a lowercase "standard library", not "conforming implementation of the ISO C++ Standard Library".
The article literally says "It's my time, and I'll waste it if I want to!" and uses "pystd" as the namespace.
What makes his library “standard”?
How do you know about a billion-dollar multinationals threatening suit?
> I maintain an implementation of the C++ standard library for a living.
I have always wondered: does the standard library have a huge test suite? For that matter, how is the language implementation itself tested against regressions, etc. If anybody has some knowledge about this...
I agree, and every time I see someone refer to the standard library as the STL, I know I’m interacting with someone who doesn’t actually know the language very well.
> I agree, and every time I see someone refer to the standard library as the STL, I know I’m interacting with someone who doesn’t actually know the language very well.
I would put away the judgment. I and many others still call it the STL despite knowing the language and history of the term. Because that ship sailed and it's what many people call it nowadays. Because writing out "the standard library" every damn time gets really tiring really fast.
It works in the reverse direction too, btw. The people who constantly nitpick on this are usually the ones who are more focused on being pedantic than being helpful. So that's the signal you send when you do that. Ask me how I know.
Pretty much every C++ developer i've ever interacted with the last 25 years -informally- refers to the standard library as 'STL'.
I do not think anyone with even a small amount of C++ experience will be confused when 'STL' is referenced in the context of C++.
Of course it might be that every C++ developer i've interacted with doesn't really know the language very well. But considering the popular axiom that says something along the lines of 'people who claim they know C++ do not really know C++ while people who know C++ do not claim they know C++' what you wrote might actually be true :-P.
I mean, I think you have unreasonable expectations for a 1K word blog post. Or maybe I have very low standards and no matter how disappointing a post is, I accept it... could be. However, if I read a blog post on "Rewrite React from Scratch", I'm expecting to see some reactivity and that's it.
As a reader, after I see I barely need to scroll to the end of the article (and the repo isn't very big either), I immediately understood that they aren't rewriting C++ standard library from scratch. Of course they can't give all the answers on how to maintain backwards compatibility with decades of legacy stuff with probably billions of devices and exotic use cases..
The section about the "perfect ABI stability" is rather naive. If you have a 3rd party library that exposes a class like this in a header:
class SomePublicClass {
pystd::HashMap<pystd::U8String, size_t> member;
/*...*/
};
It's also not a new idea either. libc++ puts everything inside an inline namespace `std::__1`. There is a reason that they never bumped that.
I think you may have misunderstood the proposal. Your 3rd party library example would have to write `pystd2025::HashMap<pystd2025::U8String, size_t> member;`. Isn't that stable?
From the post:
The sample code above used the pystd namespace. It does not actually exist. Instead it is defined like this in the cpp file:
#include <pystd2025.hpp>
namespace pystd = pystd2025;The best way to know something well is to confidently do a project like this incorrectly so that commentators correct you :)
just like there is a dynamic linker that can relocate code and fixup addresses, there should be a "dynamic class-sizer" which can recognise that something is being linked against a different version of some library, but the used fields used are all still present even if the structures have changed size, and dynamically adjust all pointers into the class to match the new size.
The title is confusing.
He is not reimplementing the STL.
He is writing some C++ classes providing functionality that is also already implemented in STL.
Yes, and he has so far reimplemented only a tiny fraction of that STL functionality.
Still interesting, despite the misleading title.
A problem I encountered while writing custom stdlib, is that certain language features expect stdlib to be there.
For example, <=> operator assumes, that std::partial_ordering exists. Kinda lame. In the newer C++ standards, more and more features are unusable without stdlib (or at least std namespace).
At least you have the chance to implement your own std::partial_ordering if necessary; in most languages those kind of features would be built into the compiler.
Sometimes standard library types defined in terms of compiler-builtins like `typedef decltype(nullptr) nullptr_t` but that doesn't always make sense. E.g. for operator<=> the only alternative would be for the compiler to define std::partial_ordering internally but what is gained by that?
There is a science to designing reusable containers and algorithms, and it’s based on research like Art of Computer programming and you can learn more by reading primary sources about the design of STL.
STL can absolutely be improved, but posts like this indicate most programmers are clueless about how it works, and not in a position to learn from its mistakes and make something better.
If we are serious about code reuse we need to study these ideas and learn how to actually write libraries. The alternative is the npm/crates model - where you throw together 100 different open source concoctions and hope it works.
Very nice! I like the tone and flippant energy of the post, of course, and also the way to get a nice scope by having a concrete case of a program to implement.
I also appreciated the comparisons against STL, very informative. It's ... interesting that if including `vector` in STL brings in 27,000 lines, and the author's implementation of the functionality for the example program was only 1,000 lines, that the compilation time difference is only 4X. Not sure I understand that, really. But benchmarking is hard, of course.
If I could come with a single suggestion it would be to include the sample program's source as text, not as a picture of text. If that means losing the pretty syntax highlighting, that's fine (by me). :)
> interesting that if including `vector` in STL brings in 27,000 lines, and the author's implementation of the functionality for the example program was only 1,000 lines, that the compilation time difference is only 4X
I imagine the time taken varies much more based on what's on the lines, rather than how many there are.
I'm not aware of specific pathological cases, but I'm sure you could make maybe 10 lines take 20 times longer than both of those vectors put together.
I unexpectedly did some cpp few days ago and I was surprised that cpp standard library doesn't have string trim function! Everybody is rolling their own. What is the reason behind that?
What do you want to trim off? ASCII 0x20? Any ASCII white-space? Any Unicode white-space? Well the latter requires defined string encodings and depends on the Unicode version and you can't just use the latest without introducing subtle compatibility issues.
The problem here isn't so much that it's not in the standard library (not everything needs to be in the standard library), but that everyone is rolling their own instead of using third party libraries.
In a similar vein, I found out that Go doesn't have a string reverse function either. Everyone online pretends reversing strings is easy (just iterate through the array backwards! The world is US ASCII only, right?).
Trimming strings isn't hard in most real world applications, on the other hand, and not putting it in the standard library means people won't confuse the way the trim method works (i.e. the user must make a choice between copying memory or reusing memory and risking memory lifetime/consistency issues). And that doesn't even include problems like "what if the string isn't utf8".
I'm more disappointed in Go, which takes a ton of questionable assumptions in the standard library to pretend difficult problems are easy. C++ wants to be correct and knowing what is or isn't whitespace is hard when you don't know the length of a single grapheme.
Exactly, same as for base64 encoding, sha256/512 hashes and many more.
In C++ frameworks it exists for ages.
Why not in ISO C++?
Welcome to the ways of ISO and committee driven development, apparently no one cared enough to submit a paper, and do the work to win the paper voting into the standard.
A bit off-topic maybe, what is a good open source library to read through to see some clean&modern C++? I've not dealt with the language in a bit and was thinking of diving back in
I've always found SerenityOS to have quite nice C++ source code. The project runs on very recent versions of C++ (to the point where the standard compilation script for Ubuntu will compile a modern compiler first) and the OS intentionally doesn't stick to POSIX, allowing some very nice API improvements that only work in a C++-first world.
Its main author moved on to Ladybird, though, so I haven't really browsed the code recently. I'm not sure if SerenityOS uses concepts and other such recent additions.
In any case, having a go at Tour of C++ book is a good way to read about modern idioms.
A bit off-topic, but as a Meson user, I would love to see C++ modules support since they start to be usable in all three big compilers.
Nice experiment for the pyStd, though, as pointed out, this would break with pre-compiled 3rd party deps that use pystd in a different version :)
I do wonder how much smaller the STL source code would be if it was pre-processed or written with only a single C++ standard in mind. So only for C++20 or only for C++23 etc. In that case how much faster would things be to compile where it doesn't need to filter through hundreds of preprocessor options?
From what I've read on mailing lists and whatnot, it seems a lot of complexity comes from explicit choices made, like iterators being unaffected by insertions[1] for maps and such, or time complexity guarantees that forces the implementation into certain corners.
[1]: https://kera.name/articles/2011/06/iterator-invalidation-rul...
> In that case how much faster would things be to compile where it doesn't need to filter through hundreds of preprocessor options?
I think most of the time spent isn’t running the preprocessor, but parsing the declarations and definitions.
Regardless, the way to speed up importing definitions in modern C++ is to use #import instead of #include.
https://news.ycombinator.com/item?id=38904758 says they could import the entire std namespace in under a second (that is long when you want to run C++ as a scripting language, but not when you compile large programs)
How will this year release scale when you need to work with newer compilers? I don‘t write cpp so I honestly don‘t know. Do you need to freeze your version of the compiler forever? Or is gcc / clang backwards compatible? Or do you sprinkle tons of pragmas on the files to control this? What I mean is how can you make sure your version one API is still compiling in the future. Take the counter example of ruby for instance. I could write a package with lots of namespaces and declare v1 frozen. But I still need to potentially
update the code so it can run in newer versions of the runtime. Edit: typos
Overwhelmingly, new compilers will compile old code just fine. IIRC, the only time old code is broken intentionally is when fixing a bug in the compiler itself causes the code to break.
Years ago every C++ project worth its salt had at least one implementation of of strings, vectors and other basic things. I hoped we finally were past that.
Probably not as the way things are defined in the STL is often not in line with what C++ programmers want for their code base.
STL adheres to zero-cost-abstraction, which often puts safety in the backseat. Many programmers, myself included, prefer safety by default with an escape route, when its really needed.
Add to that things like exceptions, locale-dependent behavior, functions with a dozen overloads, an overly complex memory allocator interface (`std::vector` vs. `std::pmr::vector`), etc.
Personally, I'd prefer a common alternative to STL that focuses on these points. ETL [1] and abseil [2] come to mind, but it's not exactly what I envision.
1: https://github.com/ETLCPP/etl 2: https://github.com/abseil/abseil-cpp
We will never be past that is there are tradeoffs in implementing things. Safety was brought out in other comments, but there are others. In some cases a slower algorithm will be faster in the real world because it is more CPU cache friendly (depending of course on what N is, but often N is small enough). In some cases you can accept a less precise answer - thus making your algorithms faster.
For most of us, in most problem spaces, the above doesn't matter and so the standard library is good enough. There will always be those who correctly have a need that is strong enough to be worth building their own standard library though.
Yes, since 1998, but apparently legacy code lives on.
At least in what concerns "strings, vectors and other basic things".
Now if you conside something like networking part of "basic things", then it is another matter.
Then again, vcpkg and conan exist now.
https://github.com/electronicarts/EASTL
Is pretty good for games.
> The C++ standard library (also know as the STL)
The C++ Standard Library is not the same as the STL.
The STL is the Standard Template Library, which provides containers such as vectors, as well as related functionality like iterators.
The C++ Standard Library includes STL, but is a lot more, including things like I/O, math, concurrency, and so on.
The maintainers of Microsoft's C++ standard library use the term interchangeably, both "STL" and "C++ standard library" refer to the same thing. https://github.com/microsoft/STL/blame/main/README.md#L3
I did that to get minimal wasm binaries, not sure if tree shaking today would sufficiently shrinks c++ wasm apps.
I recall years ago there was some kind of competition in implementing full C++ parser/front-end and standard library, or something like that, allegedly organized by nvidia.
Maybe you're thinking of cppgm.org — the "C++ Grandmaster Certification"?
- http://web.archive.org/web/20190824232557/http://www.cppgm.o...
- https://news.ycombinator.com/item?id=5148895
Nothing to do with Nvidia, though.
Is there an explanation for the py prefix? Is that some sort of cruel joke?
Rant. I do not see any improvement in the outcome code. Lets not nitpick on fast parsing and just scroll through unnecessary code actions.
> ... u8line(move(line))
We are not reusing parsed line object between iterations. Forcing fresh allocation per line.
> auto words = ...
Fresh allocation per line.
> lookup/insert
Lookup and hashing done 2 times for each word. Each unique word individually allocated on the heap.
> stats.push_back
Not preallocated. Likely doing full allocate + copy per each word.
> sort_relocatable
Could have been faster with additional memory provided. But this is minor because sorting probably was not ideal in the first place.
and the icing on the cake:
>printf("%d ... (int)count ...
As old saying goes "One can write Fortran program in any language". There are zero reasons to write non type safe text output in 2025 in C++ but here we are.
TLDR. One can name their foundation library any name and use any namespace it does not change how the code written much. Right?