▲ 371 points
back
109 comments
Hi! I'm the person who made this thing!
Thanks for reading and thanks for all the corrections, in glorious Hacker News fashion. It's definitely a pleasant surprise to see my project here. Putting the "You" in CPU is still very much a work in progress — I was hoping to polish it a lot more and even add some more content before posting on HN later next week :)
Some backstory on me: I'm 17 and left high school a year ago to work full-time at Hack Club (https://hackclub.com/). I've been programming for as long as I can remember, and started homeschooling about 6 years ago to focus more on that (and my other interests).
Since I'm entirely self-taught, I haven't taken any college systems classes — and while I had picked up a lot, I wasn't happy with my answer to "what happens when you run a thing." So I let myself spend a shit ton of time actually learning as much as possible. What I found was that:
1. Operating systems and hardware are really fun to learn about!
2. Wow, online resources on this stuff are terrible.
A decent portion of my research ended up at PDFs of lecture slides from 2014, or StackOverflow answers that were, in fact, incorrect or vastly oversimplified.
So, I wrote Putting the "You" in CPU to hopefully provide a better resource to people who wanted to start teaching themselves all this stuff! While I don't provide perfect coverage (still need to write a couple paragraphs on SMP), it's a lot better than most of what I've seen. I also had fun drawing and making diagrams for the first time, I think I definitely got progressively better and I'm really proud of some of the drawings in the latter few chapters.
P.S. The whole thing is open source on GitHub if you want to look under the hood: https://github.com/hackclub/putting-the-you-in-cpu
> Wow, online resources on this stuff are terrible.
I've been thinking this a lot lately, searching for things like: a detailed block diagram of the Linux kernel, how page tables work, a comparison of function call ABIs on different platforms, details about low-level network protocols, reserved Linux PID numbers, etc.
Occasionally I would find a Reddit thread with an answer, but it was mixed in with the usual Reddit dross, and I doubt that Reddit is a reliable place to store knowledge. StackOverflow was particularly unhelpful, as the questions are mostly oriented around accomplishing particular tasks, and the answers are necessarily brief.
Nowadays, for general knowledge, I skip Google's enshittified search and go straight to Wikipedia, but Wikipedia has limits on how much detail it will go into about technical topics.
I wonder if there's a need for a comprehensive wiki for all the details of computer guts? There are lots of scattered resources, but having them all compiled in one place would be awesome.
A wiki for operating system internals. I like this idea!
I had the same questions when I was about your age and found a lot of intersting answers in the Linux source code. of course it was a lot simpler back then. what really got me going was the osdev community (much smaller back then) and Intels system architecture documentation. iirc part 3a or 3b was where the good stuff is. it tells you a lot less about how things are and a lot more about how things could be. you also get to learn plenty of fun things in OSDEV too like how 32bit processors did 36bit addressing with PAE or how 64bit processors did 52bit addressing to squeeze money out of people. how computers start in 16bit mode and you have to learn systems level assembly code gymnastics to make it to long mode, why there's an 8bit mode around, and more. if you liked learning about loading binaries, you'll love how many surprises still exist. like 64bit mode still including memory segmentation but forcing it to flat mapping for lonh mode
Heyo, as another teenager who has really enjoyed participating in Hack Club, it's great seeing this work! This is super great. Thanks for sharing :)
Amazingly impressive! I just spent 5 minutes on your website, changing colors of adjacent tiles.
I knew I wanted my personal site to have some sort of interactive gimmick, seems like the tile grid worked out. Glad you enjoyed it!
Fantastic work! This makes for a great resource to dive into OS dev. There's a lot of detail but it's easy to digest. I suggest slightly revising the end of note #2 for chapter 2 to reflect that some async paradigms in programming languages are cooperatively multitasked. Within a process, it's generally considered safe to utilize cooperative multitasking (desirability is a different question), although I believe Erlang sticks with preemption for robustness.
> I was hoping to polish it a lot more and even add some more content before posting on HN later next week :)
That's on me, I'm a fan of your work and saw this pop up on my GitHub feed. I probably should have considered if you were ready for it to be shared. I hope the attention was mostly positive!
Hey, you’re the author of water.css! Love that project. It really helped me when I was about to launch my blog and didn’t want to get held up for months by fiddling around with CSS. Thanks a lot for providing water.css!
lol water.css never escapes me
glad you like it :3
Wow! I'm impressed!
Have you actually gone down already the rabbit hole that is the history of computing and alternative approaches to what we have today?
There are a lot of rabbit holes to go down and not enough time in my day! I cannot say in good conscience that I have gone down this one, although I've picked up a lot of factoids. I've read a couple books on the history of computing, but I'm not sure how much I actually remember — likely because I didn't go on many research tangents. Occasionally I also just get curious and Google something. So, definitely picked lots of tech-history things up, but not an expert by far. Perhaps some day I will be.
Love resources like this. I’m getting my feet wet in the RISC-V world, and it’s clear that people who want to push FOSS forward are going to need knowledge like this to get software working well on all of the SBCs that are coming out.
Just FYI, the RISC-V instruction encoding is considerably freakier than just about anything else you could be exploring except perhaps Thumb-2. They had good reasons to make it like that, but still, if you’re planning on reading hexdumps RISC-V is not going to go easy on you, even if it’s simple from most other angles.
wouldn't any programmer already know this, assuming you studied for a bachelor's degree or more?
At least if you didn't skip the computer architecture and operating systems classes.
For me it was a common thing to define an ISA, implement a simulator and assembler, and hand-write some assembly as part of learning how computers work. A dedicated man might go the extra mile and implement it in FPGA, or write a LLVM backend.
Then there is all the operating system aspect. It's quite common to implement your own kernel there to learn, or at least to modify an existing one.
For anyone looking to find out more about how computers work at a very basic level, I really recommend the book "Code" by Charles Petzold. Goes from first principles all the way up.
And Nandgame https://nandgame.com if you prefer to learn by building.
there is also Turing complete which is pretty similar.
This is amazing, thank you!
One of the things this does well is it remarks on expectations versus what was learned in the process of compiling the information. “I expected x because y but actually it’s a because b!” Clearing up misconceptions is often one of the key steps in new knowledge formation, and it’s helpful for writers to acknowledge that people often need to overcome their misconceptions as they process the information in the writing. It’s a much more reader-friendly approach than just being like, “Well, this is correct, and if you thought something else you’re a dummy.”
Btw, Chapter 6 contains wrong explanation of fork() return values. It should be opposite than stated: in the parent, child's pid is returned and in the child it's zero; just created a pull request for that.
Oh, thank you so much, and whoops :)
Fixed the code example and merged!
> For example, add eax, 512 translates to 05 00 02 00 00.
> The first byte (05) is an opcode specifically representing adding the EAX register to a 16-bit number. The remaining bytes are 512 (0x200) in little-endian byte order.
That explains only the first 3 of the 5 bytes in the instruction. What are the remaining 00s for? Did he perhaps mean that the opcode is for adding EAX to a 32-bit number?
*she and whoops, yep, I think that's supposed to say 32-bit. Good catch.
If it’s talking about x86, then yes the eax register is 32 bits wide. The low 16 bits of the same register are called “ax”.
Great article! As a self-taught web developer I find that kind of resources very valuable. It help me to move away from all the abstractions and towards a more concrete understanding of how thing work. Thank you.
This looks like a good resource, and gets into the meat very quickly, and is sort of entertaining. It is less "monad tutorial"-esque than I expected. Look forward to reading more of it.
Thanks! Glad you liked it — let me know if you have any feedback later on.
Great resource! However, one thing left me with a question about the kernel:
From [0]:
> Since shebangs are handled by the kernel, and pull from buf instead of loading the whole file, they’re always truncated to the length of buf. Apparently, 4 years ago, someone got annoyed by the kernel truncating their >128-character paths, and their solution was to double the truncation point by doubling the buffer size! Today, on your very own Linux machine, if you have a shebang line more than 256 characters long, everything past 256 characters will be completely lost.
What is the reason for silently truncating someone's filepath? The kernel, of all places, should not be doing such things. Granted, 256-byte long file paths don't make sense, but 256-byte long path+argument strings will almost certainly happen sometimes. To just silently break someone's script is wrong.
(Author here:) I strongly agree! It should not require an understanding of the kernel's execution logic to be able to understand the limitations of shebangs. I think this is stupid design and I perhaps should've highlighted it even more as such.
A couple years ago someone attempted a patch to the kernel that would at least fail loudly in the case of oversized shebangs. This ended up causing problems for NixOS which, being Nix, already was using oversized shebangs with lots of long nix-store paths. They were previously being silently truncated but still working, and suddenly all those scripts failing caused a lapse in backwards compatibility! So the patch had to be undone and we haven't gotten anything similar since.
See https://lwn.net/Articles/779997/ for more info.
you shouldn't be putting complex arguments in a shebang.
it's there to tell where the interpreter for the current file is.
shebang can't even handle more than a single argument on its own. if you have `#!/bin/program -args somescript`, it will divide it into ['/bin/program','-args somescript'], which is almost never what you actually wanted. ( this is linux specific, shebangs aren't necessarily universally portable between all unices )
if you want to do something complex, make it `#!/bin/sh` and then `exec` from the current file with whatever arguments you need.
one thing that link doesn't mention is that you can use env to lookup an interpreter via the current path.
`#!/usr/bin/env python3` for example, will find whatever `python3` the users is using, without having to know where it is.
this can be useful for scripts that you will be running from virtual environments, for example, which work by overriding the PATH to the python interpreter.
you wouldn't want to do this with anything installed, however, since you wouldn't want installed programs to let the user control what interpreter to use. instead, you should just use an absolute path.
there is an option in env, -S, that can be used to parse a command line from a single string specifically to handle this limitation.
after years of avoiding much trickery in shebang lines, it seems like a questionable practice at best.
I'm more surprised it isn't set to be whatever the disk block size is (which is often, but not always, 512 bytes).
Curious, are material such as this targeted for folks who did not do undergrad in CS? Don't typical CS grads know about this?
My article is, indeed, targeted towards folks who did not do undergrad in CS! Self-taught people need better resources to understand more low level things. I have never taken a CS class, undergrad, high school, middle school, or otherwise.
> Chapter 4: Becoming an Elf Lord
Clearly this is targets only people with multiple PhDs in electrical engineering and computer science.
Awesome work - I like the balance between the details and the high-level overview.
How do you get it all on one page? That gives you the ability to ctrl+f stuff, skip past things you know by skimming their contents, scroll back up to things you need to re-read, and way more.
This is still a WIP and I kinda wasn't expecting HN-level attention yet! I was definitely gonna make a 1-pager version, especially for offline reading (ie. public transit). Also had some people asking for PDF/EPUB.
"The bottom of every page is padded so readers can maintain a consistent eyeline."
What does this mean?
If you press page down to scroll, at the end of the page you’ll typically scroll by a smaller amount than prior to that, which changes the level at which your eyeline should be after hitting page down to continue reading uninterrupted. In Emacs and Vim hitting page down works nicely, because they just scroll “past end”. Other programs, like web browsers, often stop at the end, causing this issue and making the reader look for the point where they stopped reading for maybe even 30-45 seconds. This seems to be a workaround for that.
The page will continue to scroll until the last line can fit to the top of the page. Nobody likes it when you can't scroll past the end, and have to then start moving your eyes/head to look at the bottom of the screen. Same with IDEs.
It means that you don't have to move your eyes horizontally very much. Some people (including me) like to just scroll.
Reading the title I thought this was about owning your hardware (as opposed to the Apple approach where they effectively own your hardware).
Commenting to get back to reading this later. Thank you for this resource#
LMK what you think!
>The bottom of every page is padded so readers can maintain a consistent eyeline.
Very considerate!
These projects are really fun. On the other hand, you might want to learn in a way that lets you build hardware (esp for FPGA's). For that, I suggest a few types of books with examples:
- Computer architecture or CPU design book covering modern designs with their tradeoffs
https://www.amazon.com/Computer-Architecture-Quantitative-Ap...
- Introduction to Verilog or VHDL (languages for hardware) maybe on FPGA's or well-known boards
https://www.amazon.com/Programming-FPGAs-Getting-Started-Ver...
- High Speed Digital Design (many recommended it)
https://www.amazon.com/High-Speed-Digital-Design-Handbook/dp...
- Cookbook with many examples for FPGA's
I don't have one. Most with these titles are expensive. One that looked promising had bad reviews. I'll leave it to HN to fill in the blanks.
- High-level synthesis
Most exciting one I remember when I stopped researching this stuff was Synflow. I don't know if they're still around.
Before that, I had saved Baranov's work which showed how to turn Abstract, State Machines into actual hardware. It was like a how-to on high-level synthesis that he used commercially at Synthezza. I still share them periodically in case someone wants to build an OSS version. I also saw ASM's used in verifying compilers and hardware in separate work. There's a good chance someone can turn many things into a unified, field theory of sorts using ASM's.
https://www.amazon.com/Finite-State-Machines-Algorithmic-Com...
https://www.linkedin.com/pulse/second-ebook-samary-baranov-h...