back

by jasonpeacock·4y ago·view on hn ↗
C was invented in 1972 and became one of the most widely used languages in the world with "the compiler is the specification" - and there were many different compiler implementations too.

It wasn't standardized until 1989 - 17 years later!

I think Rust will do just fine, and the compiler error message is (IMO) the best of any language, making adhoc exploration possible.

Can you imagine trying to explore C from ad-hoc alone based on its compiler errors?!

7 comments
Sometimes doing a formal standardisation process too early can result in stifling innovation. I’m too young (at 36) to have insight in to the C development processes, but I suspect that having competing compilers trying out different things will have resulted in a better language in the long run.

It’s in many ways similar to the way the early web was developed, different browsers tried out many different additions, copied the better ones and improved them. The standardisation process we now have didn’t exist back then, and has been through a number of iterations itself since (the “xml every thing” process that ended with xhtml had a questionable outcome at best).

Some may argue that a formal standardisation process should have started earlier, before IE became too dominant. But I’m not too sure, IE was incredibly innovative and introduced some important technologies and apis, before stagnating.

In fact the “modern” html/css/js standardisation process is specifically lead by individual browsers implementing and trying out new additions, with formal acceptance dependent on other browsers also implementing it. It’s inherently a implementation lead standardisation process, not necessarily committee led.

>Sometimes doing a formal standardisation process too early can result in stifling innovation.

Realistically, I think the most likely outcome if there's a formal standard is similar to C: You end up with one compiler that adds extensions to the standard language that are too useful to ignore, and then ~everyone copies them anyway. The most useful extensions are then standardized in the next version of said standard.

Meanwhile, some poor suckers will be working in We Must Follow The Standards hellscapes and will have to stick to the ISO version, and they'll be completely left behind by the community.

The early web with every browser doing something else was terrible for website authors. A lot of the innovation in the web started when there finally was standardization so that these fancy features could actually be used reliably across browsers. You are representing that history in a completely wrong light here.
The web is a completely different beast though. You can reasonably target only one compiler, even if you lose some platform support. You cannot reasonably support only one browser.
> You cannot reasonably support only one browser.

*Looks at works best in Chrome sites.*

You sure about that?

The catch with this is the resultant ANSI C specification is extremely hard to understand because it has to encompass all of that variation.
The major difference between Rust and [Zig, C] is that the latter are essentially "cocky macro assemblers". They derive their semantics from the semantics of the underlying assembly. Take https://github.com/ziglang/zig/issues/12251 as an example. Several people from the Rust and Haskell community slacked off on twitter about how awful of a language design this is. However, when you think about what the language actually compiles to it makes perfect sense, and is pretty straightforward.

Rust has the goal of putting as much smartness between what you type and what the compiler produces, which is a perfectly fine goal, but without a specification for the input semantics it's a pretty wobbly thing, especially for a systems programming language. A lot of bit packing code is simply not writable in rust today without immediately invoking UB, that works for now but might break with every bugfix release.

> The major difference between Rust and [Zig, C] is that the latter are essentially "cocky macro assemblers". They derive their semantics from the semantics of the underlying assembly.

I don't know about Zig, but as for C, this take is not just wrong, but dangerously wrong. The behavior of C code isn't defined by the underlying assembly, but by an abstract machine model that may or may not match your real machine (narrator: It doesn't). As a result, the compiler can and will ignore the intention of your code when it can prove that your code would invoke UB. Good example: signed integer arithmetic and overflow tests. It is very difficult to write overflow tests for signed integer operations without accidentally invoking UB, and modern compilers will simply remove your overflow tests instead of translating them 1:1 to assembly instructions.

Assembly in the macro-assembler sense. Doesn't matter that it's obscure and not a direct mapping to any specific ISA, there is still relatively straightforward mapping between syntactic constructs and something von neumann-ish.

Rust simply lacks that mapping, yet.

Rust has that mapping in the same sense that C does: there exists a fairly simple way to implement every language construct (except maybe for dynamic method invocations) in assembly.

However, Rust and C alike have another whole dimension to their semantics, that of Undefined Behavior, which is not reflected in the assembly, and which needs to be taken into account for unsafe code authors (in Rust) / by all programmers (in C). See for example https://www.ralfj.de/blog/2019/07/14/uninit.html for what goes wrong when you think of C as just a macro assembler.

The distintion between Rust and C looks interesting. Are there any formal (or formal-ish) grounds for saying that only authors of unsafe code need to be aware of Undefined Behaviour in Rust? With MiniRust, would it be plausible that some sort of "black-box abstraction" theorem could be proven, in a way that makes "safe" code depending on "unsafe" code insensitive to undefined behaviours?
Rusts entire trait model is a pretty big deal. The main issue is with memory semantics however, stuff like tagged 64bit pointers are UB land pretty much immediately. MiniRust solves this by providing an explicit memory interface trait.
If we understand "straightforward" in the sense that small changes in the input lead to predictable and small changes in the output, then I can't see how this is reasonably true for pretty much any optimizing compiler.

Just today we had this nice example on the front page where initializing a variable differently completely changed the codegen for large parts of the program: https://jpieper.com/2022/08/05/debugging-bare-metal-stm32-fr...

It's also worth noting that (at least for C), the way the standard is worded an optimizing compiler and a non-optimizing compiler can generate exactly the same output in every situation. NONE of the unpredictability actually depends on optimization, it's just that optimization can (sometimes, if you're lucky) expose it. If you're not lucky, the code gen changes based on something harder to control than a single compiler option.
Rust screams UB much quicker when it comes to memory semantics than any other language that I know. Rusts lower bounds of code mangling are also much higher than in other languages.

I don't expect to have a good mental model of an optimising compiler in "gotta go fast"-mode, but a mental model of what the memory layout looks like is pretty darn relevant when designing or working with ABIs and binary data representations.

This is still wrong though, since UB doesn't exist at an assembly level, and yet C has it. Therefore there is no clear mapping from C to assembly (even considering hypothetical ISAs).
> Doesn't matter that it's obscure and not a direct mapping to any specific ISA, there is still relatively straightforward mapping between syntactic constructs and something von neumann-ish.

No there isn't. "Something von neumann-ish" would have behaviour for all inputs - maybe not desirable behaviour, maybe even different behaviour on different processor revisions, but it would have behaviour. The C abstract machine doesn't.

> Assembly in the macro-assembler sense. Doesn't matter that it's obscure and not a direct mapping to any specific ISA, there is still relatively straightforward mapping between syntactic constructs and something von neumann-ish.

That applies to pretty much all imperative languages that compile directly to assembly/machine code – including Rust.

C used to be a "cocky macroassembler", up until people started writing optimizing compilers for it.

If your compiler merely translates a source line into a series of assembly mnemonics, function calls, or interpreter gotos, then the interface is the implementation. You can rely on the underlying target language to provide your program with meaning and the only people who have to care are people reimplementing your compiler for compatibility.

The moment you start talking about optimization, then this no longer works. You no longer have a correspondence between source and compiled forms of one program. You have a many-to-many relationship where one source form can be compiled into hundreds of binaries depending on how the compiler is configured, and many source forms may actually optimize to the exact same compiled form. This requires you to provide your own semantics, else compiled programs have no meaning and -O3 becomes shorthand for "make demons fly out my nose".

In the case of C they came up with a series of rules for what-not-to-do that both did not match existing language semantics and also were dangerously incomplete. There are still C programmers who insist that you can free() memory but still touch it for a "little while"[0], or access memory "off the end" of an allocation[1], for example. And ISO C still made the mistake of retaining pointers, which are a confusing mix of value and reference type. They aren't references because you are allowed to cast them to and from integers; and they can't be values because you can use them to modify other values. Because of this tension, we keep discovering new combinations of valid transformations on valid programs that cause miscompiles, and then we have to invent things like pointer provenance to fix them.

As far as I'm concerned, the only difference between Rust and C is that Rust is honest about it's cleverness. C has to pretend to be simple while also out-clevering Rust (or at least, the safe subset of Rust).

[0] Usually in an attempt to emulate automatic memory management. Manual memory management does not work when passing complex structures across an API boundary, and the only options are to either expose custom deallocators (which means no optimizations even when they are sound), tell callers how to deallocate the data (which means no changing the data), or hack the allocator to do what you really want.

[1] It works for malware developers, it should work for me, right?

> they can't be values because you can use them to modify other values

That seems like an arbitrary criterion. They're values because they behave like values when copying them around etc. You can't even use them to modify other values implicitly - you still need to use * to get an actual dereferenceable/assignable lvalue out of the pointer.

What operator is used to modify the value doesn't matter, just the fact that you can do so. At the point where you use an assignable lvalue, the optimizer needs to know where that lvalue came from, which means that pointers need to be tracked like references.

My point is that because pointers act like both values and references, they are neither values nor references. This makes it impossible to soundly reason about them.

Rust has no explicit goal to introduce magic - instead the goal is to be safe and sound. I'd argue the code very often compiles to what one might expect it to compile to, its certainly better than C++ when it comes to magic in terms of how useful and transparent said magic is.
C++ is arguably the most complex programming language ever. There is not much to be gained by comparing against the supremum.

Take a look at the languages that rust was influenced by (https://en.wikipedia.org/wiki/Rust_(programming_language)) those aren't languages with straightforward compilation semantics.

There is a reason why rust has a datalog engine build into the compiler (https://github.com/rust-lang/datafrog). Which is imho totally rad and awesome, but really hard to fully form a mental model of without a spec.

This is pretty vague reasoning. Rust draws inspiration from each of those languages for something but that doesn't mean they affect its operational semantics, which are quite similar to C's.

The fact that you point to datafrog is illustrative: while it is not actually built into the compiler today, the use case for it is borrow checking, which famously does not impact the language's operational behavior at all! It is purely a compile-time analysis that does not influence code generation.

(For example, consider that mrustc and the GCC frontend are both able to omit borrow checking entirely and yet still produce runnable binaries!)

Of course, the borrow checker is still something that you need to build a mental model for, but by design it is okay for you to get that wrong occasionally, because the result can only ever be "your program still does what you thought, but the analysis proved that in a way you did not expect" or "your program does not compile."

I bet PL/I, Ada, and even Python 3 aren't far off.

Same applies to the Java and .NET ecosystem, because either you swim on the surface, or you really get to know how the implementations, down to bytecode, JIT, GC and standard libraries work, and now they are full speed ahead with 6 month release schedules.

after 5ys of (hobby but frequent) rust I'm pretty confident most people cannot tell what their code will compile to and what its runtime profile is going to be (aka you can't reason about performance -> so there is a lot of magic/complexity)

I've recently tried Zig and switched to it instantly, it's hard to explain but basically Andrew has a very good taste at picking important features and keeping the language complexity very low.

You know how it takes some time to learn borrow checker and macros and generics and traits and all weird rules of what you cannot do and then trait bounds and then it doesn't work exactly like you need, or the crate does not support something and you cannot implement it yourself, etc. etc.

So in Zig I had a hello world on day one, and the first thing I did was encode/decode ANY json messages for tagged union (which unfortunately is not supported in std but it was very easy to do it myself) and it worked! I did this the first day in entirely new language (I was not even doing C/C++ before) and it would probably take few days in rust and I'd probably mess up something and I know it wouldn't work for every case and every crate, because of orphan rule. In Zig it would work for any struct, internal or external. And also Debug, Display, Eq, Partial, all of that works automatically. That's huge!

And the worst thing is that recently, I've started using pointers again, and when I look at the code I don't see anything unsafe in the structure itself, it can be used unsafely but that is also very easy to fix in Zig because you have these explicit allocator and it's so easy to put everything in the same arena transparently, or use SegmentedList with stable pointers.

> Rust has the goal of putting as much smartness between what you type and what the compiler produces

No it doesn't. "smartness" is a means, not an end or a goal.

> "It really does feel like rustc is a big step toward the mythical Sufficiently Smart Compiler, letting you write nice code without worrying about a suite of low-level gotchas."

> Hear more from Reliability Project Director

https://mobile.twitter.com/rust_foundation/status/1385310806...

Literally nothing there supports what you said. Don't make shit up.
You shouldn't be giving Zig (or C, for that matter) a free pass. If your language isn't memory safe, and it compiles to LLVM with the optimizations on, then it's inheriting most of the complexity of Rust's undefined behavior. (I say "most" because I suspect that Zig doesn't use attributes like noalias as much, but there is still a lot of complexity around, for instance, provenance that are orthogonal to those attributes.) It's LLVM, not the language frontend, that actually implements most of the memory model (via lowering to machine instructions).
C had a Reference Manual (e.g. https://archive.org/details/crefman/cman74/ and https://www.bell-labs.com/usr/dmr/www/cman.pdf) which was included as an appendix in the 1st edition of K&R. That was effectively the specification through the 1980s.

At about the time K&R1 was published, the Ritchie's original C compiler was being replaced by pcc, and `lint` was another early (diagnostics-only) implementation. The K&R1 reference manual includes a few mentions of features that were deprecated by pcc, or new in pcc.

So I think C was more mature than "the compiler is the specification" by the end of the 1970s.

I would say that, on the whole, doc.rust-lang.org serves a similar purpose to K&R. It's a comprehensive tutorial, a good guide for the standard library, and a good start for a reference but not strictly a complete definition of the language.

K&R is remarkable for fitting all that into one slim book, and Rust's documentation rightly doesn't try.

Rust's documentation also has some considerable advantages from its modernity. That library documentation gets to link to the actual source code which can answer tricky questions you'd never cover in a book, yet in many cases that source is very readable, its examples are something you can try for yourself with one click, instead of one of the very best human compiled indices of any book it has text search and hyperlinks.

Beware of reading the library sources though, they can change any time without warning. Only the docs are guarantees that remain stable as Rust gets updated.
> C was invented in 1972 and became one of the most widely used languages in the world with "the compiler is the specification" - and there were many different compiler implementations too.

> It wasn't standardized until 1989 - 17 years later!

> I think Rust will do just fine,

Not based on the example you gave - using the behaviour of a single C compiler as a specification for the language implemented by that fairly easy due to how small C is. Rust is very much larger than C.

> and the compiler error message is (IMO) the best of any language, making adhoc exploration possible.

> Can you imagine trying to explore C from ad-hoc alone based on its compiler errors?!

Not much different from Rust: Gcc and clang C compiler errors are exceptionally good, and I miss them terribly when programming in some other language (like C++, or Javascript).

There was the original K&R Book on C, though, the one which was about 100 pages, was super clear, and i believe gave compilers a common idea of what is normal.

I wish i still had my late 80s copy.

The first edition of K&R is 221 pages, so roughly double what you remember.
Same in big-O though! And man I'm jealous you have one!
And we had lots of "fun" porting C code, including micro dialects like Small-C and RatC.
> I think Rust will do just fine

Not mainstream language 'fine', but fine.

Rust is already a mainstream language, unless you have a weirdly restrictive definition of mainstream.
Not sure I'd go that far. I am getting paid to work full-time in Rust right now, but... My experience is the # of companies doing this that aren't "web3" companies is pretty small. Insignificant as a % of the total industry. Rust forums on LinkedIn are dominated by chatter about Solana etc. When recruiters reach out to me about Rust, it's almost always some blockchainy thing. I wouldn't characterize that as mainstream.

Type "Rust" into a job search engine and see what comes up, it's mostly blockchainy stuff. Or a few shops that are "looking into" doing Rust for new projects (which is encouraging, but hard to say how that will shake out).

I personally won't touch anything "crypto", so I feel somewhat lucky to be getting paid to work in Rust right now for something that isn't. But I fully expect that once this gig finishes, it'll probably be back to C++ for me.

Microsoft, Amazon, Facebook/Meta, and Google are all using Rust in production. It's not just web3 nonsense.
> My experience is the # of companies doing this that aren't "web3" companies is pretty small.

Those are just the loudest people. Web3 companies depend on hype so they shout all over the web about anything they can, including how they're using the hot new language for blah blah blah and you can come and write it with us! (As long as you're willing to be paid in worthless tokens.)

In normal companies there is plenty of Rust use but you don't hear about it as much because it tends to be initiated internally by people who weren't hired as Rust programmers, and it's used for new projects that take a long time to get to the point of needing new employees. I know in my company it was at least 2 years between when we started using Rust and when we wrote "Rust" on a job advert.

it would be sad if rust gets ultimately typecast into a "blockchain" language
> belonging to or characteristic of the ideas, attitudes, or activities that are regarded as normal or conventional, or the dominant trend in opinion, fashion, or the arts

by dictionary definition, rust is not mainstream. it's not normal (common, typical, customary) for an engineer to interact with rust

I would consider it perfectly normal to discover something in my company or on Github was written in Rust. It's the 12th most popular language on Github by PRs according to this: https://madnight.github.io/githut/#/pull_requests/2022/1 (Shell and Nix don't count).

It's only 3 times less popular than C. As I said, very weird definition of mainstream.

Yet here you are on a discussion about it, of many rust discussions. Many companies use it enough that most of the MANGA companies have openings for it.

How is that not mainstream by even your own definition? It’s consistently a part of the current zeitgeist.