back
44 comments
- in about 5 yrs of time, this is not going to age well

- massive studies ll come out on how 50% of the software engineers dont know or understand what they "generated" at all and how their critical thinking skills have taken a nosedive

- every major software company out there that is busy firing juniors now is going to run into the opposite problem: finding talented juniors and they ll command a hefty pay

- the whole LLM premise runs on one little assumption that they ll improve indefinitely. you are forgetting the law of diminishing returns

- i ll come back here in 5 yrs, this whole thing wont end well

I wonder if people said the same when compilers first appeared in the software development scene.

> the whole LLM premise runs on one little assumption that they ll improve indefinitely. you are forgetting the law of diminishing returns

To be honest, I think I'd remain happy if they just stayed at the level they are now, maybe bit faster and cheaper (and ideally more energy efficient), best would be to run it locally too, but I don't really need them to get "better" to make good use of them at this point.

The difference is you can trust the compiler without knowing what System.out.println("Hello, World!") is going to ultimately look like, and the vast majority of what compilers output will never need human intervention to fix something or human oversight to ensure correctness and fitness for purpose.
>” To be honest, I think I'd remain happy if they just stayed at the level they are now, maybe bit faster and cheaper (and ideally more energy efficient), best would be to run it locally too, but I don't really need them to get "better" to make good use of them at this point.”

I have bad news … you’re gonna be mad… they’re gonna make it expensive otherwise they can’t get the hockey stick ROI.

I'd like to see them try to increase prices as long as open source models that are roughly in the same league are available
> I think I'd remain happy if they just stayed at the level they are now, maybe bit faster and cheaper

Definitely ! It's so useful as is, even without spitting a single line of code, as an analysis and documentation tool, it's already great. Now a faster and/or cheaper version: perfect. Make it runnable on commodity hardware: icing on the cake.

“FORTRAN should virtually eliminate coding and debugging.” — FORTRAN Preliminary Report, 1954.
Relative to punch cards and writing in machine code, I imagine the statement was entirely correct. And still is.
Not since the advent of the compiler have most developers understood what is being generated. Their efforts have instead gone into validation (often manual, but sometimes automated) to see that what was generated works as expected as that is all that really matters to the business goals.

Which is still where the effort is going, although there does seem to be more interest in automated validation now, which is welcome progress.

This is certainly overstating the difference between compiled code and the source code. If I write an if/else statement, I understand what the computer is going to do when it's evaluated even if I don't know or see the literal machine code.
Only if the compiler is perfectly naive. With the state of today's optimizing compilers, the output might be very different from what you'd expect. But it also doesn't matter because the business goal is the results from executing the program, not the static code. Hence why, as the earlier commenter pointed out, efforts go into validation rather than understanding the generated code. 99% of the time the code a compiler spits out simply doesn't matter. The same is now proving true as we move up a level of abstraction. The code spit out from an LLM doesn't matter any more than the code spit out from a compiler, only validation that it functions as expected matters.
compiler is not putting words together randomly with a mathematical probability but the LLM is
Yes, the difference between a compiler and an LLM is that a compiler's output correctness is binary, whereas an LLM's output correctness is probabilistic.

That may be an interesting distinction in a vacuum, but given that the input to both is words put together randomly with a mathematical probability a compiler's binary correctness isn't actually the useful property that it may first seem. In the real world, to overcome the probabilistic input, you have to validate the input with respect to the output. And it turns out once you have systems in place for that, you gain the same assurances with a probabilistic output as you do with binary output.

So, while there are some technical differences, there is no practical difference.

On your second point... that's the problem... IMO, humans need to read/understand and approve code changes generated by AI. If either through a lack of understanding or skill, they approve changes anyways, that's largely on them an they largely deserve what they get.

Like TFA, I'm a proponent of human gatekeepers in the loop. IMO, the result isn't much better at a team level when you compare dev + ai to a typical team and that level of output... but you get a lot more testing and documentation along the way just through the process refinement and iterations.

I think the harder issue is finding and growing Senior level developers that actually have a material understanding of what is happening and how things are working at a high and low level. The more junior, imo, the more likely someone is to just accept the AI result(s) that introduce more bugs in the process.

It really depends on th SDLC of the codebase. I've sat in the park with a pad and pen and wrote down ideas for a small-ish utility that had both a DB and some low level systems interaction. I designed all the interfaces, control flow, and user facing ui. A decade ago I was designing class diagrams with public and private methods, then coding it up. Today I can articulate it with loose words and have it come to life.

The secrete ingredient here is... Im an experienced programmer, I know what to ask and what to look out for. I know sockets and how devices work under the hood and which knobs are available for me to turn. Juniors just don't have enough experience. The question should really be "what do inspiring programmers do?"

People loke you're the minority. Most people use the brilliant strategy if "prompt and pray"

Because of AI speed is valued over everything else

Some of the code bases I have at work are legacy spaghetti -- hard to reason about. I too employ the prompt and pray strategy and then see if CI fails. I can only be surgical where I already have a grasp on the codebase, or if I was involved from the beginning.

Morally, I dislike what AI is doing to the profession. Rigor is suffering.

It's also possible to point the AI at legacy spaghetti and say "document this in detail" as a starting point for unspaghettifying it.
But won't it take a lot of time and resources to make it non legacy? Sometimes we don't get the luxury or option to work on stuff like this
Debugging. Lots of debugging. It will be interesting when the researchers look back in 5-10 years to determine which was easier/better - trying to understand generated code vs generating code one understands.

If a functional reasoning layer is added to the current lookup level, “do you want fries with that?” might be in their “doing”. If this doesn’t appear in a timely fashion, de-slopping code will be their lot.

> It will be interesting when the researchers look back in 5-10 years to determine which was easier/better - trying to understand generated code vs generating code one understands

We don't need to wait 5-10 years, we have plenty of experience with this already! Software devs always want to build their own because building it is way easier than grabbing something someone else built and understanding it!

There are three things programmers bring:

Salience - Agents only know what we bring to their attention

Judgment - Agents cannot make decisions in the larger business context

Responsibility - Agents do not care if they crash the system

Edge cases you never think to test: that's where AI output breaks. I deal with this daily in document processing.
Interesting question. I work on AI for document processing, and I have found the same dynamic: generating a first pass is easy, but the real engineering goes into the verification layer.
Design. And redesign.
and specs, and tests, and smoke tests, and more specs, and updated specs.

And don't forget processes!

And reading reports, endlessly.
Compilers also write code (they write machine code based on high level langs)… yet nobody asked the same question about them (or did they?)

AI needs to be instructed and then some kind of verification of the output is needed. Programmers do that

Bad analogy. Compilers don't reason, they don't hallucinate. They take the language as given and "compile" it to the equivalent binary based on strict definitions in the language. It's reproducible and consistent. Saying they write code is factually wrong, at best they translate it.

AI will do the same thing in multiple ways or veer of course and do it another way based on spoken language with no strict syntax. It will make logical leaps, it will guess, it will assume etc.

These things are not equivalent at all.

The analogy is fine. AI moves us up to a higher level language, and it is understood that higher level languages come with their own leaky abstractions, but the idea remain the same. It is different, but only to the extent that would be expected from stepping up to the next higher level. It's not something completely different. We don't yet have AGI that will magically produce the software before you even knew you needed it. We still need a programmer to write the programs.
It might not be the perfect analogy, but it does show some important parallels and differences. Compilers reason to some extent - for example type inference is definitely some deductive reasoning that I would have to do if the compiler didn't do it for me.

The hallucinations point is a difference and one that we haven't quite learnt to handle yet. Generally, compilers are deterministic (even it it might not feel like it sometimes) and there's a lot of useful properties that come from that.

I have recently put Claude into a CI/CD pipeline and it's made me realise how much I rely on pipelines being deterministic.

>> Compilers reason to some extent - for example type inference is definitely some deductive reasoning that I would have to do if the compiler didn't do it for me. <<

But this "reasoning" is done within a strict ruleset and includes "just find the optimum in X", and there is no space for "language interpretation", so I wouldnt call this reasoning compared to that type what LLMs are doing.

LLMs operate with fewer constraints and consequently a larger search space, but there is an apparent continuum seen in programming where constraints are relaxed and the search space grows. It's not just standard compilers and LLMs. There is a whole world of computer science found in-between.

In order to consider LLMs to be something different then there needs to be a clear dividing line and I'm not sure that there is. It seems more likely that it is the same type of reasoning, but that LLMs take it to a further degree.

> Generally, compilers are deterministic

Not really. Maybe more so than they used to be thanks to deterministic algorithms becoming faster, but most compilers still aren't deterministic by default. There are still advantages to non-determinism in compilers, like not having to worry about thread execution order. To be fair, most compilers these days allow you to optionally enable determinism. Then again, LLMs also allow you to optionally enable determinism.

Thread-connected non-determinism is not what the author meant by "compilers are deterministic"
If you consider hidden inputs to be part of the input space then the statement is obviously true, but then all computation is deterministic. It is fundamentally impossible for computers to be anything but deterministic. Maybe that is what the author meant, but it seems unlikely as it doesn't add anything. Mentioning determinism usually implies that there is an non-deterministic alternative, which in computing can only mean where there are hidden inputs.
> LLMs also allow you to optionally enable determinism

really? Please explain

What's in need of explanation? Computers are designed to be deterministic, and LLMs run on computers, so LLMs must also be deterministic.

There caveat to that is where external inputs give the illusion of non-determinism. Thread execution order, like we already discussed, is one such example. Technically still deterministic if you understand the external inputs, but for the sake of discussion we can consider external inputs to be non-deterministic. Which is why compilers usually end up being non-deterministic by default. However, computers are designed to be deterministic so there are ways to avoid introducing those external inputs, albeit often at the cost of performance. LLMs and compilers alike can be run deterministically.

LLMs have one additional property not typically found in a traditional compiler — a call to rand() — but rand() is also deterministic when configured with a constant seed and can also be turned off completely by setting temperature to 0.

Same as before? A fancy compiler that is able to accept code written in natural language still requires a programmer to write the source code.
Programmers collect visions to express in natural language, then loop attempts to express the vision until it is actualized
Probably dance.
I have people skills! What the hell is wrong with you people?