Vernor Vinge's concept of "focus" in "A deepness in the sky" really clicked with me and put into words what I'd always had at the back of my mind. A highly recommended read.
While I do enjoy coding, building things is what was rewarding, not actually laying down brick after brick after brick after brick.
I've done that for 10+ years. The fact that I don't have to do that anymore is a relief that cannot be overstated. I only have to check on the code that The Ideal Intern produces while I focus on what actually matters, and that is: Building The Thing.
It's all about how you use it and how careful you are with it, and the fact that today there are people IN THE FIELD that still question this baffles me.
I think this is a sort of brainrot which explains why a lot of times things take unjustifiably long to implement. LLMs can help so much with stuff like this if you know how to yield one. But it can also let you more quickly hang yourself if you don’t even care about “which modules even exist” ( to continue with my example).
This is so real!
I don't remember if I read this advice or just intuited it myself (perhaps after some hard lessons), but it's a programming habit I've kept for as long as I can remember (I started coding in the 90s). If I feel rushed, e.g. someone looking over my shoulder, and I copy+paste something, it always leaves me with a sense of unease. It creates a memory & comprehension hole that sticks out like a sore thumb, even for seemingly simple snippets. You can't really be sure it's simple without stepping through it carefully, and simple can be deceptive because it's usually the interactions and assumptions wrt surrounding code that lead to surprises. Typing out code manually gives you time and space to consider the broader picture.
Where in the heck are the efficiency gains? Couldn't you just drop the LLM part of it and save the company a trillion dollars in tokens?
People are, in heavy LLM systems, realising that the most valuable commodity is engineers knowing WTF is going on, and that loss of understanding of your codebase is the #1 blocker to actually getting things done. Any senior engineer knows this all too well
Its why I suspect we never tend to see any longer term LLM productivity stats being published, and why I extra suspect that LLMs have failed to achieve any kind of penetration into open source code. If you take away the pressure to produce bad code, writing by hand clearly wins massively in the long term. Its classic short term gains for a long term penalty
For me I feel that LLMs have exploded (in a good way) my cognitive capabilities. I'm now the general of an army, rather than playing the role of a soldier. Of course that means that I lose the experience of being the lonely soldier, but it is a no-brainer tradeoff for me.
Anyway, I have to go now so I can push my car to the grocery store (so I don't forget how to walk), me and my giant calves will be back in a few hours.
All meant in good spirit. Keep doing what you're doing, thanks for sharing, and hope people are kind and only give good natured ribbings.
Whatever this approaches, it is not a state of software engineering I find desirable.
Personally, I don't think we will ever be able to reconcile using LLMs and cognitive debt. Even before LLMs we were aware if it: we knew people moving to managerial/PM roles eventually get their coding skills rusted. Well, now we are all in those managerial roles...
But a big yes for still typing code by hand, and not leaving it to the llm. Except it has to be the code generated by your brain.
That is what will create new neurons and new connections, which is what will keep away the cognitive decline.
And the constraint of not having to use llms will enhance creativity.
Actually, the constraints llms add to your code are more in number than the former. llms code in only the specific ways they've been trained on. So you won't ever come across of other ways.
Off the top of my head.. here's RubyQuiz.com [0] which I came across when I was learning ruby more than a decade ago. Looking at the many user-submitted solutions (you have to download the zip file!) you'll see completely different ways the problems were solved.
Sure, many won't be deemed efficient or standard by today's llm or rubocop checks, but looking at their code.. and retyping them and seeing them work.. was crucial in how I was able to think in Ruby for solving coding problems.
I did the same with Go too, with the "learn go with tests" guide [1].
[0] - http://rubyquiz.com/
This is just a miserable career of "paint-by-number" because people can't be bothered to have a creative thought about their professional work or programming hobbies.
Software developers think that they are being clever with these kinds of strategies to "keep their skills sharp", but unfortunately the entire industry knows about this, and especially the upper management who are already eliminating these assembly line, JIRA-ticket-taker software jobs en masse.
I also tried where I would print out the code (with color) and then go read it with a red pen. Faster and similar results of forcing me to be able to read it enough to be "an editor" to the code.
And I have done the same for LLM-generated code and text, back in '23 this meant more taking their so-so output and then turning it into my own, but between now and then come up with a number of techniques to improve the AI output to more acceptable out of the gate so more learning than being inspired.
These are good techniques as it causes your own brain to rapidly learn the material, but no matter how good it is I have never met anyone else that does it so the real ponderable is assuming that everyone else doesn't do this and what does that mean?
Exactly as we don't write machine code letting the compiler do that, now and in the near future we won't be writing high level programming language code. We are moving towards working on a higher level of abstraction.
When I ask a frontier model to write a loop 10 different ways in Python and TypeScript and test the performance of each using a 1,000,000 iterations, it isn't creating cognitive debt. For the time being, I'm still racking my brain asking the question, how does garbage collection affect the performance.
I no longer do and blogged my updates for fear of being seen as a luddite:
https://kamens.com/blog/code-with-ai-the-hard-way-update https://kamens.com/blog/code-with-ai-the-hard-way-update-ii
That approach works well for me, so why would I retype AI-generated code? I wouldn't do that with code written by another developer, and their code can be just as buggy or weird in theory. You could argue that at least the developer knows how it works. That's true - for a week, a month, or a few years if you're lucky. Until he forgets or leaves.
The goal is to maintain the taste, for lack of a better word, that I've developed over decades of programming.
Claude put me on to the concept "Étude", so I've taken to calling it my Étude project.
I've been trying to address this by telling LLMs primarily how the code should be structured, rather than only what it should do. Still, any design I hand to an LLM will be underspecified in one way or another (if it were fully specified, it would just be code), and the LLM fills those gaps somehow - which adds to the cognitive debt, slowly but surely.
Retyping LLM-generated code is an interesting solution. You'd certainly end up understanding the generated code better than if you merely reviewed it, but I doubt it produces a mental model as reliable as the one you'd build writing the code yourself. The longer you think, the better your mental model gets - and outsourcing the thinking to an LLM means thinking less.
That said, I've started to wonder whether I'm solving the wrong problem. Should I really insist on an accurate mental model of the code I own? We'd find it strange for a non-engineering manager to try to fully understand every piece of code their reports produce. If that's the right analogy, then as LLMs' agentic capabilities improve, maybe we should stop treating LLMs as tools that boost our own productivity as a software engineer and start treating them as independent agents we manage and steer.
A debt implies repayment. But the loss here is not going to be repaid, it's permanent
Perhaps a more appropriate term might be "cognitive deficit" or "cognitive loss"
Creating while writing allowed us to build a mental context but in a unproductive way, it never scalled.
I believe we need to move onto a new way of reading codebases that go beyond reading line by line.
I know people have explored representing code in 3D spaces. I don't know the solution. But I believe that is the problem.
I tried this, long ago. I just wasn't getting Haskell, so I'd retype my programs from memory and reconstruction. Obvious for music, why not for code?
All these untested opinions? You'd think we were discussing BMWs.
Managers never got to write and internalize every line of code anyway. Arguably their essential skill is producing useful stuff without needing to do so. So is that 'cognitive debt'? When I use an agent to code, I do the same things: I validate the direction, organization, and core decisions, but delegate the details. It's an intentional calculation of risk, managing the scope of future problems if there's a failure of implementation somewhere.
It's a question of where you put the value of cognitive focus. I don't understand 'line by line' how my car's ICE works, even though it will likely impact me at some point, because while I might be able to fix it myself if it breaks, the likely applied solution will be to delegate that work anyway.
You can ask Claude to "sdoc me an annotated code walkthrough of this MR" and it will produce something like this:
https://smalldocs.org/s/Ju9GOmWZ0JXTtzqCVfgt1q#k=6HBrpcCjIu7...
I use this a lot to stay in touch with the code the LLM is producing.
[1] https://smalldocs.org [2] https://github.com/espressoplease/smalldocs
> For your next ticket or feature, engage with your LLM as you normally would to produce a design and implementation plan, but with that plan in hand, make the mechanical edits yourself...You can expect this to improve your knowledge retention compared to merely reading a diff. You want to go slower now so that later you can go at all.
Most of us read the specs, then raced into the coding phase, hands to IBM mechanical keyboards. One guy took a different approach. He took a legal pad and pencil, and wrote his whole program on paper before he ever approached a terminal. He’d do his own bug checking and syntax checking, instead of having the compiler do it ( compiles took longer in those days, and required JCL ). He avoided the entire compile/wait/read-with-dismay/quickly-try-again loop.
He was one of the top students, of course. And a lot less stressed, as I recall.
Retyping things is inefficient for learning. It's like trying to retype calculus solutions — you don't learn from it. Even if there is an explanation of why the code is written in such a way, you did not come up with it, and you don't know alternative solutions. It is a practice for memorizing, not for building your intuition.
A better option is to write it yourself first and ask LLMs for better options. They are pretty good at it, especially when you need to optimize hot loops.
At the very beginning stages you might be tempted to trace over an image but this builds a very shallow understanding. Instead you should quickly move onto replicating work you admire by sight. Consider the form, volume and values, conceptualise them in your mind and try to apply them in whatever medium you are using. This helps form your own mental model of the process. Eventually you can start using those techniques on original works.
I'm currently rethinking language learning too in a similar manner. Comprehensible input, shadowing. That sort of thing.
I'm just kind of enjoying the feeling of riding on the swell of a gigantic wave for once instead of feeling like I'm starting from behind it.
This is part of how I learned how to code! Well, sans LLMs. Instead of copypasta-ing code blocks from books, Expert Sexchange and Stack Overflow, I manually retyped everything, looked up what I didn't understand and changed what I could.
Isn't a working program itself the best textbook? It's just a difference in learning methods. Depending on Stack Overflow is also a dependency, and searching for code on GitHub is also a dependency. How much dependency you allow is purely a personal difference, and it varies depending on your own study habits and learning style. Whether your learning method is superior or not likely depends on how your brain works.
People tend to think that the more painful something is, the better it is.
I don't deny that there are talented people who can read the manual and build everything from scratch. But I think that analyzing and rebuilding a working template step by step is also valuable.
I agree with the view that LLMs may cause cognitive decline. But if you go down that path, Socrates already criticized writing for weakening human memory. And how did that turn out? Books became a universal medium for knowledge. Then the internet came along. When Stack Overflow appeared, there was opposition, but it also had explosive adoption. LLMs are just the next step in that sequence.
If there is cognitive decline, I think there's also compensation in other areas. Using LLMs clearly causes some cognitive decline. And I think there are areas that need to be reinforced to compensate.
But having a baseline to work from—modifying already-working code—is genuinely helpful. I don't see what's wrong with using that as a way to learn.
This was true when I first learned to program, and is still true today. And I do find myself manually typing in really critical code. In those cases even if I do have an LLM alongside these days, LLM suggestions also then get manually typed.
For example, when I am reading a book, I often take notes of my favorite quotes/sentences/things that stick out or perk up my ears. Usually if I own the book or it's a used copy of mine I make a note of the page number on the very last blank page(s) and write out the full sentence/quote. Then when I go back months or years later I just open to the last page and can find all my favorite/important quotes. Most 200 page books typically have ~20 useful sentences for me to reference [going forward] - it's basically like making your own index or table of contents.
That long aside, just to say, if I'm not "very aware" or "consciously mindful" of what I'm copying down or writing, I'm able to do it very well on autopilot without etching anything into my memory -- as if no recallable moment gets etched into the wet cement of the brain unless I'm actively "hip to it." I wonder if others have this "autopilot" problem.