▲ 63 points
back
84 comments
Simple rule of thumb.
Always write functional tests first. Doesn't matter if they are slow - you still want something that faithfully captures the specified behavior and allows you to detect regressions automatically.
Then, if your resulting test suite is too slow, add finer-grained tests in areas where the perf benefits of doing so dwarf the cost of necessary black-boxing.
Getting down to the level of individual classes, never mind functions - i.e. the traditional "unit tests" - should be fairly rare in non-library code.
I'm not trying to be confrontational, but even "business logic" inside a project is often clearly separated between "library" and "glue, aka controllers or whatever" - and that's kinda the point where I disagree.
Not accepting "but the unit tests work" - 100% agreed. Not being able to slice it that the thing that could be accessed by the current public API or a different public API be tested on its own (and often relatively pure) is not how I want to write my software or tests.
Thing is, trying to make everything individually testable in tiny bits takes a lot of effort that wouldn't otherwise be needed. Furthermore, with all the mocking this usually requires, you end up with tons of extraneous code that now needs to be maintained as well (and by very nature of mocks, you'll have to fix them every time the corresponding real code changes behavior observable across the boundary). In practice, the ostensible benefit of knowing exactly what the problem is as soon as you see the test failure is usually dwarfed by all these costs. And, at the same time, the more mocks you have, the less likely your tests are to catch real issues.
Now, as you rightly note, in many cases there's already an obvious component boundary that is inherent in the design. And then it makes perfect sense to write functional tests for individual components along those boundaries. And in some cases, the component may be "library-like", and those tests will end up looking a lot like your typical unit test, where individual classes and even methods are covered one by one. But most classes and methods in a typical app aren't like that.
aka the inverted test triangle.
Agree w/the author that the concept of "unit" often hurts test quality.
You should be striving to balance the long-term usefulness of your tests with the debuggability of those tests. In my experience, those tests are what most people would call "integration tests" (although that name, like so much terminology in the testing world, is confusing and poorly defined.)
You want to get the tests up at as high a level of abstraction as possible where the API and correctness assertions are likely to survive implementation detail changes (unlike many unit tests) while at the same time avoiding the opaque and difficult to debug errors that come with end-to-end testing (again, the language here is confusing, I assume you know what I mean.)
My first job interview after college. It was a pair programming session but I would not touch the keyboard. I would dictate and the interviewer would type. I had to implement a dynamic array in java using TDD in one hour. From time to time I would break protocol and jump into implementation before the tests. Overall it went good. I finish the job. The interviewer however assesed that he was afraid I would not actually use TDD on the job, he was spot on. I did not get the job.
This is hilarious because it's a perfect interview (detects precisely the thing they're testing for) but also provides total adverse selection because the thing they're testing for is ridiculous.
Ive done this too. The exercise wasnt arrays (Im militant about only setting very realistic tasks). My task required modifying existing production-like code and tests.
My hope was always that the candidates would do TDD where it seemed simple and obvious to do so. It was actually pretty rare but the candidates that defaulted to doing that always ended up being better in my opinion. They were always made offers higher than my company could afford elsewhere (so i guess in others' opinions too).
In this thread https://news.ycombinator.com/item?id=43060636 I pondered why most people dont default to TDD for production code and the answer invariably seemed to be "we didnt think TDD was a thing you could do with integration/e2e tests".
I think having tests for all your diffs at the level of published commits/change lists/etc is totally reasonable for software you really care about. What's counterproductive is practicing TDD at the level of individual editor operations.
If I'm fixing a bug, I start by writing a test that reproduces the bug. If I can't do that, I fix the test harness until I can. Then I implement the change, making mental notes of each intermediate bug I think about along the way - things like "I should be careful to name this distinctly so that it's not confused with this other value in scope that has the same type". After that, I cull down that list until it's reasonable and not totally paranoid, and write tests covering those cases. Same thing for any bugs in in-progress code caught by manual testing, fuzzers, etc.
If you have discipline and use version control, you don't need to write tests before you write the actual code to get the same level of coverage as TDD and you waste a lot less time. I've often figured out late in the game how to make something a compile time failure rather than a runtime one - time to delete all those tests written along the way? Encode them all as negative compilation tests? Fundamentally the goal of testing is to describe what behaviors of the software are intentional rather than incidental, and to detect bugs that might be introduced by future changes to the software - TDD mixes both concerns and doesn't put any emphasis on preventing future bugs specifically.
Maybe other people work on different types of things and TDD is great for them, but I write primarily infrastructure code where correctness is critical and I have the luxury of time, and TDD doesn't produce better results for me. This is a case TDD feels like it should work well for, but in my experience it doesn't improve correctness, maintainability, or speed of delivery - at least compared to the alternative I described. I'm sure there's a universe of teams with sloppy practices out there that TDD would be an improvement for, but it's not helpful for me.
Congratulations on dodging that bullet.
I mean, yeah. Now that I have a stable well paying job I can say that. Back then I was kinda sad. My first interview in english. I was extra nervous.
Does that company still exist? The engineering culture seems asinine.
Apparently. It is a London based company, just googled them, they seem to be in business
I rarely write isolated unit tests.
Most of the systems I build use a database on which all logic depends, and often a network connection.
I've worked on systems where these aspects were mocked, and they eventually grind to a halt because of the effort required to make the tiniest change.
First of all you need a way to create a pristine database from code, preferably in memory. Second nested nested transactions are nice, since you can simply rollback the outer transaction per test case; otherwise you need to drop/create the database which is slower.
For networked servers, an easy way to start/stop servers in code and send requests to them is all you need.
Given these pieces, it's easy to write integration tests that run fast enough and give a lot of bang for the buck.
TDD is even more rare for me, I typically only do that when designing API's I'm unsure about, which makes imagining user code difficult. And fixing bugs, because it makes total sense to have a failing test to verify that you fixed it, and that it remains fixed.
I still find the skepticism around TDD weird. Except for a few pretty niche scenarios (e.g. it's experimental code or manual testing is cheaper for some obscure reason) i dont really see the point of not doing it.
I especially dont see what is gained by writing the test after.
"I still find the skepticism around TDD weird."
A small community of programmers, with a disproportionately large audience, foretold that practicing test-driven development would produce great benefits; over twenty five years the audience has found that not to be the case.
Compare with "continuous integration" - here, the immediate returns of trying the proposed discipline were so good that pretty much everybody who tried the experiment got positive returns, and leaned into it, and now CI (and later CD) are _everywhere_.
As for what is gained, try this spelling: test driven development adds load to your interfaces at a time when you know the least about the problem you are trying to solve, which is to say the period where having your interfaces be flexible is valuable.
And thus, the technique gets criticism from both ends -- that design work that should have been done up front is deferred (making the design more difficult to change, therefore introducing costs/delays), and that the investment is being made in testing before you have a clear understanding for which tests are going to be sensitive to the actual errors that you introduce creating the code (thereby both increasing the amount of "waste" in the test suite, in addition to increasing the risk of needing test rewrites).
The situation is further not improved by (a) the fact that most TDD demonstrations are problems that are small, stable problems that you can solve in about an hour with any technique at all and (b) the designs produced in support of the TDD practice aren't clearly an improvement on "just doing it", and in some notable cases have been much much worse.
So if it is working for you: GREAT, keep it up; no reason for you not to reap the benefits if your local conditions are such that TDD gives you the best positive return on your investment.
Coding is not religion for me, I have no patience for fundamentalism.
I will write as many tests as I need to feel confident, which depends on context.
And integration tests give me a lot more confidence than mocked unit tests.
> I especially dont see what is gained by writing the test after.
I assume you mean versus writing it first, rather than versus not writing it at all.
I've found that TDD works well for bottom-up coding, but not so well for top-down.
With bottom up, I can write the test for a piece at the bottom, write the code to pass the test, and move on. With top-down, if I write the test first, it might be a long while before I have that top-level working, because the bottom bits don't exist yet.
When I feel it's better to write things top-down, I'll often use TDD for the bottom bits I need to write, but for the bits above that, I'll write the tests "on my way back up".
"I especially dont see what is gained by writing the test after."
The greatest value in tests is that they help prevent future changes from breaking existing functionality. Writing the test after you write the implementation is equally useful for achieving that as writing the test before you write the implementation.
> Most of the systems I build use a database on which all logic depends, and often a network connection.
Why do you believe this is relevant? Unit tests target units of code, and check if the unit of code verifies specific invariants for specific combinations and ranges of input values. The concept of a network or a database does not exist in unit tests. In fact, you are expected to abstract away these aspects.
> I've worked on systems where these aspects were mocked, and they eventually grind to a halt because of the effort required to make the tiniest change.
This scenario is totally unrealistic if you knew what you are doing.
The fact alone that you're talking about spinning up a database in the context of unit tests already raises red flags.
Even if somehow you're talking about integration and end-to-end tests, none of the perceived problems you're mentioning are even a challenge, let alone a problem.
> Given these pieces, it's easy to write integration tests that run fast enough and give a lot of bang for the buck.
That's perfectly fine. It's besides the point though, and completely ignores the whole point of unit tests, which is to have a bunch of fast tests that check if your code continues to do the right thing between changes. Unit tests also compel developers to think through how they create/update code to prevent them from introducing handled scenarios.
There are many reasons why the test pyramid is a thing, and why the unit test layer includes far more tests than any other layer.
Tests that mock databases are bad tests. This is a hill I am happy to die on.
Spinning up a fresh database instance for each testing run is trivially easy in every testing framework worth its salt.
This works fine if the difficult part of your software is in nicely abstracted boxes, but in my experience the content of the boxes is usually fairly trivial: you can write a huge number of tests for them, but they're rarely where problems occur. The integration and interaction are where you get by far the most value out of testing (it's where most errors will occur: units are usually small enough to fit in your head while working on them, but the system as a whole it not), and for that you should abstract out the bare minimum needed to control your test environment and get acceptable performance, not mock absolutely everything by default.
Unit tests are the last tests that should be added to any system, and I've yet to see any system where the effort isn't better spent on integration testing (maybe once you're as thorough as sqlite?).
Wow. If "unit" in "unit test" does indeed mean that the test itself should be able to run independent of the other tests then maybe I can get over my avoidance of calling them "unit tests"!
I dislike that term because the most valuable tests I write are inevitably more in the shape of integration tests - tests that exercise just one function/class are probably less than 10% of the tests that I write.
So I call my tests "tests", but I get frustrated that this could be confused with manual tests, so then I call them "automated tests" but that's a bit of a mouthful and not a term many other people use.
I'd love to go back to calling them "unit tests", but I worry that most people who hear me say that will still think I'm talking about the test-a-single-unit-of-code version.
"I call them 'unit tests' but they don't match the accepted definition of unit tests very well." -- Kent Beck, __Test Driven Development By Example__
The short version is that "unit test" did actually mean something (see Glenford Myers, __The Art of Software Testing__ or Boris Beizer, __Software Testing Techniques__), although it wasn't necessarily clear how those definitions applied to object-oriented programming (see Robert Binder, __Testing Object-Oriented Systems__).
The Test-First/TDD/XP community later made an effort to pivot to the language of "programmer test", but by the time that effort began it was already too late.
So I think you should continue to call your tests "tests" (or "checks", if you prefer the framing of James Bach and Michael Bolton).
As best I can tell - there's no historicity to the idea that "unit test" was a reference to the isolation of a tests from its peers; it's just a ret-con.
Ive had this experience with team-specific vocab where certain terms organically end up having terms with two or more conflicting meanings and it was horrendous. It led to all sorts of bugs, misunderstandings and even arguments.
Even worse, most people didnt realize there was a problem coz they always knew what they meant.
The only time I managed to work past it was by convincing everyone to never use that term again - burning it to the ground - and agreeing to replace it with two or more new, unambiguous terms.
Id love to burn "unit test" and "integration test" to the ground but nobody outside my team listens to me :)
Id probably replace them with:
* code coupled
* interface coupled
* high level
* low level
* xUnit
* faked infrastructural
* deployed infrastructural
* hermetic / non hermetic
* declarative / non declarative
> The argument for isolating the units from each other is that it is easier to spot a potential bug.
That's not the only argument. The important result of this, is ensuring the "unit" of code is written to be testable. This happens to require it be simple and extensible. It does not enforce making the code or tests comprehensible.
When you don't trust someone's code, have them write detailed unit tests. They will find most of their problems on their own and learn better practices, along the way.
I am, in no way, implying that unit tests are a replacement for integration or behavioral or E2E testing et al...depending on how you want to define those.
> Do not isolate code when you test it.
> Only isolate your code from truly external services
That makes tests more trustworthy but also sometimes harder to maintain I think. I have seen cases where small changes on the code base created strong ripple effects with many tests to update. Arguably, the tests were not very well written or organized and with too many high level tests. Still, this and the very large execution time of the test collection made me realized that for medium to large projects, I will be much more careful in the future before going all in with the no-mock approach.
Or, write code that _can_ work in isolation. (Pure functions). You can bet when GM builds fuel injectors, they have a harness that they can place them in to make sure that JUST the injector gets tested... putting it in a car and driving around is the wrong approach.
The same is often true for software. Ensuring that things integrate is vital. Ensuring that functions run in isolation also has value.
This makes sense when the cost of running the integration tests is high due to real hardware being in the loop. But for most business software this isn't the case and e2e tests aren't actually costly.
Looking at a previous project we ran full e2e tests from an end user perspective in three major browsers in less than 15 minutes at a cloud cost of only a few hundred dollars per month. Compared to the cost of developers on the project that was a negligible amount.
Adding more unit tests would make the feedback cycle shorter on some changes, but also add development time to create and maintain those tests. So in line with the article we did so only sparingly on a few critical paths that were likely to cause issues. The rest was well covered by e2e tests.
Absolutely, whenever there is hardware in the loop, the more of the system you test with the more expensive the test is and the harder it is to isolate issues.
That’s not to say you can skip end to end testing, but the more testing pushed to the complete system level will drive up cost and schedule.
What works great for me:
I auto test the API of the server/system/library/module I am responsible for. Nothing else. No auto testing of internal details.
It lets me completely rewrite internals without breaking the tests.
The API tests needs to be so good that another developer could implement the same server/system/library/module using the tests only.
And the API tests needs to try as hard as possible to break the code being tested.
Using this method I have had zero bugs in production for the last 5+ years.
From the article:
> The argument for isolating the units from each other is that it is easier to spot a potential bug. (...) In my opinion, this does not pay out because of the huge amount of false positive test cases you get and the time you need to fix them. Also, if you know the code base a little you should have an idea where the problem is. If not, this is your chance to get to know the code base a little better.
This is at best specious reasoning, and to me reflects that the blogger completely misses the point of having tests.
To start off, there is no such thing as a false positive test. Your tests track invariants, specially those which other components depend on. The whole point of having these tests is to have a way to automatically check for them each and every single time we touch the code, so that the tests warn us that a change we are doing will cause the application to fail.
If you somehow decide to change your code so that a few invariants break, these are not "false positives". This is your tests working as expected and warning you that you must pay attention to what you are doing so that you do to not introduce regressions.
It's also completely mind-boggling and absurd to argue that "knowing the code" is any argument to avoid tracking invariants. The whole point of automated test suites is that you do not want the app to fail because you missed any detail or corner case or failure mode. Knowing the code does not prevent bugs or errors or regressions.
I'm perplexed by the way we have people write long articles on unit tests when they don't really seem to understand what they are supposed to achieve.
It mostly comes from observing the reality of how tests are used rather than idealism. In a lot of "test-first" codebases, the majority of tests are garbage. All they check is that the code is still structured identically to the original implementation. This is the reality that people encounter, so of course it's what they talk about. Why would they talk about some ivory tower idea instead?
> It mostly comes from observing the reality of how tests are used rather than idealism. In a lot of "test-first" codebases, the majority of tests are garbage.
At most, you can only arrive at that conclusion by observing your personal reality. If your teams crank out crappy code that's error-prone and untestable with their crappy tests, and can't manage to fix either their code or the tests, then naturally they end up living in the reality they created for themselves.
Back in the real world, some projects renowned by their stability and robustness go out of their way to single out their test-drive approach to software design and development as the fundamental reason their software is stable and robust.
What do you think is the difference?
> All they check is that the code is still structured identically to the original implementation.
Unit tests don't test structure. Their whole point is that they do not reflect structe Unit tests only cover the behavior that's expected from a specific unit of code, and they only check for the invariants relevant to that specific unit of code.
Perhaps this is a telltale sign you should rethink what you're doing.
A shit test written before writing the code is still a shit test. Mimetic tests arent be any better written after the code either.
If I had to choose between 1) always writing specification-linked tests that make as few architectural assumptions as possible and 2) TDD, sure, I'd pick 1 every time.
1 and 2 is still better though.
People don't like to write tests so any argument saying you need fewer of them goes down real smooth. The details don't matter so much.
But when software actually needs to be made at scale and with high quality you almost always have lots of tests.
Statistically, the average developer is not writing software anymore where corner cases matter. They're writing various permutations on standard web apps. Most web apps are pretty buggy, even ones from major corporations. So stakes are relatively low for having failures.
People writing file systems or crypto routines are going to have an easier time understanding the value of tests.
> To start off, there is no such thing as a false positive test. Your tests track invariants, specially those which other components depend on.
They were just talking about test doubles / mocks above that. Those cause all sorts of risks with false positives and false negatives, for example by mocking out a function the code under test relies on, but the mock not being changed when the contract between those two pieces of code changes.
> They were just talking about test doubles / mocks above that. Those cause all sorts of risks with false positives and false negatives, for example by mocking out a function the code under test relies on, but the mock not being changed when the contract between those two pieces of code changes.
The point is that it's absurd to try to characterize the regressions you're introducing as "false positive"/"false negative".
Again, this is at best specious reasoning, and to me reflects that the blogger completely misses the point of having tests.
Your code has behavior. Your tests verify invariants associated with this behavior. If you purposely change that behavior, it stands to reason that your task also involves updating how your code checks for that behavior.
If you change behavior but fail to change how that behavior is checked, that is not a false positive or false negative. That represents a few problems you introduced: failing to update how that behavior is checked, and failing implement new behavior without checking if it works. Both reflect problems created by a developer failing to perform the basics of their work, and worse: blaming working tests for the mess he's creating.
Think about the issue: if a highway administration decides to redesign an intersection but fails to move/update its stoplights, does this means the old stoplights cause accidents with their false positive greens?
Maybe in the ideal case, but in practice, a lot of unit tests track much more than invariants other components depend on.
Example: a function makes two RPC calls to query two independent statuses, which are of course mocked for the unit tests. Due to the the test library used, unit tests expect query A first and query B second. The function is refactored, and it now queries B first, so unit tests start to fail. This is 100% false positive - nothing in the real world cares if A or B are queried first, those actually go to systems which don't even know each other. And yet, the change author now has to fix dozens of failed tests.
Another example: there is an event-based system, and unit tests hardcode number of main loop executions ("emit event A; poll; poll; poll; ensure event B arrives"). A change which adds one more poll cycle will have zero effect on the real app, but causes many tests to break. It's a false positive.
Yet another example: there are random numbers involved, so test driver makes sure to seed RNG to known value at start of each test. A single call to random() is added somewhere early in the code and boom! each test explodes. Another hundreds of false positives.
I could go on an on... Should those tests be rewritten to be more robust and only check things that really matter? Yep. Does real-life code has fragile unit tests like this? Yep, almost every single codebase I have seen.
> This is at best specious reasoning, and to me reflects that the blogger completely misses the point of having tests.
I mean, yeah. Most devs don't understand the point. It turns into a check-boxing exercise. And most devs are vaguely smart and can figure out how to do a check-boxing exercise with minimal effort while evading the point of doing it.
Relying on people who don't care enough to write good code to care enough to write good tests has a certain irony to it.
> To start off, there is no such thing as a false positive test. Your tests track invariants, specially those which other components depend on. The whole point of having these tests is to have a way to automatically check for them each and every single time we touch the code, so that the tests warn us that a change we are doing will cause the application to fail.
I'll push back on this. I'm a veteran of a codebase where the tests broke all the time with tons of false positives. This was mostly because of over-mocking, and an abundance of assert X called with Y and Z tests. Even pure functions were mocked.
Every time I re-wrote a function implementation, even if it was a pure function and produced the same results, it would break tens of tests. Bonus points if the implementation of the thing that was mocked out had changed since the last time someone reviewed the tests and the way it was calling the function didn't do what the test assumed anymore.
> see here and here
I believe links are significantly more useful when they include descriptive text like the title or author, rather than just 'here'.
Eh, I knew this would somehow come back to that Kent C Dodds stuff. I can respect the guy, but his opinion on valuable testing differs from mine and quite a lot of others. His opinion is "the more your tests represent the way your users use your app, the better they are". For E2E tests, yes, absolutely. For unit tests, no... not at all. As a very short example, an E2E test will prove that yes, I did indeed show a button on the screen. A unit test will prove the code that put it there worked. The code can "fail open" for many reasons. (In JavaScript for example, forgetting an `await` keyword can return a truthy value!).
The worst part about it is that he called himself a thought leader, called his approach a "best practice" and had nothing really to back that up. Now people go around repeating it all the time. It's frustrating.
The thing that drives me nuts about these threads is that people vastly overestimate their ability to write code that isn't broken. I distrust myself, and therefore I write basic unit tests which validate the code I've written to prove that I haven't screwed up.
Such unit tests are quick to write and don't change frequently because they run against low-level building blocks — so they are not a major maintenance burden. The ROI on this kind of testing is very high.
I don't like being told by Dodds and these other unit-test haters that I shouldn't be doing this. In my experience, organizations suffer much more when developers eschew such tests, being quite naturally overly optimistic about the reliability of their work.
Writing low-level unit tests doesn't prevent you from writing integration or end-to-end tests, and those are important too. I'm not here to devalue higher-level testing, and I'd appreciate it if advocates for higher-level testing likewise didn't go out of their way to devalue unit tests.
> I distrust myself, and therefore I write basic unit tests which validate the code I've written to prove that I haven't screwed up.
This drives me nuts.
You don't trust yourself to write code that isn't broken but you trust yourself to write quality tests that ensure the code isn't broken?
I don't understand this at all. Automated tests are just like any code, they are as prone to mistakes and bugs as anything else
Respectfully, I disagree, though maybe we haven’t yet articulated enough details to know exactly where we disagree.
I think underlying Kent’s statement is an observation that is undeniably true. The closer a test is to the actual way the software will be used, the better it is in a number of ways, _all else being equal_. All else is never fully equal, which is why everything is about tradeoffs.
But when a test aligns with how the software will be used, there are a whole bunch of synergies and benefits. Simplicity increases, clarity increases, you start getting multiple payoffs for each bit of effort you invest.
I’m sure people cargo cult it and lose track of the tradeoffs, like anything else, but there’s a solid point under there. And I agree with him, it is valuable enough to be called a “best practice.”
If you have an E2E test that tests the button behavior thoroughly (i.e. not just showing it, but the fact that, when it's clicked, the app state changes accordingly), then you have a test showing that "code that put it there worked".
That’s not in line with the original definition of unit test. It also sounds like a pointless test using your definition. There is no advocacy for writing pointless tests.
Disclaimer: I don't have a lot of comments on TDD as a whole, other than that most software I write is very exploratory (I'll throw away the first 3-10 drafts, and by no means does that mean it takes 3-10x as long to write), and the best language for me for that exploration is often the language of the actual software I'm writing. TDD, in that environment, doesn't seem very applicable since the whole point is that we don't know what's actually possible (or, when it's possible, if the tradeoffs are worth it).
The author has a lot of opinions about testing though which conflict with what I've found to work in even that sort of dynamic environment. Their rationale makes sense on the surface (e.g., I've never seen a "mock"-heavy [0] codebase reap positive net value from its tests), but the prescription for those observed problems seems sub-optimal.
I'll pick on one of those complaints to start with, IMO the most egregious:
> Now, you change a little thing in your code base, and the only thing the testing suite tells you is that you will be busy the rest of the day rewriting false positive test cases.
If changing one little thing results in a day of rewriting tests, then either (a) the repo is structured such that small functional changes affect lots of code (which is bad, but it's correct that you'd therefore have to inspect all the tests/code to see if it actually works correctly afterward), or (b) the tests add coupling that doesn't exist otherwise in the code itself.
I'll ignore (a), since I think we can all agree that's bad (or at least orthogonal to testing concerns). For (b) though, that's definitely a consequence of "mock"-heavy frameworks.
Why?
The author's proposal is to just test observable behavior of the system. That's an easy way to isolate yourself from implementation details. I don't disagree with it, and I think the industry (as I've seen it) discounts a robust integration test suite.
What is it about "unit" tests that causes problems though? It's that the things you're testing aren't very well thought through or very well abstracted in the middle layers. Hear me out. TFA argues for integration tests at a high level, but if you (e.g.) actually had to implement a custom sorting function at your job would you leave it untested? Absolutely not. It'd be crammed to the gills with empty sets, brute-force checking every permutation of length <20, a smattering of large inputs, something involving MaxInt, random fuzzing against known-working sorting algorithms, and who knows what else the kids are cooking up these days.
Moreover, almost no conceivable change to the program would invalidate those tests incorrectly. The point of a sorting algorithm is to sort, and it should have some performance characteristics (the reason you choose one sort over another). Your tests capture that behavior. As your program changes, you either say you don't need that sort any more (in which case you just delete the tests, which is O(other_code_deleted)), or you might need a new performance profile. In that latter case, the only tests that are broken are associated with that one sorting function, and they're broken _because_ the requirements actually changed. You still satisfy O(test_changes) <= O(code_changes); the thing the author is arguing doesn't happen because of mocks.
Let's go back to the heavily mocked monstrosities TFA references. The problem isn't "unit" testing. Integration tests (the top of a DAG), and unit tests (like our sorting example, the bottom of a DAG) are easy. It's the code in between that gets complicated, and there might be a lot of it.
What do we do then?
At a minimum, I'd personally consider testing the top and bottom of your DAG of code. Even without any thought leadership or whatever garbage we're currently selling, it's easy to argue that tests at those levels are both O(other_code_written) in cost and also very valuable. At a high level (TFA's recommendation), the tests are much cheaper than the composite product, and you'd be silly not to include them. At a low level (truly independent units, like the "sorting" case study), you'd also be silly not to include them, since your developers are already writing those tests to check if it works as they implement the feature in the first place, and the maintenance cost of the tests is both proportional to the maintenance cost of the code being tested and extremely valuable in detecting defects in that code (recall that bugs are exponentially more expensive to fix the further down the pipeline the propogate before being triaged).
Addressing the bottom of your DAG is something the article, in some sense, explicitly argues against. They're arguing against the inverted pyramid model you've seen for testing. That seems short-sighted. Your developers are already paying approximately the cost of writing a good test when they personally test a sorting function they're writing, and that test is likely to be long-lived and useful; why throw that away? More importantly, building on shaky foundations is much more expensive than most people give it credit for. If your IDE auto-complete suggests a function name that says it does the right thing and accepts the arguments you're giving it, you get an immediate 10x in productivity if that autocomplete is always right. Wizards in a particular codebase (I've been that wizard in a few, my current role as well; that isn't a derogatory assessment of "other" people) can always internalize the whole thing and immediately know the right patterns, but for everyone else with <2yrs of experience in your company in particular (keep in mind that average silicon valley attrition is 2-3yrs), a function doing what it says it's going to do is a godsend to productivity.
Back to the problem at hand though. TFA says to integration test, and so do I. I also say to test your "leaf" code in your code DAG, since it's about the same cost and benefit. What about the shit in between?
In a lot of codebases I've seen, I'd say to chock it up as a lost cause and test both the integration stuff (that TFA suggest) and also any low-level details (the extra thing I'm saying is important). Early in my career, I was implementing some CRUD feature or another and explicitly coached (on finding that the reason implementation was hard was a broken function deep in the call-stack) to do the one-liner fix to make my use case work instead of the ten-liner to make the function actually correct and the 1000-liner to then correct every caller. I don't think they were wrong in giving that advice. I'm sad that the code was in a state where that was reasonable advice.
If you're working on newer projects though (or plan to be at a place for awhile and have the liberty to do some cleanup with every new feature (a pattern I wholly endorse and which has served me very well personally)), it's worth looking at that middling code and figuring out why it's so hard to work with. 99% of the time, the reason mocks look attractive isn't because they're the only solution. It's because they're the only solution that makes sense once you've already tied your hands. You don't need something to "unit" test the shutdown handler; you need something to test the total function which processes inputs and outputs and is called by the shutdown handler. You don't need to "unit" test a UI page that requires 3 different databases to produce any output; you need to unit test the functions which turn that output into that UI page (ideally, without mocks, since although those ostensibly do the same thing they usually add an extra layer of complexity and somehow break all your tests), and for something that messy you might even just need an "integration" test around that UI page asserting that it renders approximately correctly.
What else? People sell all kinds of solutions. "Functional Programming" or "OOP" or whatever. Programming is imperative when you execute it, and the right representation for the human reader varies from problem to problem. I don't have any classes to sell or methodologies to recommend. I do strongly recommend taking a very close look at the abstractions you've chosen though. I've had no problem deleting 90% of them at new jobs, making the code faster, more correct, and easier to modify (I usually do so as part of a "coup," fixing things slowly with each new feature). When every new feature deletes code, the benefits tend to snowball. I see my colleagues doing that now to code I recently wrote, and I'd personally do it again.
[0] People typically mean one of two things when they say they're "mocking" a dependency. The first is that they want a function to be "total" and have reasonable outputs for all possible inputs. They'll mock out many different interface implementations (or equivalent blah blah blah in your favorite language) to probe that behavior and ensure that your exponential backoff routine behaves reasonably when the clock runs backward, when 1000 of them are executed simultaneously, and whatnot. That tends to make for expensive tests, so I tend to see it reserved for risky code in teams which are risk-averse, but it's otherwise very good at its job. The other case is using some sort of "mock" library which lets you treat hard dependencies as soft dependencies and modify class instantiation, method return values, and all sorts of things to fit the test you're trying to write. This latter case is much more common, so it's what I'm referring to in a "heavily mocked" codebase. It's a powerful tool which could be used for good, but IME it's always overused enough that it would be better if it didn't exist.