back

by billyp-rva·1y ago·view on hn ↗
> So, what is a test? To me, a test is a chunk of code...

This obviously excludes manual testing, and the author doesn't explain why he is excluding them.

2 comments
Presumably, they are excluded from consideration because manual tests[0] are, by definition, un-automated, and so are a bottleneck. Every manual test added to the "test suite" adds seconds (or, more likely, minutes) to execute, whereas the majority[1] of automated tests should run in milliseconds. The "confidence generated per time spent execution" of a manual test is far lower.

[0] which are discussed later - "Now, to be fair, most engineers are not skipping testing altogether. They’re probably doing some manual checks with Postman, watching logs, or visually confirming debug output. The resistance isn’t to testing itself, it’s to codifying and automating those tests."

[1] Yes, there are exceptions. That's why I said "the majority".

manual testing doesn't scale at all. the more functionality and code you have, the more people you have to hire to run tests. since those people aren't being spun up and down by an orchestrator, you need more resources to cover their coffee breaks. you also don't get testing at night or on the weekends unless you have a global presence. manual testing also limits your reach, if every time you add a new test you have to add another body to run it, you're not going to be running as many tests as you could (if those people were employed writing tests instead of running them)

worse, manual testing is considerably less repeatable than automated testing. was this result worse than it was before, or did someone just forget to set that environment variable or run it on the correct node type.

if your developers are running manual tests before every commit, that's potentially a huge waste of resources. and if you're hiring people specifically to do manual testing that's.. kind of inhuman? unless you're really hiring people that can't do anything else, in which case you'll need to hire an appropriate number of overseers.

automated testing gives you so much more leverage for less cost than manual, I struggle to think of a situation where that might be preferable.