Item — HN Reader

> Unit test suites would break all the time for silly reasons, like someone optimizing a function would mean a spy wouldn't get called with the same intermediary data, and you'd have to stop and go fix the test code that was now broken, even though the actual code worked as intended.

Can you or others speak more about this? I was taught that verifying function calls for spies/mocks was good practice. But, I encountered this problem just the other day when I refactored some Java code for a personal project. Everything still worked perfectly, but, exactly as you said, the intermediate function calls changed so the tests would fail due to spies/mocks calling different "unexpected" functions.

I'm an intermediate programmer so can someone with more experience fill me in with what's best practice here and why? Do I update the test code to reflect the new intermediate function calls? But this whole approach now seems silly since a refactoring that doesn't affect the ultimate behavior of the function that is under test will break the test and that seems wrong. So do I instead not verify function calls when using spies/mocks? In that case, what is the use case for verifying spies/mocks?

5 comments

What you want to spy on are side-effects that are part of the function's contract.

If you have a function that fetches data, you shouldn't test that its hitting the data layer, only that the correct data is returned. This way when you improve the function to not hit the data layer at all under some conditions, your tests will keep passing.

On the other hand, if that function is supposed to log metrics or details about its execution, you should test that ,as it is part of its contract and can't be inferred from the return value.

Your tests should be driven by your contracts. Is there a contract for the component being tested that says, "it calls function X if Y"? If yes, then test that. If not, then you shouldn't be testing such an implementation detail anymore so than you'd test values of local variables inside a function at some point in the middle.

The important part is to remember that not every function is a component, and not even every class by itself. Where to draw the boundary between components is itself an explicit design decision, and should be made consciously, not mechanically.

This is usually called the classical vs. mockist, or test interactions vs. test final state debate.

Instead of mocks, some people prefer to build fakes / stubs which are versions of a dependency which are "fully operative" in a sense but with a simplified internal implementation. For example a repository that keeps entities in memory. (Not the same as an in-memory database! The fake repository wouldn't use SQL at all.)

Tests would check the final state of the fakes after the interactions, or simply verify that the values returned by the tested component are correct.

The hope is that fakes, while possibly more laborious to set up, allow an style of testing that focuses less on the exact interactions between components, and therefore is less brittle.

Some links:

- Mocks aren't stubs https://martinfowler.com/articles/mocksArentStubs.html#Class...

- From interaction-based to state-based testing http://blog.ploeh.dk/2019/02/18/from-interaction-based-to-st...

What should be tested is that the function does what is promised to the caller. Which other function it uses to accomplish this task is a detail, which should generally not be part of this promise (encapsulation). So either the effects of the function must be returned by the function, or it is a side-effect, which must then be observable in some other way. Example: If a function is supposed to create new user, don't check that it calls some internal persistence or communication layer with some User info. Instead list the users in system before and after, check that the correct one was added. Try to make an action as this new user. This forces you to expose (a view of) internal state at the API surface, which makes the system more observable, usually very beneficial for debugging and fillings gaps in API.

Also, many of the assertions that are often put into unit-tests (especially at stub/mock boundaries) are better formulated as invariants, checked in the code itself. Design-by-contracts style pre/post-conditions is a sound and practical way of doing this. When this is done well, you get the localization part of low-level unit testing even when running high-level tests. Plus much better coverage, since these things are always checked (even in prod), not just in a couple of unit tests. And it is more natural when refactoring internal functions to update pre/post-conditions, since they are right there in the code. When a function disappears they also do.

I don't like the term "integration" tests though, as they hint at interactions between systems being the important thing to test. Integration between services / subsystems are just as much a detail as internal function calls. If using the real system during test is too complicated or slow, maybe it should be simplified or made faster? Only when that is not feasible do I build a mock.

If it is important to the operation of the function, from an outsider perspective, the intermediate call should be tested. Oftentimes this isn't the case, though. Most often, I see intermediate calls spied/mocked when they have side effects to be avoided. This is actually a sign of tight coupling between modules, and patterns like dependency injection can help make it easier to test.

The trick for me is focusing on what the unit does from a consumers perspective. Avoid testing implementation details (unless they are important side effects), and test the behavior that does not change. If you do that, then refactoring becomes easier, because tests will only break when the contract of the unit changes.