An overly aggressive mock can work fine, but break much later

The mock discussion still misses the real solution, which is to refactor the code so that you have a function that simple reads the file and returns json that is essentially a wrapper around open and doesn't need to be tested.

Then have your main function take in that json as a parameter (or class wrapping that json).

Then your code becomes the ideal code. Stateless and with no interaction with the outside world. Then it's trivial to test just like and other function that is simple inputs translated outputs (ie pure).

Every time you see the need for a mock, you're first thought should be "how can I take the 90% or 95% of this function that is pure and pull it out, and separate the impure portion (side effects and/or stateful) that now has almost no logic or complexity left in it and push it to the boundary of my codebase?"

Then the complex pure part you test the heck out of, and the stateful/side effectful impure part becomes barely a wrapper over system APIs.

This blog post talks as if mocking the `open` function is a good thing that people should be told how to do. If you are mocking anything in the standard library your code is probably structured poorly.

In the example the author walks through, a cleaner way would be to have the second function take the Options as a parameter and decouple those two functions. You can then test both in isolation.

Honestly I don't buy it. Worse, this is one of the reason I prefer to do "minimal integration tests" instead of unit tests. Take the example snippet of code

    def get_user_settings() -> str:
        with open(Path("~/settings.json").expanduser()) as f:
            return json.load(f)

    def add_two_settings() -> int:
        settings = get_user_settings()
        return settings["opt1"] + settings["opt2"]

>>> The thing we want to avoid is opening a real file

and then the article goes and goes around patching stdlib stuff etc.

But instead I would suggest the real way to test it is to actually create the damn file, fill it with the "normal" (fixed) content and then run the damn test.

This is because after years of battling against mocks of various sort I find that creating the "real" resource is actually less finicky than monkeypatching stuff around.

Apart from that; yeah, sure the code should be refactored and the paths / resources moved out of the "pure logical" steps, but 1) this is an example and 2) this is the reality of most of the actual code, just 10x more complex and 100x more costly to refactor.

> In Why your mock doesn’t work I explained this rule of mocking:

> Mock where the object is used, not where it’s defined.

For anyone looking for generic advice, this is a quirk of python due to how imports work in that language (details in the linked post) and shouldn't be considered universal.

If you make the function pure it will be easier to test. Pass the moving parts as function parameters, then you can pass in the mocks in the actual functions when testing. Example:

    f = () => a+b

    f = (a, b) => a+b

> An overly aggressive mock can work fine, but then break much later. Why?

Because you are testing against implementation, not specification.

You’re welcome.

Great article. In addition, updating your mocking code can often be time-consuming. To try to make this easier, I built mock[1], which streamlines the process of setting up mock services for testing.

https://dhuan.github.io/mock/latest/examples.html

If you’re doing TDD, you could just view this as moving the “open” call to your unit test. As others point out, that encourages pure functions that can pipe in input from other sources than just file paths.

Arguably this is a problem in when the patch is unapplied.

Presumably in the coverage case it’s being called by a trace function, which inevitably runs during test execution — and while we want the trace function to be called during the test function, we really want it without any patches the test function is using. But this arguably requires both an ability for the trace function to opt-out of patches and for the patcher to provide a way to temporarily disable all of them.

I feel like the #1 reason mocks break looks nothing like this and instead looks like: you change the internal behaviors of a function/method and now the mocks interact differently with the underlying code, forcing you to change the mocks. Which highlights how awful mocking as a concept is; it is of truly limited usefulness for anything but the most brittle of tests.

Don't test the wrong things; if you care about some precondition, that should be an input. If you need to measure a side effect, that should be an output. Don't tweak global state to do your testing.

Why even mock anything in this example? You need to read the source code to work out what to mock, reaching deep inside the code to name some method to mock.

But what if you just passed in the contents of the file or something?

Edit: oh wait actually this is what the very last line in the blog post says! But I think it should be emphasized more!

The main reason why your mock breaks later is because you refactored the code. You did the one thing tests are supposed to help you do, and the tests broke. If code was never modified, you wouldn't need automated tests. You'd just test it manually one time and never touch it again. The whole point of tests is you probably will rewrite internals later as you add new features, improve performance or just figure out better ways to write things. Mock-heavy tests are completely pointless in this respect. You end up rewriting the code and the test every time you touch it.

There are really only a few reasons to use mocks at all. Like avoiding network services, nondeterminism, or performance reasons. If you need to do a lot of mocking in your tests this is a red flag and a sign that you could write your code differently. In this case you could just make the config file location an optional argument and set up one in a temp location in the tests. No mocking required and you're testing the real API of the config file module.

It is worth pointing out that you can often use containerized services as an alternative to mocking.