6

I am using Python's unittest module to test a script I am writing.

The script contains a loop like this:

// my_script.py

def my_loopy_function():
    aggregate_value = 0
    for x in range(10):
        aggregate_value = aggregate_value + complicated_function(x)
    return aggregate_value

def complicated_function(x):
    a = do()
    b = something()
    c = complicated()
    return a + b + c

I have no problems using unittest to test complicated_function. But I would like to test my_loopy_function by overriding complicated_function.

I tried modifying my script so that my_loopy_function takes complicated_function as an optional parameter so that I can pass in a simple version from the test:

// my_modified_script.py

def my_loopy_function(action_function=None):
    if action_function is not None:
        complicated_function = action_function
    aggregate_value = 0
    for x in range(10):
        aggregate_value = aggregate_value + complicated_function(x)
    return aggregate_value

def complicated_function(x):
    a = do()
    b = something()
    c = complicated()
    return a + b + c

// test_my_script.py

from myscript import my_loopy_function

class TestMyScript(unittest.TestCase):
    test_loopy_function(self):
        def simple_function():
            return 1
    self.assertEqual(10, my_loopy_function(action_function=simple_function))

It has not worked as I had hoped, are there any suggestions on how I should be doing this?

4
  • 2
    You can try using the mock package. It's now part of Python 3. docs.python.org/3/library/unittest.mock.html
    – austin
    Commented Dec 19, 2014 at 17:59
  • How has it not worked as you hoped? Commented Dec 19, 2014 at 18:03
  • Yes I did see some references to mock. I get the feeling it might be the right thing for me here -- I basically want to override selected functions that get called in my code so that my tests can test the behaviour of the calling code.
    – Robert
    Commented Dec 19, 2014 at 18:06
  • @Daniel Pryden The problem that I am having is that I have a few tests running, and once I have overriden complicated_function with simple_function it stays pointing to simple_function even in a later test where I do not pass an 'action_function'.
    – Robert
    Commented Dec 19, 2014 at 18:08

4 Answers 4

8

In the end I used Python's mock, which allows me to override complicated_function without having to adjust the original code in any way.

Here is the original script, and note that complicated_function is not passed in to my_loopy_function as an 'action_function' parameter (which was what I tried in my earlier solutions):

// my_script.py

def my_loopy_function():
    aggregate_value = 0
    for x in range(10):
        aggregate_value = aggregate_value + complicated_function(x)
    return aggregate_value

def complicated_function(x):
    a = do()
    b = something()
    c = complicated()
    return a + b + c

and here is the script I am using to test it:

// test_my_script.py

import unittest
import mock
from my_script import my_loopy_function

class TestMyModule(unittest.TestCase):
    @mock.patch('my_script.complicated_function')
    def test_1(self, mocked):
        mocked.return_value = 1
        self.assertEqual(10, my_loopy_function())

This works just as I had wanted:

  1. I am able to substitute functions with a simpler version of themselves that I can more easily test,
  2. I do not need to alter my original code in any way (such as I was trying -- which was effectively by passing in function pointers), the mock module gives me post-coding access to the innards.

Thanks to austin for his suggestion to use mock. BTW I am using Python 2.7 and therefore used the pip-installable mock from PyPI.

3
  • Up vote for self answer. Welcome to the mock word ... You'll discover a very wonderful framework to help your unit testing. Commented Dec 22, 2014 at 8:50
  • @Robert: Actually, I would call what you were doing before "mocking" as well. The difference is that here you're using the mock module to do the monkeypatching for you, and it's smart enough to reset the state after the SUT is complete. Commented Dec 22, 2014 at 18:30
  • Doesn't seem to work. I tried declaring an overriding def foo() both in the top-level and inside the test function and inside some random class at the top-level — it's always just ignored and the non-overriding function gets called instead.
    – Hi-Angel
    Commented Feb 5 at 10:45
3

Don't try to override complicated_function with action_function, just use complicated_function as the default action_function:

def my_loopy_function(action_function=complicated_function):
    aggregate_value = 0
    for x in range(10):
        aggregate_value = aggregate_value + action_function(x)
    return aggregate_value
6
  • 1
    I think I considered that correctly: if I just run my_loopy_function and use the default value of None for action_function, the loop will use that module's complicated_function, which is fine. But if I run it from my test script then I am simply overriding the complicated_function to actually point to simple_function.
    – Robert
    Commented Dec 19, 2014 at 18:03
  • I would leave the note, where the bug was. Now the code is, of course right, but it might be hard to spot why. At least one person didn't =).
    – luk32
    Commented Dec 19, 2014 at 18:03
  • @Robert It will behave the same, you just over-complicated logic. And you never use anything else than complicated function in your loop. That is the problem.
    – luk32
    Commented Dec 19, 2014 at 18:04
  • @Robert I'm not sure what you meant to say in your comment here. Regardless, the pattern I'm describing is extremely common, and works — why reinvent the wheel? Commented Dec 19, 2014 at 18:05
  • @ThomasOrozco fair points, I will definitely have a re-think.
    – Robert
    Commented Dec 19, 2014 at 18:09
0

In your code, you shouldn't be able to overwrite complicated_function like that. If I try it, I get UnboundLocalError: local variable 'complicated_function' referenced before assignment.

But perhaps is the problem that in your actual code, you're referring to complicated_function in some other way (e.g. as a member of a module)? Then by overwriting it in your test, you're overwriting the actual complicated_function, so you won't be able to use it from other tests.

The correct way to do this is to overwrite the local variable with the global one, like so:

def my_loopy_function(action_function=None):
  if action_function is None:
    action_function = complicated_function
  aggregate_value = 0
    for x in range(10):
      # Use action_function here instead of complicated_function
      aggregate_value = aggregate_value + action_function(x)
    return aggregate_value
2
  • Thank you Daniel. You are right that my overwriting means that later tests cannot use the original complicated_function. I am looking into mocking the complicated_function. I like the idea of being able to overwrite functions in my code so that I can test 'from the outside in' rather than 'from the unit out'.
    – Robert
    Commented Dec 20, 2014 at 16:33
  • Robert: I don't understand what you mean by "from the outside in" here -- either way, you're injecting a mock/shim for testing purposes. (You're already mocking complicated_function by replacing the real implementation with a fake one.) The difference is that you're proposing doing this by mutating global state (the complicated_function function is a global object), and I'm proposing doing this by local logic in your my_loopy_function function. It's always best to avoid mutating global state if possible, as it leads to exactly these kinds of problems, which can be difficult to debug. Commented Dec 22, 2014 at 18:26
0

I couldn't make @mock.patch from the accepted answer work in any combination (worth noting I'm using pytest, but then the answer seems to be testing library-agnostic, so Idk), but I found a more generic way to do that, so sharing.

There's a technique called "monkey-patching". Its purpose is exactly to override a method from a 3rd party module. It works both with top-level functions as well as with class methods (in the latter case you need to add a class name in the assignment operation).

It works like this: you first do import some_module and then to override its function foo to bar you execute some_module.foo = bar.

Important: the module should be imported without from. I.e. if you do a from some_module import foo and then foo = bar, it will not work.

Example:

λ cat some_module.py
def hello():
    print("hello")

def some_func():
    hello()
λ cat test.py
import some_module

def my_override():
    print("hi")

some_module.hello = my_override

some_module.some_func()
λ python3 test.py
hi

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.