Python Mock and Patch

Mastering Python Mock and Patch: Mocking For Unit Testing

by

Have you heard about Python mock and patch as a way to improve your unit tests? You will learn how to use them in this tutorial.

Unit tests are key to software development because they ensure that your code works as planned.

Python has many robust tools for writing and running unit tests in a controlled environment by creating mocks. The Mock class is part of the unittest.mock library and it allows to create mock objects. The patch function, in the same library, allows replacing real objects with mocks.

Let’s learn about Python mocks!

Which Module Do you Use for Mocking in Python?

Mocking is a technique used in unit testing to replace parts of code that cannot be easily tested with mock objects that replicate the behavior of real objects.

The Python unittest.mock library provides a framework for building and manipulating mock objects in unit tests.

Mock objects help create a controlled test environment. They simulate the behavior of real objects, but with inputs and outputs that you can control. This allows you to isolate and test code that otherwise would be very hard to test due to external dependencies.

You can use the mock class and the patch function by importing the unittest module.

import unittest

How Do You Create a Mock using the Python Mock Class?

To create mock objects, you can use the Mock class part of the unittest module.

>>> from unittest.mock import Mock
>>> Mock
<class 'unittest.mock.Mock'>

From unittest.mock we have imported the Mock class. Now we can create a Mock object.

>>> mock = Mock()
>>> mock
<Mock id='140397919063152'>

The purpose of a Mock is to replace a real Python object but to do that the mock has to provide the same methods and attributes as the real object.

So, how do Python mocks handle this?

Let’s try to get the value of an attribute of the mock we have created or call a method of the mock:

>>> mock.length
<Mock name='mock.length' id='140397919064016'>
>>> mock.calculate_length(3, 6)
<Mock name='mock.calculate_length()' id='140398188475584'>

You can see that both worked. That’s because the Mock class dynamically creates the attribute and the method. This allows the mock to replace any real object.

Below you can see all the attributes of our mock object:

>>> mock.__dict__
{'_mock_return_value': sentinel.DEFAULT, '_mock_parent': None, '_mock_name': None, '_mock_new_name': '', '_mock_new_parent': None, '_mock_sealed': False, '_spec_class': None, '_spec_set': None, '_spec_signature': None, '_mock_methods': None, '_spec_asyncs': [], '_mock_children': {'length': <Mock name='mock.length' id='140397919064016'>, 'calculate_length': <Mock name='mock.calculate_length' id='140397919105184'>}, '_mock_wraps': None, '_mock_delegate': None, '_mock_called': False, '_mock_call_args': None, '_mock_call_count': 0, '_mock_call_args_list': [], '_mock_mock_calls': [call.calculate_length(3, 6)], 'method_calls': [call.calculate_length(3, 6)], '_mock_unsafe': False, '_mock_side_effect': None}

The interesting aspect of this is the ‘_mock_children‘ attribute that contains the attribute and the method dynamically created in the code above.

A Mock object automatically creates child objects when you access an attribute or call a method on the mock.

Asserting Calls to a Python Mock

In the previous section, we have called a method on the mock object. We can now make assertions that a specific method has been called on the mock.

Below you can see successful assertions. Those assertions test that the calculate_length() method of the mock:

  1. Has been called.
  2. Has been called once.
  3. Has been called with arguments 3 and 6.
  4. Has been called once with arguments 3 and 6.
>>> mock.calculate_length.assert_called()
>>> mock.calculate_length.assert_called_once()
>>> mock.calculate_length.assert_called_with(3, 6)
>>> mock.calculate_length.assert_called_once_with(3, 6)

If an assertion on a mock method fails, the mock raises an AssertionError.

Call the calculate_length() method on the mock again and see how the assert_called_once() assertion fails.

>>> mock.calculate_length(3, 6)
<Mock name='mock.calculate_length()' id='140398188475584'>
>>> mock.calculate_length.assert_called_once()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.8/unittest/mock.py", line 892, in assert_called_once
    raise AssertionError(msg)
AssertionError: Expected 'calculate_length' to have been called once. Called 2 times.
Calls: [call(3, 6), call(3, 6)].

The Python interpreter expects the method to be called once and instead it has been called 2 times.

You also see an AssertionError if the assert doesn’t match the arguments you use to call the mock (e.g. 4 and 6 instead of 3 and 6 in this scenario).

>>> mock.calculate_length.assert_called_with(4, 6)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.8/unittest/mock.py", line 913, in assert_called_with
    raise AssertionError(_error_message()) from cause
AssertionError: expected call not found.
Expected: calculate_length(4, 6)
Actual: calculate_length(3, 6)

Makes sense?

What is the Difference Between Mock and MagicMock?

The unittest.mock library also provides the MagicMock class. This is a subclass of the Mock class and it’s similar to the Mock class with the difference that it provides Python’s magic methods.

The statement below confirms that MagicMock is a subclass of Mock.

>>> issubclass(MagicMock, Mock)
True

Let’s create two mocks using the Mock and MagicMock classes to see the difference in their attributes.

This is a mock object that uses the Mock class:

>>> from unittest.mock import Mock
>>> mock = Mock()
>>> mock.__dict__
{'_mock_return_value': sentinel.DEFAULT, '_mock_parent': None, '_mock_name': None, '_mock_new_name': '', '_mock_new_parent': None, '_mock_sealed': False, '_spec_class': None, '_spec_set': None, '_spec_signature': None, '_mock_methods': None, '_spec_asyncs': [], '_mock_children': {}, '_mock_wraps': None, '_mock_delegate': None, '_mock_called': False, '_mock_call_args': None, '_mock_call_count': 0, '_mock_call_args_list': [], '_mock_mock_calls': [], 'method_calls': [], '_mock_unsafe': False, '_mock_side_effect': None}

Here is a mock created using the MagicMock class. It looks identical to the Mock object:

>>> from unittest.mock import MagicMock
>>> magic_mock = MagicMock()
>>> magic_mock.__dict__
{'_mock_return_value': sentinel.DEFAULT, '_mock_parent': None, '_mock_name': None, '_mock_new_name': '', '_mock_new_parent': None, '_mock_sealed': False, '_spec_class': None, '_spec_set': None, '_spec_signature': None, '_mock_methods': None, '_spec_asyncs': [], '_mock_children': {}, '_mock_wraps': None, '_mock_delegate': None, '_mock_called': False, '_mock_call_args': None, '_mock_call_count': 0, '_mock_call_args_list': [], '_mock_mock_calls': [], 'method_calls': [], '_mock_unsafe': False, '_mock_side_effect': None}

By calling magic methods on the two mock objects we can verify that the object based on MagicMock supports magic methods while the object based on the Mock class doesn’t.

The Mock Object doesn’t implement magic methods

>>> mock.__len__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.8/unittest/mock.py", line 639, in __getattr__
    raise AttributeError(name)
AttributeError: __len__
>>> mock.__iter__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/anaconda3/lib/python3.8/unittest/mock.py", line 639, in __getattr__
    raise AttributeError(name)
AttributeError: __iter__

The MagicMock Object implements magic methods

>>> magic_mock.__len__
<MagicMock name='mock.__len__' id='140397919312096'>
>>> magic_mock.__iter__
<MagicMock name='mock.__iter__' id='140397919313248'>

Now that you know the difference between the Mock and MagicMock classes, we will see how to implement a test using a MagicMock object.

An Example of Python Module to Be Tested with Mocks

Using mocks is quite useful when you want to test code that relies on external dependencies (e.g. an external API call).

For example, assume you have a Python function you want to test. This function calls another function that reads a file from the filesystem.

Create a file called names.txt in your local directory with the following contents:

Jack
Ross
Mike
Jack
Tom
Kate
Ross

Then, in a module called unique_names.py, create the function get_names_from_file() that reads the file and returns a list of names.

def get_names_from_file(filename):
    with open(filename, "r") as f:
        lines = f.readlines()
        names = []

        for line in lines:
            names.append(line.strip())

        return names

if __name__ == "__main__":
    print(get_names_from_file("names.txt"))

Verify that the output is the following:

['Jack', 'Ross', 'Mike', 'Jack', 'Tom', 'Kate', 'Ross']

Now, in the same Python module create a second function named get_unique_names() that calls the first function. This function returns unique names from the list.

def get_unique_names(filename):
    names = get_names_from_file(filename)
    unique_names = list(set(names))
    return sorted(unique_names)

Confirm that the code works:

if __name__ == "__main__":
    print(get_unique_names("names.txt"))

[output]
['Jack', 'Kate', 'Mike', 'Ross', 'Tom']

To fully understand the syntax above read the CodeFatherTech tutorial about if __name__ == “__main__”.

How Do You Write Unit Tests Using a Mock in Python?

In your test, you don’t want to rely on the fact that the file read by the function has to be available. That’s because if for some reason, the file gets deleted, your tests can start failing intermittently.

Create a file called test_unique_names.py that contains our test.

import unittest
import unique_names

class TestUnique(unittest.TestCase):
    def test_unique_names(self):
        names = unique_names.get_unique_names("names.txt")
        self.assertEqual(names, ['Jack', 'Kate', 'Mike', 'Ross', 'Tom'])

if __name__ == "__main__":
    unittest.main()

Run the test and confirm a successful execution.

(python-env) # python test_unique_names.py 
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

To learn more about how this works read the CodeFatherTech tutorial about Python unit tests.

Now, delete the file names.txt and rerun the test.

(python-env) # python test_unique_names.py
E
======================================================================
ERROR: test_unique_names (__main__.TestUnique)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_unique_names.py", line 6, in test_unique_names
    names = unique_names.get_unique_names("names.txt")
  File "/opt/tutorials/unique_names.py", line 12, in get_unique_names
    names = get_names_from_file(filename)
  File "/opt/tutorials/unique_names.py", line 2, in get_names_from_file
    with open(filename, "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'names.txt'

----------------------------------------------------------------------
Ran 1 test in 0.001s

FAILED (errors=1)

A FileNotFoundError exception will be raised. This is exactly the type of failure we want to avoid in our unit tests using Python mocks.

Let’s update our test to use a MagicMock instead of the real function get_names_from_file(). In this way, we can remove the external dependency from the file.

import unittest
import unique_names
from unittest.mock import MagicMock

class TestUnique(unittest.TestCase):
    def test_unique_names(self):
        unique_names.get_names_from_file = MagicMock()
        unique_names.get_names_from_file.return_value = ['Jack', 'Ross', 'Mike', 'Jack', 'Tom', 'Kate', 'Ross']
        names = unique_names.get_unique_names("names.txt")
        self.assertEqual(names, ['Jack', 'Kate', 'Mike', 'Ross', 'Tom'])

if __name__ == "__main__":
    unittest.main()

We have added the following two lines before calling the function get_unique_names() in our test.

unique_names.get_names_from_file = MagicMock()
unique_names.get_names_from_file.return_value = ['Jack', 'Ross', 'Mike', 'Jack', 'Tom', 'Kate', 'Ross']

The first line replaces the function get_names_from_file() with a mock. The second line sets the return value of the mock function. This is the same list returned by the function get_names_from_file() when reading the file.

This removes the dependency from the file names.txt.

This means that passing “names.txt” as an argument on the following line of the test doesn’t make any difference.

names = unique_names.get_unique_names("names.txt")

You could also pass an empty filename or any other filename. It wouldn’t make any difference because we don’t read the real file anymore as part of our test given that the mock is called instead of the original function.

Confirm that the test run is now successful.

(python-env) # python test_unique_names.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

To recap, we have seen how to replace a function in a Python module with a MagicMock object as part of a unit test.

What is the Patch Decorator in Python?

The unittest.mock library provides the patch function (unittest.mock.patch) that can replace the behavior of a target (a function, a method, or a class) as part of unit tests.

We will update the test class created in the previous section, to use the patch function instead of a MagicMock object directly.

The patch function allows patching objects. It transparently replaces the behavior of the target with a MagicMock.

import unittest
import unique_names
from unittest.mock import patch

class TestUnique(unittest.TestCase):
    @patch('unique_names.get_names_from_file')
    def test_unique_names(self, mock_get_names_from_file):
        mock_get_names_from_file.return_value = ['Jack', 'Ross', 'Mike', 'Jack', 'Tom', 'Kate', 'Ross']
        names = unique_names.get_unique_names("names.txt")
        self.assertEqual(names, ['Jack', 'Kate', 'Mike', 'Ross', 'Tom'])

if __name__ == "__main__":
    unittest.main()

To update the test from using a MagicMock to using the patch function, we have implemented the following code changes:

  1. Import patch instead of MagicMock from unittest.mock.
  2. Add the decorator @patch(‘unique_names.get_names_from_file’) to the test method.
  3. Add the argument mock_get_names_from_file to the test method. This is automatically passed into your test function by the decorator.
  4. Set the return_value of the mock object mock_get_names_from_file similarly to what we have done before.

Confirm that the test executes successfully. This means that the patched function is called.

(python-env) # python test_unique_names.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

If you add a print statement inside the test method, you can confirm that the mock created by the patch decorator is a MagicMock object.

@patch('unique_names.get_names_from_file')
    def test_unique_names(self, mock_get_names_from_file):
        print(mock_get_names_from_file)
        ...
        ...

[output]
<MagicMock name='get_names_from_file' id='4309698832'>

Using Python Patch in Python as Context Manager

The Python patch function can be used as a decorator or a context manager.

Let’s see first how the code of the unit test becomes and then we will understand what using a context manager means.

import unittest
import unique_names
from unittest.mock import patch

class TestUnique(unittest.TestCase):
    def test_unique_names(self):
        with patch('unique_names.get_names_from_file') as mock_get_names_from_file:
            mock_get_names_from_file.return_value = ['Jack', 'Ross', 'Mike', 'Jack', 'Tom', 'Kate', 'Ross']
            names = unique_names.get_unique_names("names.txt")
            self.assertEqual(names, ['Jack', 'Kate', 'Mike', 'Ross', 'Tom'])

if __name__ == "__main__":
    unittest.main()

You can see that the decorator and the additional mock argument have been removed from the test method.

We have added a “with patch” statement that creates the mock object mock_get_names_from_file to replace the function unique_names.get_names_from_file().

Once again the test is successful:

(python-env) # python test_unique_names.py
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

To understand what the context manager does, add two print statements to verify the value of the mocked function unique_names.get_names_from_file() inside and outside the “with patch” statement.

class TestUnique(unittest.TestCase):
    def test_unique_names(self):
        with patch('unique_names.get_names_from_file') as mock_get_names_from_file:
            print("Inside with patch statement: ", unique_names.get_names_from_file)
            mock_get_names_from_file.return_value = ['Jack', 'Ross', 'Mike', 'Jack', 'Tom', 'Kate', 'Ross']
            names = unique_names.get_unique_names("names.txt")
            self.assertEqual(names, ['Jack', 'Kate', 'Mike', 'Ross', 'Tom'])

        print("Outside with patch statement: ", unique_names.get_names_from_file)

From the output, you can see that the mock is applied only inside the “with patch” block. Outside of that block, the function unique_names.get_names_from_file() gets its original value coming from the unique_names Python module.

Inside with patch statement:  <MagicMock name='get_names_from_file' id='4389287920'>
Outside with patch statement:  <function get_names_from_file at 0x1059bcaf0>

Conclusion

In this tutorial, we have explained the concept of mocking in Python. It’s a useful way to create a more controlled environment for your unit tests and integration tests.

Python mock and patch, part of the unittest.mock library, enable you to test a huge variety of scenarios by removing dependencies that can make tests unstable and unreliable.

The Mock and MagicMock classes allow to create mock objects. The patch function is useful to replace the behavior of a target in your Python tests and transparently create MagicMock objects for you.

Now you know what to do if you want to mock objects in your Python tests.

Related article: in this article, we have talked a lot about unit tests. If you want to learn more about the foundations of unit tests and improve your test suites, read this tutorial about Python unit tests.

Related posts:

  1. Python Assert Statement: Learn It in 7 Steps
  2. How To Write a Unit Test in Python: A Simple Guide

Leave a Comment