Simulating External Dependencies in Pytest with pytest-mock
Wenhao Wang
Dev Intern · Leapcell

Introduction
In modern software development, applications rarely operate in isolation. They frequently interact with external services, such as REST APIs, third-party libraries, and databases. While these dependencies are crucial for an application's functionality, they introduce significant challenges to unit and integration testing. Relying on live external systems during testing can lead to slow tests, flakiness due to network issues or service outages, and even incur costs for API usage. This is where the concept of "mocking" comes into play. By simulating these external dependencies, we can create controlled, predictable, and fast test environments. This article will explore how to leverage pytest-mock—a powerful Pytest plugin—to effectively mock external API and database calls, ensuring robust and efficient testing of your Python applications.
Understanding the Core Concepts
Before diving into the practical implementation with pytest-mock, let's clarify some fundamental concepts:
- Mocking: In testing, mocking involves replacing a real object or function with a substitute that simulates the behavior of the real one. This substitute, called a "mock object" or simply a "mock," allows you to control the return values, raise exceptions, and monitor interactions, all without actually calling the original code.
- Stubbing: Often used interchangeably with mocking, stubbing specifically refers to providing pre-programmed responses to method calls made during a test. A stub's primary purpose is to return canned answers to calls made during the test, not to verify behavior.
- Spying: Spying involves wrapping a real object or function while still allowing its original behavior. The spy then records any interactions with the wrapped object, allowing you to assert that certain methods were called with specific arguments.
pytest-mockprimarily focuses on mocking, though its capabilities can be extended to observe interactions. - Unit Testing: Focuses on testing individual units or components of a software application in isolation. Mocking is crucial here to isolate the unit under test from its dependencies.
- Integration Testing: Verifies the interactions between different units or components of an application. While integration tests may involve some real dependencies, mocking can still be used for extremely volatile or costly external services.
pytest-mock is a Pytest fixture that provides a convenient wrapper around Python's built-in unittest.mock library. It integrates seamlessly with Pytest's powerful testing framework, offering a clean and intuitive way to manage mock objects within your tests.
Simulating External Dependencies with pytest-mock
The core principle behind using pytest-mock is to replace the actual objects or functions that interact with external services with controlled mock objects. This is typically done using the mocker.patch() method provided by the pytest-mock fixture.
Let's consider a practical example. Imagine we have a Python function that fetches user data from a remote API and another that saves data to a database.
Our Application Code (e.g., app.py):
import requests import sqlite3 class User: def __init__(self, user_id, name, email): self.user_id = user_id self.name = name self.email = email def __repr__(self): return f"User(id={self.user_id}, name={self.name}, email={self.email})" def fetch_user_from_api(user_id): """Fetches user data from a placeholder API.""" api_url = f"https://jsonplaceholder.typicode.com/users/{user_id}" try: response = requests.get(api_url) response.raise_for_status() # Raise an exception for bad status codes data = response.json() return User(data['id'], data['name'], data['email']) except requests.exceptions.RequestException as e: print(f"Error fetching user: {e}") return None def save_user_to_db(user): """Saves user data to a SQLite database.""" conn = None try: conn = sqlite3.connect('users.db') cursor = conn.cursor() cursor.execute(''' CREATE TABLE IF NOT EXISTS users ( id INTEGER PRIMARY KEY, name TEXT NOT NULL, email TEXT NOT NULL ) ''') cursor.execute("INSERT INTO users (id, name, email) VALUES (?, ?, ?)", (user.user_id, user.name, user.email)) conn.commit() return True except sqlite3.Error as e: print(f"Error saving user to DB: {e}") return False finally: if conn: conn.close() def get_and_save_user(user_id): """Fetches a user and saves them to the database.""" user = fetch_user_from_api(user_id) if user: return save_user_to_db(user) return False
Now, let's write tests for get_and_save_user without actually hitting jsonplaceholder.typicode.com or creating a real users.db file.
Testing with pytest-mock (e.g., test_app.py):
First, ensure you have pytest and pytest-mock installed:
pip install pytest pytest-mock requests
import pytest from unittest.mock import MagicMock import app # Our application code # Test case 1: Successful fetch and save def test_get_and_save_user_success(mocker): # Mock the requests.get method mock_response = MagicMock() mock_response.status_code = 200 mock_response.json.return_value = { 'id': 1, 'name': 'Leanne Graham', 'email': 'sincere@april.biz' } mocker.patch('requests.get', return_value=mock_response) # Mock the sqlite3.connect method and its associated cursor functions mock_conn = MagicMock() # Ensure commit and close are callable but do nothing for the mock mock_conn.commit.return_value = None mock_conn.close.return_value = None mock_conn.cursor.return_value.execute.return_value = None # For CREATE TABLE and INSERT mocker.patch('sqlite3.connect', return_value=mock_conn) # Call the function under test result = app.get_and_save_user(1) # Assertions assert result is True # Verify that requests.get was called mocker.patch.call_args_list[0].assert_called_once_with('https://jsonplaceholder.typicode.com/users/1') # Verify that sqlite3.connect was called mock_conn.cursor.assert_called_once() mock_conn.cursor.return_value.execute.assert_any_call( ''' CREATE TABLE IF NOT EXISTS users ( id INTEGER PRIMARY KEY, name TEXT NOT NULL, email TEXT NOT NULL ) ''' ) mock_conn.cursor.return_value.execute.assert_any_call( "INSERT INTO users (id, name, email) VALUES (?, ?, ?)", (1, 'Leanne Graham', 'sincere@april.biz') ) mock_conn.commit.assert_called_once() mock_conn.close.assert_called_once() # Test case 2: API call fails (e.g., network error) def test_get_and_save_user_api_failure(mocker): # Mock requests.get to raise an exception mocker.patch('requests.get', side_effect=requests.exceptions.RequestException("Network error")) mocker.patch('sqlite3.connect') # Even if API fails, ensures DB isn't touched result = app.get_and_save_user(1) assert result is False mocker.patch.call_args_list[0].assert_called_once_with('https://jsonplaceholder.typicode.com/users/1') # Assert that sqlite3.connect was NOT called app.sqlite3.connect.assert_not_called() # Test case 3: Database save fails def test_get_and_save_user_db_failure(mocker): # Mock successful API response mock_response = MagicMock() mock_response.status_code = 200 mock_response.json.return_value = { 'id': 1, 'name': 'Leanne Graham', 'email': 'sincere@april.biz' } mocker.patch('requests.get', return_value=mock_response) # Mock sqlite3.connect to raise an exception during commit mock_conn = MagicMock() mock_conn.cursor.return_value.execute.return_value = None mock_conn.commit.side_effect = sqlite3.Error("Database write error") mocker.patch('sqlite3.connect', return_value=mock_conn) result = app.get_and_save_user(1) assert result is False mocker.patch.call_args_list[0].assert_called_once_with('https://jsonplaceholder.typicode.com/users/1') # Verify DB connection and commit attempts mock_conn.cursor.assert_called_once() mock_conn.commit.assert_called_once() mock_conn.close.assert_called_once() # Ensure connection is closed even on error
Explanation of the Code Example:
mockerfixture:pytest-mockprovides themockerfixture, which automatically handles the setup and teardown of mock objects. When a test finishes, all patches created withmockerare automatically undone, preventing test pollution.mocker.patch('module.object', ...): This is the primary method for mocking.- The first argument (
'requests.get') is a string representing the fully qualified path to the object you want to mock. It's crucial to patch where the object is looked up, not necessarily where it's defined. In ourapp.py,requests.getis called directly, so we patchrequests.get. - For the database,
sqlite3.connectis called. It's important to chain the mocks for objects returned by the initial mock. For instance,mock_conn.cursor.return_value.executeallows us to mock theexecutemethod of thecursorobject thatconnectwould return.
- The first argument (
return_value: This attribute of a mock object specifies what it should return when called. Formock_response.json, we set itsreturn_valueto a dictionary mimicking the API's JSON response.side_effect: Instead of areturn_value,side_effectcan be used to make the mock raise an exception or call a function when invoked. This is useful for simulating error conditions, as shown intest_get_and_save_user_api_failureandtest_get_and_save_user_db_failure.MagicMock: This is a versatile mock class fromunittest.mockthat creates objects that automatically create attributes and methods as they are accessed. This is incredibly useful for simulating complex objects like HTTP responses or database connections where you don't care about every single attribute or method.- Assertions on Mocks: After calling the mocked function, we can use assertion methods provided by the mock object to verify its behavior:
mock.assert_called_once_with(...): Asserts that the mock was called exactly once with specific arguments.mock.assert_any_call(...): Asserts that the mock was called at least once with specific arguments.mock.assert_not_called(): Asserts that the mock was never called.mock.call_args_list: Provides a list of all calls made to the mock.
Advanced Scenarios and Best Practices
- Patching Classes: You can patch entire classes to return mock instances. For example,
mocker.patch('app.User', return_value=MagicMock())would makeapp.User()return aMagicMockobject. - Context Managers: For granular mocking within a
withstatement,mocker.patch.object()can be used, often inside awithblock for temporary patching. - Fixture Scopes: Remember Pytest's fixture scopes.
mockeris typically function-scoped, meaning patches are torn down after each test. If you need a more persistent mock (e.g., for an entire module), considerautouse=Trueon a fixture that usesmockeror configure it more globally. However, function-scoped mocks are generally preferred for isolation. - When to Mock vs. Not: Mocking is best for external, unpredictable, or expensive dependencies. For internal, stable code, it's often better to test with real implementations to catch integration issues.
- Over-mocking: Be careful not to over-mock your application. If you mock too much, your tests might pass, but your real application could still fail because the mocks don't accurately reflect the real behavior, or you're testing the mock itself rather than your logic.
Conclusion
pytest-mock provides an elegant and effective solution for isolating your Python code from external dependencies during testing. By learning to strategically use mocker.patch() and understanding the capabilities of unittest.mock objects, you can significantly improve the speed, reliability, and maintainability of your test suite. Embracing mocking allows your tests to focus purely on the logic of the unit under examination, leading to more robust and higher-quality software. Effectively simulating dependencies is key to writing resilient and efficient tests.