Thanks to visit codestin.com
Credit goes to www.scribd.com

0% found this document useful (0 votes)
14 views12 pages

Hypermodern Python Chapter 2 - Testing Claudio Jolowicz

Chapter 2 of Hypermodern Python focuses on implementing automated testing in Python projects using tools like pytest, Coverage.py, and Nox. It covers unit testing, code coverage, test automation, and mocking, providing practical examples and best practices for ensuring code reliability. The chapter emphasizes the importance of testing early and maintaining high code coverage for effective software development.

Uploaded by

teguhteja
Copyright
© Public Domain
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
14 views12 pages

Hypermodern Python Chapter 2 - Testing Claudio Jolowicz

Chapter 2 of Hypermodern Python focuses on implementing automated testing in Python projects using tools like pytest, Coverage.py, and Nox. It covers unit testing, code coverage, test automation, and mocking, providing practical examples and best practices for ensuring code reliability. The chapter emphasizes the importance of testing early and maintaining high code coverage for effective software development.

Uploaded by

teguhteja
Copyright
© Public Domain
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

Claudio Jolowicz 

Blog
Music

Hypermodern Python Chapter 2: Testing


 January 8, 2020  16 minutes read
 python • poetry • nox • pytest • coverage.py • pytest-mock

Read this article on Medium

In this second installment of the Hypermodern Python series, I’m going to discuss how to add automated testing to your project, and how to teach the random fact
generator foreign languages.1 Previously, we discussed How to set up a Python project. (If you start reading here, you can also download the code for the previous
chapter.)

Here are the topics covered in this chapter on Testing in Python:

Unit testing with pytest


Code coverage with Coverage.py
Test automation with Nox
Mocking with pytest-mock
Example CLI: Refactoring
Example CLI: Handling exceptions gracefully
Example CLI: Selecting the Wikipedia language edition
Using fakes
End-to-end testing

Here is a full list of the articles in this series:

Chapter 1: Setup
Chapter 2: Testing (this article)
Chapter 3: Linting
Chapter 4: Typing
Chapter 5: Documentation
Chapter 6: CI/CD

This guide has a companion repository: cjolowicz/hypermodern-python. Each article in the guide corresponds to a set of commits in the GitHub repository:

View changes
Download code

Unit testing with pytest

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 1/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

It’s never too early to add unit tests to a project.

Unit tests, as the name says, verify the functionality of a unit of code, such as a single function or class. While the unittest framework is part of the Python standard
library, pytest has become somewhat of a de facto standard.

Let’s add this package as a development dependency, using Poetry’s --dev option:

poetry add --dev pytest

Organize tests in a separate file hierarchy next to src, named tests:

.
├── src
└── tests
├── __init__.py
└── test_console.py

2 directories, 2 files

The file __init__.py is empty and serves to declare the test suite as a package. While this is not strictly necessary, it allows your test suite to mirror the source
layout of the package under test, even when modules in different parts of the source tree have the same name. Furthermore, it gives you the option to import modules
from within your tests package.

The file test_console.py contains a test case for the console module, which checks whether the program exits with a status code of zero.

# tests/test_console.py
import click.testing

from hypermodern_python import console

def test_main_succeeds():
runner = click.testing.CliRunner()
result = runner.invoke(console.main)
assert result.exit_code == 0

Click’s testing.CliRunner can invoke the command-line interface from within a test case. Since this is likely to be needed by most test cases in this module, let’s
turn it into a test fixture. Test fixtures are simple functions declared with the pytest.fixture decorator. Test cases can use a test fixture by including a function
parameter with the same name as the test fixture.

# tests/test_console.py
import click.testing
import pytest

from hypermodern_python import console

@pytest.fixture
def runner():
return click.testing.CliRunner()

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 2/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

def test_main_succeeds(runner):
result = runner.invoke(console.main)
assert result.exit_code == 0

Invoke pytest to run the test suite:

$ poetry run pytest


============================ test session starts =============================
platform linux -- Python 3.8.2, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
rootdir: /hypermodern-python
collected 1 item

tests/test_console.py . [100%]

============================= 1 passed in 0.03s ==============================

Code coverage with Coverage.py

Code coverage is a measure of the degree to which the source code of your program is executed while running its test suite. The code coverage of Python programs can
be determined using a tool called Coverage.py. Install it with the pytest-cov plugin, which integrates Coverage.py with pytest:

poetry add --dev coverage[toml] pytest-cov

You can configure Coverage.py using the pyproject.toml configuration file, provided it was installed with the toml extra as shown above. Update this file to
inform the tool about your package name and source tree layout. The configuration also enables branch analysis and the display of line numbers for missing coverage:

# pyproject.toml
[tool.coverage.paths]
source = ["src", "*/site-packages"]

[tool.coverage.run]
branch = true
source = ["hypermodern_python"]

[tool.coverage.report]
show_missing = true

To enable coverage reporting, invoke pytest with the --cov option:

$ poetry run pytest --cov


============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
rootdir: /hypermodern-python
plugins: cov-2.8.1
collected 1 item

tests/test_console.py . [100%]

--------------- coverage: platform linux, python 3.8.2-final-0 -----------------


Name Stmts Miss Branch BrPart Cover Missing
--------------------------------------------------------------------------------
src/hypermodern_python/__init__.py 1 0 0 0 100%
src/hypermodern_python/console.py 6 0 0 0 100%
--------------------------------------------------------------------------------
TOTAL 7 0 0 0 100%
============================== 1 passed in 0.09s ===============================

The reported code coverage is 100%. This number does not imply that your test suite has meaningful test cases for all uses and misuses of your program. Code
coverage only tells you that all lines and branches in your code base were hit. (In fact, our test case achieved full coverage without checking the functionality of the

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 3/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz
program at all, only its exit status.)

Nevertheless, aiming for 100% code coverage is good practice, especially for a fresh codebase. Anything less than that implies that some part of your code base is
definitely untested. And to quote Bruce Eckel, “If it’s not tested, it’s broken.” Later, we will see some tools that help you achieve extensive code coverage.

You can configure Coverage.py to require full test coverage (or any other target percentage) using the fail_under option:

# pyproject.toml
[tool.coverage.report]
fail_under = 100

Test automation with Nox

One of my personal favorites, Nox is a successor to the venerable tox. At its core, the tool automates testing in multiple Python environments. Nox makes it easy to run
any kind of job in an isolated environment, with only those dependencies installed that the job needs.

Install Nox via pip or pipx:

pip install --user --upgrade nox

Unlike tox, Nox uses a standard Python file for configuration:

# noxfile.py
import nox

@nox.session(python=["3.8", "3.7"])
def tests(session):
session.run("poetry", "install", external=True)
session.run("pytest", "--cov")

This file defines a session named tests, which installs the project dependencies and runs the test suite. Poetry is not a part of the environment created by Nox, so we
specify external to avoid warnings about external commands leaking into the isolated test environments.

Nox creates virtual environments for the listed Python versions (3.8 and 3.7), and runs the session inside each environment:

$ nox

nox > Running session tests-3.8


nox > Creating virtual environment (virtualenv) using python3.8 in .nox/tests-3-8
nox > poetry install
...
nox > pytest --cov
...
nox > Session tests-3.8 was successful.
nox > Running session tests-3.7
nox > Creating virtual environment (virtualenv) using python3.7 in .nox/tests-3-7
nox > poetry install
...
nox > pytest --cov
...
nox > Session tests-3.7 was successful.
nox > Ran multiple sessions:
nox > * tests-3.8: success
nox > * tests-3.7: success

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 4/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz
Nox recreates the virtual environments from scratch on each invocation (a sensible default). You can speed things up by passing the --reuse-existing-virtualenvs (-r)
option:

nox -r

Sometimes, you need to pass additional options to pytest, for example to select specific test cases. Change the session to allow overriding the options passed to
pytest, via the session.posargs variable:

# noxfile.py
import nox

@nox.session(python=["3.8", "3.7"])
def tests(session):
args = session.posargs or ["--cov"]
session.run("poetry", "install", external=True)
session.run("pytest", *args)

Now you can run a specific test module inside the environments:

nox -- tests/test_console.py

Mocking with pytest-mock

Unit tests should be fast, isolated, and repeatable. The test for console.main is neither of these:

It is not fast, because it takes a full round-trip to the Wikipedia API to complete.
It does not run in an isolated environment, because it sends out an actual request over the network.
It is not repeatable, because its outcome depends on the health, reachability, and behavior of the API. In particular, the test fails whenever the network is down.

The unittest.mock standard library allows you to replace parts of your system under test with mock objects. Use it via the pytest-mock plugin, which integrates the
library with pytest:

poetry add --dev pytest-mock

The plugin provides a mocker fixture, which functions as a thin wrapper around the standard mocking library. Use mocker.patch to replace the requests.get
function by a mock object. The mock object will be useful for any test case involving the Wikipedia API, so let’s create a test fixture for it:

# tests/test_console.py
@pytest.fixture
def mock_requests_get(mocker):
return mocker.patch("requests.get")

Add the fixture to the function parameters of the test case:

def test_main_succeeds(runner, mock_requests_get):


...

If you run Nox now, the test fails because click expects to be passed a string for console output, and receives a mock object instead. Simply “knocking out”
requests.get is not quite enough. The mock object also needs to return something meaningful, namely a response with a valid JSON object.

When a mock object is called, or when an attribute is accessed, it returns another mock object. Sometimes this is sufficient to get you through a test case. When it is not,
you need to configure the mock object. To configure an attribute, you simply set the attribute to the desired value. To configure the return value for when the mock is
called, you set return_value on the mock object as if it were an attribute.

Let’s look at the example again:

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 5/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz
with requests.get(API_URL) as response:
response.raise_for_status()
data = response.json()

The code above uses the response as a context manager. The with statement is syntactic sugar for the following slightly simplified pseudocode:

context = requests.get(API_URL)
response = context.__enter__()

try:
response.raise_for_status()
data = response.json()
finally:
context.__exit__(...)

So what you have is essentially a chain of function calls:

data = requests.get(API_URL).__enter__().json()

Rewrite the fixture, and mirror this call chain when you configure the mock:

@pytest.fixture
def mock_requests_get(mocker):
mock = mocker.patch("requests.get")
mock.return_value.__enter__.return_value.json.return_value = {
"title": "Lorem Ipsum",
"extract": "Lorem ipsum dolor sit amet",
}
return mock

Invoke Nox again to see that the test suite passes. 🎉


$ nox -r
...
nox > Ran multiple sessions:
nox > * tests-3.8: success
nox > * tests-3.7: success

Mocking not only speeds up your test suite, or lets you hack offline on a plane or train. By virtue of having a fixed, or deterministic, return value, the mock also enables
you to write repeatable tests. This means you can, for example, check that the title returned by the API is printed to the console:

def test_main_prints_title(runner, mock_requests_get):


result = runner.invoke(console.main)
assert "Lorem Ipsum" in result.output

Additionally, mocks can be inspected to see if they were called, using the mock’s called attribute. This provides you with a way to check that requests.get was
invoked to send a request to the API:

# tests/test_console.py
def test_main_invokes_requests_get(runner, mock_requests_get):
runner.invoke(console.main)
assert mock_requests_get.called

Mock objects also allow you to inspect the arguments they were called with, using the call_args attribute. This allows you to check the URL passed to
requests.get:

# tests/test_console.py
def test_main_uses_en_wikipedia_org(runner, mock_requests_get):
runner.invoke(console.main)
args, _ = mock_requests_get.call_args
assert "en.wikipedia.org" in args[0]

You can configure a mock to raise an exception instead of returning a value by assigning the exception instance or class to the side_effect attribute of the mock. Let’s
check that the program exits with a status code of 1 on request errors:

# tests/test_console.py
def test_main_fails_on_request_error(runner, mock_requests_get):
mock_requests_get.side_effect = Exception("Boom")
result = runner.invoke(console.main)
assert result.exit_code == 1

You should generally have a single assertion per test case, because more fine-grained test cases make it easier to figure out why the test suite failed when it does.

Tests for a feature or bugfix should be written before implementation. This is also known as “writing a failing test". The reason for this is that it provides confidence
that the tests are actually testing something, and do not simply pass because of a flaw in the tests themselves.

Example CLI: Refactoring

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 6/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

The great thing about a good test suite is that it allows you to refactor your code without fear of breaking it. Let’s move the Wikipedia client to a separate module.
Create a file src/hypermodern-python/wikipedia.py with the following contents:

# src/hypermodern-python/wikipedia.py
import requests

API_URL = "https://en.wikipedia.org/api/rest_v1/page/random/summary"

def random_page():
with requests.get(API_URL) as response:
response.raise_for_status()
return response.json()

The console module can now simply invoke wikipedia.random_page:

# src/hypermodern-python/console.py
import textwrap

import click

from . import __version__, wikipedia

@click.command()
@click.version_option(version=__version__)
def main():
"""The hypermodern Python project."""
data = wikipedia.random_page()

title = data["title"]
extract = data["extract"]

click.secho(title, fg="green")
click.echo(textwrap.fill(extract))

Finally, invoke Nox to see that nothing broke:

$ nox -r
...
nox > Ran multiple sessions:
nox > * tests-3.8: success
nox > * tests-3.7: success

Example CLI: Handling exceptions gracefully

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 7/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

If you run the example application without an Internet connection, your terminal will be filled with a long traceback. This is what happens when the Python interpreter
is terminated by an unhandled exception. For common errors such as this, it would be better to print a friendly, informative message to the screen.

Let’s express this as a test case, by configuring the mock to raise a RequestException. (The requests library has more specific exception classes, but for the
purposes of this example, we will only deal with the base class.)

# tests/test_console.py
import requests

def test_main_prints_message_on_request_error(runner, mock_requests_get):


mock_requests_get.side_effect = requests.RequestException
result = runner.invoke(console.main)
assert "Error" in result.output

The simplest way to get this test to pass is by converting the RequestException into a ClickException. When click encounters this exception, it prints the
exception message to standard error and exits the program with a status code of 1. You can reuse the exception message by converting the original exception to a string.

Here is the updated wikipedia module:

# src/hypermodern-python/wikipedia.py
import click
import requests

API_URL = "https://en.wikipedia.org/api/rest_v1/page/random/summary"

def random_page():
try:
with requests.get(API_URL) as response:
response.raise_for_status()
return response.json()
except requests.RequestException as error:
message = str(error)
raise click.ClickException(message)

Example CLI: Selecting the Wikipedia language edition

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 8/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

In this section, we add a command-line option to select the language edition of Wikipedia.

Wikipedia editions are identified by a language code, which is used as a subdomain below wikipedia.org. Usually, this is the two-letter or three-letter language code
assigned to the language by ISO 639-1 and ISO 639-3. Here are some examples:

fr for the French Wikipedia


jbo for the Lojban Wikipedia
ceb for the Cebuano Wikipedia

As a first step, let’s add an optional parameter for the language code to the wikipedia.random_page function. When an alternate language is passed, the API
request should be sent to the corresponding Wikipedia edition. The test case is placed in a new test module named test_wikipedia.py:

# tests/test_wikipedia.py
from hypermodern_python import wikipedia

def test_random_page_uses_given_language(mock_requests_get):
wikipedia.random_page(language="de")
args, _ = mock_requests_get.call_args
assert "de.wikipedia.org" in args[0]

The mock_requests_get fixture is now used by two test modules. You could move it to a separate module and import from there, but Pytest offers a more
convenient way: Fixtures placed in a conftest.py file are discovered automatically, and test modules at the same directory level can use them without explicit
import. Create the new file at the top-level of your tests package, and move the fixture there:

# tests/conftest.py
import pytest

@pytest.fixture
def mock_requests_get(mocker):
mock = mocker.patch("requests.get")
mock.return_value.__enter__.return_value.json.return_value = {
"title": "Lorem Ipsum",
"extract": "Lorem ipsum dolor sit amet",
}
return mock

To get the test to pass, we turn API_URL into a format string, and interpolate the specified language code into the URL using str.format:

# src/hypermodern-python/wikipedia.py
import click
import requests

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 9/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

API_URL = "https://{language}.wikipedia.org/api/rest_v1/page/random/summary"

def random_page(language="en"):
url = API_URL.format(language=language)

try:
with requests.get(url) as response:
response.raise_for_status()
return response.json()
except requests.RequestException as error:
message = str(error)
raise click.ClickException(message)

As the second step, we make the new functionality accessible from the command line, adding a --language option. The test case mocks the
wikipedia.random_page function, and uses the assert_called_with method on the mock to check that the language specified by the user is passed on to the
function:

# tests/test_console.py
@pytest.fixture
def mock_wikipedia_random_page(mocker):
return mocker.patch("hypermodern_python.wikipedia.random_page")

def test_main_uses_specified_language(runner, mock_wikipedia_random_page):


runner.invoke(console.main, ["--language=pl"])
mock_wikipedia_random_page.assert_called_with(language="pl")

We are now ready to implement the new functionality using the click.option decorator. Without further ado, here is the final version of the console module:

# src/hypermodern-python/console.py
import textwrap

import click

from . import __version__, wikipedia

@click.command()
@click.option(
"--language",
"-l",
default="en",
help="Language edition of Wikipedia",
metavar="LANG",
show_default=True,
)
@click.version_option(version=__version__)
def main(language):
"""The hypermodern Python project."""
data = wikipedia.random_page(language=language)

title = data["title"]
extract = data["extract"]

click.secho(title, fg="green")
click.echo(textwrap.fill(extract))

You now have a polyglot random fact generator, and a fun way to test your language skills (and the Unicode skills of your terminal emulator).

Using fakes

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 10/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

Mocks help you test code units depending on bulky subsystems, but they are not the only technique to do so. For example, if your function requires a database
connection, it may be both easier and more effective to pass an in-memory database than a mock object. Fake implementations are a good alternative to mock objects,
which can be too forgiving when faced with wrong usage, and too tightly coupled to implementation details of the system under test (witness the
mock_requests_get fixture). Large data objects can be generated by test object factories, instead of being replaced by mock objects (check out the excellent
factoryboy package).

Implementing a fake API is out of the scope of this tutorial, but we will cover one aspect of it: How to write a fixture which requires tear down code as well as set up
code. Suppose you have written the following fake API implementation:

class FakeAPI:
url = "http://localhost:5000/"

@classmethod
def create(cls):
...

def shutdown(self):
...

The following will not work:

@pytest.fixture
def fake_api():
return FakeAPI.create()

The API needs to be shut down after use, to free up resources such as the TCP port and the thread running the server. You can do this by writing the fixture as a
generator:

@pytest.fixture
def fake_api():
api = FakeAPI.create()
yield api
api.shutdown()

Pytest takes care of running the generator, passing the yielded value to your test function, and executing the shutdown code after it returns. If setting up and tearing
down the fixture is expensive, you may also consider extending the scope of the fixture. By default, fixtures are created once per test function. Instead, you could create
the fake API server once per test session:

@pytest.fixture(scope="session")
def fake_api():
api = FakeAPI.create()
yield api
api.shutdown()

End-to-end testing

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 11/12
6/1/24, 3:15 PM Hypermodern Python Chapter 2: Testing · Claudio Jolowicz

Testing against the live production server is bad practice for unit tests, but there is nothing like the confidence you get from seeing your code work in a real
environment. Such tests are known as end-to-end tests, and while they are usually too slow, brittle, and unpredictable for the kind of automated testing you would want
to do on a CI server or in the midst of development, they do have their place.

Let’s reinstate the original test case, and use Pytest’s markers to apply a custom mark. This will allow you to select or skip them later, using Pytest’s -m option.

# tests/test_console.py
@pytest.mark.e2e
def test_main_succeeds_in_production_env(runner):
result = runner.invoke(console.main)
assert result.exit_code == 0

Register the e2e marker using the pytest_configure hook, as shown below. The hook is placed in the conftest.py file, at the top-level of your tests package.
This ensures that Pytest can discover the module and use it for the entire test suite.

# tests/conftest.py
def pytest_configure(config):
config.addinivalue_line("markers", "e2e: mark as end-to-end test.")

Finally, exclude end-to-end tests from automated testing by passing -m "not e2e" to Pytest:

# noxfile.py
import nox

@nox.session(python=["3.8", "3.7"])
def tests(session):
args = session.posargs or ["--cov", "-m", "not e2e"]
session.run("poetry", "install", external=True)
session.run("pytest", *args)

You can now run end-to-end tests by passing -m e2e to the Nox session, using a double dash (--) to separate them from Nox’s own options. For example, here’s how
you would run end-to-end tests inside the testing environment for Python 3.8:

nox -rs tests-3.8 -- -m e2e

Thanks for reading!


The next chapter is about linting your project.

Continue to the next chapter

1. The images in this chapter come from Émile-Antoine Bayard’s Illustrations for From the Earth to the Moon (De la terre à la lune) by Jules Verne (1870) (source:
Internet Archive via The Public Domain Review) ↩︎

https://cjolowicz.github.io/posts/hypermodern-python-02-testing/#test-automation-with-nox 12/12

You might also like