59

I'm working on a presentation about python testing options, and one of the technologies I want to demo is pytest. I'm planning to do the presentation from an jupyter/ipython notebook. Ideally I'd like to be able to define a test function in a cell and then use pytest to run just that function; that way I can demonstrate individual features of pytest, etc.

Is there an existing way to do this? None of the interactive features I saw in the pytest api seem to fit the bill, but I'm certainly no pytest expert. I'd be happy to write a plugin to do it, so any advice on the best approach in that direction would be great.

I did see pytest-ipynb, but it does not seem to do what I need. If that assessment is wrong, advice on how to use that would be great, too.

1

4 Answers 4

34

There is a similar module that looks very mature: https://github.com/chmp/ipytest

It is referenced in the module developed by @akaiola and is more active and regularly updated / maintained.

In particular, the tests could use some functions defined in other cells, which is the whole purpose of performing tests inside the Notebook (see example below).

To use it:

  1. Put in a cell an import and configuration of ipytest:
import ipytest
ipytest.autoconfig()
  1. Write some tests in cell(s)
  2. Either run cells with test with %%ipytest magic for each cell with test ... or put a ipytest.run() in a cell after the tests

You can check the official example: https://github.com/chmp/ipytest/blob/main/Example.ipynb

Or an example I did:

example 1

And another example:

another example

2
  • 1
    Also - one can use the %%writefile magic, then call !pytest - demonstrated in this talk by Florian Bruhin. Commented Aug 15, 2023 at 7:16
  • 1
    @YinonEhrlich Interesting addition. However, this would be limited if we want to test functions defined in other cells of your notebook... unless you duplicate writing of this code as well. Commented Aug 16, 2023 at 7:22
25

I created ipython_pytest for this purpose and used it in my PyCon Helsinki 2016 presentation.

You would use the tool like this:

%%pytest

def test_my_stuff():
    assert 42 == 42

Unfortunately I couldn't find a way to make functions and classes defined in the notebook available to test functions in the test cell. I'd appreciate if someone comes up with a solution for that.

6
  • That looks perfect! That's what I get for not coming to the conference.
    – abingham
    Commented Dec 26, 2016 at 1:28
  • Oops, sorry, thought I had.
    – abingham
    Commented Dec 26, 2016 at 13:45
  • @akaihola found your talk. Would you like to post it too? )
    – pylang
    Commented Dec 28, 2016 at 20:57
  • @pylang I'm planning to turn the slides into a series of blog posts. I seem to need to be kicked a bit though... Oh, and the talk seems to be now on Youtube, too: youtube.com/watch?v=jOvkYnSBKeI
    – akaihola
    Commented Dec 28, 2016 at 21:32
  • 1
    could someone find a solution to reach objects out of the test cell ?
    – karfkars
    Commented Dec 30, 2017 at 21:08
7

My 2 cents here since I've already solved this issue for myself 2x and both times come back to this site. For my future self:

For a project structured in the following way

prod_folder:
* data
* code
    * run_notebook.ipynb
* tests
   * test_some_test.py

If you want to run pytest from within run_notebook.ipynb the following cell-magic worked for my purposes:

!pytest ../tests
5
  • What kernel/tool do you refer to here? Commented Dec 18, 2020 at 12:44
  • a jupyter python kernel cell
    – Aus_10
    Commented Dec 22, 2020 at 1:26
  • 2
    The ! makes is a shell call. You are instantiating a new Python session from the shell and running pytest. Commented Mar 1, 2021 at 4:53
  • That's a great point. Hadn't considered that thanks. Mine was very much "it seems to work and go with it"
    – Aus_10
    Commented Mar 1, 2021 at 18:49
  • I also often come back to this site and ends up in this page where there is my answer ;) Commented Oct 25, 2021 at 11:10
6

In Google Colab I usually do the following procedure:

  1. Create a file name for the respective cell:

Here's an example:

%%file test_list.py

# find the mismatched elements in list `l1` which are not in list `l2`

def solution(l1: list, l2: list) -> list:
    return [
        e
        for e in l1
        if e not in [e for e in l2]
    ]

import pytest

a = [1,2,3]
b = [3,4,None,None]
TEST_CASES = [[a,b]]

class TestCase:
    @pytest.mark.parametrize("test_case", TEST_CASES)
    def test_events(self, test_case):
        assert solution(test_case[0], test_case[1]) == [1,2]
  1. Run pytest in another cell:
!python -m pytest test_list.py 

Out:

============================= test session starts ==============================
platform linux -- Python 3.8.16, pytest-3.6.4, py-1.11.0, pluggy-0.7.1
rootdir: /content, inifile:
plugins: typeguard-2.7.1
collected 1 item                                                               

test_list.py .                                                           [100%]

=========================== 1 passed in 0.02 seconds ===========================

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.