23

I have a global object in my main file

# reporter.py

from os import environ
from influxdb import InfluxDBClient

influxdb_client = InfluxDBClient(host=environ['INFLUXCLOUD_HOST'],
                                 username=environ['INFLUXCLOUD_USERNAME'],
                                 password=environ['INFLUXCLOUD_PASSWORD'],
                                 ssl=True,
                                 timeout=4*60)

def foo():
    pass

I'm using pytest and I want to set faux values to these environment variables. I have the following in my conftest.py:

# conftest.py

import pytest

@pytest.fixture(scope='session', autouse=True)
def setup_env(monkeypatch):
    monkeypatch.setenv('INFLUXCLOUD_HOST', 'host')
    monkeypatch.setenv('INFLUXCLOUD_USERNAME', 'username')
    monkeypatch.setenv('INFLUXCLOUD_PASSWORD', 'password')

However, when I import reporter in my test file, I get a KeyError that INFLUXCLOUD_HOST is missing in the env.

Why does not pytest execute the setup_env and monkeypatch my environment? Is there a way to do so?

1
  • 1
    Can you please show your test file and how do your import reporter? Commented Oct 14, 2017 at 13:34

3 Answers 3

14

As of pytest 6.2, you can use a MonkeyPatch object directly instead of the monkeypatch fixture, either as an instance or a context manager.

(Sergey already provided solid background on the "Why" question; this attempts to address the "How".)

Context manager (recommended):

since unlike the monkeypatch fixture, an instance created directly is not undo()-ed automatically.

# test_reporter.py

from pytest import MonkeyPatch


def test_get_client_username():
    with MonkeyPatch.context() as mp:
        mp.setenv('INFLUXCLOUD_HOST', 'host')
        mp.setenv('INFLUXCLOUD_USERNAME', 'username')
        mp.setenv('INFLUXCLOUD_PASSWORD', 'password')

        from src.reporter import influxdb_client

        assert influxdb_client._username == 'username'

Direct use of instance:

# conftest.py

from pytest import MonkeyPatch


mp = pytest.MonkeyPatch()
mp.setenv('INFLUXCLOUD_HOST', 'host')
mp.setenv('INFLUXCLOUD_USERNAME', 'username')
mp.setenv('INFLUXCLOUD_PASSWORD', 'password')

Assumed file structure for reference:

src/
    reporter.py
    __init__.py
test/
    conftest.py
    test_reporter.py
12

The problem here lies in misunderstanding what the session-scoped fixture is.

To know which tests & autoused fixtures do exist, pytest needs to import the test files & conftest plugins. Then it scans the imported modules, and looks for the fixtures & test functions & test classes & etc. This is called "collection" in pytest terms.

Only after all tests are collected, pytest decides to execute them, and arranges the execution plan, and specifically when the fixtures are prepared. The session-scoped fixtures are prepared first and teared down last — before any tests start, and after all tests have finished.

However, importing of the test files & conftest assumes the execution of these modules — as importing of any other Python module, unrelated to pytest.

So, when you do import reporter from your test file, or even if you put that global variable directly to your test file, this module is executed, and it attempts to use the env vars. But the fixtures are not yet executed (and pytest does not know about their existence yet). Therefore, it fails.

Even if you will import reporter from inside of the test function, this will not help much, as pytest may try to import that reporter.py module before during the collection stage. Pytest would filter it out due to the absence of the test functions/classes, but the import attempt will be done and will fail.

The best solution here is to "pack" the client into a fixture and use that fixture instead of the global variable.

1
  • 7
    could you provide an example of how "packing" the client into a fixture would look like? I didn't quite get that. When mocking the calls ("client", if you will) to os.environ[<<my_var>>] I still get a keyerror, because env variables are loaded before my client is.
    – Chris
    Commented Nov 22, 2018 at 10:51
0

As an addition to both the excellent answers that Sergey and Nathaniel provided, I followed Nathaniel's approach but my environment variable was still not being set correctly.

In my case, I had the following setup: lots of tests where I am using monkeypatch.setattr to mock attributes and methods of packages imported in the module to be tested. Therefore I need a top level import of the module so monkeypatch has access to that scope to patch the attributes and methods there, where they are imported (I believe this is not so different from https://docs.python.org/3/library/unittest.mock.html#where-to-patch, but others may correct me). I then import the functions I want to test.

In this setup, and using Nathaniel's suggestion (which is not wrong, I am setup differently), I have something like:

# test.py
from package_to_test import module_to_test
from package_to_test.module_to_test import (
   function_to_test1
   function_to_test2
   ...
   
)

...
# other tests
...

def test_function_using_env_var(monkeypatch):
   
    with monkeypatch.context() as monkeycontext:
        monkeycontext.setenv("ENV_VAR_TO_PATCH", "mocked_env_var")

        from package_to_test.module_to_test import function_using_env_var

        results = function_using_env_var()
        assert results == ...

In this case, I was still having the environment variable not correctly set. Now there could be other solutions. I tried to split up the test into another test module but that did not seem to do it.

The way I solved it was by simply reloading the module via importlib.reload. In this manner, we can also move the import of the function back up to the top:

# test.py
from package_to_test import module_to_test
from package_to_test.module_to_test import (
   function_to_test1
   function_to_test2
   ...
   
)
import importlib

...
# other tests
...

def test_function_using_env_var(monkeypatch):
   
    with monkeypatch.context() as monkeycontext:
        monkeycontext.setenv("ENV_VAR_TO_PATCH", "mocked_env_var")
        
        importlib.reload(module_to_test)
        

        results = function_using_env_var()
        assert results == ...

There might be simpler ways of solving this, so if anybody knows, please do tell!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.