12

When writing project-specific pytest plugins, I often find the Config object useful to attach my own properties. Example:

from _pytest.config import Config


def pytest_configure(config: Config) -> None:
    config.fizz = "buzz"

def pytest_unconfigure(config: Config) -> None:
    print(config.fizz)

Obviously, there's no fizz attribute in _pytest.config.Config class, so running mypy over the above snippet yields

conftest.py:5: error: "Config" has no attribute "fizz"
conftest.py:8: error: "Config" has no attribute "fizz"

(Note that pytest doesn't have a release with type hints yet, so if you want to actually reproduce the error locally, install a fork following the steps in this comment).

Sometimes redefining the class for typechecking can offer a quick help:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from _pytest.config import Config as _Config

    class Config(_Config):
        fizz: str

else:
    from _pytest.config import Config



def pytest_configure(config: Config) -> None:
    config.fizz = "buzz"

def pytest_unconfigure(config: Config) -> None:
    print(config.fizz)

However, aside from cluttering the code, the subclassing workaround is very limited: adding e.g.

from pytest import Session


def pytest_sessionstart(session: Session) -> None:
    session.config.fizz = "buzz"

would force me to also override Session for typechecking.

What is the best way to resolve this? Config is one example, but I usually have several more in each project (project-specific adjustments for test collection/invocation/reporting etc). I could imagine writing my own version of pytest stubs, but then I would need to repeat this for every project, which is very tedious.

2 Answers 2

7

One way of doing this would be to contrive to have your Config object define __getattr__ and __setattr__ methods. If those methods are defined in a class, mypy will use those to type check places where you're accessing or setting some undefined attribute.

For example:

from typing import Any

class Config:
    def __init__(self) -> None:
        self.always_available = 1

    def __getattr__(self, name: str) -> Any: pass

    def __setattr__(self, name: str, value: Any) -> None: pass

c = Config()

# Revealed types are 'int' and 'Any' respectively
reveal_type(c.always_available)
reveal_type(c.missing_attr)

# The first assignment type checks, but the second doesn't: since
# 'already_available' is a predefined attr, mypy won't try using
# `__setattr__`.
c.dummy = "foo"
c.always_available = "foo"

If you know for certain your ad-hoc properties will always be strs or something, you could type __getattr__ and __setattr__ to return or accept str instead of Any respectively to get tighter types.

Unfortunately, you would still need to do the subtyping trick or mess around with making your own stubs -- the only advantage this gives you is that you at least won't have to list out every single custom property you want to set and makes it possible to create something genuinely reusable. This could maybe make the option more palatable to you, not sure.

Other options you could explore include:

  • Just adding a # type: ignore comment to every line where you use an ad-hoc property. This would be a somewhat precise, if intrusive, way of suppressing the error messages.
  • Type your pytest_configure and pytest_unconfigure so they accept objects of type Any. This would be a somewhat less intrusive way of suppressing the error messages. If you want to minimize the blast radius of using Any, you could maybe confine any logic that wants to use these custom properties to their own dedicated functions and continue using Config everywhere else.
  • Try using casting instead. For example, inside pytest_configure you could do config = cast(MutableConfig, config) where MutableConfig is a class you wrote that subclasses _pytest.Config and defines both __getattr__ and __setattr__. This is maybe a middle ground between the above two approaches.
  • If adding ad-hoc attributes to Config and similar classes is a common kind of thing to do, maybe try convincing the pytest maintainers to include typing-only __getattr__ and __setattr__ definitions in their type hints -- or some other more dedicated way of letting users add these dynamic properties.
3
  • As always, great answer! I like the incomplete stubs suggestion - having a def __getattr__(name: str) -> Any: ... on stub module level allows to redefine only a small subset of what the actual module declares so only the project-specific stuff lands there, all rest is looked up in the library's source code. Based on that, it wasn't that hard to prepare a small stub template to include in each project (basically extracted all if TYPE_CHECKING: blocks), from there they can be customized to include project-specific stuff. Awesome!
    – hoefling
    Apr 15, 2020 at 8:44
  • IMO mypy docs could promote the incomplete stubs feature more - didn't find any mentioning of it in Stub files or Automatic stub generation. The one doc that actually mentions them with an example is typeshed's contributing guide, but it's worth to have that in the official docs too.
    – hoefling
    Apr 15, 2020 at 8:50
  • +1: Adding def __getattr__(self, name): pass and def __setattr__(self, name, value): pass is the only way to typecheck dynamically object attributes on Python 2.x Apr 28, 2023 at 10:46
3

You can extend the Config class by a single new attribute which is a dict and stores all the custom information. For example:

def pytest_configure(config: Config) -> None:
    config.data["fizz"] = "buzz"  # `data` is the custom dict

This way one custom stub file fits all your projects. Sure, it won't help your old projects immediately since you'd need to rewrite the relevant parts to use data['fizz'] instead of fizz. However an additional advantage of using a dict is that it prevents possible name clashes between already existing and custom attributes.

If attaching custom data to Config objects is common practice maybe it's worth an attempt to standardize this in form of a data dict and open a corresponding issue at the pytest project.

If you don't like rewriting code but nevertheless want to use a static type checker, you could still use custom per-project stub files, generated from some template. You could list all the custom attributes directly as annotations on a custom class and then have a script generate the corresponding stub file from it:

from _pytest.config import Config as _Config

class Config(_Config):
    fizz: str

# The above code can be used by a script to generate custom stub files.
1
  • Thanks, the idea with a stub template is actually a nice idea - combined with the incomplete stub marking from @michael0x2a's answer, this gives a nice small template to start with. This is what I actually ended doing - enhance the cookiecutter template with the minimal incomplete stubs and adjusting the template mypy config.
    – hoefling
    Apr 15, 2020 at 8:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.