60

This is how I do it currently:

import os
from setuptools import setup, find_packages
here = os.path.abspath(os.path.dirname(__file__))

requires = [
    'pyramid',
    'pyramid_debugtoolbar',
    'waitress',
    'requests',
    'mock',
    'gunicorn',
    'mongoengine',
    ]

setup(name='repoapi',
      version='0.0',
      description='repoapi',
      packages=find_packages(),
      include_package_data=True,
      zip_safe=False,
      install_requires=requires,
      tests_require=requires,
      test_suite="repoapi",
      entry_points="""\
      [paste.app_factory]
      main = repoapi:main
      """,
      )

Is this an okay way? I have some troubles. For example, for pyramid, I cannot use the system-wide nosetests plugin to run tests. I need to install pyramid in the global python site-packages!

But I don't want that. So I must install nose in the virtualenv of this project. But I don't want it to be a dependency. I don't feel like it should belong to requires. It isn't. Yet, I also don't want to install by hand all the time. Yeah I know I have a lot of I don't want to do this and that...

But how would you solve that? I don't want to tamper the global python site-packages, but I want to install nose as part of the virtualenv.

Also, pip install requirement files. It's slightly more accurate because I don't need to specify the version manually and I don't need to be afraid of updating setup.py manually. Just throw pip freeze > file.txt and done.

However, pip can return garbage because we throw garbage packages into virtualenv.

So many blades. What's the best practice? How do you deal with these issues?

Maybe I missed it, but https://github.com/django/django/blob/master/setup.py, how did Django do it?

1

2 Answers 2

87

You can split up your requirements into "install" dependencies and "test" dependencies like this:

import os
from setuptools import setup, find_packages
here = os.path.abspath(os.path.dirname(__file__))

install_requires = [
    'pyramid',
    'pyramid_debugtoolbar',
    'waitress',
    'requests',
    'gunicorn',
    'mongoengine',
    ]

tests_require = [
    'mock',
    'nose',
    ]

setup(name='repoapi',
      ...
      install_requires=install_requires,
      tests_require=tests_require,
      test_suite="nose.collector",
      ...
      )

This way, when someone installs the package, only the "install" dependencies are installed. So, if someone only wants to use the package (and they aren't interested in running the tests), then they don't have to install the test dependencies.

When you do want to run the tests, you can use this:

$ python setup.py test

Per the docs:

Note that these required projects will not be installed on the system where the tests are run, but only downloaded to the project’s setup directory if they’re not already installed locally.

Once the "test" dependencies are in place, then it will run the "test_suite" command. Since you mentioned nose as your preferred test runner, I showed how you use "nose.collector" to configure that.

Incidentally, the Django setup.py is not the cleanest example for understanding the basics of setuptools. I think the Sentry setup.py is a better example to learn from.

4
  • Thanks for sharing. But wouldn't nose actually get installed into virtualenv? So at the end of the day, the best practices is still documentation, bug ticket to keep setup.py in sync? Thanks.
    – CppLearner
    Mar 15, 2013 at 0:38
  • Thanks for linking to the docs for tests_require. Having lots of difficult managing dependencies in Python. Couldn't find the property in the docs I was originally referring to: packaging.python.org/guides/…
    – Dennis
    Nov 27, 2018 at 17:32
  • What if I want to install the test dependencies but not run the tests just yet? Is there a python setup.py install --all option? Apr 3, 2020 at 18:33
  • 3
    setuptools tests_require is deprecated since 41.5.0 (October 2019). We are now at 50.3.0. Sep 18, 2020 at 8:25
5

If you are using requirement files, then an alternative would be to read their content instead of duplicating it:

import pathlib
from setuptools import setup, find_packages

HERE = pathlib.Path(__file__).parent
INSTALL_REQUIRES = (HERE / "requirements.txt").read_text().splitlines()
TESTS_REQUIRE = (HERE / "test-requirements.txt").read_text().splitlines()[1:]

setup(...,
      install_requires=INSTALL_REQUIRES,
      tests_require=TESTS_REQUIRE,
      ...
      )

I find this approach much more stable and less prone to errors, since requirements change sometimes and one often forgets to update two places.

Note: notice the TESTS_REQUIRE start from the second line, this is due to the fact that the first line of test-requirements.txt is often -r requirements.txt. Feel free to change it if your case is different.

4
  • 8
    setuptools tests_require is deprecated since 41.5.0 (October 2019). We are now at 50.3.0. Sep 18, 2020 at 8:25
  • 1
    @MartinThoma So what should we use instead? Docs don't seem to say. May 29, 2021 at 8:00
  • 1
    Just document how to run your tests, e.g. in the readme May 29, 2021 at 11:19
  • 2
    Pytest is the most common choice. tox is also pretty well known. If I see a pytest.ini or a tox.ini I would know how to run the tests. May 29, 2021 at 11:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.