67

I am new to coverage and ran into a strange problem. My coverage is taking my virtual environment site packages into account. Here is the output of the coverage run:

coverage run test.py
....................
----------------------------------------------------------------------
Ran 20 tests in 0.060s

OK
(atcatalog)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -jmfrank63@fullstack-audio-text-catalog:~/workspace (git master)       [19:58:45]
$ coverage report
Name                                                                              Stmts   Miss  Cover
-----------------------------------------------------------------------------------------------------
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/__init__               18      0   100%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/_compat                38     20    47%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/app                   528    255    52%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/flask/blueprints            156    118    24%
                             .
                             .
                             .
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/urls               412    215    48%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/utils              242    175    28%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/wrappers           568    298    48%
/home/ubuntu/Envs/atcatalog/lib/python2.7/site-packages/werkzeug/wsgi               448    352    21%
atcatalog/__init__                                                                    7      0   100%
atcatalog/views/__init__                                                              0      0   100%
atcatalog/views/publang                                                               7      0   100%
atcatalog/views/pubtext                                                               1      0   100%
atcatalog/views/userlang                                                             13      0   100%
atcatalog/views/users                                                                 5      0   100%
atcatalog/views/usertext                                                             14      0   100%
test                                                                                120      0   100%
-----------------------------------------------------------------------------------------------------
TOTAL                                                                             12530   8044    36%
(atcatalog)- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -jmfrank63@fullstack-audio-text-catalog:~/workspace (git master)       [19:58:55]

Here is the structure of my project directory which resides under home:

workspace/
├── README.md
├── atcatalog
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── static
│   ├── templates
│   └── views
│       ├── __init__.py
│       ├── __init__.pyc
│       ├── publang.py
│       ├── publang.pyc
│       ├── pubtext.py
│       ├── pubtext.pyc
│       ├── userlang.py
│       ├── userlang.pyc
│       ├── users.py
│       ├── users.pyc
│       ├── usertext.py
│       └── usertext.pyc
├── requirements.txt
├── run.py
└── test.py

I had the virtual environment at first inside the project directory and now moved it out to ~/Envs with virtualenvwrapper, but the problem persisted. run.py and test.py are not special in any way, they both import app from atcatalog. I also tried to find ways to omit the virtual environment directory, but google gave no answer (surprisingly). I don't think it is the purpose of coverage to test already well tested site-packages. So I would exclude them from the run.

How can I accomplish to avoid coverage having testing my site-packages?

9
  • I would try using py.test with coverage.
    – tknickman
    Aug 22, 2015 at 20:38
  • Of course it is an option to switch to a different testing framework. But that doesn't explain what is going wrong here. None of the tutorials showed this behavior. And there are no records of other people having this problem. I am probably doing something stupid wrong. Aug 22, 2015 at 20:45
  • When you moved the virtual env, did you delete the old one? What is the result of ls -a in workspace/
    – tknickman
    Aug 22, 2015 at 20:47
  • Yes the old is deleted. I also got rid of the .coverage file. Also the output shows coverage is using my new virtual environment which is clearly outside my project directory. Aug 22, 2015 at 20:50
  • Have you given this a read? coverage.readthedocs.org/en/coverage-4.0b1/source.html#source
    – tknickman
    Aug 22, 2015 at 20:53

4 Answers 4

83

Thanks to tknickman I figured it out: Use either

coverage run --source <path to project dir> test.py

or create a configuration file .coveragerc which resides in the directory you run coverage from, with the following content:

[run]
source =
    <path to project dir>

This provides you do not have your virtual environment installed under the project directory. If you have the virtual environment installed under the project dir you can use

coverage run --source <project path> --omit <pattern> test.py

Note that omit wants a file pattern like

~/projectdir/venv/*

instead of a path.

The corresponding .coveragerc would look like this:

[run]
source=
    <path to project dir>
omit=
    <path to project dir>/<name of virtual env>/*

I still think that like packages of the standard library any packages installed under site-packages should not be covered by default.

5
  • Hello @JohannesMariaFrank, I have my folder with my python files and inside is venv - was trying to run it, but I do not know what to put as <project path> or <path to project dir> cause I am running it inside that folder where are the files - so nothing works for me, and if I create the .coveragerc file - still I would not know what to put as <path to project dir> - should I put "" - like empty string or? thank you
    – Radek
    Sep 2, 2018 at 19:44
  • Hi @Radek An absolute path should be perfectly fine, in fact ~/projectdir/venv is an absolute path as ~ resolves to /home/radek or C:\Users\radek. Sep 5, 2018 at 8:57
  • I have did @JohannesMariaFrank as you said and got this error: Can't find '_ main _' module in '/[path-to-my-directory]/venv/include'
    – Radek
    Sep 5, 2018 at 20:04
  • Hi @Radek sounds like stackoverflow.com/questions/24723547/… Sep 7, 2018 at 22:38
  • btw, if your .coveragerc file is in the same location, then do source=. omit=./venv/* in your .coveragerc file
    – cryanbhu
    Dec 19, 2020 at 12:00
9

In your setup.cfg file include:

[coverage:run]
omit=*/site-packages/*,*/tests/*,*/.eggs/*

Or any other folders that show up in your results that you want to hide from coverage.

8

Try using py.test and then specifiying your test options in a setup.cfg file. You will need to pip install pytest first.

For example:

[pytest]
norecursedirs = build docs/_build *.egg .tox *.venv
python_files = tests/functional* tests/integration*
addopts =
    #--verbose
    --tb short
    # Turn on --capture to have brief, less noisy output
    # You will only see output if the test fails
    # Use --capture no if you want to see it all or have problems debugging
    --capture fd
    # --capture no
    # show extra test summary info as specified by chars (f)ailed, (E)error,      (s)skipped, (x)failed, (X)passed.
    - rfEsxX
    --junitxml junit.xml
    --cov workspace --cov-report xml --cov-report term-missing

You can read more about configuring py.test here: https://pytest.org/latest/customize.html

1
  • 5
    This answer doesn't have anything to do with the original question which was about coverage.py and not py.test.
    – Jupiter
    Jun 4, 2019 at 15:45
1

If using pytest, you can specify exclusive paths or files to test in setup.cfg (see docs):

[pytest]
# a directory
testpaths = tests

# exact file(s)
python_files = tests/test1.py tests/test2.py

It looks like if you include the python_files and testpaths parameters, then the python_files will only be used.

1
  • testpaths = tests - is tests a folder?
    – Radek
    Sep 8, 2018 at 20:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.