210

I work with conda environments and need some pip packages as well, e.g. pre-compiled wheels from ~gohlke.

At the moment I have two files: environment.yml for conda with:

# run: conda env create --file environment.yml
name: test-env
dependencies:
- python>=3.5
- anaconda

and requirements.txt for pip which can be used after activating above conda environment:

# run: pip install -i requirements.txt
docx
gooey
http://www.lfd.uci.edu/~gohlke/pythonlibs/bofhrmxk/opencv_python-3.1.0-cp35-none-win_amd64.whl

Is there a possibility to combine them in one file (for conda)?

3
  • 2
    There is a GitHub pull request for using requirements.txt in conda. github.com/conda/conda-env/pull/172 Feb 6, 2016 at 19:29
  • 3
    conda env export will generate the text for a .yml based on the currently installed packages.
    – Kenny
    May 10, 2016 at 1:24
  • 3
    from inspecting my env.yml file it seems that conda env export > environment.yml already includes the pip packages I need. That's great! Apr 24, 2021 at 1:40

4 Answers 4

252

Pip dependencies can be included in the environment.yml file like this (docs):

# run: conda env create --file environment.yml
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- numpy=1.13.3  # pin version for conda
- pip:
  # works for regular pip packages
  - docx
  - gooey
  - matplotlib==2.0.0  # pin version for pip
  # and for wheels
  - http://www.lfd.uci.edu/~gohlke/pythonlibs/bofhrmxk/opencv_python-3.1.0-cp35-none-win_amd64.whl

It also works for .whl files in the same directory (see Dengar's answer) as well as with common pip packages.

6
  • 3
    It did well! conda env create -f environment.yml, and environment.yml file has to be in the current directory, otherwise, it will be thrown oddly errors.
    – Johnny
    Mar 25, 2018 at 5:49
  • 2
    is there a way to generate the yaml file automatically? e.g. I used to do pip freeze > requirements4pip.txt and conda list --export > requirements4conda.txt Apr 24, 2021 at 0:09
  • 2
    from inspecting my env.yml file it seems that conda env export > environment.yml already includes the pip packages I need. That's great! Apr 24, 2021 at 1:40
  • 1
    conda env create -f environment.yml does not include pip packages installed using local code such as pip install -e ., but the requirements.txt file generated using pip has these packages included. So I think the best option is to use -r requirements.txt in the yaml file.
    – omsrisagar
    Mar 15, 2022 at 21:53
  • 1
    You can also specify the channel for packages, e.g., - conda-forge::scikit-learn. Oct 18, 2022 at 13:06
151

One can also use the requirements.txt directly in the YAML. For example,

name: test-env
dependencies:
  - python>=3.5
  - anaconda
  - pip
  - pip:
    - -r requirements.txt

Basically, any option you can run with pip install you can run in a YAML. See the Advanced Pip Example for a showcase of other capabilities.


Important Note

A previous version of this answer (and Conda's Advanced Pip Example) used a substandard file URI syntax:

    - -r file:requirements.txt

Pip v21.2.1 introduced stricter behavior for URI parsing and no longer supports this. See this answer for details.

11
  • 4
    Found it, I should have looked at the pip install options here. -r is for the requirement file and the longer equivalent is --requirement
    – dumbledad
    Feb 26, 2020 at 7:19
  • 5
    I tried combining - -r requirements.txt -e . on one line, but it seemed not to work. Using two lines, - -r requirements.txt followed by - -e . seems to work.
    – Jasha
    Apr 3, 2021 at 18:18
  • 2
    @merv done. Just for completness I post the link here ;) stackoverflow.com/questions/68571543/…
    – Stefan
    Jul 29, 2021 at 7:08
  • 1
    Won't this only look for whatever is in the requirements.txt on PyPI, rather than sourcing stuff from the conda repos? Also, does this still make a difference (e.g. for MKL-linked numpy) in 2022? Aug 28, 2022 at 7:44
  • 1
    @DominikStańczak0 partially. Technically, this YAML also has the whole anaconda metapackage installed through Conda which will install lots of stuff, like MKL, through Conda. If you want a pure PyPI install (which seems like many users intent) I would leave off anaconda. Otherwise, list what you want installed from Conda explicitly under the Conda dependencies section and remove anaconda. That would be my personal approach.
    – merv
    Aug 28, 2022 at 18:26
19

Just want to add that adding a wheel in the directory also works. I was getting this error when using the entire URL:

HTTP error 404 while getting http://www.lfd.uci.edu/~gohlke/pythonlibs/f9r7rmd8/opencv_python-3.1.0-cp35-none-win_amd64.whl

Ended up downloading the wheel and saving it into the same directory as the yml file.

name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
  - opencv_python-3.1.0-cp35-none-win_amd64.whl
1
  • 2
    I mentioned this in the answer above.
    – bastelflp
    Jan 4, 2017 at 17:45
4

If you want to do it automatically it seems that if you do:

conda env export > environment.yml

already has the pip things you need. No need to run pip freeze > requirements4pip.txt separately for me or include it as a

  - pip:
    - -r file:requirements.txt

as another answer has mentioned.

See my yml file:

$ cat environment.yml
name: myenv
channels:
  - pytorch
  - dglteam
  - defaults
  - conda-forge
dependencies:
  - _libgcc_mutex=0.1=main
  - absl-py=0.12.0=py38h06a4308_0
  - aiohttp=3.7.4=py38h27cfd23_1
  - async-timeout=3.0.1=py38h06a4308_0
  - attrs=20.3.0=pyhd3eb1b0_0
  - beautifulsoup4=4.9.3=pyha847dfd_0
  - blas=1.0=mkl
  - blinker=1.4=py38h06a4308_0
  - brotlipy=0.7.0=py38h27cfd23_1003
  - bzip2=1.0.8=h7b6447c_0
  - c-ares=1.17.1=h27cfd23_0
  - ca-certificates=2021.4.13=h06a4308_1
  - cachetools=4.2.1=pyhd3eb1b0_0
  - cairo=1.14.12=h8948797_3
  - certifi=2020.12.5=py38h06a4308_0
  - cffi=1.14.0=py38h2e261b9_0
  - chardet=3.0.4=py38h06a4308_1003
  - click=7.1.2=pyhd3eb1b0_0
  - conda=4.10.1=py38h06a4308_1
  - conda-build=3.21.4=py38h06a4308_0
  - conda-package-handling=1.7.3=py38h27cfd23_1
  - coverage=5.5=py38h27cfd23_2
  - cryptography=3.4.7=py38hd23ed53_0
  - cudatoolkit=11.0.221=h6bb024c_0
  - cycler=0.10.0=py38_0
  - cython=0.29.23=py38h2531618_0
  - dbus=1.13.18=hb2f20db_0
  - decorator=4.4.2=pyhd3eb1b0_0
  - dgl-cuda11.0=0.6.1=py38_0
  - dill=0.3.3=pyhd3eb1b0_0
  - expat=2.3.0=h2531618_2
  - filelock=3.0.12=pyhd3eb1b0_1
  - fontconfig=2.13.1=h6c09931_0
  - freetype=2.10.4=h7ca028e_0
  - fribidi=1.0.10=h7b6447c_0
  - gettext=0.21.0=hf68c758_0
  - glib=2.66.3=h58526e2_0
  - glob2=0.7=pyhd3eb1b0_0
  - google-auth=1.29.0=pyhd3eb1b0_0
  - google-auth-oauthlib=0.4.4=pyhd3eb1b0_0
  - graphite2=1.3.14=h23475e2_0
  - graphviz=2.40.1=h21bd128_2
  - grpcio=1.36.1=py38h2157cd5_1
  - gst-plugins-base=1.14.0=h8213a91_2
  - gstreamer=1.14.0=h28cd5cc_2
  - harfbuzz=1.8.8=hffaf4a1_0
  - icu=58.2=he6710b0_3
  - idna=2.10=pyhd3eb1b0_0
  - importlib-metadata=3.10.0=py38h06a4308_0
  - intel-openmp=2021.2.0=h06a4308_610
  - jinja2=2.11.3=pyhd3eb1b0_0
  - joblib=1.0.1=pyhd3eb1b0_0
  - jpeg=9b=h024ee3a_2
  - kiwisolver=1.3.1=py38h2531618_0
  - lcms2=2.12=h3be6417_0
  - ld_impl_linux-64=2.33.1=h53a641e_7
  - libarchive=3.4.2=h62408e4_0
  - libffi=3.2.1=hf484d3e_1007
  - libgcc-ng=9.1.0=hdf63c60_0
  - libgfortran-ng=7.3.0=hdf63c60_0
  - libglib=2.66.3=hbe7bbb4_0
  - libiconv=1.16=h516909a_0
  - liblief=0.10.1=he6710b0_0
  - libpng=1.6.37=h21135ba_2
  - libprotobuf=3.14.0=h8c45485_0
  - libstdcxx-ng=9.1.0=hdf63c60_0
  - libtiff=4.1.0=h2733197_1
  - libuuid=1.0.3=h1bed415_2
  - libuv=1.40.0=h7b6447c_0
  - libxcb=1.14=h7b6447c_0
  - libxml2=2.9.10=hb55368b_3
  - lz4-c=1.9.2=he1b5a44_3
  - markdown=3.3.4=py38h06a4308_0
  - markupsafe=1.1.1=py38h7b6447c_0
  - matplotlib=3.3.4=py38h06a4308_0
  - matplotlib-base=3.3.4=py38h62a2d02_0
  - mkl=2020.2=256
  - mkl-service=2.3.0=py38h1e0a361_2
  - mkl_fft=1.3.0=py38h54f3939_0
  - mkl_random=1.2.0=py38hc5bc63f_1
  - multidict=5.1.0=py38h27cfd23_2
  - ncurses=6.2=he6710b0_1
  - networkx=2.5.1=pyhd3eb1b0_0
  - ninja=1.10.2=hff7bd54_1
  - numpy=1.19.2=py38h54aff64_0
  - numpy-base=1.19.2=py38hfa32c7d_0
  - oauthlib=3.1.0=py_0
  - olefile=0.46=pyh9f0ad1d_1
  - openssl=1.1.1k=h27cfd23_0
  - pandas=1.2.4=py38h2531618_0
  - pango=1.42.4=h049681c_0
  - patchelf=0.12=h2531618_1
  - pcre=8.44=he6710b0_0
  - pillow=8.2.0=py38he98fc37_0
  - pip=21.0.1=py38h06a4308_0
  - pixman=0.40.0=h7b6447c_0
  - pkginfo=1.7.0=py38h06a4308_0
  - protobuf=3.14.0=py38h2531618_1
  - psutil=5.8.0=py38h27cfd23_1
  - py-lief=0.10.1=py38h403a769_0
  - pyasn1=0.4.8=py_0
  - pyasn1-modules=0.2.8=py_0
  - pycosat=0.6.3=py38h7b6447c_1
  - pycparser=2.20=py_2
  - pyjwt=2.0.1=pyhd8ed1ab_1
  - pyopenssl=20.0.1=pyhd3eb1b0_1
  - pyparsing=2.4.7=pyhd3eb1b0_0
  - pyqt=5.9.2=py38h05f1152_4
  - pysocks=1.7.1=py38h06a4308_0
  - python=3.8.2=hcf32534_0
  - python-dateutil=2.8.1=pyhd3eb1b0_0
  - python-libarchive-c=2.9=pyhd3eb1b0_1
  - python_abi=3.8=1_cp38
  - pytorch=1.7.1=py3.8_cuda11.0.221_cudnn8.0.5_0
  - pytz=2021.1=pyhd3eb1b0_0
  - pyyaml=5.4.1=py38h27cfd23_1
  - qt=5.9.7=h5867ecd_1
  - readline=8.1=h27cfd23_0
  - requests=2.25.1=pyhd3eb1b0_0
  - requests-oauthlib=1.3.0=py_0
  - ripgrep=12.1.1=0
  - rsa=4.7.2=pyhd3eb1b0_1
  - ruamel_yaml=0.15.100=py38h27cfd23_0
  - scikit-learn=0.24.1=py38ha9443f7_0
  - scipy=1.6.2=py38h91f5cce_0
  - setuptools=52.0.0=py38h06a4308_0
  - sip=4.19.13=py38he6710b0_0
  - six=1.15.0=pyh9f0ad1d_0
  - soupsieve=2.2.1=pyhd3eb1b0_0
  - sqlite=3.35.4=hdfb4753_0
  - tensorboard=2.4.0=pyhc547734_0
  - tensorboard-plugin-wit=1.6.0=py_0
  - threadpoolctl=2.1.0=pyh5ca1d4c_0
  - tk=8.6.10=hbc83047_0
  - torchaudio=0.7.2=py38
  - torchtext=0.8.1=py38
  - torchvision=0.8.2=py38_cu110
  - tornado=6.1=py38h27cfd23_0
  - typing-extensions=3.7.4.3=0
  - typing_extensions=3.7.4.3=py_0
  - urllib3=1.26.4=pyhd3eb1b0_0
  - werkzeug=1.0.1=pyhd3eb1b0_0
  - wheel=0.36.2=pyhd3eb1b0_0
  - xz=5.2.5=h7b6447c_0
  - yaml=0.2.5=h7b6447c_0
  - yarl=1.6.3=py38h27cfd23_0
  - zipp=3.4.1=pyhd3eb1b0_0
  - zlib=1.2.11=h7b6447c_3
  - zstd=1.4.5=h9ceee32_0
  - pip:
    - aioconsole==0.3.1
    - lark-parser==0.6.5
    - lmdb==0.94
    - pexpect==4.6.0
    - progressbar2==3.39.3
    - ptyprocess==0.7.0
    - pycapnp==1.0.0
    - python-utils==2.5.6
    - sexpdata==0.0.3
    - tqdm==4.56.0
prefix: /home/miranda9/miniconda3/envs/myenv

Note that at the time of this writing doing conda env create --file environment.yml to create the yml env results in an error:

$ conda env create --file environment.yml

CondaValueError: prefix already exists: /home/miranda9/miniconda3/envs/myenv

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.