I have created a conda virtual environment on a Windows 10 PC to work on a project. To install the required packages and dependencies, I am using conda install <package>
instead of pip install <package>
as per the best practices mentioned in https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#using-pip-in-an-environment
In order to distribute my software, I choose to create an environment.yml and a requirements.txt file targeting the conda and non-conda users respectively. I am able to export the current virtual environment into a yml file, so the conda users are taken care of. But, for the non-conda users to be able to replicate the same environment, I need to create and share the requirements.txt file. This file can be created using conda list --export > requirements.txt
but this format is not compatible with pip and other users can't use pip install -r requirements.txt
on their systems.
Using pip freeze > requiremens.txt
is a solution that is mentioned here and here. This means that non-conda users can simply execute pip install -r requirements.txt
inside a virtual environment which they may create using virtualenv in the absence of conda.
However, if you generate a requiremets.txt file in the above style, you will end up with a requirements.txt file that has symbolic links. This is because we tried to create a requirements.txt file for packages that are installed using conda install
and not pip install
.
For example, the requirements.txt file that I generated in a similar fashion looks like this.
certifi==2020.6.20
cycler==0.10.0
kiwisolver==1.2.0
matplotlib @ file:///C:/ci/matplotlib-base_1603355780617/work
mkl-fft==1.2.0
mkl-random==1.1.1
mkl-service==2.3.0
numpy @ file:///C:/ci/numpy_and_numpy_base_1596215850360/work
olefile==0.46
pandas @ file:///C:/ci/pandas_1602083338010/work
Pillow @ file:///C:/ci/pillow_1602770972588/work
pyparsing==2.4.7
python-dateutil==2.8.1
pytz==2020.1
sip==4.19.13
six==1.15.0
tornado==6.0.4
wincertstore==0.2
These symbolic links will lead to errors when this file is used to install the dependencies.
Steps I took that landed me to the above requirements.txt file:
- Created a new conda virtual environment using
conda create -n myenv python=3.8
- Activated the newly created conda virtual environment using
conda activate myenv
- Installed pip using
conda install pip
- Installed pandas using
conda intall pandas
- Installed matplotlib using
conda install matplotlib
- generated a pip compatible requrements.txt file using
pip freeze > requirements.txt
So, my question is how do you stick to the best practice of using conda install
instead of pip install
while still being able to distribute your software package to both conda and non-conda users?