3

I'm working with a team. We each have our own Windows system. We have shared drives and a shared git repository. We want to have a shared virtual environment (in Python).

My understanding (from previous questions from myself and others) is that virtual environments do not include all files necessary for running python, in particular, the shared VE does not include the Python interpreter.

I can see how we can create a shared VE and it seems we could just copy that around, or put it on the shared drive, or put it in a git repository. But my understanding of this is that it does not eliminate the need for individuals to install their own local versions of python. Is that correct?

One of my colleagues has heard (or read) that "there is a package that allows teams to share their virtual environment configuration through a git-like interface. That way you can “pull” the updated configuration and it will install the new packages automatically. This allows each person to change the configuration and test it before releasing it to the team."

So is there a special package to enable this? Or is it just a regular venv that is included in the git repository with the other files? If we do this, then we must all put the venvs in the same place in on our file systems OR we have to go in and manually change the VIRTUAL_ENV variable in activate.bat. Is that correct?

In any case, we do all have to install our own local versions of python anyway. Is that correct?

5
  • 1
    If the virtualenv is on a shared drive, then everyone can access it. You just have to make sure its on a shared user directory, and is group readable. Virtual environments have their own python executable, which you can see when you run which python inside the virtual env after it is activated.
    – RoadRunner
    Nov 5, 2020 at 14:53
  • 1
    What@RoadRunner and @DanielFarrell said should be your main guidelines. Sharing current state of venv on Google Drive, some other cloud storage, or even pushing it all to some git seem like much simpler options.
    – Apollo
    Nov 5, 2020 at 14:55
  • @Apollo in this case, are the venvs self-contained? Does each person still have to install some version of python on their own machine?
    – elbillaf
    Nov 5, 2020 at 15:06
  • @RoadRunner testing with just myself right now, before I bother others. I installed latest pycharm and python3.85 venv. I have a venv on C: (local) and on shared drive. When I run the program, it correctly runs with the shared version. (Pycharm tells me what it's using.) But when I click on Open Terminal it keeps using the C: drive version, despite the fact that I set both Project->Interpreter and Console->python console to use the shared version.
    – elbillaf
    Nov 5, 2020 at 18:04
  • 1
    @elbillaf Venvs have their own python exe.
    – Apollo
    Nov 6, 2020 at 16:43

2 Answers 2

5

If the virtual environment is on a shared drive(group readable), then your team members should be able to access it. A virtual environment is just a directory.

But my understanding of this is that it does not eliminate the need for individuals to install their own local versions of python. Is that correct?

Virtual environments have their own python binaries, which you can see when you run which python inside the virtual environment after it is activated.

So is there a special package to enable this? Or is it just a regular venv that is included in the git repository with the other files? If we do this, then we must all put the venvs in the same place in on our file systems OR we have to go in and manually change the VIRTUAL_ENV variable in activate.bat. Is that correct?

I would advise against uploading a virtual environment directory to version control, since it contains binaries, configuration files that don't belong in there. Its also unnecessary to do this because the dependencies are tracked in a requirements.txt file, which list the pip dependencies and is committed to version control. Additionally, When the virtual environment is activated, the VIRTUAL_ENV environment variable is automatically exported, so there is no need to modify it.

Conclusion

For simplicity, its probably best to have each user create their own virtual environment and install the dependencies from requirements.txt on their local machines. This also ensures users don't make a change to the virtual environment that will affect other users, which is a drawback of the above shared drive approach.

If they want to pull the latest requirements, then pulling the latest change using git pull and reinstalling the dependencies with pip install -r requirements.txt is good enough. You just have to ensure the virtual environment is activated, otherwise the dependencies will get installed system wide. This is where the pipenv package also comes in handy.

Usually in my team projects, the README contains instructions to get this setup for each team member.

Additionally, as Daniel Farrell helpfully mentioned in the comments, pip won't be able to manage packages like libffi, openssl, python-devel etc. inside a virtual environment. This is where using Docker containers become useful, since you can install dependencies inside a isolated environment built on top of the host operating system. This ensures the dependencies don't mess with the system wide packages, which is a good practice to follow in any case.

An example Dockerfile I have used in the past:

FROM python:3.8-slim-buster

# Set environment variables:
ENV VIRTUAL_ENV=/opt/venv
ENV PATH="$VIRTUAL_ENV/bin:$PATH"

# Create virtual environment:
RUN python3 -m venv $VIRTUAL_ENV

# Install dependencies:
COPY requirements.txt .
RUN pip install -r requirements.txt

# Run the application:
COPY app.py .
CMD ["python", "app.py"]

Which I modified from this Elegantly activating a virtualenv in a Dockerfile article.

1
  • 1
    The biggest drawback to not using docker is the system packages that your pip requirements will likely need, but since they aren't python, they can't come from pip. I'm thinking of libffi, openssl, etc. Docker keeps that in the repo, but it's out of band of virtualenv or pipenv managment, iirc. A great answer regardless
    – erik258
    Nov 5, 2020 at 18:21
5

containerization aims to solve the "from where comes python?" problem. My developers' teams usually use a Dockerfile that installs their requirements within a docker-compose that spins ups a development environment for their applications . Unlike a virtual environment, containers offer a complete userspace solution that works pretty well in windows and osx.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.