62

I checked the activate script and it looks to me all it does is:

  • set VIRTUAL_ENV env
  • append $VIRTUAL_ENV/bin in front of PATH

How does virtualenv provide that magical virtual environment by these? What do I miss?

1

2 Answers 2

60

I will describe the basic process, which I learned from the presentation @jcollado linked to.

When Python starts, it looks at the path of the binary, and the prefixes thereof.

So let's say your virtualenv is /home/blah/scratch. The Python process knows it was executed from /home/blah/scratch/bin/python (which is usually just a copy of your system python binary /usr/bin/python) and it knows its own version X.Y because it's compiled into it. Then Python looks for lib/pythonX.Y/os.py in this order:

/home/blah/scratch/bin/lib/pythonX.Y/os.py
/home/blah/scratch/lib/pythonX.Y/os.py    <-- this file should exist
/home/blah/lib/pythonX.Y/os.py
/home/lib/pythonX.Y/os.py
/lib/pythonX.Y/os.py

It stops at /home/blah/scratch/lib/pythonX.Y/os.py because it's the first file that actually exists. If it didn't, Python would keep looking. It then sets sys.prefix based on this. It uses a similar process to set sys.exec_prefix, and then sys.path is constructed based on these.

4
  • 1
    X.X is derived from the Python version, so it'll be 2.7 or something similar.
    – Max
    Sep 14, 2012 at 12:55
  • Never thought about this path discovering for sys.prefix!
    – Drake Guan
    Sep 14, 2012 at 13:57
  • 3
    Thank you for clearly explaining in the body of your answer rather than linking to an external source which could change or disappear in the future. Jun 18, 2015 at 23:05
  • Before this answer, where is the doc stating all this? Even this was very hard to find.
    – Gulzar
    Apr 18, 2021 at 10:42
22
  1. First the user creates a new virtualenv with the command virtualenv myenv. This creates a directory called myenv and copies the system python binary to myenv/bin. It also adds other necessary files and directories to myenv, including a setup script in bin/activate and a lib subdirectory for modules and packages.
  2. Then the user sources the activate script with . myenv/bin/activate, which sets the shell’s PATH environment variable to start with myenv/bin.
  3. Now when the user runs python from this shell, it will execute the copy of the binary stored in myenv/bin. Even though the binary is identical to the one in /usr/bin/python, the standard python binary is designed to search for packages and modules in directories that are relative to the binary’s path (this functionality is not related to virtualenv). It looks in ../lib/pythonX.Y where X and Y are the major and minor version numbers of the python binary. So now it is looking in myenv/lib/pythonX.Y.
  4. The myenv/bin directory also contains a script named pip so that when the user installs new packages using pip from the virtualenv, they will be installed in myenv/lib/pythonX.Y
3
  • 2
    I'm wondering where do you guys know the secret path looking of something like ../lib/pythonX.Y and even other stuff related to sys.prefix and etc?
    – Drake Guan
    Feb 1, 2014 at 20:33
  • 2
    I haven't seen any good documentation about it yet, but I noticed this behavior when building Python from source. You can also test it like this: mkdir /tmp/bin; mkdir /tmp/lib; cp /usr/bin/python /tmp/bin; cp -r /usr/lib/pythonX.Y/ /tmp/lib/pythonX.Y; /tmp/bin/python. Then type import sys; sys.prefix. This will show "/tmp". If you remove the /tmp/lib directory, it will revert to "/usr".
    – clark800
    Feb 3, 2014 at 23:11
  • 2
    Step 3 is the key thing I was missing--that the python binary looks in it's executable's neighborhood for associated files. Thanks.
    – Dave X
    Mar 22, 2018 at 3:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.