5

I've always used #!/usr/bin/env python3 for my python scripts, as Debian distributions typically use python for version 2.7 and python3 for version 3.x, and changing this convention can cause problems in systems (e.g. see this SO question How to update-alternatives to Python 3 without breaking apt?).

However, I cannot get this to work properly on Windows. Consider a machine with the standard python (downloaded from https://www.python.org/downloads/) and installed as administrator for all users. Here's a script that prints the executable and (base) prefix:

#!/usr/bin/env python3
import os, sys, platform
print('EXECUTABLE: ' + sys.executable)
print('PREFIX: ' + sys.prefix)
print('BASE PREFIX: ' + sys.base_prefix)

Now look at the following:

PS C:\pytest> gcm python | select Source
C:\Program Files\Python37\python.exe

PS C:\pytest> gcm python3 | select Source
C:\Users\user\AppData\Local\Microsoft\WindowsApps\python3.exe

As you can see, python3.exe on Windows is a "stub" which brings up the Microsoft Store on the page for installing Python from it. I mention this in case it is significant. However, running the above script gives a proper result:

PS C:\pytest> .\script.py
EXECUTABLE: C:\Program Files\Python37\python.exe
PREFIX: C:\Program Files\Python37
BASE PREFIX: C:\Program Files\Python37

Unfortunately, if I create and activate a virtual environment:

PS C:\pytest> python -m venv .venv
PS C:\pytest> . .\.venv\Scripts\Activate.ps1
(.venv) PS C:\pytest> .\script.py
EXECUTABLE: C:\Program Files\Python37\python.exe
PREFIX: C:\Program Files\Python37
BASE PREFIX: C:\Program Files\Python37

I don't understand why this is happening as according to PEP-486 the python launcher should be able to detect active virual environments. It does seem to be related to the version qualifier in the shebang. If I modify the script to use #!/usr/bin/env python it seems to work fine:

(.venv) PS C:\pytest> .\script.py
EXECUTABLE: C:\pytest\.venv\Scripts\python.exe
PREFIX: C:\pytest\.venv
BASE PREFIX: C:\Program Files\Python37

As explained in the beginning, I do not want to modify my scripts to use the shebang without the version qualifier. Is there any way to configure the python launcher to properly detect the virtual environment when using the python3 shebang? The documentation on shebang lines explicitly states:

Any of the above virtual commands can be suffixed with an explicit version (either just the major version, or the major and minor version).

Furthermore it states:

The /usr/bin/env form of shebang line has one further special property. Before looking for installed Python interpreters, this form will search the executable PATH for a Python executable.

However, even attempting to copy python.exe as python3.exe for my virtual environmment, it does not seem to work:

(.venv) PS C:\pytest> copy .\.venv\Scripts\python.exe .\.venv\Scripts\python3.exe
(.venv) PS C:\pytest> gcm python3 | select Source

Source
------
C:\pytest\.venv\Scripts\python3.exe


(.venv) PS C:\pytest> .\script.py
EXECUTABLE: C:\Program Files\Python37\python.exe
PREFIX: C:\Program Files\Python37
BASE PREFIX: C:\Program Files\Python37

The python launcher still ignores the virtual environment even though python3.exe is visible in the PATH and picks the system installation...

UPDATE: I found in another StackExchange question how to remove the python3.exe stub that takes you to the Microsoft Store and deleted it to see if it makes any difference. Nothing changed so this problem lies purely in the python launcher.

UPDATE 2: This is the debug information after enabling it by setting PYLAUNCH_DEBUG=1

(.venv) PS C:\pytest> $env:PYLAUNCH_DEBUG=1
(.venv) PS C:\pytest> .\script.py
launcher build: 32bit
launcher executable: Console
File 'C:\Users\user\AppData\Local\py.ini' non-existent
File 'C:\WINDOWS\py.ini' non-existent
Called with command line: "C:\pytest\script.py"
maybe_handle_shebang: read 168 bytes
maybe_handle_shebang: BOM not found, using UTF-8
parse_shebang: found command: python3
locating Pythons in 64bit registry
locate_pythons_for_key: unable to open PythonCore key in HKCU
locate_pythons_for_key: "C:\Program Files\Python37\python.exe" is a 64bit executable
locate_pythons_for_key: "C:\Program Files\Python3\PCbuild\win32\python.exe: The filename, directory name, or volume label syntax is incorrect.
locate_pythons_for_key: "C:\Program Files\Python3\PCbuild\amd64\python.exe: The filename, directory name, or volume label syntax is incorrect.
locate_pythons_for_key: "C:\Program Files\Python3\PCbuild\python.exe: The filename, directory name, or volume label syntax is incorrect.
locating Pythons in native registry
locate_pythons_for_key: unable to open PythonCore key in HKCU
locate_pythons_for_key: unable to open PythonCore key in HKLM
found no configured value for 'python3'
search for Python version '3' found '"C:\Program Files\Python37\python.exe"'
run_child: about to run '"C:\Program Files\Python37\python.exe" "C:\pytest\script.py" '
EXECUTABLE: C:\Program Files\Python37\python.exe
PREFIX: C:\Program Files\Python37
BASE PREFIX: C:\Program Files\Python37
child process exit code: 0

UPDATE 3: After Mark's suggestion that it's just a bug, I've raised an issue in the python bug tracker and am awaiting confirmation.

5
  • 1
    I can reproduce. It looks like a bug. If I run a script implicitly (script.py) or explicitly (py script.py) with an active virtual environment (script.py) I get the globally installed Python, but py -c "import sys;print(sys.executable)" gives me the Python from the virtual environment. If I remove the shebang from the script, I get the virtual environment's Python. Commented Dec 8, 2019 at 18:36
  • Thank you Mark for confirming. I will raise a bug report and post back once I have confirmation from developers there to close the question....
    – Alexandros
    Commented Dec 8, 2019 at 18:44
  • From the debug messages, it like it doesn't bother checking for virtual environments when a shebang is present Commented Dec 8, 2019 at 18:54
  • Just ran using #!/usr/bin/env python because that seems to work and I can see what you mean: parse_shebang: found command: python searching PATH for python executable Python on path: C:\pytest\.venv\Scripts\python.exe
    – Alexandros
    Commented Dec 8, 2019 at 19:04
  • Not a fix, rather a workaround for now: If I run the script explicitly with PS > python .\script.py the venv interpreter is called, even with the shebang python3 in place. Like this at least the script can stay unchanged, keeping the shebang.
    – lcnittl
    Commented Apr 8, 2020 at 7:29

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.