5

Is it possible to change execution version of Python within a script?

I have Python 2.6.6 which gets loaded by default. I want to change this version to 3.6.0, which is installed at custom location(not /usr/bin), inside a script. So, in the script, I will check for Python version with sys.version , will load Python 3.6.0 module in script. It is not getting reflected in the run environment. Here is the code:

import sys, os
pyVersion = int(sys.version.split(" ")[0].replace(".", ""))
exec(open(os.environ['MODULESHOME']+"/init/python.py").read())
if pyVersion < 360:
    print("Python 3.6.0 version required")
    print("Loading utils/python module")
    module(['load', 'utils/python/3.6.0'])
    module('li')

It is listing modules as expected. Output:

Python 3.6.0 version required
Loading utils/python module
Currently Loaded Modulefiles:
1) licenses                    2) cliosoft/6.32.p3(default)   3)utils/python/3.6.0

Now, when I check python version in next line, It is still python 2.6.6

print(sys.version)

Output:

2.6.6 (r266:84292, Jul 23 2015, 15:22:56) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)]

Now, how can I make Python to use 3.6.0 for all the lines of code after loading module?

NOTE: I can change version before loading the script. But, it will be used by multiple users and I don't have root access. So, not possible to change version for every one. And I'm interested in doing it this way only.

10
  • Why can't you just give your script an appropriate shebang line, eg #!utils/python/3.6.0 ?
    – PM 2Ring
    Feb 1, 2017 at 11:54
  • 1
    As I installed python at different location, I need to point to bin and lib files from this location. I use environmental modules to do this. Also, I don't want to hard code the path in script.
    – srand9
    Feb 1, 2017 at 11:59
  • 1
    What about checking the Python version and just re-executing the script with a different interpreter? Also, I think you can make version checking easier by using sys.version_info, for example as if sys.version_info.major == 2: ...
    – TidB
    Feb 1, 2017 at 12:15
  • I've also installed Python 3.6.0 to a non-standard location: it's in /opt/python3.6. I do have root access, so I've got a symlink to the python3.6 binary in /usr/bin/. However, when I use #!/opt/python3.6/bin/python3.6 as the shebang everything works fine, and sys.path contains what it should.
    – PM 2Ring
    Feb 1, 2017 at 12:18
  • 2
    Shebangs should be #!/usr/bin/env python3.6 for maximum compatibility, especially cross-plattform use. But to answer the seemingly original question: No, when you execute a script using Python 2, you cannot switch to a Python 3 interpreter midway. You would have to start a new process with the correct interpreter for that.
    – poke
    Feb 1, 2017 at 12:51

3 Answers 3

2

In your latest comment, you alluded to the usage of a "HERE DOC". Within Python, these are called 'docstrings' and are written as three quotes ("""). In respect to your problem, they can be used as follows:

#!/usr/bin/env python

# Python 2
import os
import sys
print "1. Python version: ", sys.version
print "2. Process id: ", os.getpid()

os.execlp("python3", "python3", "-c", """

# Python 3
import os
import sys
print("3. Python version: ", sys.version)
print("4. Process id: ", os.getpid())
""")

print "5. Does not execute"

On my machine (Ubuntu 16.04), the output is:

1. Python version:  2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609]
2. Process id:  3085
3. Python version:  3.5.2 (default, Nov 17 2016, 17:05:23) 
[GCC 5.4.0 20160609]
4. Process id:  3085

(The versions may vary on your machine.)

Notice two things:

  • The version number changes from 2.* to 3.*
  • The process id remains the same 3085
  • Lines after the exec are not run. In fact, the exec command "unloads" the Python 2 interpreter. It is no longer around to execute the remainder of the file.

By some definition, this can be considered "changing Python 2.* to 3.*". The same method can be used to change from 2.6.6 to 3.6.0.

Finally, a word of caution. Unless your needs are dire, I recommend against using the above technique. The following limitations may apply:

  • The behaviour of exec varies widely between platforms.
  • The maximum arg size (size of the Python 3 string) is hard-coded into the kernel. It can be as small as a few kilobytes.
  • And finally, I find placing a majority of the script within a docstring nauseating.
1
  • I just read @TidB's comment, "include the actual logic in a string and exec it. I'd strongly recommend to stay away from it though." This answer is an expansion on the idea.
    – Ryan
    Feb 8, 2017 at 6:14
1

Expanding on the solution suggested by Ryan. You can avoid having the entire script be a string by doing something like

import sys, os

if sys.version[0] == '2':
    with open(__file__, 'r') as f:
        _script_text = ''.join(f.readlines())
        os.execlp('python3', 'python3', '-c', _script_text)
        exit()

# Rest of your code
# | | | | | | | | |

Of course avoiding this entirely would be better, but I think this is ever so slightly less aweful to look at.

1

env and python scripts: controlling the python version On Unix-like systems, python scripts will often start with this line:

#!/usr/bin/env python

How can you manually test this script under different versions of Python?

This "env-shebang" idiom effectively selects the primary Python interpreter installed on the user's system, at script execution time. On any given system, this will resolve to a particular version of Python. For instance, on stock Debian Lenny GNU/Linux, it's Python 2.5; the first python in the default path is /usr/bin/python, which is a symbolic link to /usr/bin/python2.5.

Now, if you're developing a Python script that is supposed to work on several different Python versions, and want to test it under each, the "env-shebang" idiom can get in your way. Temporarily editing the file, to change the argument to env, may do the trick. But that's error prone and difficult to automate.

So here's a better solution.

Let's say you want to test under Pythons version 2.5, 2.6 and 2.7, and you already have an interpreter for each version installed. For example, on my current development machine, python 2.7 is at $HOME/opt/python2.7/bin/python2.7, python 2.6 is at /usr/bin/python2.6, and python 2.5 is at /usr/bin/python2.5.

What you can do is create a subdirectory for each version in some convenient location. I name them like "env_pythonX.Y" and just place them under $HOME/bin (where X and Y are the major and minor version numbers). Then, inside each of these directories, make a symbolic link named "python" to the appropriate version.

Again using my dev machine as an example, $HOME/bin/env_python2.7/python is a symbolic link to $HOME/opt/python2.7/bin/python2.7, $HOME/bin/env_python2.6/python is a symbolic link to /usr/bin/python2.6, and $HOME/bin/env_python2.5/python is a symbolic link to /usr/bin/python2.5.

Now you're prepared. To run the script program.py from the Unix command line, using (say) python version 2.6, just define the PATH so that $HOME/bin/env_python2.6/python is first. At the prompt, you can type something like:

PATH=$HOME/bin/env_python2.6:$PATH ./program.py

Here, I've used a syntax provided by most modern shells for per-command variable assigments. Prepending a command with FOO=BAR means "Run the command in an environment where the variable FOO has the value BAR". This conveniently lets us set the search path to produce our python, for just that one invokation.

This is easy to repeat with the others. To manually run the script with each Python version in sequence, use this series of commands:

PATH=$HOME/bin/env_python2.7:$PATH ./program.py
PATH=$HOME/bin/env_python2.6:$PATH ./program.py
PATH=$HOME/bin/env_python2.5:$PATH ./program.py

Of course, this is all highly automatable in your test scripts.

By the way, if the symbolic link is wrong, a different python version may be silently used. You can quickly verify with python -V:

PATH=$HOME/bin/env_python2.6:$PATH python -V

This should emit something like Python 2.6.5.

You're all set. Now get to testing!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.