148

What I'm trying to do is ship my code to a remote server, that may have different python version installed and/or may not have packages my app requires.

Right now to achieve such portability I have to build relocatable virtualenv with interpreter and code. That approach has some issues (for example, you have to manually copy a bunch of libraries into your virtualenv, since --always-copy doesn't work as expected) and generally slow.

There's (in theory) a way to build python itself statically.

I wonder if I could pack interpreter with my code into one binary and run my application as module. Something like that: ./mypython -m myapp run or ./mypython -m gunicorn -c ./gunicorn.conf myapp.wsgi:application.

8
  • Do you mean something like cx_freeze? Oct 7, 2016 at 9:47
  • @TigerhawkT3, I've never heard of cx_freeze until today. Looked through docs, seems like it's what I want, but it fails to build on Ubuntu 12.04.5 with python3.5.1. I'm looking for alternatives, thanks for kick in the right direction.
    – roboslone
    Oct 7, 2016 at 11:30
  • 1
    Sadly, pyinstaller doesn't work with Django 1.9 and py2app/py2exe don't support linux :(
    – roboslone
    Oct 7, 2016 at 11:39
  • 2
    if cx_Freeze and pyinstaller don't work, there is still hope. You can create package(s) for your target distribution(s) which will resolve all dependencies on installation. Or, maybe use something like docker to run your app.
    – KeatsPeeks
    Oct 11, 2016 at 21:29
  • 1
    other options are nuitka and cython
    – denfromufa
    Oct 11, 2016 at 22:14

6 Answers 6

190
+300

There are two ways you could go about to solve your problem

  1. Use a static builder, like freeze, or pyinstaller, or py2exe
  2. Compile using cython

This answer explains how you can go about doing it using the second approach, since the first method is not cross platform and version, and has been explained in other answers. Also, using programs like pyinstaller typically results in huge file sizes, while using cython will result in a file that's much smaller

  1. First, install cython.

    sudo -H pip3 install cython
    
  2. Then, you can use cython to generate a C file out of the Python .py file (in reference to https://stackoverflow.com/a/22040484/5714445)

    cython example_file.py --embed
    
  3. Use GCC to compile it after getting your current python version (Note: The below assumes you are trying to compile it to Python3)

    PYTHONLIBVER=python$(python3 -c 'import sys; print(".".join(map(str, sys.version_info[:2])))')$(python3-config --abiflags)
    gcc -Os $(python3-config --includes) example_file.c -o output_bin_file $(python3-config --ldflags) -l$PYTHONLIBVER
    

You will now have a binary file output_bin_file, which is what you are looking for

Other things to note:

  1. Change example_file.py to whatever file you are actually trying to compile.
  2. Cython is used to use C-Type Variable definitions for static memory allocation to speed up Python programs. In your case however, you will still be using traditional Python definitions.
  3. If you are using additional libraries (like opencv, for example), you might have to provide the directory to them using -L and then specify the name of the library using -l in the GCC Flags. For more information on this, please refer to GCC flags
  4. The above method might not work for anaconda python, as you will likely have to install a version of gcc that is compatible with your conda-python.
6
  • 39
    Hi, what if my application spans across multiple files. This approach seems to only compile the current script and not all the Python dependencies.
    – RChat
    Dec 14, 2016 at 13:19
  • 6
    no need to rename your python file (say test.py) into a pyx right now
    – iMath
    Jun 2, 2019 at 4:15
  • 6
    This was of great help to me. To answer the question about multiple files, you could use cythonize: cythonize -i file_0.py [...] file_n.py for all your python modules, then cython main_file.py --embed for your main script. You'll end up with an executable and .so files (.dll, .dylib depending on your OS) that you can use together as a standalone solution.
    – afp
    Jun 16, 2019 at 19:51
  • I am getting "No modules named encodings". I got to know I require certain files from python installation. However, my target system doesn't have python. Please help.
    – abhiarora
    Oct 18, 2019 at 9:40
  • 16
    Warning: Cython requires dynamically loading stuff, and it DOES NOT produce self-contained binaries. Don't waste your time!
    – gonzo
    Feb 24, 2021 at 0:24
16

You might wish to investigate Nuitka. It takes python source code and converts it in to C++ API calls. Then it compiles into an executable binary (ELF on Linux). It has been around for a few years now and supports a wide range of Python versions.

You will probably also get a performance improvement if you use it. Recommended.

4
  • I couldn't find any information about using Nuitka with Django. Do you, by any chance, have such experience?
    – roboslone
    Oct 18, 2016 at 12:42
  • 1
    @roboslone, Prakhar Agarwal Cython supports external libraries really well, when compared to Nuitka PyPy's static builder. For instance, from the link at the end of this comment, "people use Cython because it helps them solve a problem. Which is either that they want to connect to external non-Python libraries from Python code or that they want to be able to manually optimise their code, or both.", the link being blog.behnel.de/posts/indexp241.html Oct 18, 2016 at 13:03
  • 3
    R. S. Nikhil Krishna, I'm not trying to make my code faster, I'm trying to ship it in one binary with python interpreter inside.
    – roboslone
    Oct 18, 2016 at 18:06
  • Aah okay. I thought you wanted to bind it with some external library. My bad. Oct 18, 2016 at 18:56
3

You're probably looking for something like Freeze, which is able to compile your Python application with all its libraries into a static binary:

PyPi page of Freeze

Python Wiki page of Freeze

Sourceforge page of Freeze

2
  • Actually, all current python packaging tool is doing fake package. They strongly rely on libc. Once you put those fake binary file into ubuntu7 and run them, you'll see many errors. Sep 21 at 11:25
  • I simply wish someone could do a full static compilling for python. So that the final binary excutable will get running even without glibc. Sep 21 at 11:26
2

If you are on a Mac you can use py2app to create a .app bundle, which starts your Django app when you double-click on it.

I described how to bundle Django and CherryPy into such a bundle at https://moosystems.com/articles/14-distribute-django-app-as-native-desktop-app-01.html

In the article I use pywebview to display your Django site in a local application window.

2
  • My application runs on Linux mostly, so py2app doesn't apply
    – roboslone
    Oct 19, 2016 at 16:10
  • Though the article uses py2app, it can be done using pyinstaller, too. Oct 19, 2016 at 16:44
1

Freeze options:

However, your target server should have the environment you want -> you should be able to 'create' it. If it doesn't, you should build your software to match the environment.

I found this handy guide on how to install custom version of python to a virtualenv, assuming you have ssh access: https://stackoverflow.com/a/5507373/5616110

In virtualenv, you should be able to pip install anything and you shouldn't need to worry about sudo privileges. Of course, having those and access to package manager like apt makes everything a lot easier.

2
  • I'm already using virtualenv, this is not the case. My question is about statically compiled python witn required modules and applications.
    – roboslone
    Oct 13, 2016 at 12:24
  • Then you have to freeze it, no other way around that.
    – iScrE4m
    Oct 13, 2016 at 12:25
1

I have created a docker image that relies on Nuitka and a custom statically linked python3.10 to create a static binary.

Did not test it extensively, if you have the chance please let me know if it works for your use case.

You can check it at: https://github.com/joaompinto/docker-build-python-static-bin

2

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.