4

I'm trying to compile Python 3.6 on Linux statically with OpenSSL.

My build happens in a dockerfile, but essentially does:

$ ./configure --prefix=/task/build --disable-shared LDFLAGS="-static"
$ make altinstall

With an update to Modules/Setup.local to make it look like:

*static*

# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
SSL=/usr/local/ssl
_ssl _ssl.c \
 -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
 -L$(SSL)/lib -lssl -lcrypto

However, on the configure step, I get the error:

Step 9/14 : RUN ./configure --prefix=/task/build --disable-shared LDFLAGS="-static"
     ---> Running in cb79ee47052b
    checking for git... found
    checking build system type... x86_64-pc-linux-gnu
    checking host system type... x86_64-pc-linux-gnu
    checking for python3.6... no
    checking for python3... no
    checking for python... python
    checking for --enable-universalsdk... no
    checking for --with-universal-archs... no
    checking MACHDEP... linux
    checking for --without-gcc... no
    checking for --with-icc... no
    checking for gcc... gcc
    checking whether the C compiler works... no
    configure: error: in `/task/cpython':
    configure: error: C compiler cannot create executables
    See `config.log' for more details
The command '/bin/sh -c ./configure --prefix=/task/build --disable-shared LDFLAGS="-static"' returned a non-zero code: 77

If I change the configure command to:

$ ./configure --prefix=/task/build --disable-shared

I get a compiled binary, but it isn't statically linked to OpenSSL.

What am I doing wrong?

Thanks!


Build dockerfile:

FROM amazonlinux:2017.03.1.20170812

ARG python_version=3.6.8

WORKDIR /task
COPY Modules-Setup.local /task/Modules-Setup.local

# Install requirements
RUN yum install -y \
  gcc \
  git \
  gzip \
  openssl-devel \
  tar \
  zlib \
  zlib-devel

# Get openssl and python source
RUN git clone https://github.com/python/cpython.git
WORKDIR /task/cpython
RUN git checkout tags/v${python_version}

# Configure the build
RUN ./configure --prefix=/task/build --disable-shared LDFLAGS="-static"

# Append modules setup with custom values
RUN cat /task/Modules-Setup.local >> /task/cpython/Modules/Setup.local
RUN cat /task/cpython/Modules/Setup.local

# Build
RUN make altinstall

# Zip the results
WORKDIR /task/build
RUN tar --create --gzip --file=/task/python-${python_version}.tar.gz \
  lib/ bin/
8
  • configure command is incorrect - try: LDFLAGS="-static" ./configure --prefix=/task/build --disable-shared. Could you detail what statically linked to OpenSSL means?
    – CristiFati
    Commented Apr 30, 2019 at 20:49
  • Statically linked means include the OpenSSL archive in the python lib bundle to me, lmk if that clarifies.
    – Jay
    Commented Apr 30, 2019 at 21:28
  • I thought so, 1st thing you'll need is the static OpenSSL libs. There's nothing (in terms of build flags, options) you can do on Python side if you don't have those. stackoverflow.com/questions/725472/….
    – CristiFati
    Commented Apr 30, 2019 at 21:36
  • Gotcha - thanks. I ultimately want a folder with everything I need to run python (with some linked modules available, like openssl) that I can copy to a machine and use immediately. Should compiling (statically) those packages and linking them as described by jww below get me there?
    – Jay
    Commented May 1, 2019 at 19:58
  • This might be an XY Problem. There are tools (at least on Win) that collect anything needed for a piece of Python code to run (e.g. py2exe). You got the static linking wrong: it refers to the python executable: if static, it will have ~5MB, otherwise, it will have ~10KB (but you'll also have a libpython3.6m.so that the .exe depends on, and that contains all the Python core functionality). @jww explained it very well. Bottom line: if you want to statically link to something, that something should be (also) built for static linking.
    – CristiFati
    Commented May 1, 2019 at 20:17

3 Answers 3

8

I'm trying to compile Python 3.6 on Linux statically with OpenSSL.
...

# Socket module helper for SSL support; you must comment out the other
# socket line above, and possibly edit the SSL variable:
SSL=/usr/local/ssl
_ssl _ssl.c \
 -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
 -L$(SSL)/lib -lssl -lcrypto

Change -lssl and -lcrypto to -l:libssl.a and -l:libcrypto.a:

SSL=/usr/local/ssl
_ssl _ssl.c \
  -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
  -L$(SSL)/lib -l:libssl.a -l:libcrypto.a

You can also use the full path to the archive:

SSL=/usr/local/ssl
_ssl _ssl.c \
  -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
  $(SSL)/lib/libssl.a $(SSL)/lib/libcrypto.a

Archives (*.a) are just a collection of object files (*.o), so you can use an archive wherever you use an object file.

Also see -l:filename in the ld(2) man page:

--library=namespec

Add the archive or object file specified by namespec to the list of files to link. This option may be used any number of times. If namespec is of the form :filename, ld will search the library path for a file called filename, otherwise it will search the library path for a file called libnamespec.a.

If you have other components in /usr/local you are using, then you might want to add -L/usr/local/lib -Wl,-R,/usr/local/lib -Wl,--enable-new-dtags to your LDFLAGS. The new-dtags embeds a RUNPATH (as opposed to RPATH) in the ELF headers. RUNPATH can be overridden with LD_LIBRARY_PATH.


I get a compiled binary, but it isn't statically linked to OpenSSL.

The way to check is to use ldd with the paths you use at runtime. For example, here is from a local OpenSSL build on Fedora:

$ ldd /usr/local/bin/openssl
    linux-vdso.so.1 (0x00007fff3cde6000)
    libssl.so.1.0.0 => /usr/local/lib64/libssl.so.1.0.0 (0x00007f043dc4e000)
    libcrypto.so.1.0.0 => /usr/local/lib64/libcrypto.so.1.0.0 (0x00007f043d9df000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007f043d9c0000)
    libc.so.6 => /lib64/libc.so.6 (0x00007f043d7fa000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f043dcc0000)

Here are a couple of related questions, but it does not look like they cover static linking with Python.


And to be clear, config.log has the error but you did not show the relevant portion from it:

checking whether the C compiler works... no
configure: error: in `/task/cpython':
configure: error: C compiler cannot create executables
See `config.log' for more details

Static OpenSSL may (or may not) fix the problem.

1
  • Is there a way (similar to ldd) to discover the shared objects that my compiled binary will import during execution (not just at startup)?
    – Jay
    Commented May 2, 2019 at 14:01
3

I ran across the same issue and solved it by installing the static glibc libraries:

yum install glibc-static
0

You can have a look at the python repository, where it has a list of modules and c filename that python need to compile python:

https://github.com/python/cpython/blob/51863b7d6ea183167da09fc6b3f2745a1aaa4ef5/Modules/Setup#LL232

Something like this:

*static*

# Modules that should always be present (POSIX and Windows):

#_asyncio _asynciomodule.c
#_bisect _bisectmodule.c
#_contextvars _contextvarsmodule.c
#_csv _csv.c
#_datetime _datetimemodule.c
#_decimal _decimal/_decimal.c
#_heapq _heapqmodule.c
#_json _json.c
#_lsprof _lsprof.c rotatingtree.c
#_multiprocessing -I$(srcdir)/Modules/_multiprocessing _multiprocessing/multiprocessing.c _multiprocessing/semaphore.c
#_opcode _opcode.c
#_pickle _pickle.c
#_queue _queuemodule.c
#_random _randommodule.c
#_socket socketmodule.c
#_statistics _statisticsmodule.c
...

Do an uncomment, and put it into 'Setup.local' file, then you are ready to go.


If you really don't know how to compile it, I have a dockerfile for you:

FROM alpine:3.12

ENV PYTHON_VER 3.10.4
ENV PYTHON_LIB_VER 3.10

RUN mkdir /build /package
WORKDIR /build

RUN wget https://www.python.org/ftp/python/$PYTHON_VER/Python-$PYTHON_VER.tgz && tar -xzf Python-$PYTHON_VER.tgz
WORKDIR Python-$PYTHON_VER

RUN apk add musl-dev 
RUN apk add build-base
RUN apk add linux-headers
RUN apk add bash

RUN apk add python3-dev py3-pip 
RUN apk add git vim wget make

RUN apk add openssl-dev
RUN apk add sqlite-dev
RUN apk add zlib-dev
RUN apk add bzip2-dev
RUN apk add libffi-dev
RUN apk add util-linux-dev
RUN apk add xz-dev
RUN apk add libnsl-dev libtirpc-dev
RUN apk add gdbm-dev
RUN apk add tk-dev

RUN apk add ncurses-dev
RUN apk add readline-dev

# RUN python3 -m ensurepip
# RUN pip3 install readline

ADD Setup.local Modules/
ENV LDFLAGS "-static"
RUN ./configure LDFLAGS="-static -static-libgcc" CPPFLAGS="-static" --disable-shared --with-ensurepip=install --with-ssl --with-zlib --enable-nis --with-tirpc-include-dir=/usr/include/tirpc --with-gdbm
RUN make -j$(nproc) LDFLAGS="-static" LINKFORSHARED=" "

# RUN cp libpython$PYTHON_LIB_VER.a /usr/lib
# WORKDIR /build
# ENV LDFLAGS "-static -l:libpython3.10.a"


CMD sleep 999999999999
# Manually copy /build to host storage
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.