24

How do I specify the header files in a setup.py script for a Python extension module? Listing them with source files as follows does not work. But I can not figure out where else to list them.

from distutils.core import setup, Extension
from glob import glob

setup(
    name = "Foo",
    version = "0.1.0",
    ext_modules = [Extension('Foo', glob('Foo/*.cpp') + glob('Foo/*.h'))]
)
1
  • Try importing setuptools rather than distutils.core. Then I believe it will pick up header files automatically. If that doesn't work, add a MANIFEST.in as below Commented May 27, 2016 at 12:22

6 Answers 6

22
+50

Add MANIFEST.in file besides setup.py with following contents:

graft relative/path/to/directory/of/your/headers/
9
  • Doesn't that tell distutils to install the headers? I don't want to install the headers; they should only be used as input for the compiler. Only the pyd file produced by the compiler should be installed. Commented Jul 14, 2011 at 7:28
  • 2
    No, whey won't be installed. MANIFEST is not for installation, but for source distribution (setup.py sdist). Commented Jul 18, 2011 at 11:38
  • 3
    Great! But I'm surprised that you need a MANIFEST at all to do something this simple. Commented Jul 18, 2011 at 12:17
  • Gah, I can't believe I forgot about MANIFEST.in. >.<
    – John Doe
    Commented Jul 18, 2011 at 19:36
  • 2
    @Éric Araujo: I'm suprised to learn that distutils knows about source files but not about header files. Commented Mar 26, 2012 at 11:58
8

Try the headers kwarg to setup(). I don't know that it's documented anywhere, but it works.

setup(name='mypkg', ..., headers=['src/includes/header.h'])
4
  • That should indeed work. I have no idea whether the doc was omitted by oversight or if it was supposed to be an internal detail.
    – merwok
    Commented Mar 29, 2012 at 2:41
  • 3
    @Éric Araujo: I tried it, and it did not work. I did "setup.py sdist", and the headers were not included in Foo-0.1.0.zip. Commented Mar 30, 2012 at 11:02
  • 3
    There is actually a note in the code of the sdist command that says headers are not included, and another note in build_ext (which defines a get_source_file function used by sdist to build a part of its list of files) that says “Wouldn't it be neat if we knew the names of header files too”. distutils is closed to new features (we work on distutils2 now), but I am inclined to consider this a bug and fix it for the next releases of Python 2.7 and 3.2. I will open a report on bugs.python.org later.
    – merwok
    Commented Mar 31, 2012 at 23:14
  • @ÉricAraujo could you add a link to the bug report? I can't find it.
    – marcin
    Commented Aug 13, 2016 at 17:01
6

I've had so much trouble with setuptools it's not even funny anymore. Here's how I ended up having to use a workaround in order to produce a working source distribution with header files: I used package_data.

I'm sharing this in order to potentially save someone else the aggravation. If you know a better working solution, let me know.

See here for details: https://bitbucket.org/blais/beancount/src/ccb3721a7811a042661814a6778cca1c42433d64/setup.py?fileviewer=file-view-default#setup.py-36

    # A note about setuptools: It's profoundly BROKEN.
    #
    # - The header files are needed in order to distribution a working
    #   source distribution.
    # - Listing the header files under the extension "sources" fails to
    #   build; distutils cannot make out the file type.
    # - Listing them as "headers" makes them ignored; extra options to
    #   Extension() appear to be ignored silently.
    # - Listing them under setup()'s "headers" makes it recognize them, but
    #   they do not get included.
    # - Listing them with "include_dirs" of the Extension fails as well.
    #
    # The only way I managed to get this working is by working around and
    # including them as "packaged data" (see {63fc8d84d30a} below). That
    # includes the header files in the sdist, and a source distribution can
    # be installed using pip3 (and be built locally). However, the header
    # files end up being installed next to the pure Python files in the
    # output. This is the sorry situation we're living in, but it works.

There's a corresponding ticket in my OSS project: https://bitbucket.org/blais/beancount/issues/72

4

If I remember right you should only need to specify the source files and it's supposed to find/use the headers.

In the setup-tools manual, I see something about this I believe.

"For example, if your extension requires header files in the include directory under your distribution root, use the include_dirs option"

Extension('foo', ['foo.c'], include_dirs=['include'])

http://docs.python.org/distutils/setupscript.html#preprocessor-options

4
  • 1
    Tried ext_modules = [Extension('Foo', glob('Foo/*.cpp'), include_dirs=['Foo'])]. Does not work; the header files are not added to the package. Commented Jul 9, 2011 at 8:34
  • Have you tried the code in your questions example? It might work, at least I hope it will. :) I couldn't find anything else related to your problem in the manual; sorry. :(
    – John Doe
    Commented Jul 9, 2011 at 8:40
  • I added the include_dirs parameter. It seems to require an absolute directory and not a relative one as the build is carried out in the build dir and .h files are not copied across.
    – Ben
    Commented May 11, 2015 at 20:57
  • Seems like an option is proposed. Ref : bugs.python.org/issue34012
    – xtreak
    Commented Jun 30, 2018 at 17:12
1
Extension('foo', ['foo.c'], include_dirs=['include'])

This works! Thanks John

1
0

I can confirm that using the MANIFEST.in file to specify the relative path that contains headers works. I had a folder called kernels that contained 4 CUDA kernels, each in a separate folder. Additionally, I had header file utils.h which was shared across all 4 folders by using #include "../utils.h". When I was running the command python3 -m build, the header file utils.h was not copied to the build folder and the include statement was failing because it was not finding the header file. In the end, using the MANIFEST.in file worked, I could see that the header file was copied.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.