71

I have a simple pandas pyinstaller exe which is over 40MB.

My exe example:

import collections
import csv
import selenium
import pandas

print('hi')

40MB+ for this seems a bit overkill.

How can I reduce this as much as possible?

One method:

pyinstaller --onefile --exclude matplotlib --exclude scipy --exclude pandas --exclude numpy.py

This however is not practical considering how big the exclusion list would be.

How do I select a folder for pyinstaller to get modules from and exclude everything else so I may have a small application?

Spec file:

a = Analysis(['123.py'],
             pathex=['C:\\Users\\AA\\ZZ'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          name='123',
          debug=False,
          strip=False,
          upx=True,
          runtime_tmpdir=None,
          console=True )

It's also worth mentioning. By default, Pyinstaller does not detect pandas.

Add:

hiddenimports = ['pandas._libs.tslibs.timedeltas']

To: C:\Users\<NAME>\AppData\Local\Programs\Python\Python36\Lib\site-packages\PyInstaller\hooks

A possible solution when using multiple executables, could be to link each executable to a separate folder or executable with all imports.

5
  • 1
    If you remove the dependencies, the exe is not going to be able to run your script, or am I missing something?
    – Florent B.
    Dec 14, 2017 at 15:06
  • 2
    @FlorentB. Pyinstaller is packaging other stuff not needed. Also, 40mb for that exe? I tried to UPX it, but it's not changing much for me. Must be doing it wrong.
    – user9062171
    Dec 14, 2017 at 15:07
  • When I do pyinstaller --onefile test.py (test.py contains your example), I get an executable that is about 5mb. Dec 17, 2017 at 2:43
  • 4
    Try using the --onedir option instead of --onefile, and see what pyinstaller is putting in the dist folder. Dec 17, 2017 at 2:44
  • In general, check your dependency tree. For example, numpy may be using using MKL, matplotlib may be using Qt, both of which can lead to very large executables. See e.g. stackoverflow.com/a/70113449, stackoverflow.com/a/67954011
    – djvg
    Feb 14, 2022 at 9:58

5 Answers 5

52
+50

try setting up your environment with a virtualenv, and install in there only the required libraries

some details on working with virtual env are here: https://virtualenv.pypa.io/en/stable/

9
  • 2
    I am hesitant to do this as I have recently had to reinstall my entire os because of file corruption issues and python. Is this safe? I will see how this goes in a virtual box.
    – user9062171
    Dec 14, 2017 at 10:17
  • 5
    yes - it somewhat like a python virtualbox - all packages are installed in a separate virtual env, and don't effect the main python installation. removing it is just deleting a directory, and you cn have sevral virtualenvs with different combinations of libraries and versions Dec 14, 2017 at 10:20
  • 1
    This method saved 50% file size in my case. I was importing os, pandas, sqlalchemy.
    – mccbala
    Oct 30, 2018 at 2:41
  • 1
    Hm I installed pytorch in my virtualenv and it has 1.4 GB!!! Now if I use not all of pytorch (just some libs) is it possible to decrease the exe file size in this way?
    – sqp_125
    Mar 2, 2020 at 21:16
  • 2
    This method actually made my exe 10.4 MB from 155 MB. lol, thanks.
    – Amin Pial
    Jun 8, 2021 at 8:17
16

For me, it is a simple case of using pandas that the exe is huge.

Though removing certain directories was helpful, as was UPXING that helped a great deal also.

I got it reduced a lot and it was not doing this by default.

That being said, the final and most import solution is talked about here: Importing Python modules from a select location . So there was a feature that did all this, but for now there is some manual handling involved because: multipackage-bundles is broken.

Now to the simple solution for lots of exe's

If you have many executables, I highly recommend this approach:

pyinstaller -F abc.py --onedir (Have all imports of both scripts)
pyinstaller -F abd.py --onedir (Have all imports of both scripts)

Now put abd.exe in the one directory of abc.py folder as well as any other external scripts. Be sure they are differently named or only one script will run.

This works really well because all the dependencies are in one folder. This is how it should be. So in this example say you had a 40mb one folder. For each additional exe afterwards, it will only be +5mb(or how big the exe is) rather than 40mb each.

5

The python interpreter and all imported modules are included in the executable.

You can try adding modules you want to exclude to the excludes list under Analysis in your spec file.

You could also try compressing the executable using UPX. See A note on using UPX

2
  • 2
    Does it not upx by default? I have a large number of modules to exclude so probably not ideal. There is no exclude all modules except for a given directory?
    – user9062171
    Dec 14, 2017 at 10:15
  • 1
    I thought it was upxed automatically, my mistake. Something like: pyinstaller -F --upx-dir C:\Users\DD\Downloads\upx394w\upx394w\123\upx308w\upx.exe zz.spec does not work though.
    – user9062171
    Dec 14, 2017 at 13:50
3

I had a similar problem and found a solution. I used Windows terminal preview. This program allows creation of various virtual environments like Windows Power Shell (btw. Linux Ubuntu too. Also, worth noting: you can have many terminals in this program installed and, even, open a few at once. Very cool stuff).

Inside Windows Power Shell in Windows terminal preview I installed all the necessary libraries, then I opened the path to my file and tried to use this command:

pyinstaller --onefile -w 'filename.py'

...but, the output exe didn't work. For some reason, the console said that there is a lack of one library (which I had installed earlier). I've found the solution in mimic the auto-py-to-exe library. The command used by this GUI is:

pyinstaller --noconfirm --onedir --console "C:/Users/something/filename.py"

And this one works well. I reduced the size of my output exe program from 911MB to 82,9MB !!!

BTW. 911MB was the size of output made by auto-py-to-exe.

I wonder how is it possible that no one yet has created a compressor that reads the code, checks what libraries are part of the code, then putting only them inside the compression. In my case, auto-py-to-exe probably loaded all libraries that I ever installed. That would explain the size of this compressed folder.

Some suggest using https://virtualenv.pypa.io/en/stable/ but in my opinion, this library is very difficult, at least for me.

1

I use the Anaconda environment and so the virtualenv solution isn't an option. my way was to exclude unnecessary modules in my spec file, ex.:

in Analysis(...)

excludes=['pandas', 'numpy'],

(this are modules whose raise the size of the files extraordinarily)

For every build i'm using this adjusted spec file to create the exe.

pyinstaller "mySpec.spec" --distpath="<path>"
1
  • 1
    You can create a clean new conda environment. This brought the executable size down to 12 MB from 266 MB Aug 6, 2022 at 10:30