pyVideoTrans

Appearance

PyInstaller Packaging Pitfalls: From Silent Crashes to a Clear Path Forward ​

In software development, packaging a smoothly running Python project into a standalone executable (EXE) is a crucial step for delivering it to users. PyInstaller is undoubtedly the king in this domain. However, sometimes this king can bring us quite a few headaches.

I recently encountered a typical problem. My project had always been packaged smoothly with PyInstaller. That was until I upgraded the torch2.7 library. Suddenly, the packaging command pyinstaller sp.spec failed. It would run for a moment and then exit silently, leaving no error messages.

This article documents my journey of finding the root cause and ultimately solving this "silent crash" dilemma. At the same time, I'll take this opportunity to systematically discuss my experience with PyInstaller, especially regarding its use on the Windows platform.

1. The Mysterious "Silent Crash" ​

After running the packaging command in my virtual environment, the log output stopped abruptly halfway:

... (previous logs omitted)
127592 INFO: Loading module hook 'hook-numba.py' ...
127608 INFO: Loading module hook 'hook-llvmlite.py' ...

(venv) F:\python\pyvideo>_

The cursor blinked quietly—no Error, no Traceback, nothing.

This situation is very tricky. An exit without an error usually means the problem lies deeper, likely a crash of the Python interpreter itself. Libraries like PyTorch, NumPy, and Numba contain a lot of C/C++ compiled code at their core. When PyInstaller analyzes these libraries, version incompatibilities or conflicts can cause memory errors, leading the entire process to crash directly without generating a Python-level error report.

Since the problem appeared after upgrading torch, the root cause could almost certainly be pinned on version compatibility.

2. Finding Clues: Making the Error Speak ​

Faced with a "silent lamb," our top priority is to make it "speak."

My first thought was: could the PyInstaller version be too old to recognize the new torch? So I executed:

pip install --upgrade pyinstaller

After upgrading to the latest version, I ran the packaging command again. A miracle happened—the previous "silent crash" disappeared! But it was replaced by a brand new, clear error message:

... (logs)
182529 INFO: Processing standard module hook 'hook-h5py.py' ...

=============================================================
A RecursionError (maximum recursion depth exceeded) occurred.
...

RecursionError! The problem finally became clear.

This error tells us that PyInstaller got stuck in excessively deep recursive calls while analyzing the project dependencies.

Imagine PyInstaller as a detective. To find all the files needed for the program to run, it starts from your main script sp.py, sees who it imports (like torch), then looks at who torch imports (like scipy), then who scipy imports... and so on, tracing layer by layer.

For a massive library like torch, the internal module interdependencies form a huge, intricate web. After an upgrade, this web might have become even more complex, causing the detective to loop around too many times during the trace, eventually exceeding Python's set "recursion depth" safety limit, forcing the program to stop.

3. The Winning Move: Increasing the Recursion Limit ​

Fortunately, PyInstaller's error message thoughtfully provided the solution. We just need to relax this limit in the .spec file.

The .spec file is PyInstaller's "blueprint" for packaging, giving us fine-grained control over the packaging process.

I opened my sp.spec file and added two lines of code at the very beginning:

python

# Add these two lines to solve RecursionError
import sys
sys.setrecursionlimit(5000)

This line of code tells Python: "Increase the recursion depth limit from the default 1000 to 5000."

After saving and running pyinstaller sp.spec again, the packaging process smoothly passed the previously stuck point and finally succeeded in generating the executable file. Problem solved.

This experience teaches us that when packaging projects containing large scientific computing libraries (like PyTorch, TensorFlow, SciPy, etc.), RecursionError is a common issue, and increasing the recursion limit is the most direct and effective solution.

4. Diving into the .spec File: The Art of Packaging ​

Since the .spec file was key to solving the problem, let's take a deeper look at it. Simply typing pyinstaller script.py on the command line is easy, but for complex projects, carefully editing a .spec file is the more professional and reliable approach.

We can use the pyi-makespec script.py command to generate a basic .spec file and then modify it from there.

Below, let's examine the intricacies of my sp.spec file.

python
# sp.spec

import sys
sys.setrecursionlimit(5000)

import os, shutil
from PyInstaller.utils.hooks import collect_data_files

# --- 1. Define resources to include and hidden imports ---
hidden_imports = [
    'funasr', 'modelscope', 'transformers', # etc.
    'scipy.signal',
]
datas = []
datas += collect_data_files('funasr')
datas += collect_data_files('modelscope')

# --- 2. Analysis Phase (Analysis) ---
a = Analysis(
    ['sp.py'],  # Your main program entry point
    pathex=[],
    binaries=[],
    datas=datas, # Include all non-code files
    hiddenimports=hidden_imports, # Tell PyInstaller about libraries it might miss
    excludes=[], # Explicitly exclude certain libraries if needed
    # ... other parameters
)

# --- 3. Package Python Modules (PYZ) ---
pyz = PYZ(a.pure, a.zipped_data)

# --- 4. Create the Executable (EXE) ---
exe = EXE(
    pyz,
    a.scripts,
    name='sp',          # Define your program name here
    console=False,      # False means a GUI program without a console window
    icon='videotrans\\styles\\icon.ico', # Specify your icon here
    # ... other parameters
)

# --- 5. Collect All Files into the Final Directory (COLLECT) ---
coll = COLLECT(
    exe,
    a.binaries,
    a.datas,
    name='sp',
)

# --- 6. Custom Post-Build Operations ---
# This is a very powerful feature of the .spec file, allowing execution of arbitrary Python code after packaging
os.makedirs("./dist/sp/videotrans", exist_ok=True)
shutil.copytree("./videotrans/prompts", "./dist/sp/videotrans/prompts", dirs_exist_ok=True)
shutil.copy2("./voice_list.json", "./dist/sp/voice_list.json")
# ... more file copy operations

Core Concept Explanation:

  • hiddenimports: Some libraries use dynamic imports (e.g., importlib.import_module()), which PyInstaller's static analyzer might not "see." Adding the names of these libraries as strings to the hiddenimports list directly tells PyInstaller: "Don't miss these guys."
  • datas: Your program might need some non-.py files, like .json configuration files, images, model files, etc. datas is used to package these data files.
    • collect_data_files('some_library') is a convenient helper function that automatically collects all data files accompanying a specific library.
    • You can also add them manually using the format [('source_file_path', 'relative_path_in_packaged_directory')]. For example, [('config.json', '.')] places config.json in the same directory as the executable.
  • EXE: This is the key to customizing the executable file.
    • name: Defines the name of sp.exe.
    • icon: Specifies an .ico file as the program's icon. This is a crucial step for making the program look more professional on Windows.
    • console: True creates a program with a black console window (suitable for command-line tools), False creates one without (suitable for GUI programs).
  • Post-build Script: After COLLECT, you can write any Python code. In my example, I used shutil.copytree and shutil.copy2 to copy directories and files that don't need to be packaged into the EXE but need to be placed alongside it, such as configuration files, documentation, model weights, etc. This provides great flexibility.

5. Other Common Issues on the Windows Platform ​

Besides RecursionError, you might encounter other issues when using PyInstaller on Windows:

  1. Missing DLLs: Sometimes PyInstaller misses certain dynamic link libraries (.dll). The program crashes immediately upon running, reporting a missing DLL. The solution is to find that DLL file and manually add it in the binaries parameter of Analysis, using a format similar to datas.

  2. File Path Issues: After packaging, the program can't find resource files. This is because the behavior of relative paths changes in packaged mode. The correct approach is to use the following code to get a reliable base path:

    python
    import sys
import os

if getattr(sys, 'frozen', False):
    # If in packaged state (.exe)
    base_path = os.path.dirname(sys.executable)
else:
    # If in normal running state (.py)
    base_path = os.path.dirname(os.path.abspath(__file__))

# Then use base_path to construct your resource paths
config_path = os.path.join(base_path, 'config.ini')

    Note: This code is very reliable for one-folder mode. For one-file mode, the program extracts to a temporary directory when run, and sys._MEIPASS points to that temporary directory.

  3. False Positives by Antivirus Software: This is a persistent problem, especially when using the one-file (--onefile) mode. This is because PyInstaller's bootloader needs to extract files into memory before execution, which resembles the behavior of some malware.

    • Recommendation: Prioritize using the one-folder mode (the default in .spec). It's more stable, starts faster, and is less likely to trigger false positives. Then send the entire folder to the user.
    • If a single file is necessary, try updating PyInstaller to the latest version or compile the bootloader from source yourself, though that's more complex.

Conclusion ​

PyInstaller is a powerful tool, but it inevitably faces challenges when dealing with the increasingly complex Python ecosystem. This troubleshooting journey from a "silent crash" to a RecursionError reaffirms a few simple truths:

  1. Clear error messages are half the battle. When encountering a problem, first find a way to make it "speak." Updating the relevant toolchain (like PyInstaller itself) can sometimes provide unexpected clues.
  2. The .spec file is important. Spend some time learning it, and you'll be able to handle various complex packaging needs with ease.
  3. Be cautious when upgrading major libraries. After upgrading core dependencies (like torch, tensorflow), your packaging script will likely need adjustments.

References ​