pyVideoTrans

Appearance

pyVideoTrans Documentation ​

Core Features at a Glance

  • Video Translation: Recognizes spoken audio in videos or audio files, generates subtitles in the original language, translates them into target language subtitles, creates a voiceover (dubbing), and embeds the new voiceover and target subtitles back into the original video. (Left panel: Translation V/A)
  • Speech Transcription / Recognition: Transcribes spoken human voice in video or audio files in batches into SRT subtitle files with timestamps. (Left panel: Speech-To-Text)
  • Speech Synthesis / Voiceover (TTS): Uses various TTS services to generate high-quality, natural-sounding voiceovers for SRT subtitles or plain text (.txt) files. (Left panel: Text-To-Speech)
  • Subtitle Translation: Batch-translates SRT subtitle files while preserving original timestamps and formatting. (Left panel: Translation Srt)
  • Transcribe & Translate Subtitles: Transcribes spoken audio from video or audio files into SRT subtitles and translates them into the specified target language in batches. (Left panel: STT-Translation)

1. How It Works ​

The core feature of this software is Video Translation—translating a video in one language into a new video with voiceover and subtitles in another language:

[Original Video] --> [Speech to Text] --> [Generate SRT subtitle files] --> 
[Translation SRT subtitle] --> [SRT to Speech] -->
[Align subtitles, audio, and video] --> [Output the final result file]

Plain English Explanation:

  1. Speech Transcription / Recognition: Like a stenographer, it listens to what people say in the video and writes it down word for word as subtitles.
  2. Subtitle Translation: Translates the transcribed subtitles into your target language.
  3. Subtitle Dubbing (TTS): Uses Text-to-Speech (TTS) engines to "read out" the translated subtitles with a selected voice, generating a new voiceover.
  4. Sync & Alignment: Re-aligns the new voiceover with the original video footage to make sure the audio and video stay synchronized, embeds the subtitles, and outputs the final video.

The entire process is fully automated. All you need to do is select your file, configure a few settings, and click Start.

  • Supported: Any audio or video containing clear human speech (whether it already has subtitles or not).
  • Not Supported: Videos with only background music/visuals and no spoken dialogue.

2. Download & Installation ​

Windows Users ​

  1. Download the pre-packaged version: Click to download Windows version
  2. We recommend extracting it to a short path containing only English letters and numbers (e.g., D:\pyVideoTrans).
  3. Double-click sp.exe to launch.

⚠️ Do NOT run sp.exe directly inside the ZIP archive without extracting first.

For GPU acceleration, make sure CUDA 12.8 and cuDNN 9 are installed

Click to view important notes (avoid most common errors)
  1. Do NOT extract the program to system folders like C:/Program Files or C:/Windows that require special administrator permissions, as generated output and temporary files need to be written inside the program directory.
  2. It is recommended to extract the software to a folder path containing only English characters and numbers (e.g., D:/videotrans). Avoid very deep folder paths to prevent Windows file path length limit errors.
  3. Use short file names for videos to be translated. Overly long file names and deep directory paths can exceed Windows path limits and cause errors.

Videos downloaded from YouTube often have extremely long file names and various special symbols. Using them directly on Windows can easily cause errors. It is strongly recommended to rename them to simple, short names without special characters.

  1. On Windows, it is recommended to turn on "File name extensions" (hidden by default). This helps avoid errors, especially when working with reference audio, .srt, and .wav files.

Open any folder, click View -> Show -> File name extensions in the top navigation bar and check it. Once enabled, MP4 videos will show .mp4 and WAV audio will show .wav.

MacOS / Linux Installation Guide

  1. Install dependencies:

    bash
    # macOS
brew install libsndfile git

brew tap homebrew-ffmpeg/ffmpeg
 
brew install homebrew-ffmpeg/ffmpeg/ffmpeg

# Ubuntu/Debian
sudo apt-get install ffmpeg libsndfile1-dev

  2. Install uv:

    bash
    curl -LsSf https://astral.sh/uv/install.sh | sh

  3. Clone and run:

    bash
    git clone https://github.com/jianchang512/pyvideotrans.git
cd pyvideotrans
uv sync
uv run sp.py

View Detailed Source Code Deployment Guide (macOS/Linux/Windows)

3. Interface Overview ​

After launching the software, the main interface is organized from top to bottom into the following rows:

RowContentDescription
1Select video to translate (Required)Supports mp4, mkv, avi, mov, wav, mp3, etc. Batch translation supported.
2Speech RecognitionSelect the recognition engine and model to convert speech into subtitles (Default: faster-whisper / large-v3-turbo)
3Subtitle TranslationSelect the translation service, source language, and target language to translate the subtitles (Default: Google Translate)
4Subtitle Dubbing (TTS)Select the TTS service and voice character to generate voiceovers for translated subtitles (Default: Edge-TTS)
5Sync & AlignmentAudio speedup, video slowdown, speech speed & volume, subtitle embedding (Default: Audio Speedup)
6Start ProcessingClick to start processing
7Progress BarDisplays progress; click to open the output folder
8Advanced SettingsNoise reduction, vocal/BGM separation and re-embedding, and other advanced options
What do Free, LocalAPI, and Built-in mean?
  • Free: Online services such as Google Translate, Microsoft Translate, and Edge-TTS. They are free to use out-of-the-box without extra setup. Note that rate limits may apply, which could cause occasional errors under high-frequency usage.
  • Built-in: Models that can be conveniently integrated into pyVideoTrans without separate external deployment (e.g., VITS, Piper, Qwen3-TTS, OmniVoice, F5-TTS, Qwen3-ASR, SuperionTTS, ChatterBox, etc.). To prevent the software package from becoming excessively large, only the execution code is built-in; model weights will be downloaded automatically upon first use.
  • LocalAPI: Open-source models deployed locally by yourself (e.g., GPT-SoVITS, CosyVoice). Once deployed and started, enter the API or WebUI URL into the pyVideoTrans settings, and the software will call your local service via that address.

View all model download links and manual download instructions

4. Translation video: Step-by-Step Guide ​

Step 1: Select Video Files ​

1

Click the "Select Audio or Video" button in the top left to choose the files you want to translate. Supported formats include mp4, mkv, avi, mov, wav, mp3, etc. You can select one or multiple files at once.

  • Folder: Check this to batch-process all videos in an entire folder.

  • Clear Output: Check this if you want to re-translate the same video from scratch. Otherwise, the software will reuse previously generated files (such as already transcribed or translated SRT subtitles). If you are re-translating a file and want to skip the time-consuming transcription step, uncheck this.

  • Save To...: Click this button to customize the output directory for the translated video. If not specified, the default directory is used:

    • When Folder is unchecked: Saved to a _video_out folder in the same directory as the selected video. For example, if the selected video is D:/videos/001.mp4, it outputs to D:/videos/_video_out/. A subfolder named {video_name}-{extension} is created for each video.
  • When Folder is checked: Saved to _video_out/{folder_name} alongside the selected folder. For example, if the selected folder is D:/videos/ceshi, it outputs to D:/videos/_video_out/ceshi/, creating a {video_name}-{extension} subfolder for each video.
  • Output MP4 Only: If checked, only the final translated video file is kept in the output folder; intermediate subtitles, audio files, etc., are automatically deleted.
  • Auto Shutdown: Automatically shuts down your computer after all tasks are completed—ideal for large batch or overnight tasks.

Step 2: Choose Speech Recognition Engine ​

2

Engine / ChannelRecommended ForDescription
faster-whisper(Built-in)Default recommendation; tiny is fastest with lowest accuracy, large-v3 has highest accuracy but is slowestFast and high quality
openai-whisper(Built-in)High accuracy requirements; tiny is fastest with lowest accuracy, large-v3 has highest accuracy but is slowestSlightly higher accuracy, slower speed
Qwen-ASR(Built-in)Chinese and 10+ common languagesGreat Chinese recognition, slower speed
Moss-Diarize(Built-in)Chinese and 10+ common languagesSupports speaker diarization within 90 minutes without a separate diarization model
Whisper.cpp(WinBuilt-in)Built-in on Windows for direct use; requires manual build on macOS and LinuxC++ port of the Whisper model
Alibaba FunASR(Built-in)Great for Chinese; supports paraformer, sensevoice, Fun-ASR-Nano models
Firered Chinese(Built-in)Chinese and Chinese dialect videosOpen-sourced by Xiaohongshu (RED)
parakeet Japanese(Built-in)Japanese videosOpen-sourced by NVIDIA
Dolphin Asia(Built-in)East Asian, Southeast Asian, and Middle Eastern languagesFocused on Asian minor languages
OmnilingualASR(Built-in)Broad language coverage across all built-in languagesOpen-sourced by Meta (Facebook); supports many languages but average accuracy
Model Selection for faster-whisper / openai-whisper
  • tiny → Fastest, lowest accuracy
  • base / small → Balanced choice
  • medium → Better performance
  • large-v3 → Best accuracy, requires 8GB+ VRAM (GPU memory)
  • large-v3-turbo → Recommended, great balance of speed and quality
  • tiny.en/base.en/small.en/medium.en/distil-large-v3/distil-large-v3.5 → Distilled models optimized for English; only available when the Spoken language is English
SecondarySTT and LLM Correction Errors

  • SecondarySTT: When a voice character is selected and single subtitle embedding is chosen, you can enable secondary recognition. After dubbing is complete, it re-transcribes the generated dubbing audio to create concise subtitles embedded into the video, ensuring exact alignment between subtitles and voiceover.

In Advanced Options -> Speech Recognition Settings, you can set the Max Speech Duration and Min Speech Duration for secondary recognition. Smaller values help produce shorter subtitle lines.

  • LLM Correction Errors:

    Speech transcription will inevitably contain various errors such as typos and missing punctuation. LLM error correction works by sending the recognition results to an AI model after the transcription is complete to correct typos, add punctuation, etc., to achieve a smoother and more fluent result. It defaults to using the DeepSeek channel, which can be switched in Menu - Tools - Advanced Options - General - LLM Error Correction Channel. The LLM error correction prompts are located in software directory/videotrans/prompts/resegment/llm.txt, which can be modified and adjusted.

Click to view all supported Speech Recognition Engines

Step 3: Choose Translation Engine (Translate subtitles into target language) ​

3

Channel / ServiceDescription
Google Translate(Free)Default. Decent translation quality; may require a proxy/VPN in certain regions
DeepSeekAI LLM translation with excellent quality; cost-effective and highly recommended
Hy-MT2(Built-in)Tencent Hunyuan model translation
M2M100(Built-in)Local offline model translation
Microsoft(Built-in)No proxy needed; may have rate limits

Then select Spoken (the language spoken in the video) and Target (the language you want to translate into).

Click to view all supported Translation Services

Step 4: Choose Dubbing / TTS Engine (Generate voiceover for subtitles) ​

4

Channel / ServiceDescription
Edge-TTS(Free)Default recommendation. Microsoft free service, natural voices, supports all built-in languages
Qwen3-TTS(Built-in)Alibaba local model, high quality but slower, supports voice cloning, 10+ languages
F5-TTS(Built-in)Supports Chinese, English, French, German, Italian, Japanese, Korean, Vietnamese, etc.; supports voice cloning
OmniVoice(Built-in)Supports all built-in languages, supports voice cloning
Confucius(Built-in)Supports Chinese, English, and 14 languages; supports voice cloning
ChatterBox(Built-in)Multilingual with great European language performance; supports voice cloning
ZipVoice ZH/EN(Built-in)Chinese & English; supports voice cloning
Moss-TTS-Nano(Built-in)Multilingual; supports voice cloning
Higgs-audio-v3(Built-in)Multilingual; supports voice cloning (Requires 10GB VRAM for CUDA, 20GB RAM for CPU)
gTTS(Built-in)Google TTS, supports all built-in languages, requires proxy in some regions, average quality

After selecting the engine, pick a speaker in the Voice Character dropdown. Selecting clone means the software will clone the voice from the original video.

clone

Click to view all supported Dubbing / TTS Engines

Step 5: Subtitle, Dubbing, and Video Synchronization ​

Translating a sentence into another language inevitably changes the sentence length, syllable count, and speaking duration. This is the root cause of desynchronization between subtitles, dubbing, and video.

You can adjust synchronization using the following options:

5

ParameterDefaultDescription
TTS Speedup✅ CheckedSpeeds up the voiceover to match the original video duration when dubbing is longer
Slow Video☐ UncheckedSlows down video footage to match the voiceover when dubbing is longer
Subtitle EmbeddingSoft SubtitlesEmbeds subtitles into the video (select Hard Subtitles if playing on web pages)

Subtitle Types:

  • No Subtitles: Replaces audio only without displaying subtitles.
  • Hard Subtitles: Burns subtitles permanently into video frames; subtitles will display anywhere (including web players).
  • Soft Subtitles: Subtitles are added as a separate track; can be toggled on/off in media players (won't display in standard web players).
  • Hard Subtitles(Bilingual): Shows both original and translated subtitles burned into video frames (SecondarySTT will be disabled).
  • Soft Subtitles(Bilingual): Bilingual subtitle track that can be toggled on/off in media players (SecondarySTT will be disabled).

Click to learn more about Audio, Subtitle, and Video Alignment Principles

Step 6: Start Processing ​

6

Click the Start button. The bottom progress bar will display real-time status. Click it to open the output folder. Once complete, the translated video is automatically saved to the output folder.

CUDA Acceleration: If you have an NVIDIA graphics card with CUDA 12.8 and cuDNN 9 configured, enable this to speed up speech recognition significantly.

Single Video Mode: If you select only one video, an editing window will pop up 3–4 times during processing, allowing you to edit subtitles, re-dub, preview the video, etc. Learn more about Single Video Mode.

5. More Settings ​

For noise reduction, preserving original background audio, speaker diarization/separation, adjusting voiceover speed/volume, etc., click More Settings.

Click More.. to expand:

ParameterDescription
Noise ReductionCleans background noise, improving speech recognition accuracy
Default Punct / Restore Punct / Remove Punct"Remove Punct" replaces all punctuation with spaces; "Restore Punct" attempts to restore missing punctuation marks
Separate Vocal/BGMSeparates vocals from background music to improve speech transcription
Re-embed BGMMixes the original background music back into the new voiceover after dubbing
BGM VolumeAdjusts background music volume (0.0 – 2.0)
Speed+%Adjusts dubbing speed (-50% to +100%)
Volume+%Adjusts dubbing volume (-95% to +200%)
Pitch+Adjusts voice pitch (-100Hz to +100Hz)
Identify SpeakersIdentifies different speakers (only effective in single-video translation)
No LimitWhen speaker identification is enabled, specifying the exact number of speakers improves accuracy

6. Lossless Video Output ​

To ensure the output video retains original picture quality without re-encoding, make sure all of the following conditions are met:

  1. The original video is an MP4 file encoded in H.264 (libx264).
  2. "Slow Video" is unchecked.
  3. Subtitle type is set to "No Subtitles", "Soft Subtitles", or "Soft Subtitles(Bilingual)".
  4. In Advanced Options, the 264/265 codec is set to 264.

7. Advanced Options Reference ​

Access via Menu Bar -> Tools/Options -> Advanced Options:

Click to view full guide on Advanced Options parameters

8. Frequently Asked Questions (FAQ) ​

Q: Can I run multiple sp.exe instances at the same time? ​

Yes, but it is not recommended. Multiple instances share the same tmp temporary folder. When any instance closes, it attempts to clear this temporary folder, which may cause errors in other running instances.

If you really need multiple instances, copy the software to another folder (e.g., one in D:/aivideo and another in D:/aivideo2) and launch sp.exe separately from each folder so they operate independently.

Q: Why is processing very slow? ​

  1. Make sure GPU acceleration (CUDA) is enabled.
  2. Use smaller models.
  3. Ensure your graphics card drivers are up to date.

Q: Recognition results are inaccurate? ​

  1. Check if the "Spoken" language is selected correctly.
  2. Try using a larger model.
  3. Enable "Noise Reduction".
  4. Adjust the "Voice Threshold" setting.

Q: Voice, subtitles, and video are out of sync after translation? ​

This is normal. Different languages have different syllable counts and sentence structures, which inevitably changes dubbing duration. Solutions:

  1. Enable "TTS Speedup" (enabled by default).
  2. You can also enable "Slow Video".
  3. Adjust "Speed+%" to increase overall voiceover speed.

Q: The output video file is too large? ​

  1. Increase the "Video Output Quality Control (CRF)" value (e.g., 25–30).
  2. Change the video codec from 264 to 265.
  3. Turn off "Slow Video".

Q: How do I use GPU acceleration? ​

Ensure you have installed the NVIDIA graphics driver, CUDA 12.8, and cuDNN 9.11, then check "CUDA Acceleration" on the main interface. Note: AMD GPUs do not support CUDA acceleration.

Q: What are the conditions for lossless video output? ​

Original video is H.264 MP4 + "Slow Video" unchecked + No hard subtitles embedded + Codec set to 264.

Q: Can it extract or erase hard subtitles? ​

This software cannot directly extract or erase hardcoded subtitles burned into video frames.