pyVideoTrans

Appearance

Best Practices for Video Translation ​

This article recommends the best configuration for each of the four stages of the processing pipeline.

Stage 1: Speech Recognition ​

Goal: Convert speech in the video into subtitles in the corresponding language.

πŸ’‘ Tip: If the original audio/video has background noise, click "More Settings" on the main interface and check "Separate Vocal/BGM" to eliminate noise interference and improve recognition accuracy.

Non-Chinese Video ​

LevelServiceModelNotes
Free Recommendedfaster-whisper (built-in)large-v3Speed and quality balance
Free Alternativeopenai-whisper (built-in)large-v3Slightly more accurate
Free Alternativewhisper.cpp (Win built-in)large-v3Fast, built-in for Windows
Paid RecommendedOpenAI STT APIβ€”Excellent results

Chinese Video ​

LevelServiceModelNotes
Free RecommendedQwen-ASR (built-in)β€”Great for Chinese
Free AlternativeAlibaba FunASR (built-in)paraformer-zhGreat for Chinese
Free AlternativeFirered Chinese (built-in)Xiaohongshu Chinese & dialectsGreat for Chinese
Free AlternativeHuggingface_ASR (built-in)zai-org/GLM-ASR-Nano-2512Zhipu AI model, great for Chinese
Paid RecommendedByteDance STT Turboβ€”Great for Chinese
Paid RecommendedXiaomi mimo-v2.5-asrβ€”Great for Chinese
Paid AlternativeAlibaba Bailian ASRβ€”Chinese optimized

Japanese Video ​

LevelServiceModelNotes
Free Recommendedopenai-whisper (built-in)large-v3Good general results
Free RecommendedParakeet Japanese (built-in)NVIDIA Japanese modelGood results
Free AlternativeHuggingface_ASRjapanese-wav2vec2-largeJapanese-specific
Paid RecommendedOpenAI STT APIβ€”Excellent results

Lesser-used Language Video ​

LevelServiceNotes
Free Recommendedopenai-whisper (built-in) large-v3General model, supports dozens of languages
Free RecommendedDolphin (built-in)Designed for Asian languages (select "Auto Detect" in batch STT)
Free RecommendedOmnilingual ASR (built-in)1600+ languages (select "Auto Detect" in batch STT)
Paid RecommendedGemini / OpenAI APIGood for lesser-used languages

Note: With local models, no NVIDIA GPU or CUDA acceleration will result in slow processing. Insufficient VRAM may cause crashes.

View speech recognition service usage guides

Stage 2: Subtitle Translation ​

Goal: Translate the subtitles generated in Stage 1 into the target language.

LevelServiceNotes
First ChoiceDeepSeek / OpenAI ChatGPT / Gemini (latest models)Best AI translation quality
FreeGoogle Translate / Microsoft TranslateTraditional translation, fast
LocalM2M100Fully offline translation
LocalHy-MT2-1.8BTencent open-source translation model

Key settings:

  • Check "Send full subtitle" β€” gives AI full context for more natural translation
  • For AI services, set "AI translation batch size" to 100 or higher (with long-context models)

Stage 3: Dubbing ​

Goal: Generate dubbing audio from the translated subtitles.

LevelServiceNotes
Free RecommendedEdge-TTSMicrosoft free API, natural, supports all languages
Local RecommendedQwen-TTS, F5-TTS, OmniVoice, ConfuciusBuilt-in, supports cloning
Paid RecommendedByteDance TTS 2.0 / Qwen-TTS (Bailian) / Xiaomi / Minimaxi / OpenAI-TTSHigh-quality commercial APIs
Voice CloneOmniVoice / Confucius / Qwen-TTS / GPT-SOVITS / CosyVoice / F5-TTS / Index-TTS / ChatterBox / ZipVoiceUse original video voice

View detailed dubbing service information

Stage 4: Subtitle, Dubbing & Video Sync ​

Goal: Synchronize subtitles, dubbing, and video content.

ConfigurationNotes
Check "Second-pass recognition"Re-transcribe dubbed audio for precise timestamps
Set "Dubbing speed"Chinese→English: set +10 or +15 to speed up
Check "Audio speedup"Auto speed up when dubbing is longer than original
Also check "Slow video"Best results combined with audio speedup
Check "Separate vocal/BGM"Embed original background audio
Check "Noise reduction"Improve audio quality and recognition accuracy

Stage 5: Quality Improvement Tips ​

Basic Settings ​

  1. Check "Send full subtitle"
  2. Check "Menu β†’ Tools β†’ Advanced Options β†’ AI translation with full original subtitle"
  3. Set "AI translation batch size" to 100 or higher
  4. Must use an online AI model that supports long context

When Using clone for Voice Cloning ​

  1. Do NOT use LLM re-segmentation β€” re-segmentation disrupts reference audio extraction.
  2. Set Menu β†’ Tools β†’ Advanced Options β†’ Speech Recognition:
    • Min speech duration: 3000 ms
    • Max speech duration: 10 sec
    • Check Whisper pre-split audio
    • Check Merge short subtitles
  3. If many subtitles are under 3s, use OmniVoice-TTS service.
  4. Use AI translation (DeepSeek / OpenAI ChatGPT), check Send full subtitle.
  5. Speech recognition: Chinese β†’ ByteDance STT Turbo / Qwen-ASR / FunASR / Firered Chinese; English β†’ Faster-whisper + large-v3, select Default segmentation.
  6. To re-embed background audio, click "More Settings" and check "Separate vocal/BGM". If not needed, check "Noise reduction".

FAQ ​

Q: Conditions for lossless video output? ​

Original video encoded as H.264 MP4 + Slow video unchecked + No hard subtitles + Encoding set to 264.

Q: Processing is very slow? ​

  1. Ensure GPU acceleration (CUDA) is enabled
  2. Use a smaller model
  3. Update GPU drivers

Q: How to use GPU acceleration? ​

Ensure NVIDIA GPU drivers, CUDA 12.8, and cuDNN 9.11 are installed. Check "CUDA Acceleration" on the main interface. AMD GPUs are not supported.