When a language is translated into another and dubbed, due to differences in syllable count and grammatical structure, the duration inevitably changes, naturally causing issues with subtitle, voice, and video synchronization. This is a normal phenomenon.
Specifically, it manifests as: the original video character has finished speaking, but the dubbing is only halfway through; or the next sentence in the original video has already started, but the dubbing is still on the previous sentence.
Example: Translation Leading to Character Count Changes
Translating the following Chinese sentences into English and German results in significant changes in length and syllable count, which inevitably alters the corresponding audio duration:
Chinese: 得国最正莫过于明
English: There is no country more upright than the Ming Dynasty
German: Der gerechteste Weg, ein Land zu erobern, führt über die Ming-Dynastie.
Chinese: 我一生都在研究宇宙
English: I have been studying the universe all my life
German: Ich habe mein ganzes Leben dem Studium des Universums gewidmet.
Chinese: 北京圆明园四只黑天鹅疑被流浪狗咬死
English: Four black swans in Beijing's Yuanmingyuan Garden suspected of being bitten to death by stray dogs
German: Vier schwarze Schwäne im Yuanmingyuan-Park in Peking stehen im Verdacht, von streunenden Hunden getötet worden zu sein.
As shown by the examples above, when Chinese subtitles are translated into other languages, the sentence structure and syllable count change. Reading these sentences in the corresponding language will inevitably result in unequal durations. Generally, Chinese is shorter, English is longer, and German is even longer. Moreover, even for the same sentence, using different voice actors for dubbing will produce different durations, as some characters speak faster and others slower.
The video footage is fixed, but the dubbing duration changes, naturally causing an audio-video mismatch. To solve this problem, several strategies are typically employed:
How to Achieve Synchronization After Translation and Dubbing?
Increase Dubbing Speed/Audio Acceleration: Theoretically, as long as there is no upper limit on speech rate, matching the voice duration to the subtitle duration can always be achieved. For example, if the original voice duration is 1 second and the dubbing duration is 3 seconds, increasing the dubbing speed to 300% can synchronize them. However, this method makes the voice sound rushed and unnatural, fluctuating in speed, leading to a less-than-ideal overall effect.
Condense the Translation: Reduce the dubbing duration by shortening the translated text. For example, translate "我一生都在研究宇宙" into the more concise "Cosmology is my life's work." While this method yields the best effect, it requires modifying subtitles sentence by sentence, making it very inefficient.
Adjust Silence Between Subtitles: If there is silence between two subtitle segments in the original audio, reduce or remove some of that silence to compensate for the duration difference. For instance, if there is a 2-second silence between two subtitle segments in the original audio, and the first translated subtitle is 1.5 seconds longer than the original, the silence can be shortened to 0.5 seconds, aligning the dubbing time of the second subtitle with the original audio time. However, not all subtitle segments have enough silence to adjust, limiting the applicability of this method.
Remove Silence Before and After Dubbing: Typically, some silence is retained before and after dubbing. Removing this silence can effectively shorten the overall dubbing duration.
Slow Down Video Playback/Video Slow Motion: If simply speeding up the dubbing is unsatisfactory, consider combining it with video slowdown. For example, if an original subtitle segment has a voice duration of 1 second, but the dubbing becomes 3 seconds. We can shorten the dubbing duration to 2 seconds (1x speed increase) while simultaneously reducing the playback speed of the corresponding video segment to half (extending the duration to 2 seconds), thus achieving synchronization.
Each of these methods has its pros and cons, and none can perfectly solve all problems. Achieving optimal synchronization often requires fine manual adjustment, which contradicts the goal of software automation. Therefore, video translation software typically combines the above strategies to achieve the best possible result.
Implementation in Video Translation Software
In the software, these strategies are usually controlled through the following settings:
- Main Interface Settings:
Dubbing Speedup: Used to automatically increase the dubbing duration to match the subtitles;
Video Slowdown: Used to automatically reduce the video playback speed to match the dubbing duration;
Remove Inter-Subtitle Silence: When neither 'Dubbing Speedup' nor 'Video Slowdown' is selected, you can optionally remove silence between subtitles.
Align Subtitles and Audio: When neither 'Dubbing Speedup' nor 'Video Slowdown' is selected, the audio and subtitles will be out of sync. Selecting this option forces the subtitles and audio to align.
Dubbing Speed: Used to accelerate the overall dubbing speed;
Secondary Recognition: If embedding a single subtitle track, select this. It will perform a second voice transcription on the dubbed file to create subtitles, ensuring precise alignment between subtitles and dubbing.
- Advanced Options Settings (Menu Bar - Tools/Options - Advanced Options - Subtitle/Audio/Video Alignment):
Maximum Audio Speedup Factor/Video Slowdown Factor limits the extent of speedup and slowdown to prevent voice distortion or excessively slow video playback.
Translate Only One Video at a Time: After dubbing, an editing dialog will pop up. In this dialog, you can view the actual dubbing duration and the offset relative to the original audio (especially if it exceeds the original duration). You can manually modify and adjust the subtitle text before re-dubbing to get as close to the original audio duration as possible.
By flexibly using the above settings, video translation software can automate the process of synchronizing subtitles and dubbing as much as possible, improving translation efficiency.