pyVideoTrans

Appearance

OmniVoice is an advanced zero-shot multilingual TTS model that supports over 600 languages. It can generate high-quality speech with exceptional inference speed and supports voice cloning.

Starting from pyVideoTrans v3.98-0403, you can use OmniVoice for dubbing. The following describes how to deploy OmniVoice from source and download the OmniVoice integration package.

Download the OmniVoice Integration Package for Windows ​

Baidu Netdisk download: https://pan.baidu.com/s/1e6nHkq69TvUQe5VE02V-eg?pwd=1234

huggingface.co download: https://huggingface.co/mortimerme/repocollect/resolve/main/omnivoice-0528.7z?download=true

  1. After downloading the .7z file, extract it, then double-click start.bat. Wait for it to start successfully. The success indicator is when the interface shown below appears and stays.

On the first start, the model will be downloaded. By default, it will download from the domestic mirror site https://hf-mirror.com. The official site cannot be accessed directly from China, so you need a VPN (https://huggingface.co)

  1. Open the pyVideoTrans software, go to Menu -- TTS Settings -- OmniVoice-URL, and enter http://127.0.0.1:8081. Then, in the character list below, fill in the reference audio you want to use and its corresponding text.

The reference audio refers to the wav file you want to use for voice cloning. The ideal duration is 3-10 seconds, with clear pronunciation.

For example, if you have an audio file nverguo.wav containing the clear voice of the daughter of the king, place it in the f5-tts folder under the pyVideoTrans software. Then, in the settings interface, fill in the corresponding text content for this audio file in the character text box.

nverguo.wav#Fill in the spoken text content here

  1. Click Test. If it prompts OK, save and start using it.

Deploy OmniVoice from Source on MacOSX/Linux ​

  1. Click to go to the OmniVoice official code repository to download the source code. After opening, click the green <> Code button in the middle of the homepage, then click Download ZIP. This will download a zip file. Extract it to your desired folder. Then click into OmniVoice-master to enter the folder containing pyproject.toml.
  2. Download uv, open the terminal command line, and execute the uv installation command curl -LsSf https://astral.sh/uv/install.sh | sh

For Windows, you can click to download the uv compressed package and extract it. Place uv.exe together with pyproject.toml.

  1. Open the terminal in the folder containing pyproject.toml. For Windows, you can type cmd in the folder address bar and press Enter to automatically open it. Then execute the command uv sync to install the modules. Domestic installation may be slow; you can use the following command to speed up installation: uv sync --default-index "https://mirrors.aliyun.com/pypi/simple"

  2. Download the model. The model is hosted on https://huggingface.co, which is not directly accessible from China. If you don't have a VPN, set up a mirror as follows.

Enter the following command in the terminal to enable the model download mirror: For Windows cmd, type set HF_ENDPOINT=https://hf-mirror.com and press Enter. For MacOSX and Linux, type export HF_ENDPOINT=https://hf-mirror.com.

  1. Start the interface by executing the command uv run omnivoice-demo --ip 0.0.0.0 --port 8081 in the terminal.
  2. Open the pyVideoTrans software, go to Menu -- TTS Settings -- OmniVoice-URL, and enter http://127.0.0.1:8081. Then, in the character list below, fill in the reference audio you want to use and its corresponding text.

The reference audio refers to the wav file you want to use for voice cloning. The ideal duration is 3-10 seconds, with clear pronunciation.

For example, if you have an audio file nverguo.wav containing the clear voice of the daughter of the king, place it in the f5-tts folder under the pyVideoTrans software. Then, in the settings interface, fill in the corresponding text content for this audio file in the character text box.

nverguo.wav#Fill in the spoken text content here

  1. Click Test. If it prompts OK, save and start using it.

Using OmniVoice on Google Colab ​

You can also open this link to use it directly on Colab.

Then click Runtime > Run all at the top. Wait for the bottom cell to display a URL like https://xxxxxxxx.gradio.live. Click to open it in a browser to use. Copy this address to pyVideoTrans Menu--TTS Settings--OmniVoice-URL to use it in the software.

Frequently Asked Questions ​

  1. After extracting the source code, I can't see pyproject.toml?

After extraction, a folder named OmniVoice-master will be displayed. Click into it, and you will see pyproject.toml or pyproject. All operations need to be performed in this folder.

  1. Prompt uv is not recognized as an internal or external command?

This means uv is not installed or is not in the system environment. For Windows, click to download the uv compressed package and extract it. Copy uv.exe and paste it into the directory where pyproject.toml is located.

For other systems, simply execute the command to install: curl -LsSf https://astral.sh/uv/install.sh | sh

  1. The model download is very slow and eventually fails?

By default, it downloads from the original model site, which is inaccessible from China. You either need a VPN (ensuring access to https://huggingface.co) or set up a model mirror.

On Windows, execute the command set HF_ENDPOINT=https://hf-mirror.com in the terminal.

On MacOSX and Linux, execute export HF_ENDPOINT=https://hf-mirror.com.

  1. uv sync execution fails?

This is the module installation process. By default, it also downloads from abroad. You can try using the domestic Aliyun mirror source. Use uv sync --default-index "https://mirrors.aliyun.com/pypi/simple" to replace the uv sync command.

  1. It still fails even with the Aliyun mirror source?

Your computer may be missing the MSVC environment. Please click here to install the MSVC environment according to the plan.

  1. How to directly use OmniVoice for dubbing on the web page?

After startup is complete, you can directly open http://127.0.0.1:8081 in your browser.

  1. How to modify the startup port and model mirror in the integration package?

Open start.bat with Notepad. If you do not want to use the domestic download mirror, simply delete the line set "HF_ENDPOINT=https://hf-mirror.com".

If you want to modify the startup port, directly modify the 8081 in set "PORT=8081" to your desired port.