pyVideoTrans

Appearance

pyvideotrans Channel Extension Development Guide ​

This guide applies to the modular channel architecture introduced in the refactored pyvideotrans. Developers can quickly add custom Speech Recognition (STT), Subtitle Translation (Trans), or Text-to-Speech (TTS) channels by inheriting base classes and registering them properly.

1. Development Workflow Overview ​

Here is the standard workflow to integrate a new channel (using {name} as an internal identifier, such as newai):

text
[1. Constant Definition & Registration] 
    videotrans/{module}/_constants.py  --> Assign an ID and declare ChannelProvider
              ↓
[2. UI & Settings Configuration (if settings are required)]
    videotrans/ui/{name}.py             --> UI layout
    videotrans/winform/{name}.py        --> Window interaction logic (WinForm)
    videotrans/configure/_app_params.py --> Inject default parameters
    videotrans/ui/menu_list.py          --> Add to the top settings menu
              ↓
[3. Core Business Logic Implementation]
    videotrans/{module}/_{name}.py      --> Inherit the Base class and implement abstract methods
              ↓
[4. Voice Role Configuration (TTS only)]
    videotrans/configure/constant.py or videotrans/voicejson/{name}.json

Step 1: Define and Register Channel Constants ​

Module directories:

  • Speech Recognition: videotrans/recognition/
  • Subtitle Translation: videotrans/translator/
  • Text-to-Speech: videotrans/tts/

Open _constants.py in the corresponding directory:

1. Rules & Constraints ​

  • Unique ID: Add a new constant at the end of the list with an incremented number (e.g., NEWAI_API = 32).
  • Module Identifier {name}: Must start with a letter and contain only lowercase letters, numbers, and underscores (e.g., newai).
  • Implementation File: Create a new _{name}.py file in the same directory (e.g., _newai.py).

2. Register in ID_NAME_DICT ​

python
# videotrans/{module}/_constants.py

# 1. Add an incremented constant
NEWAI_API = 32

# 2. Register at the end of the ID_NAME_DICT dictionary
ID_NAME_DICT[NEWAI_API] = ChannelProvider(
    name="NewAI Channel",       # Display name in the dropdown menu
    key_name="newai_key",       # Required parameter for validation (checks if settings are configured)
    win="newai",                # Configuration window name (matches winform/{name}.py)
    imp="._newai"               # Path to dynamically import the implementation file (relative to current package)
)

About key_name: If the channel requires an API Key, Token, or Base URL, enter the name of the main required field here. If the user tries to use the channel without filling in this field, the system will intercept the request and show a warning popup.

Step 2: Settings Window & Menu Integration (Optional) ​

If your new channel does not require users to configure API keys or model parameters (e.g., a zero-config local model), you can skip this step.

1. Create UI Layout and Controller ​

  • UI Interface: Create videotrans/ui/{name}.py (defines inputs, save buttons, etc. We recommend copying and modifying an existing file like deepseek.py in the same folder).
  • Window Controller: Create videotrans/winform/{name}.py (handles displaying saved parameters, saving data, connection testing, etc.).

2. Inject Default Configuration Parameters (Optional) ​

Add default parameters in the _get_defaults() method in videotrans/configure/_app_params.py:

python
# videotrans/configure/_app_params.py
def _get_defaults():
    return {
        # ...Existing configuration items
        "newai_key": "",
        "newai_url": "https://api.newai.com/v1",
        "newai_model": "newai-v1",
    }

3. Add to the Menu Bar ​

Open videotrans/ui/menu_list.py and add your channel to the corresponding list based on its type:

  • Translation channels: Append to MENU_CFG_TRANS
  • TTS channels: Append to MENU_CFG_TTS
  • Speech Recognition channels: Append to MENU_CFG_STT

The menu item is a 3-element tuple: ("{name}", "Menu Display Label", None) (when the 3rd element is None, the system automatically finds and opens videotrans/winform/{name}.py):

python
# videotrans/ui/menu_list.py
MENU_CFG_TRANS = [
    # ...
    ("newai", "NewAI Settings", None),
]

Step 3: Implement Core Channel Logic ​

3.1 Subtitle Translation Channel (videotrans/translator/) ​

Create file: videotrans/translator/_{name}.py.

  • Reference examples: For OpenAI-compatible APIs, see _deepseek.py; for custom or standard HTTP APIs, see _google.py; for local models, see _hymt2.py.
python
from dataclasses import dataclass
from typing import Union
from videotrans.configure.config import ROOT_DIR, params
from videotrans.translator._base import BaseTrans

@dataclass
class NewAITrans(BaseTrans):
    """
    NewAI Translation Channel Implementation
    """

    def _item_task(self, data: str) -> str:
        """
        Core translation function (must be implemented)
        
        :param data: Text content to be translated
                     - If "Send full SRT" is checked: data is a multi-line plain text string in standard SRT format
                     - If unchecked / traditional channels: data is a multi-line string separated by newlines
        :return: Translated text string (must strictly match the format of the input)
        """
        api_key = params.get("newai_key")
        
        # Check exit signal
        if self._exit():
            return ""
        
        # TODO: Send HTTP request for translation
        translated_text = self._request_translate(api_key, data)
        return translated_text

    def _download(self):
        """
        Model download function (required for local models, can pass for API channels)
        Standard model storage path: f"{ROOT_DIR}/models/{model_name}"
        """
        pass

3.2 Speech Recognition Channel (videotrans/recognition/) ​

Create file: videotrans/recognition/_{name}.py.

  • Reference examples: For API recognition, see _openrouter.py; for lightweight local models, see _fireredasr.py; for memory-heavy or multi-process models, see _whisper.py.
python
from dataclasses import dataclass
from typing import List, Union
from videotrans.configure.config import ROOT_DIR, params
from videotrans.recognition._base import BaseRecogn
from videotrans.util.tools import SrtItem

@dataclass
class NewAIRecogn(BaseRecogn):
    """
    NewAI Speech Recognition Channel Implementation
    """

    def _exec(self) -> Union[List[SrtItem], None]:
        """
        Core recognition logic (must be implemented)
        :return: List of SrtItem populated with transcribed text, or None on error/exit
        """
        if self._exit():
            return None

        # self.cut_audio() automatically slices audio based on VAD
        # raws: List[SrtItem], each item contains fields like 'filename' (absolute path of the audio chunk), 'from_time', 'to_time', etc.
        raws: List[SrtItem] = self.cut_audio()
        
        for it in raws:
            if self._exit():
                return None
            
            # Call transcription logic to populate text
            audio_file = it["filename"]
            it["text"] = self._transcribe_file(audio_file)

        return raws

    def _download(self):
        """If a local model is required, download it to the {ROOT_DIR}/models directory"""
        pass

    def _transcribe_file(self, audio_path: str) -> str:
        # TODO: Actual audio chunk transcription logic
        return ""

3.3 Text-to-Speech Channel (videotrans/tts/) ​

After dubbing each subtitle line, you must reformat the audio file to 48000 Hz sample rate, stereo (2 channels), and pcm_s16le format. This ensures seamless audio alignment and concatenation later and prevents format mismatch errors.

You can call this helper method directly: self.convert_to_wav(dubbed_audio_file, data_item['filename'])

Create file: videotrans/tts/_{name}.py. TTS channels are categorized into two types based on how they run, each requiring a different core method:

Mechanism A: API-Based Streaming/Concurrent Requests (Must Implement _run) ​

Reference examples: _openrouter.py, _xiaomi.py.

python
from dataclasses import dataclass
from typing import Dict, List, Union
from videotrans.configure.config import ROOT_DIR, params
from videotrans.tts._base import BaseTTS

@dataclass
class NewAITTS(BaseTTS):

    def __post_init__(self):
        super().__post_init__()
        self.api_key = params.get("newai_key")
        self.speed = self.get_speed()  # Get speech rate set by the user in the UI

    def _run(self, data_item: Union[Dict, List, None], idx: int = -1) -> Union[str, None]:
        """
        Handle TTS for a single subtitle line
        
        :param data_item: Subtitle metadata dictionary
                          {
                              "filename": "/path/to/target.wav",  # Absolute path for output audio
                              "role": "RoleName",                 # Selected voice role in UI
                              "text": "Text to synthesize"        # Subtitle text
                          }
        :param idx: Subtitle index
        :return: Absolute path to the generated file on success, or None on failure
        """
        if self._exit():
            return None

        out_path = data_item["filename"]
        role = data_item["role"]
        text = data_item["text"]

        # TODO: Send TTS request and write to out_path
        self._generate_voice(text, role, out_path)
        return out_path

    def _download(self):
        pass

Mechanism B: Local Models with Local Weights (Must Implement _exec) ​

Reference examples: _omnivoice.py, _zipvoice.py.

python
from dataclasses import dataclass
from videotrans.configure.config import ROOT_DIR
from videotrans.tts._base import BaseTTS

@dataclass
class NewAILocalTTS(BaseTTS):

    def _exec(self):
        """
        Batch / queued local inference task
        All queued TTS tasks are stored in self.queue_tts
        """
        for item in self.queue_tts:
            if self._exit():
                break
            
            out_path = item["filename"]
            role = item["role"]
            text = item["text"]
            
            # TODO: Model batch inference
            self.model_infer(text, role, out_path)

    def _download(self):
        # Download weights to {ROOT_DIR}/models
        pass

5. TTS Voice Role Configuration Guidelines ​

Voice role lists in the UI dropdown can be configured in two ways:

1. Global Roles (Roles do not change with source/target language) ​

Define a comma-separated string of roles directly in videotrans/configure/constant.py, and return it in the role_menu() function in videotrans/util/help_role.py:

python
# videotrans/configure/constant.py
NEWAITTS_ROLES = "voice_a,voice_b,voice_c"

Design Principle: We recommend using the exact Voice ID expected by the API (e.g., zh-CN-YunxiNeural) as the role name. Avoid displaying human-friendly aliases in the UI that require a second mapping step, as this keeps maintenance simple.

2. Language-Specific Roles (Role list changes based on selected language) ​

Used when each language has its own distinct set of voices:

  1. Create a new file {name}.json in the videotrans/voicejson/ directory (e.g., newai.json).
  2. Use standard language codes referenced in EDGE_LANGUANGES_CODE from videotrans/configure/_languages_dict.py:
json
{
  "zh-cn": ["xiaoxiao", "yunxi"],
  "en": ["jenny", "guy"],
  "ja": ["nanami", "keita"]
}

6. Common Pitfalls & Best Practices ​

  1. Respond to Exit Signals (Crucial): In long-running loops or network requests, make sure to check if self._exit(): return frequently. Otherwise, the task will not stop immediately when the user clicks "Stop" on the main interface.
  2. Isolate Heavy Local Models: For local models that consume significant VRAM/memory or use PyTorch/C++ bindings (such as Whisper, Kokoro, OmniVoice), never load them directly in the main process. Instead, spawn them in an isolated subprocess (using multiprocessing or subprocess) as demonstrated in _whisper.py. This prevents GUI freezing and memory leaks.
  3. Standard Model Paths: Downloaded weights must always be saved under {ROOT_DIR}/models/{model_name}. Do not create relative paths directly in the current working directory.
  4. Automatic Download Retries: When downloading models, implement retry logic and resumable downloads to prevent failures caused by occasional network timeouts on unstable connections.
  5. SRT Structure and Alignment Risks: In translation channels with "Send full SRT" enabled, LLMs may sometimes break line breaks or drop timestamps. Always validate that the returned format and line count are correct before saving. If corrupted, fall back gracefully or show an informative error message.