Slicer translation workflow using crowdin

lassoan
Mon, 24 Jan 2022 19:30:33 +0000
Development

During the project week we explored crowding and various options for implementing a translation workflow. I’ve read up on various documentations and see how other projects do this and tested a couple of approaches and came up with the followings.
Requirements (click for more details)
Proposed implementation:

Create a separate repository: SlicerLanguagePacks

Stores .ts files (synchronized with crowdin’s github integration)

Contains an extension: LanguagePacks

Offers a module for helping translators (so that they can test out everything without waiting for nightly build actions and offline): update ts template files, download updated ts files, generate qm files, install qm files immediately.

Extension package contains compiled qm files of all extensions. Users can install latest translations via the extensions manager the same way as any other extensions. Later extension auto-update can ensure latest translations are installed.

During nightly builds on factory:

Extract translatable strings into source .ts files from Slicer core and all extensions

Commit source .ts files into SlicerLanguagePacks repository

Trigger crowdin synchronization (may just wait for the automatic synchronization): this uploads the updated source .ts files from SlicerLanguagePacks to Crowdin and creates a git pull request to update localized .ts files in SlicerLanguagePacks

Merge Crowdin’s SlicerLanguagePacks pull request

Compile localized .ts files are compiled into .qm files

Include Slicer core qm files into Slicer installer package. The qm files included in the Slicer core install package will be overridden by translations downloaded via SlicerLanguagePacks extension.

Include latest Slicer core qm files and extensions qm files into SlicerLanguagePacks extension.

Rationale (click for more details)
Questions:

How to translate Python scripted module’s .py files and CLI module descriptor XML files?

Option A. Pre-process the .py and .xml files to create intermediate files that Qt’s lupdate tool can parse

Option B: Generate .ts files directly. For example, there are tools that can extract text from _() function calls in Python files. We could probably create an XSL transform that would create a .ts file from the module descriptor .xml file directly (very similarly to how .md files are generated from these files).

How to organize the .ts file and folder structure (what to put in a .ts file, how to organize .ts files into folders)?

Option A: Create a separate .ts for each module. There are 150 modules in Slicer core and there are about 200 extensions, each containing typically a couple of modules. A new folder can be added for each extension. Challenges: There would be many files and the file list would change as modules are added/removed/renamed. It is not always clear which module provides text for a specific message on the GUI. It is slightly more complicated to generate per-module .ts files without building the application (creating .ts files without requiring building the application is useful, as it makes continuous/on-demand translation updates much simpler).

Option B: Create a single .ts file for Slicer core, and a separate .ts file for each extension. The Slicer core translation file is about 700KB, which is not too big. Challenges: Slicer core uses about 2600 terms, so it cannot be browsed, only filtered/searched. It is not possible to see percentage completion per module or meaningfully use Crowdin’s priority feature (you can specify low/normal/high priority for each file).

Option C: Have a mix between options A and B - modules could be organized into categories (such as module categories) to have a few dozens of files for Slicer core instead of 150. It would probably still make sense to have a single .ts file per extension.

Any comments and suggestions are welcome. I also hope we can discuss this tomorrow at the Slicer Weekly meeting.

5 posts - 3 participants