xxnuo/MTranServer

Offline translation model server with low resource consumption, fast speed, and private deployment capability. 低资源占用速度快可私有部署的离线翻译模型服务器

MTranServer – Offline Translation Server

What it is

  • A self‑hosted translation service that runs on a regular PC (no GPU required). It uses ONNX‑runtime models (derived from Mozilla’s translation models) to translate between several language pairs, most notably English‑Chinese and German‑English.
  • Provides a local HTTP API that mimics popular translation APIs (DeepL, Google Translate, etc.) so existing plugins and scripts can point to it.

Key features

  • Zero‑GPU, CPU‑only inference via ONNX Runtime; automatically picks the best CPU thread count and execution mode.
  • Cross‑platform desktop client (Windows, macOS, Linux) with a system‑tray UI and a built‑in Swagger UI for testing.
  • Docker support – can be run as a container with a simple docker compose file.
  • Plugin compatibility – offers drop‑in endpoints (/deepl, /google/..., /kiss, /imme, etc.) so VS Code, browser extensions, and other translation tools work without code changes.
  • Offline mode – after the first online download of a language pair, the service can run completely offline.
  • Command‑line control – flags for host/port, UI toggle, cache size, worker idle timeout, and explicit model download.

How to get it

  1. Desktop app – download the appropriate installer from the GitHub Releases page; it adds a tray icon and launches the server automatically.
  2. npm package – npm i -g mtranserver && mtranserver (or bunx, pnpm, etc.).
  3. Docker – pull xxnuo/mtranserver:latest and run with the provided compose.yml (bind a models and config directory).

Running the server

# quick start via npm
npx mtranserver@latest

# or with Docker
docker pull xxnuo/mtranserver:latest
docker compose up -d
  • The first time a language direction is requested, the server downloads the required ONNX model files (needs internet). Subsequent requests reuse the cached files.
  • Use mtranserver --help to see all options (host, port, UI, offline flag, cache size, etc.).

Ecosystem

  • MTranCode – VS Code/Cursor extension that translates code comments by calling http://localhost:8989.
  • MTranBrowser – a browser extension (still under development) that can also talk to the local server.
  • The server’s API is documented in API.md and exposed via a Swagger UI at http://localhost:8989 when the UI flag is enabled.

Performance (v5 ONNX)

  • On an Intel i5‑14400, English→Chinese short sentences average 11 ms (≈86 req/s) with 6 CPU threads; paragraphs take ~28 ms.
  • Compared to the previous 5.0.1 release, throughput roughly doubled (e.g., English→Chinese short‑sentence throughput ↑ 1.78×).
  • Memory footprint for a loaded English→Chinese model peaks around 392 MiB.

Why it matters

  • Provides a privacy‑preserving alternative to cloud translation services—your text never leaves your machine.
  • Works on modest hardware, making local translation accessible for developers, writers, and anyone who needs quick gist‑level translations without a GPU.

Links

Related

  • Project
    facebookincubator/AITemplate

    A Python framework that transforms deep neural networks into optimized CUDA/HIP C++ code for high-performance inference serving on NVIDIA and AMD GPUs.

  • Project
    OpenNMT/CTranslate2

    A C++ and Python library for efficient Transformer model inference, using quantization and custom runtimes to accelerate execution and reduce memory usage on CPU and GPU.

  • Project
    dusty-nv/jetson-containers

    jetson‑containers is a modular Docker‑based build system that supplies ready‑made, CUDA‑compatible AI/ML, robotics, and simulation packages for NVIDIA Jetson edge devices. It lets you pick frameworks (PyTorch, TensorFlow, LLM toolkits, ROS, etc.), build a single container with a single command, and run it with NVIDIA runtime defaults. Ideal for developers and researchers who need reproducible, high‑performance AI environments on Jetson hardware.

  • Project
    arrayfire/arrayfire

    A general-purpose tensor library that provides a high-level abstraction for parallel computing across CPUs, GPUs, and other hardware accelerators.