Architecture

This page is for users and contributors who want to understand the implementation. Module responsibilities, dependency direction and the error model in more detail are in docs/architecture.md in the repository.

Pipeline

altgo's voice processing is a linear pipeline driven by key events:

Key listener → state machine → recorder → transcriber → polisher (optional) → clipboard + overlay + history

The whole pipeline runs on a dedicated OS thread with its own current_thread tokio runtime, isolated from Tauri's main runtime. The main loop waits on five kinds of events with tokio::select!: key events, state machine timeouts, recordings waiting for retry, the idle reclaim tick and the stop signal. Transcription and polishing run sequentially inside the main loop, which keeps a single transcription in flight. See ADR-0003.

History stores text only, with the raw transcript recorded as rawText and the currently displayed text as text. Audio is never stored. The data file is ~/.config/altgo/history.json. A successful transcription appends one record and broadcasts a history-updated event, which refreshes the history list on the home page.

Key state machine

The state machine is a pure synchronous module. It only returns StartRecord or StopRecord commands, and the main loop performs the side effects.

State	Entered when	Left when
Idle	Idle	A press moves to PotentialPress
PotentialPress	The key is pressed	The long press threshold moves to Recording, an early release moves to WaitSecondClick
Recording	Long press triggered	Release stops and returns to Idle
WaitSecondClick	Short press released	A second press inside the double press interval moves to ContinuousRecording, a timeout returns to Idle
ContinuousRecording	Double press triggered	Another press stops and returns to Idle

The long press threshold defaults to 200 ms, the double press interval to 300 ms and the minimum press duration to 100 ms. A press shorter than the minimum counts as IME chatter and is dropped.

Modules

Main modules under the Tauri core directory src-tauri/src/:

src-tauri/src/
├── main.rs                 # thin entry point, calls lib::run()
├── lib.rs                  # Tauri assembly, managed state and pipeline thread
├── cmd.rs                  # IPC commands
├── config.rs               # TOML config loading and validation
├── config_store.rs         # active config and patches
├── history.rs              # history JSON
├── state_machine.rs        # key state machine
├── pipeline_controller.rs  # pipeline lifecycle
├── voice_pipeline/         # main loop, orchestration, dispatch and sink
├── transcriber.rs          # Transcriber trait
├── sherpa.rs               # local sherpa-onnx engine
├── online_asr.rs           # online ASR, dispatched by protocol family
├── polisher.rs             # Polisher trait and remote implementation
├── polisher/               # local polishing, fidelity check, prompts and protocol details
├── polish_runtime/         # llama-server child process lifecycle
├── polish_model.rs         # local polishing model catalog
├── model.rs                # transcription model catalog and validation
├── download.rs             # shared download core
├── prompt_store.rs         # polishing prompt templates
├── key_listener/           # key listening
├── key_capture/            # key capture for Settings
├── recorder/               # recording
├── output/                 # clipboard and text injection
├── overlay/                # overlay window management and phases
├── tauri_sink.rs           # pipeline events to the frontend
├── updater.rs              # automatic updates
├── display_backend.rs      # XWayland switch for Wayland sessions
└── audio.rs / error.rs / resource.rs

Transcription engine

Transcription is abstracted behind the Transcriber trait, and [transcriber] backend decides which implementation runs.

The local implementation is SherpaTranscriber, which embeds sherpa-onnx. The model loads once when the pipeline starts and stays in memory. The model registry provides SenseVoice, SenseVoice Cantonese-enhanced and FireRedASR2 CTC. Inference is CPU-bound and synchronous, and runs on the blocking thread pool through spawn_blocking.

The online implementation is OnlineAsr, which dispatches between three protocol families according to [transcriber.online] protocol:

Protocol	Request shape	Text source
mimo_chat	chat/completions with the WAV as base64 in a single input_audio block and asr_options	choices[0].message.content
chat_input_audio	Same chat shape without asr_options, with the language as a top-level field when set	choices[0].message.content
transcriptions	OpenAI /audio/transcriptions multipart	the top-level text field

Online transcription does not retry. A failure is reported directly and never falls back to local. See ADR-0013.

Both backends accept WAV bytes at 16 kHz, mono, 16-bit PCM. The recorder always produces this format.

Polishing engine

Polishing is abstracted behind the Polisher trait and is off by default. When the level is not none, [polisher] backend selects the remote or local implementation.

The remote implementation LLMFormatter supports the OpenAI-compatible chat API and the Anthropic Messages API, with exponential backoff retries. The local implementation LocalPolisher talks to a llama-server child process. The runtime and models are downloaded on demand, started lazily and reclaimed after ten idle minutes.

Local output passes a fidelity check before it is used. Numbers, URLs or paths changed, the text reduced to a summary, content added, or formatting markers introduced all count as failures and fall back to the raw transcript. See ADR-0011.

Platform strategy

Linux calls system tools as child processes: xinput or evtest for key listening, parecord for recording, and xclip, xsel or wl-copy for the clipboard. Wayland sessions switch to XWayland before the GUI initialises, otherwise the overlay cannot be positioned.

Windows uses native APIs: a WH_KEYBOARD_LL low-level keyboard hook for key listening, cpal/WASAPI for recording, arboard for the clipboard and SendInput to optionally inject text into the focused window. Text injection is off by default.

Platform capabilities sit behind traits, and each module implements them in files named linux.rs or windows.rs. The pipeline only consumes Box<dyn Trait> and stays independent of the platform.

Frontend and IPC

The frontend is a React application with three windows: the main window, the overlay and the update notes window. The main window hosts the home and settings pages, the overlay shows only the recording, processing and result phases, and the update notes window displays categorised changes for a new version.

The frontend and backend communicate over Tauri IPC. IPC and history.json use camelCase, while altgo.toml uses snake_case, and serde converts at the boundary. The frontend receives pipeline status, transcription results, polishing failures, history updates, download progress and other changes as events.

Further reading

• docs/architecture.md: module responsibilities, dependency direction, error model and IPC contract
• docs/testing.md: test layers, test doubles and CI coverage
• docs/adr/: architecture decision records