Caption Live

Production-grade captioning requires low-latency partial results, stable final segments for downstream translation, and a trust boundary that never ships vendor master keys to end users. This application implements that split: ephemeral Speechmatics realtime JWTs minted on the server, translation executed in Next.js route handlers, and presentation logic isolated in React hooks and typed services.

The client acquires microphone input via getUserMedia, then runs a Web Audio graph to produce 16 kHz mono floating-point PCM aligned with Speechmatics "raw" binary frames. Chunks are pushed over a realtime WebSocket using a configurable regional endpoint, with explicit stream lifecycle messages (start / end-of-stream) to bound sessions and recover cleanly.

• Short-lived JWT acquisition through a dedicated API route calling Speechmatics temporary-key infrastructure—no long-lived API key in client storage when the server is configured.
• Streaming partial and final transcript events, normalized into an append-only segment model for UI stability.
• Optional diarization profiles to separate speakers in the caption surface.
• Post-processing toggles including disfluency removal and configurable caption density / phrase buffering so operators can trade subtitle cadence against latency.

Each finalized transcript segment can be sent to POST /api/translate, which supports both Cloud Translation v2 (basic) and v3 (advanced) clients based on environment configuration. Source and target BCP-47-style language codes are validated against a curated picker; translation can be disabled when the target is "none" for pure captioning workflows.

• Dual-column live layout with source vs. translation ordering persisted per device.
• Density presets, scalable caption typography, and a fullscreen-friendly subtitle bar suitable for event projection.
• Settings surface for API connectivity, regional Speechmatics WebSocket host, and caption behavior—gated UX when neither server nor local Speechmatics credentials are available.

• IndexedDB archival of ended sessions for private replay on the same browser profile.
• ZIP export bundling transcript text, optional parallel translation file, AI-generated summaries, and a machine-readable manifest (timestamps, languages, segment counts).
• Server-backed publish workflow producing shareable read-only URLs under /share/[id].
• Optional AI session summaries (source and translated) via a dedicated API route, with idempotent client-side persistence back into the local session record.

A web app manifest and service worker registration enable install-to-homescreen flows and static asset caching appropriate for field deployments on tablets or dedicated captioning hardware—without compromising the realtime requirement that live audio still traverse the network during active sessions.

The codebase is structured as a Next.js App Router project with colocated Vitest coverage on transcript coalescing, phrase buffering, PCM helpers, and Speechmatics client protocol edges. Cross-cutting concerns—language metadata, speaker display strings, caption layout storage—live in small modules with explicit types for transcript segments and publish payloads, which keeps UI components thin and testable.

This page is the default route; the interactive studio runs under /live. Use the sidebar logo anytime to return here.