I built a real-time translation booth for a government service counter in the UAE. It needed to support 32 languages, deliver translated speech in under two seconds, and work reliably in a noisy public environment. Here is how it was architected.
The pipeline
Audio in → Speech Recognition → Text → Translation → Speech Synthesis → Audio out. Simple in a diagram. The hard part is doing all of this in under two seconds while handling interruptions, background noise, and 32 different language pairs.
Streaming everything
The key insight was to stream every stage. Do not wait for the full sentence before translating — start translating as soon as you have a phrase. The speech recognition model outputs partial transcripts. The LLM starts translating before the sentence ends. The TTS begins speaking as soon as it has a complete clause.
Directional audio
In a public booth, you do not want the translated audio leaking to everyone. I used beamforming microphone arrays and directional speakers to create an audio cone. The speaker hears the translation clearly; bystanders hear almost nothing.
What broke in production
The demo worked perfectly in a quiet office. At the government counter, the background noise from a busy hall broke the speech recognition. The fix was a noise gate combined with a fine-tuned acoustic model for the specific environment. Two weeks of iteration for what took two days to demo.
Cost per language pair
Running 32 languages means 992 possible pairs (31 × 32). I did not build 992 pipelines. Instead, every language translates through English as a hub. 31 recognition models in, one translation through English, 31 synthesis models out. Higher latency for uncommon pairs, but manageable infrastructure.