Back to projects
Real-Time Speech AI (ASR / TTS)
Abstract.A speech interface for a client's internal data system, built at a corporate R&D lab.
- Role
- AI Engineer
- Period
- Mar 2024 to Mar 2025
- Context
- Electronics manufacturer, R&D lab (via Human Resocia)
- Team
- 5
| int8 | CTranslate2 inference for low latency |
| Hands-free | Register, edit and delete by voice |
| Integrated | Into the client's existing application |
01From problem to approach
The problem
Staff needed to register, edit and delete records without a keyboard, and recognition had to feel instant.
The approach
Run Faster-Whisper on int8 CTranslate2 for low latency, map transcripts to actions with string matching, and confirm every action out loud.
02How it works
- Input Voice input The user speaks a command; Streamlit records live audio.
- Step 02 ASR · Faster-Whisper Speech to text; int8 CTranslate2 keeps inference fast and light.
- Step 03 Intent matching String matching maps the transcript to a data action.
- Step 04 Data action Register, edit or delete a record, entirely hands-free.
- Step 05 TTS · VITS Synthesizes a spoken confirmation of the result.
- Output Audio response Auto-plays the response, closing the loop.
03What I built
- Shipped a hands-free voice data-entry system (register, edit and delete by voice) with Faster-Whisper ASR on int8 CTranslate2 for low-latency recognition and VITS text-to-speech read-back.
- Raised input accuracy with string matching and spoken confirmation.
- Built a Streamlit UI with live recording and auto-play, and resolved audio-library failures under WSL.
- Integrated the system into the client's existing application.
04Technologies
- Python
- PyTorch
- Faster-Whisper
- CTranslate2
- VITS
- ESPnet
- Conformer
- FastSpeech2
- HiFi-GAN
- Parallel WaveGAN
- Streamlit