VYAAS AI
A voice assistant that runs on hosted and local models.
Why it exists
01I wanted to talk to a computer the way I talk to a person, in Hindi and English, without opening five different apps. Every assistant I tried was either locked to one model, slow at Hindi, or wrapped in an interface that felt like a chat box with a microphone attached.
VYAAS started as a way to separate the voice loop from the model layer. The same conversation can run through a hosted model when the internet is good, or a small local model when it isn't. That decision shaped the whole architecture.
Recorded 2025–2026
Kaushambi, India
The constraints
02- 01A mid-range Android phone and a laptop with no dedicated GPU.
- 02Free-tier API limits for speech and language models.
- 03School from early morning, so most work happened after 9 p.m.
- 04A target of roughly 1.5 seconds from end of speech to first audio response.
The approach
03Keep the voice loop separate from the model
LiveKit carries the audio room and the Python agent handles turn detection. The model is chosen behind a small adapter, so switching from a hosted model to Ollama does not rewrite the frontend.
Use Hindi TTS as a first-class path
Sarvam's Bulbul voices handle Hindi output, with an English fallback. This was not an afterthought translation layer; the prompts and the UI copy were written for both languages from the start.
Offer a local mode, with honest trade-offs
The offline path uses a quantised GGUF model through Ollama. It is slower on CPU and weaker at Hindi, but it keeps audio and text on the machine. The UI labels which mode is active.
What it actually does
04- 01Real-time voice conversation with interruptible responses.
- 02Switch between hosted and local models without changing interface.
- 03Hindi and English speech output with natural voices.
- 04Account login and conversation history.
- 05Local fallback when the network or API quota fails.
Technical notes
05The frontend is a Next.js app. A Python agent joins the LiveKit room, handles speech-to-text, calls the selected model, and streams speech back. Conversation state lives in Supabase; authentication uses Firebase.
The hardest bug was not the model. It was turn detection on cheap Bluetooth headsets, where the microphone keeps sending noise after the user stops speaking. A small energy gate and a longer silence window fixed most of it.
- Role
- Product, frontend, voice pipeline, backend integration
- Year
- 2025–2026
- Platform
- Web, Desktop
- Stack
- Next.js, React, Python, LiveKit, OpenRouter, Sarvam TTS, Supabase, Firebase Auth
- Status
- In progress
- Links
- Repository ↗ · Live demo ↗
Honest limits
06- 01Barge-in is still unreliable on Bluetooth headsets.
- 02The local model is noticeably weaker at Hindi and slower on CPU.
- 03The desktop packaging exists but is not a finished release.
- 04The public demo is web-first and may be rate-limited.
What it taught me
07“Real-time audio is 20% model work and 80% timing, buffering and error handling. I also learned to design the fallback before the happy path, because the happy path is the part that works least often.”