Multimodal Pronunciation Analysis: Scoring Vowels and VAD under the Hood
A technical overview of how nthoughts utilises multimodal AI models to evaluate speech clarity without expensive third-party speech assessment APIs.
July 18, 2026 · 7 min read
Editorial & Research Faculty
Ankit Kumar
Co-Founder & ML Systems
Ankit is a former speech synthesis research engineer. He builds and maintains the multimodal AI audio evaluation pipelines that power nthoughts' pronunciation scoring.
MS in Computer Science & Speech Signal Processing, IIT MadrasBS in Electrical Engineering, NIT TrichyAuthor of 6 Peer-Reviewed Papers on Acoustic Modeling & TTS Latency
Background & Academic Research
Ankit Kumar is an audio systems engineer and machine learning practitioner with deep expertise in neural speech synthesis, acoustic modeling, and low-latency client-side audio pipelines. Before co-founding nthoughts, Ankit spent seven years architecting speech evaluation engines at top audio research labs, where he built real-time phoneme alignment tools and browser-based Web Audio DSP architectures.
Having grown up bilingual in Chennai and speaking Tamil and English natively, Ankit personally experienced the cognitive friction of switching between Dravidian agglutinative sentence structures and English syntax during high-stakes technical standups. This lived experience led him to design automated systems capable of recognizing language transfer mistakes without introducing awkward conversational latency.
At nthoughts, Ankit leads the ML and audio infrastructure team. He oversees the sub-300ms audio streaming pipeline that links the browser Web Audio API to Maya's ElevenLabs voice engine, ensuring conversational turn-taking feels immediate, fluid, and natural.
Guiding Pedagogical & Technical Philosophy
"Conversational practice fails the moment audio latency exceeds 500 milliseconds. If the AI pauses too long before answering, the human brain disengages and the illusion of a genuine conversation collapses. Speed, precision, and privacy are non-negotiable prerequisites for calm learning."
Areas of Expertise
- •Real-Time Speech-to-Text & Sub-Second Turn-Taking Latency
- •Web Audio API & Browser Digital Signal Processing (DSP)
- •Acoustic Phonetic Alignment & Syllabic Stress Analysis
- •Privacy-First Audio Processing & Ephemeral Stream Pipelines
- •LLM Guardrails for Pedagogical Linguistic Remediation
Contributions to nthoughts
- •Engineered nthoughts' sub-300ms turn-taking streaming architecture.
- •Built the local-first audio transcription pipeline with zero raw audio retention.
- •Created the real-time grammar clause isolation parser.
- •Designed the low-bandwidth mobile web voice stream protocol.