Feasibility summary
Indriya completed an initial feasibility study of Moshi-based spoken interaction. It examined the available adaptation workflow and local inference path, focusing on conversational timing, interruptions, spoken corrections and recovery from misunderstood instructions.
The work established a prototype environment for controlled interaction experiments and identified data and evaluation requirements for further fine-tuning. It remains exploratory research. Broader conversational validation and pilot readiness are subsequent milestones.
1. Interaction beyond the transcript
A spoken system has to contend with when a person speaks as well as what they say. An interruption, a correction or a pause can change the next useful response. Transcribing the words alone does not describe the whole interaction.
The feasibility study examined these questions using Moshi, a speech-text foundation model and full-duplex spoken-dialogue framework from Kyutai. The upstream project provides tools for controlled spoken interaction.
The scope was to establish an experiment environment, examine the adaptation workflow and identify what further work would require.
2. Controlled interaction experiments
The prototype environment supports controlled experiments around conversational timing, interruptions and spoken corrections. Recovery after a misunderstood instruction was another focus. These situations give the next phase specific behaviors to investigate.
Further evaluation will need to document interaction scenarios, timing measurements and review criteria. That record will make it possible to compare behavior across trials, rather than relying on a selected interaction.
Further validation needs to assess whether a behavior remains consistent across representative interactions. A single successful correction or interruption is not a sufficient description of that consistency.
3. Requirements for further adaptation
The study examined the available adaptation workflow and identified data and evaluation requirements for further fine-tuning.
For the Moshi track, the established assets are the prototype environment and the feasibility work. The next phase will require suitable interaction data, review criteria and a bounded evaluation design.
Human oversight governed research design and quality assessment across the programme. AI-assisted engineering helped establish and document the environment; decisions about data acceptance and interpretation of outcomes remained with human reviewers.
4. Local inference and deployment questions
The study examined the local inference path. Moshi’s upstream project includes an MLX implementation for macOS, which provides a relevant route for local experiments. The broader Indriya programme investigated Apple Silicon inference alongside AWS GPU training and controlled evaluation.
Deployment preparation included runtime compatibility, checkpoint conversion, memory profiling and assessment of accuracy after compression. The selected runtime and device conditions will need to accompany later performance findings.
Timing and recovery need to be assessed in the actual runtime as well as in the model. The planned wider validation therefore remains a separate milestone from establishing a working experiment environment.
5. The next research phase
The feasibility work gives the next phase a defined set of questions: conversational timing, interruption handling, correction and recovery. The next phase will evaluate these behaviors across a wider range of conversations.
The initial six-month programme also produced multilingual transcription and speaker-attributed conversation-analysis prototypes.
Speech understanding and spoken output
Ongoing comparisons include NVIDIA and Google multimodal models alongside Apple AFM 3 work on the cloud Mac. Nemotron 3 Nano Omni and the audio-capable Gemma variants produce text; a spoken response needs an additional synthesis path. NVIDIA Magpie is a specialist synthesis candidate.
Apple describes Core Advanced capabilities including dictation and expressive voices. The interface available to an application must be established on the selected operating system; a system feature alone does not establish equivalent framework access. Compare complete interaction pipelines for latency and interruptions.
References and publication status
References checked on 14 October 2026.