The problem
What needed
to change.
Voice, language-model and avatar components are difficult to assemble into a coherent real-time conversation pipeline.
Project story
A modular framework links speech detection, transcription, response generation, speech synthesis and avatar rendering through interchangeable handlers.
A modular framework links speech detection, transcription, response generation, speech synthesis and avatar rendering through interchangeable handlers.

About the project
A modular digital-human conversation platform that combines speech recognition, an AI language model, speech generation, and an animated avatar. Its architecture is intended to support a complete interactive conversation experience on a single PC, depending on the selected models and hardware.
Developers building virtual assistants, demonstrations, conversational kiosks, or avatar-based interfaces.
The problem
Voice, language-model and avatar components are difficult to assemble into a coherent real-time conversation pipeline.
The solution
A modular framework links speech detection, transcription, response generation, speech synthesis and avatar rendering through interchangeable handlers.
What we do
The capabilities and workflows that bring this project together.
Why we do it
Make AI interaction feel closer to speaking with a person, while allowing speech, model, and avatar components to be swapped rather than rebuilding the entire pipeline.
Inside the workflow
Microphone audio passes through voice activity detection and speech recognition; an LLM generates a response; text-to-speech and avatar components produce the spoken, animated reply. Handler folders include multiple speech and avatar options.
A closer look
A visual guide to the project’s scope. An application screenshot is not available for this entry yet.
Technology
Python, PyTorch, FastAPI, Gradio, WebRTC/aiortc, FastRTC, and modular VAD, ASR, LLM, TTS, and avatar handlers.
Project scope
An existing modular framework checkout. The folder does not by itself establish a particular client deployment or which model configuration is operational.
Keep exploring
From project to possibility