- Home
- Knowledge graph
- avatar_lipsync
avatar_lipsync
Open-source Flutter package that makes a face talk from one photo or a drawn face, lip-synced to streamed speech in real time.
avatar_lipsync is an open-source Flutter package I built for real-time lip sync: it makes a face talk from a single photo (or a drawn face), in sync with speech that arrives as a stream. I designed it for voice chatbots on realtime models, and the README has recipes for OpenAI Realtime and Google Gemini Live. It is a personal project I work on at night. This is an early version and it will keep improving.
Photo: @micaela_valente, used with permission. Right: the rig overlay, with the mesh and a live-motion heat map.
How it works
- Ahead of the sound. Each audio chunk is analysed before it plays: an FFT every 10 ms measures loudness, formants and sibilance.
- A JALI-style mouth model. The jaw follows loudness, the lips follow articulation, the lips close on m/b/p, and springs keep the motion natural.
- One photo is enough. MediaPipe Face Landmarker finds 478 face points in the browser, and a Delaunay mesh animates the photo.
- Alive. It blinks, nods and shifts its gaze.
![]()
Mouth shapes on "Portrait de Blanche Angers, actrice" (c. 1880), Paris Musées, CC0.
Status
Version 0.3.0 on pub.dev with 160/160 pub points, under the verified publisher azanello.com. A drawn face works on every platform and the photo mode on the web. Every tag push publishes to pub.dev from GitHub Actions via OIDC. MIT licensed. Links: live demo, where the photo lip-syncs to your microphone · pub.dev/packages/avatar_lipsync · GitHub.
Related
- Ale Zanello: that's me, author and maintainer
- Flutter · Dart · OpenAI · Google Gemini · GitHub Actions
- Akashi: my other open-source Dart/Flutter package
