KeySync
KeySync is a free, open-source project for matching the lip movements in an existing video to new audio.
What KeySync is
KeySync is a free, open-source project for matching the lip movements in an existing video to new audio. Its two-stage framework is designed to improve timing consistency, while masking methods address expression leakage from the source video and facial occlusions. The project reports results for lip reconstruction and cross-synchronization, and introduces LipLeak as a measure of expression leakage. An interactive demo accepts video and audio uploads, but videos are limited to 6 seconds because of GPU restrictions on Hugging Face Spaces. For local installation, the listed prerequisites are a CUDA-compatible GPU, Python 3.11, and Conda. The quick start uses MP4 video and WAV audio. Provided scripts can prepare video at 25 fps and audio at 16000 Hz, and crop video around the face. Users can enable occlusion handling and specify coordinates for an object to mask. The repository provides instructions for downloading pretrained models and includes separate scripts for training keyframe and interpolation models. KeySync is licensed under Apache-2.0.
Good
- Free and licensed under Apache-2.0.
- Interactive demo accepts video and audio uploads.
- Includes masking options for facial occlusions.
- Scripts prepare video and audio inputs.
Watch out
- Online demo limits videos to 6 seconds.
- Local installation requires a CUDA-compatible GPU.
- Quick start uses MP4 video and WAV audio.
Who it suits
KeySync suits people seeking an open-source way to synchronize an existing video's lips with new audio. Local users need a CUDA-compatible GPU, Python 3.11, and Conda.
Verdict
KeySync offers an online demo and a local workflow with controls for occlusions and input preparation. The demo's six-second limit and the listed local prerequisites are important constraints to consider.
The facts
Each one read from KeySync's own pages, with the date it was checked.
What it does
- Architecture
- KeySync is a two-stage framework designed to improve temporal consistency in lip synchronization.checked 9 Oct 2026
- Demo limit
- The Hugging Face demo is limited to videos of up to 6 seconds because of GPU restrictions.checked 9 Oct 2026
- Evaluation
- The repository includes a LipScore metric for measuring synchronization quality between generated and ground-truth videos.checked 9 Oct 2026
- Expression leakage
- Its masking strategy is designed to reduce expression leakage from the input video.checked 4 Oct 2026
- Framework
- KeySync uses a two-stage framework to improve temporal consistency in lip synchronization.checked 4 Oct 2026
- Input formats
- The quick start guide uses MP4 video files and WAV audio files as inputs.checked 4 Oct 2026
- Input preparation
- The repository recommends converting video to 25 fps and audio to 16000 Hz for compatibility with the models.checked 9 Oct 2026
- Interactive demo
- The demo accepts video and audio uploads to create synchronized videos and is limited to videos up to 6 seconds by Hugging Face Spaces GPU restrictions.checked 4 Oct 2026
- Leakage and occlusion handling
- The framework uses a masking strategy to address expression leakage and facial occlusions.checked 9 Oct 2026
- License
- The GitHub repository identifies its license as Apache-2.0.checked 9 Oct 2026
- Local requirements
- The repository lists a CUDA-compatible GPU, Python 3.11, and Conda as prerequisites for local use.checked 9 Oct 2026
- Model weights
- The repository instructs users to download pretrained models from the toninio19/keysync Hugging Face repository.checked 4 Oct 2026
- Occlusion controls
- The command-line options include a setting to enable occlusion handling and coordinates for the object to mask.checked 4 Oct 2026
- Occlusion option
- The inference parameters include an option to mask objects that block the face.checked 9 Oct 2026
- Occlusions
- The framework includes a masking approach for handling facial occlusions.checked 4 Oct 2026
- Online demo
- The interactive demo accepts video and audio uploads to create synchronized videos.checked 9 Oct 2026
- Preprocessing
- Provided scripts can convert videos to 25 fps and audio to 16000 Hz, and crop videos around the face.checked 4 Oct 2026
- Pretrained models
- The repository provides instructions to download pretrained models from Hugging Face.checked 9 Oct 2026
- Purpose
- KeySync synchronizes lip movements in an existing video with new input audio.checked 9 Oct 2026
- Reported results
- The authors report improved lip reconstruction and cross-synchronization, and reduced expression leakage measured with their LipLeak metric.checked 9 Oct 2026
- Training
- The repository includes separate scripts for training the keyframe and interpolation models.checked 9 Oct 2026
Alternatives to KeySync
- 1
Sync Labs
Free planAndroidiOSWebAPI8.0From $5 a month - 2
Morph Studio
Free planiOSWeb7.8From $12 a month - 3
VEED
Free planAndroidiOSWebExtension7.5Free plan - 4
Muse
Free planOpen sourceLinuxmacOSSelf-hostedWindows7.4Free plan - 5
CHAMELAION LipSync API
Free planWebAPI7.3From €5 a month - 6
lipsyn.cc
Free planWebAPI7.3€11.99 one-time