FoleyCrafter
The facts below are what the maker publishes about FoleyCrafter, checked and dated. Our editors have not written about it yet.
The facts
Each one read from FoleyCrafter's own pages, with the date it was checked.
What it does
- Base model
- FoleyCrafter is built on a pretrained text-to-audio generator, and the project page says the T2A model remains fixed during the framework's training.checked 9 Oct 2026
- Checkpoints
- The repository says checkpoints download automatically when running `inference.py` and also documents manual downloads.checked 9 Oct 2026
- Code and models
- The project provides its models and code through its GitHub repository.checked 9 Oct 2026
- Dependencies
- The repository setup instructions use a Conda environment and Git LFS to download checkpoints.checked 9 Oct 2026
- Hosted demo
- The linked Hugging Face Space is paused, and its page says users can ask its authors to restart it.checked 10 Oct 2026
- How to run
- The repository documents a Gradio interface launched with `python app.py --share` and command-line inference through `inference.py`.checked 9 Oct 2026
- Implementation
- The framework has a Gradio interface and command-line inference usage.checked 10 Oct 2026
- Integrations
- The project is built upon Auffusion, CondFoleyGen, and SpecVQGAN.checked 10 Oct 2026
- Intended use
- The project describes sound effects as useful for cinema and gaming to enhance realism, impact, and emotional depth.checked 10 Oct 2026
- License
- The repository lists an Apache-2.0 license and advises commercial users to check the Auffusion license as well.checked 10 Oct 2026
- Prompt control
- Text prompts can be used to control video-to-audio generation according to user intent.checked 9 Oct 2026
- Prompt controls
- Text prompts support controllable video-to-audio generation, and the project demonstrates both positive and negative prompts.checked 10 Oct 2026
- Prompt options
- The command-line interface includes prompt and negative-prompt options, a random seed, temporal alignment controls, input and checkpoint paths, and an output path.checked 9 Oct 2026
- Purpose
- FoleyCrafter is a text-based video-to-audio generation framework that produces high-quality audio semantically relevant and temporally synchronized with input video.checked 10 Oct 2026
- Semantic alignment
- Its semantic adapter conditions audio generation on video features to produce sounds relevant to the visual content.checked 10 Oct 2026
- Setup
- The README provides a Conda environment setup and says model checkpoints can be downloaded automatically by running inference.py.checked 10 Oct 2026
- Sound generation
- The framework uses a pretrained text-to-audio model to generate sound effects for video.checked 10 Oct 2026
- Support
- The repository README lists contact email addresses for Yiming Zhang, YiCheng Gu, and Yanhong Zeng.checked 10 Oct 2026
- Temporal alignment
- Its temporal controller uses an onset detector and timestamp-based adapter to align audio with video.checked 10 Oct 2026
- Temporal synchronization
- Its temporal controller uses an onset detector and timestamp-based adapter for audio-video alignment.checked 9 Oct 2026
- What it does
- FoleyCrafter is a text-based video-to-audio generation framework that generates high-quality audio semantically relevant to and temporally synchronized with input videos.checked 9 Oct 2026
The company
- Makers
- The project page lists authors affiliated with Shanghai AI Laboratory and the Chinese University of Hong Kong, Shenzhen.checked 9 Oct 2026
Alternatives to FoleyCrafter
- 1
Resemble AI
Free planAndroidiOSWebAPI8.3From $280 a month - 2
ElevenLabs
Free planWebAPI7.6From $6 a month - 3
Morphic
Free planWeb7.4From $4 a month - 4
Sonilo
Free planAndroidiOSWebAPI7.4From $11.99 a month - 5
Creen
Free planAndroidiOSWeb7.3From $6 a month - 6
Mirelo
Free planWebAPImacOSWindows7.1From €20 a month