Robust Video Matting

Robust Video Matting is a free, open-source project for separating people from video backgrounds. It best suits developers and technically confident creators who want to run or integrate a matting model; its real-time, no-input approach is compelling, while file conversion carries a practical speed penalty.

Overview

A recurrent neural network uses temporal memory across frames to perform human matting without additional inputs. That design is useful for continuous footage and live capture, where preparing guidance for each frame would get in the way. Background replacement is supported up to 4K, but this is a focused matting project rather than a general video editor.

The repository includes source code and pretrained models under GPL-3.0, with training and evaluation documentation for teams that want to adapt or integrate the model. The license makes the software free, though teams distributing it as part of another product should account for GPL-3.0 obligations. The project was developed at ByteDance Inc.

Key features

• Live matting: The project says it can process video in real time without extra inputs. A browser webcam demo lets users see it running live and visualize recurrent states, making it a useful starting point for evaluating the model.
• Two model sizes: MobileNetv3 is recommended for most uses. ResNet50 is larger and brings small performance improvements, so it is best reserved for cases where those gains justify the larger model.
• Broad model support: Implementations or exports cover PyTorch, TorchHub, TorchScript, ONNX, TensorFlow, TensorFlow.js, and CoreML. This gives developers several deployment paths, though compatibility depends on the target runtime: the ONNX models use opset 12 and are tested with CPU and CUDA backends.
• File conversion: The conversion API accepts a video or image sequence and can produce a composited video or PNG sequence, with optional alpha and foreground outputs. The provided conversion script is expected to run much slower than the headline model throughput because it lacks hardware video encoding and decoding and parallel tensor transfers.
• Try it on your footage: A Colab demo lets users test their own videos using a free GPU, while linked inference and training guides cover usage, training, and evaluation.

The README reports 172 FPS at HD and 154 FPS at 4K on an RTX 3090 with FP16, batch size 1, and frame chunk 1; it also reports 104 FPS at HD and 76 FPS at 4K on a GTX 1080 Ti. Those figures describe model processing, not the slower end-to-end conversion script, so they should not be treated as a promise of export speed.

Pricing

The Robust Video Matting plan costs nothing and includes GPL-3.0 source code and pretrained models. There is no paid tier or usage quota described for this project. The free software is a strong fit for developers who can run it in their own environment, but it does not come with a packaged consumer editing workflow.

Platforms

The project is marked for Android, API, iOS, Linux, self-hosted use, and web. CoreML models require iOS 13 or later, and CoreML does not support dynamic resolution, which can constrain iOS workflows. Android support is linked through a third-party NCNN C++ project, so it is a separate implementation path rather than one of the listed model exports.

Who it's for

Choose Robust Video Matting if you need human-focused video separation, want to integrate a model into software, or prefer a free self-hosted approach with several framework options. It also suits technically confident creators willing to use Colab or conversion tooling. Look elsewhere if you need a turnkey editing app, predictable fast batch exports, or flexible resolution handling on iOS.

Pros and cons

• Pro: Free source code and pretrained models under GPL-3.0 make experimentation and self-hosted use accessible.
• Pro: Real-time matting without additional inputs and support up to 4K suit live and high-resolution workflows.
• Pro: Multiple model formats, plus both webcam and file demos, offer several ways to evaluate or integrate the project.
• Con: The conversion script runs much slower than the reported tensor speeds, limiting the value of those figures for export planning.
• Con: CoreML's fixed-resolution limitation and iOS 13 minimum narrow compatibility for some Apple-device workflows.
• Con: Android relies on a third-party NCNN project, which adds another implementation to assess.

Alternatives

Video Background Removal Software and AI Video Background Removers are category roundups for comparing other options.

• Cutout.Pro is a freemium option across mobile, desktop, web, and API platforms; its monthly plan starts at $5 a month for 80 credits/month. Choose it if you prefer a paid credit plan to running an open-source model.
• BackgroundRemover is free and open source, with local installation through pip or source code. Choose it if you want a locally installable background-removal project rather than a video matting workflow.
• Bria Video Background Removal offers a free cloud plan with 100 generations, a 10 requests/minute limit, queue-based latency, and Discord community support. Choose it if a cloud API workflow and bounded free generation quota suit your needs.
• Pixelcut has a free tier with limited background removal and free exports without a watermark. Choose it if you want a consumer-oriented mobile or web option with limited free removal.
• VCam offers a free starter plan with a watermark, up to 500MB storage, and three workspace members. Choose it if those workspace and storage limits fit your needs.
• EZ-CorridorKey is a free desktop app for Linux, macOS, Windows, and self-hosted use. Choose it if a desktop workflow fits better.
• Knockout API has a recurring free tier capped at 30 images per month. Choose it if your workload is image-based and fits that quota.
• Transparent Background is a free open-source Python package with GUI and webcam support. Choose it if you want a Python-based background tool with those interfaces.

Verdict

Robust Video Matting is a strong choice for developers and technically capable creators who need free, human-focused matting with real-time processing and model exports for several frameworks. Its main advantage is the combination of no-input video matting and flexible integration; look elsewhere if you need a polished editing app or rely on the provided script for fast file conversion.