semdup
semdup is an open-source fuzzy-search tool for finding near-duplicate code by semantic meaning rather than token sequences.
What semdup is
semdup is an open-source fuzzy-search tool for finding near-duplicate code by semantic meaning rather than token sequences. It scans repositories and is available for Linux, macOS, and Windows, with a self-hosted option. The default embedding model runs locally on CPU or CUDA; the project says data stays local unless an external backend is selected. Supported languages include Rust, TypeScript, Python, Go, Java, C#, PHP, Ruby, C, and C++, though heavily macro-based C/C++ may perform less well. Users can tune similarity thresholds, set separate warning and error levels, and require a chosen number of matches. A provided GitHub Actions workflow and action can run checks on pull requests and pushes. The default model is nomic-ai/CodeRankEmbed; compatible models and a Python sidecar are options for other backends. An optional Amazon Bedrock backend uses Titan Text Embeddings V2 and requires the AWS SDK credential chain and bedrock:InvokeModel permission. Initializing, refreshing, or scanning downloads the model into a user cache and verifies it against compiled BLAKE3 hashes. The project is licensed under MIT or Apache-2.0.
Good
- Detects near-duplicates by semantic meaning.
- Default inference runs locally on CPU or CUDA.
- Supports ten listed programming languages.
- Includes GitHub Actions workflow and action.
- Available under MIT or Apache-2.0.
Watch out
- Fuzzy search can produce false positives.
- Sparse search may miss pairs found by exact search.
- Heavily macro-based C/C++ may perform less well.
- First use downloads a model into the user cache.
Who it suits
semdup suits developers looking for repository-level semantic duplicate checks, including in pull request workflows. It may also suit users who want local inference and configurable matching thresholds.
Verdict
semdup offers configurable semantic duplicate checks with local inference and CI integration. The project cautions that it is young and may contain bugs, and its sparse search can miss matches that exact search finds.
The facts
Each one read from semdup's own pages, with the date it was checked.
What it does
- Bedrock integration
- An optional Amazon Bedrock backend uses Titan Text Embeddings V2 through the AWS SDK credential chain and requires bedrock:InvokeModel permission.checked 4 Oct 2026
- Caveat
- The project says it is still young, may have bugs, and can produce false positives because it uses fuzzy search.checked 9 Oct 2026
- Code review
- The CLI includes a diff mode that compares changed code against a Git base reference, and the project provides a GitHub Actions workflow and action.checked 9 Oct 2026
- Configuration
- Users can configure similarity thresholds, warning and error thresholds, and the minimum number of duplicates in a cluster.checked 9 Oct 2026
- Download integrity
- The first initialization, refresh, or scan downloads the model into a user cache and verifies files against compiled BLAKE3 hashes before loading them.checked 4 Oct 2026
- Index limitation
- Sparse search is approximate and can miss pairs found by exact search, while exact search compares every pair and avoids dropping matches.checked 4 Oct 2026
- Known limitation
- The maker says the tool is still young, may contain bugs, and can produce false positives because it uses fuzzy search.checked 4 Oct 2026
- Languages
- It supports Rust, TypeScript, Python, Go, Java, C#, PHP, Ruby, and C/C++, with a note that heavily macro-based C/C++ may perform less well.checked 9 Oct 2026
- License
- The project is available under either the MIT or Apache-2.0 license.checked 9 Oct 2026
- Local inference
- The default model runs on the user’s machine using CPU or CUDA.checked 9 Oct 2026
- Maintainer support
- The maker welcomes contributions to the project.checked 4 Oct 2026
- Matching controls
- Users can tune similarity thresholds, configure separate warning and error thresholds, and require matches to appear a chosen number of times.checked 4 Oct 2026
- Model choice
- The default embedding model is nomic-ai/CodeRankEmbed, and users can configure a compatible model or use a Python sidecar for other backends.checked 4 Oct 2026
- Optional backend
- An optional Amazon Bedrock backend uses Titan embeddings and the AWS SDK credential chain.checked 9 Oct 2026
- Purpose
- Semdup detects near-duplicate code by comparing semantic meanings rather than token sequences.checked 9 Oct 2026
- Search limits
- Sparse search is approximate and can miss matches that exact search would find; automatic indexing switches to sparse search above 20,000 scannable units.checked 9 Oct 2026
- Storage
- Semdup stores its index and cached embeddings in SQLite, keyed by content hash and model ID.checked 9 Oct 2026
- Use case
- The maker describes it as useful for catching AI-generated code duplication and preventing agents from reimplementing existing code.checked 4 Oct 2026
- Workflow integration
- The project provides a GitHub Actions workflow and action that can run semdup checks on pull requests and pushes.checked 4 Oct 2026
