Apache Arrow DataFusion

Visit site Alternatives

Score6.9

Rank#5 of 25

PriceFree

Free planYes

Runs onLinux, macOS, Self-hosted

Summary

Apache Arrow DataFusion is a free, extensible query engine written in Rust that uses Apache Arrow as its in-memory format. It is designed for developers building database and analytics systems for specific workloads, rather than as a ready-made hosted database. Queries run in-process, with threads for parallel execution. DataFusion provides SQL and DataFrame APIs, while related projects offer Python and Java interfaces. Its documented capabilities include SQL parsing and planning, parallel and streaming execution, and query optimization. Built-in sources include CSV, Parquet, JSON, Avro, and Arrow; developers can add sources through the TableProvider trait and tailor functions, operators, and other components. Runtime controls include enforced memory limits and disk spilling for sorts, grouping, and joins. The core engine is for in-process use; Ballista is a related distributed-processing extension. Rust users commonly add DataFusion from crates.io, while official Apache releases are source artifacts. The project recommends checking releases with an OpenPGP signature or SHA-512 checksum.

Who it is for

DataFusion suits developers building custom database or analytics systems, especially those who want to extend query sources or engine components. Its core in-process design fits workloads that do not require distribution through the core project.

What is good

SQL and DataFrame APIs are available.
Supports CSV, Parquet, JSON, Avro, and Arrow sources.
Includes memory limits and disk spilling.
Developers can customize sources, functions, and operators.

What to know first

Core DataFusion is designed for in-process use.
Official Apache releases are source artifacts.
The engine is intended for developers building systems.

Verdict

DataFusion provides query execution and extension points for teams creating their own data systems. For distributed processing, the related Ballista extension is separate from the core project.

Apache Arrow DataFusion plans and pricing

All plans

Apache DataFusion Free Open source project; official releases are source artifacts; distributed as a Rust library and CLI datafusion.apache.org · 29 Sept 2026

Compared on query engine software

Free plan: Yesdatafusion.apache.org
Federated queries: Yesdatafusion.apache.org
Heterogeneous sources: Yesdatafusion.apache.org
Deployment: self_hosteddatafusion.apache.org
SQL support: fulldatafusion.apache.org
Streaming sources: Yesdatafusion.apache.org

Facts

Product: Apache DataFusion is an extensible query engine written in Rust that uses Apache Arrow as its in-memory format.datafusion.apache.org · 29 Sept 2026
Intended users: The core project provides libraries and binaries for developers building database and analytics systems customized to particular workloads.datafusion.apache.org · 29 Sept 2026
Execution: DataFusion runs queries in-process using threads for parallel query execution.datafusion.apache.org · 29 Sept 2026
APIs: DataFusion offers SQL and DataFrame APIs, and related subprojects provide Python and Java interfaces.datafusion.apache.org · 29 Sept 2026
Query features: Documented features include SQL parsing and planning, parallel and streaming execution, and query optimization.datafusion.apache.org · 29 Sept 2026
Formats: Built-in data source support includes CSV, Parquet, JSON, Avro, and Arrow.datafusion.apache.org · 29 Sept 2026
Extensibility: Developers can add data sources through the TableProvider trait and customize functions, operators, and other components.datafusion.apache.org · 29 Sept 2026
Runtime limits: Documented runtime features include enforced memory limits and disk spilling for sorts, grouping, and joins.datafusion.apache.org · 29 Sept 2026
Distribution: The core DataFusion project is designed for in-process use; Ballista is a related distributed processing extension.datafusion.apache.org · 29 Sept 2026
Downloads: Rust users commonly add DataFusion from crates.io, while official Apache releases are provided as source artifacts.datafusion.apache.org · 29 Sept 2026
Release verification: The download page recommends verifying release artifacts with an OpenPGP signature or SHA-512 checksum.datafusion.apache.org · 29 Sept 2026
Support: The project directs users to its community communication channels for getting in touch.datafusion.apache.org · 29 Sept 2026
Governance: The project is governed through the Apache Software Foundation process.datafusion.apache.org · 29 Sept 2026