OpenLineage

What it does

Airflow version support

The Airflow integration has moved to the official provider; the legacy integration for Airflow versions earlier than 2.7 is no longer maintained and receives no bug fixes, security updates, or new features.checked 11 Oct 2026

Backend requirement

OpenLineage is a specification with integrations, examples, and tools, rather than a metadata server; the FAQ points to Marquez as a server that can receive and analyze its events.checked 11 Oct 2026

Capability limit

The integrations capability matrix is explicitly marked incomplete, and not every mechanism gathers every metadata facet.checked 11 Oct 2026

Client libraries

OpenLineage provides Python and Java clients for creating and emitting lineage events.checked 11 Oct 2026

Community support

The project directs users to its Slack community for help and provides open technical steering committee meetings to discuss project updates.checked 11 Oct 2026

Custom integrations

Python and Java clients can emit events to HTTP, Kafka, and other consumers, and the project provides a SQL parser available as Rust, Java, or Python libraries.checked 11 Oct 2026

dbt modes

The dbt integration can parse generated artifacts after a run or consume structured JSON logs in real time; artifact parsing waits for the dbt command to finish.checked 11 Oct 2026

Deployment

The project describes simple deployments with single consumers and complex deployments with multiple consumers.checked 11 Oct 2026

Event types

The specification includes runtime job run state events and design-time job and dataset metadata events.checked 11 Oct 2026

Extensible model

Its model represents datasets, jobs, and runs, and can be extended with user-defined facets.checked 11 Oct 2026

Integration limits

The integrations capability matrix is explicitly incomplete, and the compatibility matrix lists minimum versions for some supported platforms.checked 11 Oct 2026

Integrations

Documented integrations include Apache Airflow, Flink, Hive, Spark, dbt, Feast, Great Expectations, SQL, and Trino.checked 11 Oct 2026

Intended users

The project describes its audience as organizations that rely on data in their operations and organizations building products or technology to manage data and its processing.checked 11 Oct 2026

Not a metadata server

OpenLineage is a specification with integrations, examples, and tools; the FAQ points to Marquez for receiving and analyzing events.checked 11 Oct 2026

Open standard

The project describes its specification as an open standard that enables systems to interoperate with lineage metadata.checked 11 Oct 2026

Project governance

OpenLineage is an LF AI & Data Foundation project, and its site welcomes contributions and community feedback.checked 11 Oct 2026

Purpose

OpenLineage is an open framework for collecting and analyzing data lineage metadata about datasets, jobs, and runs.checked 11 Oct 2026

Security

The legacy Airflow integration is explicitly described as no longer receiving security updates.checked 11 Oct 2026

Security and compliance

The opened official pages do not state a security certification or compliance claim.checked 11 Oct 2026

Spark metadata

The Spark integration uses SparkListener events to collect job and dataset metadata from sources including filesystems, JDBC backends, and data warehouses.checked 11 Oct 2026

Standard

Its extensible open standard defines a generic model for dataset, job, and run entities, with user-defined facets for additional metadata.checked 11 Oct 2026

Transport options

The clients can emit events to HTTP, Kafka, and other consumers.checked 11 Oct 2026

What it tracks

It tracks metadata about datasets, jobs, and runs to help identify root causes and understand the impact of changes.checked 11 Oct 2026