Apache Iceberg

Apache Iceberg is an open table format for analytic data, not a hosted database or a general-purpose data workspace. It is best suited to teams operating large tables across multiple query engines. Its snapshot history and write safeguards are compelling; adopting it also means running the stack yourself and checking engine compatibility before upgrades.

Overview

Iceberg provides a shared table layer over storage you supply and manage. Spark, Trino, Flink, Presto, Hive, and Impala can work with the same tables, which makes it useful when different parts of an analytics pipeline need to read or change a common dataset. That interoperability is the central reason to choose it, rather than a hosted service that manages the surrounding infrastructure.

Table snapshots add operational control: users can query by version or timestamp and restore an earlier state. Atomic updates, serializable isolation, and consistent snapshot reads help keep changes coherent. Multiple writers are supported through optimistic concurrency; when a commit fails, Iceberg retries it against the current table state. These controls suit active shared datasets, though they do not remove the need to operate storage, catalogs, and compatible engines.

Key features

• SQL changes: Merge new data, update rows, and make targeted deletes. This gives teams a way to apply selective changes instead of replacing an entire table.
• Schema evolution: Add, rename, or reorder columns without rewriting the table. That is valuable as analytic datasets change shape, because structural updates need not trigger a full rewrite.
• Hidden partitioning: Iceberg derives partition values and skips irrelevant partitions and files automatically. Teams can benefit from partition pruning without managing those values directly.
• Snapshot history: Query a specific table version or timestamp, or roll back to an earlier state. Table-level history is useful for investigating changes and recovering from unwanted updates.
• File maintenance: Rewrite data files with bin-packing or sorting strategies. This gives operators ways to reorganize files as part of table management.
• Encryption: Use predefined AWS, Azure, or GCP KMS clients, or a custom KMS client. Catalogs must preserve the table's encryption key ID property throughout its lifetime, so catalog behavior is part of the security design.
• Integrations and artifacts: The project documents integrations with Spark, Flink, Kafka Connect, Hive, and catalogs such as AWS Glue, JDBC, and Nessie. Downloads include source archives, Spark and Flink runtime JARs, and AWS, GCP, and Azure bundles.

Pricing

Apache Iceberg is free and open source under the Apache License, Version 2.0. The Apache Iceberg plan costs nothing. There is no free trial because there is no paid entry tier to trial; the cost advantage comes with a self-hosted deployment and bring-your-own storage rather than a managed service.

The plan covers table-level data scope, dataset branching, and point-in-time rollback, with table snapshots. There are no seat or storage quotas stated for the plan. Teams should account for the operational work of running their own environment and choosing compatible engines and catalogs.

Platforms

Apache Iceberg supports API, Linux, macOS, Windows, and self-hosted use. Its self-hosted model gives teams control over deployment, while placing setup and ongoing operation with them.

Who it's for

Iceberg is a strong fit for data teams maintaining large analytic tables that must be shared across supported engines, or that need schema changes, versioned snapshots, and concurrent writes. It is less suitable for readers who want a hosted application, managed storage, or a graphical workspace rather than a table format they incorporate into their own stack.

Before upgrading, teams should check engine versions against the release they plan to use. Version 1.12.0 removes support for Spark 3.4 and Flink 2.0, a concrete compatibility constraint for environments still relying on either version. Community discussions are available through the dev mailing list and an Apache Iceberg Slack workspace, with public user and developer mailing lists also provided.

Pros and cons

• Pro — Shared tables across engines: Spark, Trino, Flink, Presto, Hive, and Impala can work with the same tables, reducing the need to make each engine use a separate table representation.
• Pro — Useful table history: Snapshot queries and rollback provide a direct recovery and inspection path at table granularity.
• Pro — Controlled writes and schema changes: Optimistic concurrency, atomic updates, and schema evolution support changing datasets without requiring a rewrite for column changes.
• Con — Self-hosted responsibility: Teams provide storage and operate the deployment; Iceberg is not a managed data service.
• Con — Version compatibility needs care: The removal of Spark 3.4 and Flink 2.0 support in 1.12.0 may block an upgrade until those engine versions change.
• Con — Catalogs have a security obligation: Encryption depends on catalogs preserving the encryption key ID property for the table's lifetime.

Alternatives

Choose Weights & Biases instead if a freemium tool with a free plan and trial, plus listed iOS and web platforms, better matches the workflow; its free plan includes five model seats, 5 GB per month of storage, and 1 GB per month of Weave data ingestion.

Dolt is another free, open-source option, with a roughly 100 MB program; choose it if that package fits better than Iceberg's focus on large analytic tables. git-annex is worth considering when the need is to manage large files without storing their contents in Git.

Apache Paimon is a free, open-source data lake platform; consider it when choosing connectors and storage plugins for an environment is the priority. ClearML offers a freemium model and describes its self-hosted version as fully open source on GitHub, so it may suit readers seeking that alternative. Nile has a free local plan that runs entirely on the user's machine without a cloud account.

DataLad is a free, open-source option for readers comparing other dataset tools. Kart is another free, GPL-licensed open-source alternative. Browse the Data Version Control Tools category for more options.

Verdict

Choose Apache Iceberg if your team needs a shared, versioned table format across analytic engines and can take responsibility for deployment, storage, and compatibility. Its strongest case is the combination of table history, controlled writes, and schema evolution in a free open format. Look elsewhere if you need a managed service, or if your engine versions cannot accommodate the support changes in 1.12.0.