https://maladroit.dev/kb/computer-engineering/databases/Recent content in Databases on maladroit.devHugoen-ushttps://maladroit.dev/kb/computer-engineering/databases/apache-iceberg/Mon, 01 Jan 0001 00:00:00 +0000https://maladroit.dev/kb/computer-engineering/databases/apache-iceberg/<p>Apache Iceberg is an open table format for huge, slow-changing, analytic data-sets. Iceberg aims to support the following:</p> <ul> <li>Schema evolution with no side effects</li> <li>Hidden partitioning to avoid performance penalties and user error</li> <li>Partition layout evolution as data and patterns evolve over time</li> <li>Time travel to reproduce historical queries</li> <li>Version rollback to correct problems by resetting to a known good state</li> <li>Readers isolated from concurrent writes</li> <li>Writers add and remove files atomically</li> <li>Planning and scanning takes <code>O(1)</code> and is not impacted by table size</li> </ul> <p>Because Iceberg has been designed for large data-sets it can be used in production where a single table is measured in petabytes. Iceberg is also designed to solve correctness problems in eventually-consistent cloud object stores. It is cloud platform agnostic, table changes are atomic so readers never see partial changes, and concurrent writers when experiencing conflicts will retry to ensure updates are compatible.</p>