Tag: Data Engineering

All the articles with the tag "Data Engineering".

• 31 MIN READ•Aug 2, 2026

  Apache Polaris 1.7.0 and the Quiet Work of Making a Catalog Trustworthy

  Apache Polaris 1.7.0 deep dive: idempotent writes, semantic models, stricter credential vending, orphan cleanup, and what the upgrade asks of you.

  Apache PolarisApache IcebergCatalog
• 31 MIN READ•Aug 2, 2026

  Designing Batch Pipelines That Write Well Into Apache Iceberg

  How to design batch pipelines that write well into Apache Iceberg: commit strategy, partitioning, sort order, write-audit-publish, and maintenance done right.

  Apache IcebergData EngineeringBatch Pipelines
• 31 MIN READ•Jul 28, 2026

  Running an Apache Iceberg Lakehouse With No Internet Connection

  A practical guide to deploying an Iceberg lakehouse in air-gapped environments: component choices, artifact pipelines, identity without a cloud, and the operational realities that surprise teams.

  Apache IcebergAir-GappedData Engineering
• 31 MIN READ•Jul 28, 2026

  Why Iceberg V4 Wants to Retire Equality Deletes, and What Streaming Teams Should Do About It

  Equality deletes made streaming upserts into Iceberg practical at the cost of read performance. V4 proposes retiring them in favor of deletion vectors with an async conversion path.

  Apache IcebergStreamingData Engineering
• 31 MIN READ•Jul 28, 2026

  Surviving Commit Conflicts When Dozens of Writers Hit the Same Iceberg Table

  Commit conflicts multiply with writer count, and AI agents introduce unpredictable write patterns. Here's how to diagnose, tune, and architect around Iceberg's optimistic concurrency.

  Apache IcebergConcurrencyData Engineering
• 31 MIN READ•Jul 28, 2026

  The Parquet Versioning Problem, and Why Iceberg Cares About It

  Parquet files have a version field that doesn't reliably signal feature requirements. A new versioning discipline is coming, borrowing from Iceberg's format version model.

  ParquetApache IcebergData Engineering
• 31 MIN READ•Jul 25, 2026

  The Apache Iceberg Market in the Middle of 2026

  A survey of the Apache Iceberg market in July 2026: the state of the specification, platform support, the acquisition wave, the catalog contest, and how to evaluate real Iceberg support.

  apache iceberglakehousedata engineering
• 31 MIN READ•Jul 25, 2026

  Three Vendors Are Rebuilding the Path From Transaction to Agent

  Databricks, Snowflake, and SAP are closing the gap between operational databases and analytical platforms through acquisition, betting on different layers of the same five-part architecture.

  data engineeringapache iceberglakehouse
• 20 MIN READ•Jul 13, 2026

  Active Analytics Loops for AI Action Agents

  An in-depth exploration of active analytics loops for ai action agents

  AI AgentsAnalyticsData Engineering
• 18 MIN READ•Jul 13, 2026

  Hybrid Lakehouse Design for Regulated Markets

  An in-depth exploration of hybrid lakehouse design for regulated markets

  Hybrid LakehouseRegulated MarketsData Engineering
• 18 MIN READ•Jul 13, 2026

  Iceberg Concurrency for AI Agent Writes

  An in-depth exploration of iceberg concurrency for ai agent writes

  Apache IcebergConcurrencyAI Agents
• 18 MIN READ•Jul 13, 2026

  REST Catalog V2: Fixing Iceberg Protocol Debt

  An in-depth exploration of rest catalog v2: fixing iceberg protocol debt

  REST Catalog v2IcebergProtocol