Amazon S3 Tables now support all Apache Iceberg V3 data types

Summary

• Amazon S3 Tables now support all data types in the Apache Iceberg V3 specification, including variant, nanosecond timestamps, unknown, geometry, & geography.
• Users can create new V3 tables or upgrade existing V2 tables in place to access features like deletion vectors & row lineage.
• Deletion vectors replace V2 positional delete files with a compact binary format to reduce compaction time & delete file overhead.
• Row lineage automatically adds _row_id & _last_updated_sequence_number fields to records for efficient downstream pipeline queries.
• The variant data type stores semi-structured data in columnar format to enable file pruning & reduce I/O compared to parsing JSON strings.
• Teams can create tables with the variant type to store events with different structures without predefined schemas.
• Deletion vectors require merge-on-read mode configuration for write operations.
• Upgrading from V2 to V3 is an atomic, one-way operation that does not support downgrading.
• Existing V2 readers remain compatible with upgraded tables until fully adopting V3 features.
• Row lineage enables incremental pipelines by allowing queries for rows modified after a specific sequence number.
• AWS analytics services including Amazon EMR Spark, AWS Glue, & Amazon Redshift support Iceberg V3 tables.
• Both S3 Tables & AWS Glue Data Catalog support the Iceberg REST Catalog API for interoperability.
• V3 data types require an engine built on Apache Spark 4.0 or later, such as AWS Glue 6.0 or later, or Amazon EMR release 8.1 or later.
• New V3 data types are supported only for tables using the Parquet file format.
• Columns of type variant, geometry, geography, or nanosecond timestamp cannot be included in a table's sort order for compaction.
• Amazon S3 Tables support for Apache Iceberg V3 data types is now available in all AWS Regions where S3 Tables are supported.
• Apache Iceberg V3 support is available at no additional charge beyond standard S3 Tables pricing.