- AWS now lets Aurora PostgreSQL read Apache Iceberg and Apache Parquet data from S3 and S3 Tables without first loading it into the database.
- The feature uses an embedded DuckDB engine, the aurora_analytics extension and an IAM role with access to S3 and AWS Glue.
- Supported versions are Aurora PostgreSQL 17.11 and later, plus Aurora PostgreSQL 18.6 and later, across commercial and GovCloud regions.
- Teams can join live transactional rows with lakehouse data, cache results and materialize hot datasets for low-latency reads.
Amazon Web Services has opened a new path for Aurora PostgreSQL clusters that need to mix everyday application records with files sitting in a data lake. The managed database can now read Apache Iceberg tables and Apache Parquet files without first copying them into Aurora, so a single SQL statement can cover both live operational rows and archived information. The move is aimed at teams that have grown tired of building and babysitting extract, transform and load jobs just to answer questions that cross current and historical data.
Instead of treating the data lake as a separate world, the service lets existing PostgreSQL clients, tools and endpoints reach lakehouse content through foreign tables. That means no duplicate copy of the historical dataset has to live inside the operational database, and application developers can keep using the syntax they already know. AWS is pitching this as a way to cut pipeline maintenance, reduce infrastructure sprawl and make it easier to enrich transactions with older context.
What the new Aurora PostgreSQL capability changes
The feature is not a separate analytics product that customers must stand up elsewhere. It is enabled through the aurora_analytics extension and an AWS Identity and Access Management role that gives the cluster permission to reach Amazon S3 and the AWS Glue Data Catalog. Once that role and extension are in place, administrators create foreign tables that point at Iceberg or Parquet data and query them much like ordinary PostgreSQL tables.
A notable part of the architecture is that DuckDB is embedded directly in Aurora. The open-source analytical engine handles scans over lakehouse files while Aurora continues to serve operational workloads. Query processing stays within the database environment, which avoids handing the job off to a separate cluster and helps keep latency and network overhead under control.
The supported sources include Iceberg tables registered in the AWS Glue Data Catalog, Parquet and Iceberg objects in Amazon S3, and data held in S3 Tables. External catalogs that follow the Iceberg REST Catalog interface can also be reached through federation with Glue, so teams do not have to abandon catalog investments they already have. A query can join rows from Aurora with Iceberg tables spread across multiple catalogs, giving applications a unified view without moving files.
Versions, setup and supported services
Aurora PostgreSQL 17 starting at version 17.11 and Aurora PostgreSQL 18 starting at version 18.6 are the supported releases. After creating or choosing a cluster, an administrator attaches an IAM role with the AuroraAnalytics feature and turns on the extension. The setup can be done through the Amazon RDS console or a PostgreSQL client such as psql.
Foreign table definitions can be created one by one, but larger environments can use IMPORT FOREIGN SCHEMA to generate many foreign tables in one step from an AWS Glue Data Catalog database. For Parquet files, Aurora can read the schema from file metadata, which removes the need to spell out every column manually. That detail matters for teams with wide or frequently changing datasets.
Read queries may run on the writer instance or on a read replica, so analytical scans can be shifted away from the primary operational workload. Commands that write data into Aurora, such as CREATE TABLE AS SELECT, INSERT INTO … SELECT or MERGE INTO, run on the writer instance. This gives teams a way to materialize hot lakehouse data into a native table when they need consistently fast responses.
To see what a query is doing, administrators can call aurora_analytics_stat_statements(). The function reports metrics such as rows scanned, bytes read from Amazon S3 and cache hits. Frequently used data is cached on the Aurora instance, so repeated access to the same lakehouse objects can return faster without repeatedly pulling the same bytes from object storage.
Performance, caching and low-latency paths
Aurora applies predicate pushdown and column pruning, which means only relevant rows and columns are requested from the data lake. As the underlying dataset grows, that filtering helps keep queries from turning into full scans. The cached results and query statistics give operators a practical way to tune workloads instead of guessing where time is being spent.
For applications that demand single-digit-millisecond latency, AWS points to materialization as the escape hatch. Selected data lake records can be copied into a native Aurora PostgreSQL table with familiar SQL commands, after which they are queried like any other local table. This keeps a fast path for the hottest data while still leaving the full data lake available for broader analysis.
The company lists real-time dashboards, transactions enriched with historical context, and AI agents that need to reason over both current and archived records as typical use cases. The common thread is a workload that cannot wait for a nightly batch or a reverse ETL job. By letting an application ask one database for both fresh and old information, AWS hopes to remove a recurring source of pipeline complexity.
Availability and pricing
The capability is available in all commercial AWS Regions and AWS GovCloud (US) Regions, according to AWS. There is no separate feature charge. Customers still pay for the additional Aurora compute that queries consume and for Amazon S3 request costs tied to reading data lake files. That pricing model keeps the entry point low, though heavy analytical use will still show up on the bill.
AWS has framed the launch as part of a broader effort to bring DuckDB performance into its managed services. The DuckLabs team, which maintains the DuckDB project, joined Amazon, and the company says future improvements to the open-source engine can flow into Aurora and other AWS offerings. That connection suggests the integration is meant to evolve rather than remain a one-off feature.
For database teams, the practical appeal is familiar: fewer moving parts, less duplicated data and a shorter path from question to answer. The trade-off is that access controls, IAM permissions and catalog configuration still need to be handled with care. A cluster that can see both production rows and lakehouse files becomes a powerful single query surface, so governance remains part of the job.
The practical effect is a tighter link between Aurora PostgreSQL and the data lake. Current transactions and historical files can meet in one SQL query, DuckDB handles the analytical scan, and materialization offers a fast lane when needed. Teams already invested in PostgreSQL tooling gain another option for analytics-heavy workloads, while those with strict security or cost controls will want to review permissions, caching behavior and S3 usage before rolling it out widely.
