DEA-C01 Data Store Management Practice Question

A data engineer is designing a data lake on Amazon S3 and needs to catalog data using the AWS Glue Data Catalog. The data is stored in Parquet format, partitioned by year/month/day. The engineer wants to query the data using Amazon Athena and ensure that partition pruning occurs to minimize query costs. Which action should the engineer take?

⚠ Common exam trap

The trap here is assuming that running MSCK REPAIR TABLE or using Glue crawlers is sufficient for partition pruning, when in fact partition projection provides a more scalable and cost-effective solution.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Configure the table in AWS Glue Data Catalog with partition projection using the appropriate storage descriptor and table properties.

Partition projection in AWS Glue Data Catalog enables Athena to dynamically compute partition locations, avoiding the need to manage partitions manually. It supports efficient partition pruning, reducing the amount of data scanned and lowering query costs. This is the recommended approach for highly partitioned datasets with a predictable structure.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Use AWS Glue crawlers to automatically discover the schema and partitions, then run MSCK REPAIR TABLE in Athena.
Why it's wrong here
Glue crawlers can discover partitions, but they may not capture all partitions if the S3 structure is not standard. Running MSCK REPAIR TABLE adds partitions to the Athena table, but it can be slow and may not be efficient for large numbers of partitions. This approach does not guarantee optimal partition pruning without additional configuration.
✗
Use AWS Glue ETL jobs to write data to S3 in a non-partitioned layout and rely on Athena's predicate pushdown.
Why it's wrong here
Non-partitioned data prevents partition pruning, forcing Athena to scan all data and increasing query costs. Predicate pushdown on Parquet can skip row groups but is less effective than partition pruning. This approach contradicts the requirement to minimize query costs through partitioning.
✗
Manually define the table in AWS Glue Data Catalog with partition keys and use Athena's ALTER TABLE ADD PARTITION for each partition.
Why it's wrong here
Manually adding partitions is error-prone and not scalable for a large number of partitions. It also requires updating the catalog whenever new partitions are added, increasing operational overhead. While it ensures partitions exist, it does not automate the process and can lead to missing partitions and inefficient queries.
✓
Configure the table in AWS Glue Data Catalog with partition projection using the appropriate storage descriptor and table properties.
Why this is correct
Partition projection allows Athena to calculate partition locations dynamically based on table properties, eliminating the need to manually add partitions or run crawlers. This enables efficient partition pruning and reduces query costs, especially for highly partitioned datasets like year/month/day.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Store Management Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question