DEA-C01 Data Store Management Practice Question

A data engineer is managing an Amazon S3 data lake that contains millions of small files. The engineer needs to optimize query performance in Amazon Athena and reduce costs. The data is stored in Parquet format and is partitioned by date. Which action should the engineer take to improve performance and reduce costs?

⚠ Common exam trap

The trap here is thinking that adding more partitions or enabling S3 Versioning will help, when the core issue is the number of small files, which requires compaction.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Use AWS Glue ETL to compact small files into larger files.

Compacting small files into larger files using AWS Glue ETL reduces the number of files and S3 requests, improving Athena query performance and reducing costs. Other options either do not address the small file issue or would degrade performance. This is a best practice for optimizing data lakes.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Use AWS Glue ETL to compact small files into larger files.
Why this is correct
Compacting small files into larger files reduces the number of S3 GET requests and metadata overhead, improving Athena query performance. It also reduces the amount of data scanned if the compaction results in better compression and columnar storage. This is a common optimization for data lakes with many small files.
✗
Convert the data to CSV format to enable faster scans.
Why it's wrong here
CSV is not columnar and does not support efficient compression or column pruning. Converting from Parquet to CSV would increase the amount of data scanned and degrade performance. Parquet is already optimized for query performance, so this action is counterproductive.
✗
Enable S3 Versioning on the bucket to improve read performance.
Why it's wrong here
S3 Versioning protects against accidental deletion and overwrites but does not improve read performance or reduce the number of files. It actually increases storage costs by retaining multiple versions. This action does not address the small file problem.
✗
Increase the number of partitions to reduce the amount of data scanned per query.
Why it's wrong here
While partitioning can reduce data scanned, over-partitioning with millions of small files exacerbates the small file problem. Each partition adds metadata overhead and more S3 requests. The engineer should first compact files before considering additional partitioning.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Store Management Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question