DEA-C01 Data Ingestion and Transformation Practice Question

A company uses AWS Glue DataBrew for data preparation. The data source is an S3 bucket with millions of small CSV files (each < 1 MB). The DataBrew project takes a long time to load the sample data. What is the most likely cause and solution?

⚠ Common exam trap

The DEA-C01 exam often tests the misconception that increasing DPUs or switching to a different AWS service will fix performance issues, when the real root cause is S3's small-file overhead and the LIST API's pagination limit.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

The large number of small files causes S3 LIST overhead; concatenate files into larger files

DataBrew loads a sample of the data by listing objects in the S3 bucket. With millions of small CSV files, the S3 LIST API call becomes a bottleneck because each list operation has a 1000-object limit per response, requiring multiple paginated requests. Concatenating the small files into larger files reduces the number of objects, dramatically decreasing LIST overhead and speeding up sample loading.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Use Amazon Athena to query the data instead of DataBrew
Why it's wrong here
Athena queries data in place but performs no data preparation, so it cannot replace DataBrew's profiling, cleaning and transformation of millions of small CSVs. It is tempting because Athena handles large S3 datasets efficiently, and would be correct for ad-hoc SQL querying rather than preparing data.
✗
The DataBrew job is under-provisioned; increase the number of DPUs
Why it's wrong here
DataBrew sampling reads a subset of files, so DPU count does not govern sample-load latency; the bottleneck is the per-file overhead of millions of tiny objects. It is tempting because scaling DPUs speeds up job runs, and would be correct for a compute-bound transformation job rather than a sampling bottleneck.
✓
The large number of small files causes S3 LIST overhead; concatenate files into larger files
Why this is correct
DataBrew must LIST and open each object individually, so millions of sub-1 MB files impose heavy S3 LIST and per-request overhead during sampling. Concatenating them into larger files reduces request count and dramatically speeds up sampling.
✗
Use AWS Glue ETL instead of DataBrew for this volume
Why it's wrong here
Glue ETL is a code-based transformation service, not a visual data-preparation tool, so it does not address DataBrew's slow sampling of millions of small files. It is tempting because Glue handles large-scale ETL, and would be correct when building scheduled transformation pipelines rather than interactive preparation.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Ingestion and Transformation Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question