DEA-C01 Data Ingestion and Transformation Practice Question
A data engineer runs an AWS Glue for Apache Spark job that writes partitioned Parquet output to Amazon S3. The engineer notices thousands of small output files in each partition, which slows downstream Amazon Athena queries. The job reads from a large S3 source and uses default partitioning. Which change should the engineer make to reduce the number of small files?
⚠ Common exam trap
The trap here is assuming that changing the output format or adding compute reduces file count, when file count is governed by the number of written partitions.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use repartition or a controlled partition count before writing, or enable Glue's grouping of output files.
The number of output files equals the number of partitions written, so consolidating partitions with repartition or using Glue's output file grouping reduces small files without sacrificing parallelism. Coalescing to one partition kills scalability, bookmarks and DPUs do not affect file count, and CSV does not change partitioning while harming query efficiency.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Set the Glue job's output to a single partition using coalesce(1) before writing.
Why it's wrong here
Coalescing to a single partition forces all output through one task, which eliminates parallelism and can cause memory pressure or failure on large datasets. It produces one file per write, but at the cost of dramatically slower writes and potential out-of-memory errors. This is the wrong trade-off for a large source and does not scale as data volume grows.
- ✗
Enable Glue job bookmarks and increase the number of DPUs allocated to the job.
Why it's wrong here
Job bookmarks track previously processed input to support incremental runs, and more DPUs add compute, but neither reduces the number of files written per partition. The small-file problem is caused by the number of output partitions relative to data size, not by reprocessing or insufficient compute. This combination leaves the downstream Athena performance issue unresolved.
- ✗
Convert the output to CSV so fewer files are created per partition.
Why it's wrong here
Changing the format to CSV does not change how many files Spark writes; the file count is determined by the number of output partitions, not the format. CSV also loses the columnar benefits that make Athena queries efficient and increases storage size. This does not address the small-file problem and degrades downstream query performance further.
- ✓
Use repartition or a controlled partition count before writing, or enable Glue's grouping of output files.
Why this is correct
Repartitioning to a sensible number of partitions, or using Glue's built-in grouping of output files, consolidates many small files into fewer larger ones while preserving parallelism. This reduces per-file overhead and improves Athena query performance by lowering the number of objects scanned and listed. It scales with data volume, unlike forcing a single partition, and directly targets the small-file cause.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
One of 1,321 original DEA-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Amazon Web Services exam blueprint
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.