MLS-C01 Data Engineering Practice Question
A data engineer is designing a data pipeline that transforms raw JSON files (each 50-200 KB) in Amazon S3 into Parquet format using AWS Glue. The pipeline must minimize data processing costs and handle a high volume of small files (millions per day). The engineer configures a Glue ETL job with Spark, but the job is slow and expensive due to overhead of reading many small files. Which optimization should the engineer implement to reduce cost and improve performance?
⚠ Common exam trap
Watch out — candidates often assume scaling up resources (more memory or DPUs) will fix performance issues, but the real problem is the small-file overhead, which is a data layout issue that cannot be solved by adding compute power.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use S3 object grouping or batch operations to combine small files before Glue processing.
The primary performance bottleneck with many small files in S3 is the overhead of listing, opening, and reading each file individually in Spark. By grouping or batching small files into larger objects (e.g., using S3 Batch Operations or a pre-processing step), you reduce the number of input splits and task launches, which dramatically lowers the cost and runtime of the Glue ETL job. This directly addresses the root cause of the inefficiency rather than merely scaling resources.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the worker type to G.2X for more memory per worker.
Why it's wrong here
Larger workers increase cost; does not fix the small file issue.
- ✗
Increase the number of DPUs allocated to the Glue job.
Why it's wrong here
More DPUs increase parallelism but also cost; does not address small file overhead.
- ✗
Change the output format from Parquet to CSV to reduce compression overhead.
Why it's wrong here
CSV is not columnar and may increase I/O; Parquet is optimized for analytics.
- ✓
Use S3 object grouping or batch operations to combine small files before Glue processing.
Why this is correct
Combining small files reduces task overhead, leading to faster and cheaper jobs.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.