DEA-C01 Data Ingestion and Transformation Practice Question
A retail company uses AWS Glue ETL jobs to process sales data from an S3 data lake. The source data is partitioned by year/month/day in CSV format. The Glue job reads the latest day's data, performs transformations (e.g., cleaning, aggregating), and writes the results to a separate S3 bucket. The job runs on a schedule every day at 2 AM. Recently, the job has been failing intermittently with the error 'AnalysisException: Path does not exist: s3://source-bucket/year=2024/month=02/day=30/'. The engineer verifies that the folder 'day=30' does not exist because February has only 28 days in 2024. The job is reading data from a hardcoded path. The company expects the job to handle variable days per month automatically. What should the engineer do to fix the issue?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a Glue crawler to populate the Data Catalog and use dynamic frame from_catalog with partition predicates
Using a Glue crawler to populate the Data Catalog and then using dynamic frame with from_catalog allows Glue to automatically discover all existing partitions. This eliminates the need for hardcoded paths and handles variable days per month. Option A (Spark SQL with manual partition pruning) still requires manual handling of partitions. Option B (try-catch) is a workaround but does not fix the root cause. Option C (increasing retries) does not address the missing partition issue.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Modify the script to use Spark SQL with manual partition pruning based on current date
Why it's wrong here
Manual pruning still requires handling missing partitions; dynamic frame is more robust.
- ✗
Add a try-catch block in the script to skip missing partitions
Why it's wrong here
This is a workaround; the job should not attempt to read non-existent partitions.
- ✗
Increase the job's retry count and set a timeout
Why it's wrong here
Retries will fail again on the same missing partition.
- ✓
Use a Glue crawler to populate the Data Catalog and use dynamic frame from_catalog with partition predicates
Why this is correct
The crawler discovers existing partitions, and dynamic frame reads only available partitions.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
Courseiva writes every DEA-C01 question from scratch — 1,711 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.