DEA-C01 Data Operations and Support Practice Question
A data engineer at a financial services company manages an AWS Glue ETL pipeline that processes transaction data from Amazon S3 to Amazon Redshift for reporting. The pipeline runs every hour and uses a Glue job that reads Parquet files, performs transformations in Spark, and writes to Redshift using the JDBC connector. Recently, the job has been failing intermittently with the error: 'java.sql.BatchUpdateException: ERROR: null value in column "transaction_id" violates not-null constraint'. The data engineer has verified that the source Parquet files do contain non-null values for transaction_id. The job uses a DynamicFrame and applies a mapping to rename columns. The engineer also noticed that the failure occurs only during peak hours when there is high concurrency on Redshift. Which course of action should the engineer take to resolve this issue?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Review the Glue job's mapping transformation to ensure transaction_id is correctly mapped and not dropped.
The error indicates that transaction_id is being nullified or dropped during the Glue job's mapping transformation. Even though source files have non-null values, the mapping could be incorrectly mapping or omitting the column, causing nulls to be written to Redshift. The failure during peak hours is coincidental; the root cause is the mapping logic. Option A is incorrect because filtering nulls would not fix the mapping error and could discard valid data. Option B is incorrect because increasing Redshift WLM concurrency scaling does not address the null constraint violation. Option D is incorrect because more Glue workers do not fix the transformation issue; the problem is data quality, not capacity.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Add a filter in Glue to remove rows with null transaction_id.
Why it's wrong here
Adding a filter to remove rows with null transaction_id would not resolve the root cause; the nulls are introduced by the mapping transformation, and filtering them out could also remove valid data that only appears null due to the mapping error.
- ✗
Increase the Redshift WLM concurrency scaling to handle more queries.
Why it's wrong here
Increasing Redshift WLM concurrency scaling helps with query performance under high load, but the error is a null constraint violation, not a performance issue. Concurrency scaling does not fix the data transformation error.
- ✓
Review the Glue job's mapping transformation to ensure transaction_id is correctly mapped and not dropped.
Why this is correct
The error shows that transaction_id is being written as null to Redshift despite source files having non-null values. Reviewing and correcting the Glue job's mapping transformation to ensure transaction_id is correctly mapped and not dropped will resolve the issue.
- ✗
Increase the number of Glue workers to handle peak-hour load.
Why it's wrong here
Increasing the number of Glue workers can improve job performance but does not address the null value problem; the issue lies in the data transformation logic, not resource constraints.
Visual reference
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
One of 1,711 original DEA-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.