← DEA-C01·Amazon Web Services

Question 1,133 of 1,786

Data Ingestion and Transformation →hardMultiple ChoiceObjective-mapped

Quick Answer

The answer is to set up an S3 event notification that invokes an AWS Lambda function, which triggers an AWS Glue job with a parameter containing the S3 object key of the new file, and modify the Glue job to process only that file while using the file key to avoid duplicates. This approach eliminates duplication because each file is processed exactly once by its unique key, and it improves efficiency by eliminating full scans of the input folder—only the newly arrived file is transformed. On the AWS Certified Data Engineer Associate DEA-C01 exam, this scenario tests your understanding of incremental ETL with S3 events and AWS Glue, specifically how to handle late-arriving data and achieve exactly-once processing; a common trap is assuming a scheduled job or Glue Workflows alone can track individual files. Remember the memory tip: “Key to one file, one run, no rerun”—the S3 object key is the single source of truth for idempotent processing.

DEA-C01 Data Ingestion and Transformation Practice Question

This DEA-C01 practice question tests your understanding of data ingestion and transformation. The scenario asks you to isolate a root cause — eliminate options that address a different problem before choosing. After answering, compare your reasoning against the explanation and wrong-answer breakdown below. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.

A financial services company ingests stock trade data from multiple exchanges into an Amazon S3 bucket (trade-bucket). Each exchange sends a CSV file every 5 minutes. The data must be transformed into Parquet format and partitioned by exchange and date (trade_date) for efficient querying using Amazon Athena. The pipeline must handle late-arriving data (files up to 2 hours late) and ensure exactly-once processing to avoid duplicates. Currently, a scheduled AWS Glue ETL job runs every hour, reads new CSV files, converts them to Parquet, and writes to an output bucket. However, the team is experiencing data duplication: if the job fails midway, upon retry it reprocesses all files in the input folder, causing duplicates in the output. Additionally, the job takes too long because it scans all files each run. The engineer must redesign the pipeline to eliminate duplicates and improve efficiency. What should the engineer do?

Question 1hardmultiple choice

Read the full NAT/PAT explanation →

A
Use AWS Glue Workflows to orchestrate the job and add a condition to check for duplicates before writing.
Why wrong: Glue Workflows do not provide file-level idempotency; duplicate detection would require additional logic and scans.
B
Set up an S3 event notification to invoke an AWS Lambda function that starts a Glue job with a parameter containing the S3 object key of the new file; modify the Glue job to process only that file and use the file key to avoid duplicates.
This ensures each file is processed exactly once, and the job runs only on new files, improving efficiency.
C
Modify the Glue job to move processed CSV files to an archive folder after successful transformation, and process only unprocessed files.
Why wrong: Moving files does not guarantee exactly-once if the job fails after moving some files; also, moving files adds latency and cost.
D
Replace Glue with Amazon EMR and use Spark Structured Streaming with checkpointing to process files incrementally.
Why wrong: EMR is more complex to manage and may be overkill; checkpointing in streaming would still need file tracking.

Full breakdown with real-world context →

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Set up an S3 event notification to invoke an AWS Lambda function that starts a Glue job with a parameter containing the S3 object key of the new file; modify the Glue job to process only that file and use the file key to avoid duplicates.

Option D is the best approach: S3 event notifications trigger Lambda to start a Glue job passing the file key; the job processes only that file, using the file key to avoid reprocessing. Option A still processes all files and may cause duplicates if a file arrives after the job starts. Option B (Glue Workflows) does not solve the file-level tracking. Option C (EMR) is overkill and still scans the entire input folder.

Key principle: NAT direction and interface roles matter as much as the IP address mapping. Inside/outside designation controls which traffic is translated.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Use AWS Glue Workflows to orchestrate the job and add a condition to check for duplicates before writing.
Why it's wrong here
Glue Workflows do not provide file-level idempotency; duplicate detection would require additional logic and scans.
✓
Set up an S3 event notification to invoke an AWS Lambda function that starts a Glue job with a parameter containing the S3 object key of the new file; modify the Glue job to process only that file and use the file key to avoid duplicates.
Why this is correct
This ensures each file is processed exactly once, and the job runs only on new files, improving efficiency.
Related concept
Static NAT maps one inside address to one outside address.
✗
Modify the Glue job to move processed CSV files to an archive folder after successful transformation, and process only unprocessed files.
Why it's wrong here
Moving files does not guarantee exactly-once if the job fails after moving some files; also, moving files adds latency and cost.
✗
Replace Glue with Amazon EMR and use Spark Structured Streaming with checkpointing to process files incrementally.
Why it's wrong here
EMR is more complex to manage and may be overkill; checkpointing in streaming would still need file tracking.

Common exam traps

Common exam trap: NAT rules depend on direction and matching traffic

NAT is not only about the public address. The inside/outside interface roles and the ACL or rule that matches traffic are just as important.

Detailed technical explanation

How to think about this question

NAT questions usually test address translation, overload/PAT behaviour, static mappings and whether the right traffic is being translated. Read the interface direction and address terms carefully.

KKey Concepts to Remember

Static NAT maps one inside address to one outside address.
PAT allows many inside hosts to share one public address using ports.
Inside local and inside global describe the private and translated addresses.
NAT ACLs identify traffic for translation, not always security filtering.

TExam Day Tips

Identify inside and outside interfaces first.
Check whether the scenario needs static NAT, dynamic NAT or PAT.
Do not confuse NAT matching ACLs with normal packet-filtering intent.

Key takeaway

NAT direction and interface roles matter as much as the IP address mapping. Inside/outside designation controls which traffic is translated.

Real-world example

How this comes up in practice

A media company stores terabytes of video archives that are accessed once a year for audit purposes. Moving these objects to a cold storage tier (Azure Archive, S3 Glacier, or Google Nearline) costs a fraction of hot storage. Questions like this test whether you understand storage tiers, access frequency tradeoffs, and retrieval latency requirements.

What to study next

Got this wrong? Here's your next step.

Review the four NAT address types (inside local, inside global, outside local, outside global), PAT port overload, and static vs dynamic NAT use cases. Then practise related DEA-C01 NAT questions on configuration and troubleshooting.

Related DEA-C01 practice-question pages

Use these pages to review the topic behind this question. This is how one missed question becomes focused revision.

Data Ingestion and Transformation practice questions

Practise DEA-C01 questions linked to Data Ingestion and Transformation.

Data Operations and Support practice questions

Practise DEA-C01 questions linked to Data Operations and Support.

Data Security and Governance practice questions

Practise DEA-C01 questions linked to Data Security and Governance.

Data Store Management practice questions

Practise DEA-C01 questions linked to Data Store Management.

DEA-C01 fundamentals practice questions

Practise DEA-C01 questions linked to DEA-C01 fundamentals.

DEA-C01 scenario practice questions

Practise DEA-C01 questions linked to DEA-C01 scenario.

DEA-C01 troubleshooting practice questions

Practise DEA-C01 questions linked to DEA-C01 troubleshooting.

Practice this exam

Start a free DEA-C01 practice session

Short sessions build daily habit. Longer sessions build exam-day stamina. Try a timed session to simulate real conditions.

10 questions 20 questions 30 questions 50 questions Timed 30

DEA-C01 practice-test guide →Study guide →Browse all practice tests

FAQ

Questions learners often ask

What does this DEA-C01 question test?

Data Ingestion and Transformation — This question tests Data Ingestion and Transformation — Static NAT maps one inside address to one outside address..

What is the correct answer to this question?

The correct answer is: Set up an S3 event notification to invoke an AWS Lambda function that starts a Glue job with a parameter containing the S3 object key of the new file; modify the Glue job to process only that file and use the file key to avoid duplicates. — Option D is the best approach: S3 event notifications trigger Lambda to start a Glue job passing the file key; the job processes only that file, using the file key to avoid reprocessing. Option A still processes all files and may cause duplicates if a file arrives after the job starts. Option B (Glue Workflows) does not solve the file-level tracking. Option C (EMR) is overkill and still scans the entire input folder.

What should I do if I get this DEA-C01 question wrong?

What is the key concept behind this question?

Static NAT maps one inside address to one outside address.

About these practice questions

Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →

How Courseiva writes practice questions · Editorial policy

Keep practising

Question Discussion

Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.

Loading comments…

This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.