DP-203 Design and implement data storage Practice Question
A data engineering team is designing a batch processing pipeline that reads from Azure Data Lake Storage Gen2, transforms data using Azure Databricks, and writes to Azure Synapse Analytics. The pipeline must process data incrementally and handle late-arriving data up to 2 hours. Which approach should they use to track processed files?
⚠ Common exam trap
Candidates often choose Azure Data Factory with watermark columns (Option C) because it is a common incremental load pattern, but they overlook that watermark columns apply to row-based sources with change tracking, not to file-based sources where the challenge is tracking which files have been processed.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Store processed file names in a Delta table and compare with source folder listing
Storing processed file names in a Delta table allows the pipeline to track which files have already been ingested, supporting incremental processing and handling late-arriving data up to 2 hours. By comparing the current source folder listing against the Delta table, the pipeline can identify only new or late-arriving files, avoiding reprocessing and ensuring exactly-once semantics. This approach integrates seamlessly with Azure Databricks and Delta Lake's ACID transactions, providing reliable state management for batch pipelines.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use Blob Storage event triggers to invoke Azure Functions
Why it's wrong here
Event triggers are for real-time processing, not batch incremental processing with late-arriving data.
- ✗
Use Azure Synapse Pipelines with a schedule and full load each time
Why it's wrong here
Full loads are inefficient and not incremental; they do not handle late-arriving data efficiently.
- ✗
Use Azure Data Factory with watermark columns in the source
Why it's wrong here
Watermark columns require a column indicating last modified time, which may not exist in file-based sources.
- ✓
Store processed file names in a Delta table and compare with source folder listing
Why this is correct
Delta table provides a reliable way to track processed files and can be updated incrementally.
Visual reference
Go deeper
Related to this question
Learn chapter
Introduction to Azure Data Engineering
Key term
Azure Synapse Analytics
Azure Synapse Analytics is a cloud-based data integration, warehousing, and analytics service that brings together big data and data warehouse capabilities under one platform.
Key term
Azure Databricks
Azure Databricks is a fast, easy, and collaborative Apache Spark-based analytics platform optimized for Azure that lets data teams prepare data, run machine learning models, and build data pipelines using a single workspace.
About these practice questions
This DP-203 question is part of Courseiva's 760-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DP-203 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DP-203 exam.