Databricks-DE-Pro Data Transformation, Cleansing, Quality Practice Question
A Data Engineer is using Delta Live Tables (DLT) to build a pipeline that ingests JSON files from cloud storage. The engineer defines a streaming table with expectations to enforce data quality. The expectation `@dlt.expect_or_drop("valid_timestamp", "timestamp IS NOT NULL")` is applied. During a pipeline run, 5% of records have a NULL timestamp. What is the outcome for those records, and how does it affect the pipeline?
⚠ Common exam trap
The trap here is assuming that `expect_or_drop` quarantines records or fails the pipeline, when it simply drops them and continues.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The records with NULL timestamp are dropped from the target table, and the pipeline continues processing without failure.
The `expect_or_drop` expectation drops records that violate the condition and allows the pipeline to continue. It does not quarantine records, retain them, or fail the pipeline. This behavior is designed to handle data quality issues without interrupting processing, while still providing metrics on dropped records.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
The records with NULL timestamp are dropped from the target table, and the pipeline continues processing without failure.
Why this is correct
The `expect_or_drop` decorator instructs DLT to drop records that violate the expectation and continue processing. The dropped records are not written to the target table, but the pipeline does not fail. Metrics are recorded to track the number of dropped records. This allows the pipeline to maintain data quality while handling invalid data gracefully, which is the intended behavior for this expectation.
- ✗
The records with NULL timestamp are dropped, and the pipeline fails after processing the batch due to the drop threshold being exceeded.
Why it's wrong here
DLT does not have a built-in drop threshold that triggers pipeline failure. The `expect_or_drop` action drops records unconditionally and the pipeline continues. There is no automatic failure based on the percentage of dropped records unless you implement custom logic, such as using `expect_or_fail` with a condition that checks the drop rate. Thus, this outcome is not correct.
- ✗
The records with NULL timestamp are quarantined in a separate table, and the pipeline fails with an error.
Why it's wrong here
DLT does not automatically quarantine records; that would require custom logic. The `expect_or_drop` action drops records and does not fail the pipeline. Quarantining is a pattern that must be implemented manually, for example by writing invalid records to another table. Therefore, this outcome is incorrect for the given expectation.
- ✗
The records with NULL timestamp are retained in the target table, but the pipeline fails with a data quality error.
Why it's wrong here
`expect_or_drop` does not retain invalid records; it drops them. Also, it does not fail the pipeline. The `expect_or_fail` decorator would cause a failure, but that is not used here. Retaining records while failing would be contradictory and is not how DLT expectations work.
About these practice questions
One of 267 original Databricks-DE-Pro practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-DE-Pro practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-DE-Pro exam.