Databricks-DE-Assoc Data Ingestion and Loading Practice Question
A data engineer is using Auto Loader to ingest JSON files from a cloud storage directory into a Delta table. The directory receives files continuously, and the engineer wants to ensure that the ingestion process can handle schema drift where new columns are added to the JSON files over time. The engineer also wants to minimize the need to reprocess all data when the schema changes. Which configuration should the engineer use?
⚠ Common exam trap
The trap here is assuming that setting the schema evolution mode alone is sufficient, without also enabling 'mergeSchema' on the write operation to propagate schema changes to the Delta table.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Set the option 'cloudFiles.schemaEvolutionMode' to 'addNewColumns' and enable 'mergeSchema' on the write stream.
To handle schema drift with Auto Loader, the 'cloudFiles.schemaEvolutionMode' should be set to 'addNewColumns' to automatically add new columns to the schema. Additionally, the write stream must have 'mergeSchema' enabled to update the Delta table schema. The other modes either fail or rescue data without adding columns, and without 'mergeSchema' on write, the new columns won't be persisted.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Set the option 'cloudFiles.schemaEvolutionMode' to 'failOnNewColumns' and enable 'mergeSchema' on the write stream.
Why it's wrong here
'failOnNewColumns' causes the stream to fail when new columns are detected, which is the opposite of what is needed. The engineer wants to automatically handle new columns, not fail. Even with 'mergeSchema', the stream would stop, requiring manual intervention. This mode is used when strict schema enforcement is required.
- ✗
Set the option 'cloudFiles.schemaEvolutionMode' to 'addNewColumns' and set 'mergeSchema' to 'false' on the write stream.
Why it's wrong here
While 'addNewColumns' allows Auto Loader to infer new columns, if 'mergeSchema' is set to 'false' on the write stream, the Delta table will not update its schema to include these new columns. This would cause the write to fail or drop new columns, depending on the write mode. Therefore, this configuration does not fully support schema evolution.
- ✓
Set the option 'cloudFiles.schemaEvolutionMode' to 'addNewColumns' and enable 'mergeSchema' on the write stream.
Why this is correct
Auto Loader's 'cloudFiles.schemaEvolutionMode' controls how schema changes are handled. Setting it to 'addNewColumns' allows new columns to be added to the schema automatically. Additionally, enabling 'mergeSchema' on the write operation ensures that the Delta table schema is updated to include these new columns. This combination effectively manages schema drift without reprocessing all data.
- ✗
Set the option 'cloudFiles.schemaEvolutionMode' to 'rescue' and enable 'mergeSchema' on the write stream.
Why it's wrong here
The 'rescue' mode in Auto Loader captures unexpected data in a rescued data column instead of evolving the schema. While it prevents data loss, it does not add new columns to the schema. Therefore, it does not meet the requirement of handling schema drift by adding new columns. 'mergeSchema' alone cannot add columns if they are not in the inferred schema.
About these practice questions
Courseiva writes every Databricks-DE-Assoc question from scratch — 276 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-DE-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-DE-Assoc exam.