Databricks-DE-Assoc Troubleshooting, Monitoring, and Optimization Practice Question
A data engineer is investigating why a Databricks job that writes to a Delta table is experiencing performance degradation over time. The job performs frequent small appends. Which TWO actions should the engineer take to improve write performance? (Choose two.)
⚠ Common exam trap
The trap here is assuming that general Spark tuning parameters like shuffle partitions or AQE will solve Delta-specific small file issues, when the real fix is Delta Lake maintenance operations.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Run OPTIMIZE on the Delta table to compact small files.
Frequent small appends create many small files, which degrade performance. Running OPTIMIZE compacts these files into larger ones, and enabling auto compaction automates this process. Together, they reduce the number of files and improve write and read efficiency. These are standard Delta Lake maintenance practices.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use partitioning on a high-cardinality column.
Why it's wrong here
Partitioning on a high-cardinality column can lead to a large number of directories and small files, exacerbating the small file problem. Partitioning should be done on low-cardinality columns that are commonly used in filters. For frequent small appends, compaction is the appropriate fix.
- ✗
Increase the number of shuffle partitions to 2000.
Why it's wrong here
Shuffle partitions affect the parallelism of shuffles during operations like joins or aggregations, but they do not directly address the issue of small files in Delta Lake. Increasing shuffle partitions might even create more small files if the write is parallelized. The focus should be on file compaction.
- ✓
Run OPTIMIZE on the Delta table to compact small files.
Why this is correct
Frequent small appends create many small files, which degrade read and write performance. Running OPTIMIZE compacts these small files into larger ones, reducing the number of files and improving I/O efficiency. This is a recommended maintenance operation for Delta tables with many small files.
- ✗
Set spark.sql.adaptive.enabled to true.
Why it's wrong here
Adaptive Query Execution optimizes query plans at runtime, such as coalescing shuffle partitions, but it does not automatically compact small files in Delta tables. While AQE can improve overall job performance, it is not a solution for the small file problem caused by frequent appends.
- ✓
Enable auto compaction on the Delta table.
Why this is correct
Auto compaction automatically compacts small files after a write when the number of small files exceeds a threshold. This reduces the need for manual OPTIMIZE and helps maintain performance without manual intervention. It is a best practice for tables that receive frequent small writes.
About these practice questions
Courseiva writes every Databricks-DE-Assoc question from scratch — 276 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-DE-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-DE-Assoc exam.