Databricks-DE-Assoc Troubleshooting, Monitoring, and Optimization Practice Question
A data engineer is troubleshooting a slow-running Spark job on Databricks. Which TWO metrics in the Spark UI are most useful for identifying data skew?
⚠ Common exam trap
Candidates often look at 'Executor Memory' or 'CPU usage' as primary indicators of skew. These metrics are symptoms, but the Spark UI specifically identifies skew via task distribution metrics.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Task Duration (max vs median)
Identifying data skew requires looking for significant disparities in task execution times and data distribution across partitions. When one or two tasks take drastically longer than the others while processing similar data volumes, or when partition sizes are highly uneven, skew is likely present. Monitoring these metrics allows engineers to implement techniques like salting or broadcast joins to redistribute the data and balance the workload across executors.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Task Duration (max vs median)
Why this is correct
Comparing the maximum task duration to the median or minimum duration reveals tasks that are processing significantly more data than others. A large gap between the median and max duration is a classic indicator that specific tasks are encountering skewed data partitions and slowing down the entire stage.
- ✓
Shuffle Read Size
Why this is correct
The Shuffle Read Size metric shows how much data each task receives during a shuffle. If the distribution of read sizes is highly imbalanced—with one task reading gigabytes while others read kilobytes—it confirms that the partition keys are causing data to aggregate disproportionately into specific executor tasks.
- ✗
Executor CPU Usage
Why it's wrong here
While high CPU usage is a sign of intense computation, it does not directly identify data skew. An executor might be at 100% CPU because of inefficient code or complex transformations, not necessarily because the data distributed to that executor is skewed relative to the rest of the cluster.
- ✗
Total Cluster Memory Usage
Why it's wrong here
Cluster memory usage provides a high-level view of resource consumption, but it lacks the task-level granularity needed to diagnose skew. It shows if the cluster is under memory pressure but does not explain which specific keys or tasks are causing the imbalance that triggers the memory pressure.
- ✗
Driver Memory Consumption
Why it's wrong here
Driver memory is relevant for monitoring 'collect' operations or broadcast variable sizes, but it is unrelated to task-level data skew. If the driver is running out of memory, it is usually due to improper use of driver-side operations rather than imbalanced partitions occurring within the worker tasks.
About these practice questions
Courseiva writes every Databricks-DE-Assoc question from scratch — 276 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-DE-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-DE-Assoc exam.