Databricks-DE-Pro Debugging and Deploying Practice Question
A data engineer is troubleshooting a Databricks job that fails with a 'TaskFailed' error. The job uses a cluster with autoscaling enabled. The engineer suspects that the failure is due to memory issues on the workers. Which TWO actions should the engineer take to diagnose and resolve the issue? (Choose two.)
⚠ Common exam trap
The trap here is assuming that driver memory or autoscaling settings are the cause, when the issue is likely per-executor memory and requires inspection of Spark UI and event logs.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Check the cluster's event log for 'ExecutorLostFailure' events indicating out-of-memory errors.
To diagnose memory issues, the engineer should use the Spark UI to examine memory-related metrics such as garbage collection and spill, and check the cluster event log for executor out-of-memory errors. These steps confirm whether memory is the bottleneck. Once confirmed, the engineer can adjust cluster configuration, such as increasing executor memory or optimizing the job. Disabling autoscaling or increasing driver memory does not address worker memory problems, and increasing partitions is a potential fix but not a primary diagnostic step.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the driver node's memory to handle the failing tasks.
Why it's wrong here
The driver node manages the Spark application but does not execute tasks. Memory issues on workers are not resolved by increasing driver memory. While driver memory is important for coordination, the error suggests worker memory problems. This action would not address the root cause.
- ✗
Increase the number of partitions when reading the data to reduce per-task memory usage.
Why it's wrong here
Increasing partitions can reduce per-task memory by processing smaller chunks, but it is not a direct diagnostic action and may not resolve the issue if the problem is skewed data or inefficient operations. It is a potential optimization but not one of the first two steps to diagnose memory issues. The question asks for actions to diagnose and resolve, but this is more of a tuning step after diagnosis.
- ✗
Disable autoscaling to ensure the cluster has a fixed number of workers.
Why it's wrong here
Disabling autoscaling would not directly address memory issues; it might even reduce available resources if the cluster scales down. Autoscaling helps handle variable workloads. The problem is likely per-executor memory, not the number of workers. Disabling autoscaling is not a diagnostic step and could worsen performance.
- ✓
Check the cluster's event log for 'ExecutorLostFailure' events indicating out-of-memory errors.
Why this is correct
The cluster event log records executor losses and out-of-memory errors. 'ExecutorLostFailure' often indicates that an executor was killed due to memory limits. Reviewing the event log helps confirm if memory is the culprit and provides details such as which executor failed and why. This is a key diagnostic step.
- ✓
Review the Spark UI for the job's run to identify stages with high garbage collection time or spill.
Why this is correct
The Spark UI provides detailed metrics on memory usage, including garbage collection time and disk spill. High GC time or spill indicates memory pressure. By examining the stages, the engineer can pinpoint which operations are memory-intensive and adjust accordingly, such as increasing executor memory or optimizing the query.
Visual reference
About these practice questions
Courseiva writes every Databricks-DE-Pro question from scratch — 267 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-DE-Pro practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-DE-Pro exam.