Databricks-Spark-Assoc Spark Architecture and Components Practice Question
A Spark application is running in cluster mode on Databricks. The driver program is running on a worker node, and the application has been running for several hours. Suddenly, the driver node experiences a hardware failure and crashes. What happens to the running tasks and the application?
⚠ Common exam trap
The trap here is assuming that Spark has built-in driver fault tolerance or automatic recovery from checkpoints, which is not the default behavior.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The application fails completely, and all running tasks are lost; the application must be restarted from scratch.
The driver is the single point of failure in a Spark application. It hosts the SparkContext, DAG Scheduler, and Task Scheduler. If the driver crashes, the application fails, and all running tasks are lost. While the cluster manager may restart the driver, the application does not automatically resume from checkpoints unless explicitly configured. Thus, the correct outcome is complete failure and restart from scratch.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
The application fails completely, and all running tasks are lost; the application must be restarted from scratch.
Why this is correct
In Spark's architecture, the driver is the central coordinator. If the driver crashes, the entire application fails because there is no other component to manage the DAG Scheduler, Task Scheduler, or SparkContext. Running tasks on executors will be killed or eventually time out. Without a driver, the application cannot continue, and it must be restarted. Databricks may attempt to restart the driver if configured with automatic restart, but otherwise, it fails.
- ✗
The executors continue to run tasks and complete the job, and a new driver is automatically elected from the executors.
Why it's wrong here
Spark does not have a mechanism for electing a new driver from executors. The driver is a single point of failure. Executors cannot coordinate among themselves to complete the job; they rely on the driver for task scheduling and job management. Without the driver, executors will eventually stop or be terminated by the cluster manager. Thus, this statement is incorrect.
- ✗
The running tasks continue on executors until they finish, and then the application terminates gracefully.
Why it's wrong here
Tasks running on executors are managed by the driver. If the driver crashes, the executors lose their connection and will not be able to report task status or receive new tasks. The cluster manager will typically kill the executors after a timeout. Tasks cannot complete the job without the driver to coordinate stages and handle shuffle dependencies. Therefore, the application does not terminate gracefully; it fails.
- ✗
The cluster manager restarts the driver on another node, and the application resumes from the last checkpoint.
Why it's wrong here
While the cluster manager (e.g., YARN, Kubernetes) may restart the driver if configured with a restart policy, Spark does not automatically resume from the last checkpoint unless the application is designed with checkpointing and recovery logic. By default, the application restarts from scratch, losing all in-memory state. The cluster manager does not inherently provide checkpoint-based recovery; that requires application-level code or Databricks-specific features like automatic restart with checkpointing, which is not default.
About these practice questions
Courseiva writes every Databricks-Spark-Assoc question from scratch — 295 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-Spark-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-Spark-Assoc exam.