Databricks-ML-Pro Model Development Practice Question
A data scientist is using Hyperopt with SparkTrials on a Databricks cluster to tune an XGBoost classifier. After several trials, they notice that each trial runs on a single executor and the overall tuning job takes much longer than expected. They want to speed up hyperparameter tuning without changing the search space. Which adjustment is most likely to improve performance?
⚠ Common exam trap
The trap here is assuming that merely adding cluster resources will speed up SparkTrials without adjusting the parallelism parameter that governs concurrent trials.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Increase the number of Spark executors and set `parallelism` in `SparkTrials` to a value greater than 1.
SparkTrials parallelizes hyperparameter trials across Spark executors when `parallelism` is set above 1. The data scientist observed that each trial used a single executor, indicating that parallelism was effectively 1 or the cluster lacked executors. By adding executors and increasing parallelism, multiple trials run simultaneously, cutting total tuning time. The search space and model logic remain unchanged, so this is the direct fix.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Enable autoscaling on the cluster and set `parallelism` to 1 in `SparkTrials`.
Why it's wrong here
Autoscaling can add executors, but with `parallelism` set to 1, SparkTrials runs only one trial at a time regardless of available executors. This serializes the search and does not leverage additional cluster resources. The single-executor symptom would persist because parallelism controls concurrent trials, not cluster size alone.
- ✗
Increase the `max_evals` parameter in the `fmin` call to run more trials.
Why it's wrong here
Increasing `max_evals` simply runs more trials, which takes longer and does not improve parallelism. It does not change how trials are distributed across executors. The problem is not the number of evaluations but the lack of concurrent execution, so this adjustment would not speed up the tuning job.
- ✓
Increase the number of Spark executors and set `parallelism` in `SparkTrials` to a value greater than 1.
Why this is correct
SparkTrials distributes trials across Spark executors when parallelism is greater than one. By increasing executors and setting parallelism, multiple hyperparameter configurations run concurrently, reducing wall-clock time. This is the intended use of SparkTrials for single-node ML libraries like XGBoost, where each trial is independent. The search space remains unchanged, so model quality is not compromised.
- ✗
Switch from `SparkTrials` to `Trials` and run the search on the driver node only.
Why it's wrong here
Using `Trials` runs all hyperparameter trials sequentially on the driver, which would eliminate distributed execution and likely increase tuning time. The driver node also has limited resources compared to the cluster, making it a poor choice for speeding up a search. This change would not address the single-executor behavior and would worsen performance.
About these practice questions
This Databricks-ML-Pro question is part of Courseiva's 300-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-ML-Pro practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-ML-Pro exam.