Databricks-ML-Pro Model Development Practice Question
A machine learning engineer is using Databricks AutoML to train a classification model. They notice that the best model from AutoML has a high F1 score on the validation set but performs poorly on a holdout test set. They suspect that the data has a temporal component and that the default train/validation split is causing data leakage. What should they do to address this?
⚠ Common exam trap
The trap here is assuming that increasing validation size or enabling cross-validation automatically respects temporal order, when only specifying the time column triggers a chronological split.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Re-run AutoML with the `time_col` parameter set to the timestamp column, so that AutoML uses a chronological split for training and validation.
Databricks AutoML provides the `time_col` parameter to handle temporal data. When set, AutoML performs a chronological split, ensuring that training data precedes validation data, which prevents leakage from future data. This is the correct way to address the poor holdout performance caused by a random split. Other options do not fix the temporal leakage issue.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Enable cross-validation with 5 folds in AutoML by setting `n_folds` to 5, which will ensure that each fold respects temporal order.
Why it's wrong here
AutoML's cross-validation does not automatically respect temporal order unless a time column is specified. Setting `n_folds` alone does not prevent leakage; the folds are still random. To get temporal cross-validation, you must set `time_col`. This option misinterprets the effect of `n_folds` on data splitting.
- ✗
Use `mlflow.log_param` to record the timestamp column and then manually retrain the best model with a custom time-based split outside of AutoML.
Why it's wrong here
While manually retraining with a time-based split can work, it bypasses AutoML's built-in support for temporal data. Logging the timestamp as a parameter does not affect AutoML's splitting. This approach is inefficient and unnecessary since AutoML provides the `time_col` parameter. The correct solution is to leverage that parameter rather than manually intervening.
- ✓
Re-run AutoML with the `time_col` parameter set to the timestamp column, so that AutoML uses a chronological split for training and validation.
Why this is correct
Databricks AutoML supports a `time_col` parameter that specifies a time column for temporal data. When set, AutoML performs a chronological split, ensuring that training data precedes validation data. This prevents leakage from future data into the training set. It is the correct approach for time-series or temporally ordered data to get realistic validation performance.
- ✗
Increase the size of the validation set by setting `train_validation_split` to 0.5, which will reduce overfitting and improve generalization.
Why it's wrong here
Increasing the validation set size does not address temporal leakage. If the split is random, future data may still leak into training. A larger validation set may reduce variance but will not fix the fundamental issue. The problem is the splitting strategy, not the proportion. This change would not resolve the poor holdout performance.
About these practice questions
This Databricks-ML-Pro question is part of Courseiva's 300-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-ML-Pro practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-ML-Pro exam.