Databricks-ML-Assoc Practice Test 5 — 25 Questions

Question 1

Which technique should be used to prevent data leakage in ML workflows when performing cross-validation on time-series data?

Accepted Answer

Expanding window cross-validation (time-series split).. In time-series forecasting, standard k-fold cross-validation is inappropriate because it would allow the model to 'see' the future during training, leading to overly optimistic performance estimates. Instead, 'time-series split' or 'walk-forward' validation should be used, where the training set only consists of data points that occur chronologically before the validation set. This preserves the temporal order, providing a realistic assessment of the model's performance on unseen future data.

Answer

Random sampling with replacement across the entire dataset.

Answer

Standard k-fold cross-validation with 5 folds.

Answer

Removing all features that have high correlation with the target variable.

Question 2

When logging a model using MLflow in Databricks, which component is required to capture the environment dependencies so that the model can be accurately reproduced on a different cluster?

Accepted Answer

A conda.yaml or requirements.txt file specifying library versions.. Capturing environment dependencies is essential for reproducibility, as models often rely on specific library versions. MLflow uses the conda.yaml or requirements.txt files to snapshot the Python environment during the log_model process. This ensures that when the model is loaded in a deployment environment, it functions exactly as it did during development, preventing runtime errors caused by version mismatches or missing dependencies in the target production environment.

Answer

The raw input dataset in Parquet format.

Answer

The cluster's Spark configuration JSON string.

Answer

The raw model object without additional metadata.

Question 3

Which TWO of the following practices are recommended when performing feature engineering on Databricks using Feature Store to ensure consistency between training and inference?

Accepted Answer

Register feature tables in the Databricks Feature Store with unique primary keys.. Feature Store ensures that the exact same transformation logic used during model training is accessible during low-latency inference. By utilizing the Feature Store's unified interface, data scientists prevent training-serving skew, which is a common cause of poor model performance in production. This practice promotes reproducibility and streamlines the deployment pipeline by decoupling data preparation from model training code, ensuring that production features are calculated reliably using the same definitions as training features.

Answer

Hardcode all feature transformation logic directly into the model training notebook.

Answer

Calculate features in real-time within the model inference endpoint only.

Answer

Avoid using time-based features to ensure the model remains simple.

Question 4

Which technique is most effective for handling high-cardinality categorical features when training a tree-based model on Databricks?

Accepted Answer

Target encoding, replacing labels with the mean of the target variable.. Target encoding or utilizing native categorical support in libraries like LightGBM or CatBoost is highly effective for high-cardinality features. These methods prevent the dimensionality explosion associated with one-hot encoding, which would otherwise lead to sparse, massive feature matrices that degrade training performance and memory efficiency. By handling categories internally, models maintain predictive power while remaining performant in distributed Databricks environments.

Answer

One-hot encoding for all categorical variables.

Answer

Standardization using a StandardScaler.

Answer

Removing all categorical features entirely.

Question 5

A data scientist is training a Random Forest model on a 500GB dataset using Databricks. They notice that the model training process is consistently running out of memory on the driver node. Which approach should be taken to resolve this memory bottleneck while maintaining model performance?

Accepted Answer

Use the Spark MLlib RandomForestRegressor implementation to distribute computation across workers.. Random Forest models often store large ensembles and intermediate metadata in memory. When training on distributed data, using Spark ML's native distributed algorithms is essential. By moving computation to the worker nodes and leveraging the Spark RDD-based implementation, the driver node is relieved from holding the entire dataset or model state, allowing for successful training on larger datasets without frequent OOM errors that occur when pulling data locally.

Answer

Increase the driver node instance size to match the total dataset volume.

Answer

Convert the dataset to a Pandas DataFrame and train using the Scikit-Learn library directly.

Answer

Enable broadcast joins on the feature table before training the model.

Question 6

When deploying a model using Model Serving, how does Databricks ensure that the environment remains consistent between the training workspace and the serving environment?

Accepted Answer

By capturing the model's dependencies during log_model. Databricks uses Conda or virtual environment dependencies captured during the MLflow logging process. When the model is logged, the environment details, including library versions, are saved. During deployment, the serving infrastructure creates a container that replicates this environment. This practice is crucial for avoiding 'dependency hell', where models fail in production due to subtle library version mismatches compared to the original training environment.

Answer

By requiring the user to provide a Dockerfile

Answer

By strictly enforcing the use of the latest stable libraries

Answer

By running the model on the same training cluster

Question 7

Refer to the exhibit. What is the impact of setting 'auto_capture_request_payload' to true?

Accepted Answer

It stores incoming request data in a table for analysis and monitoring.. Enabling 'auto_capture_request_payload' is a powerful feature for debugging, auditing, and continuous model improvement. It allows teams to log the exact data received by the endpoint, enabling them to analyze prediction inputs, perform retrospective bias analysis, and verify data quality. This is a critical component of a robust MLOps strategy, as it provides the raw data needed to identify the causes of poor predictions and justify model decisions for compliance.

Answer

It automatically retrains the model whenever new data is captured.

Answer

It limits the endpoint to only accept JSON payloads.

Answer

It encrypts the payload before it reaches the model.

Question 8

A data scientist is using MLflow to track a hyperparameter tuning experiment with Spark MLlib's CrossValidator. They notice that each run in the MLflow UI shows only a single set of metrics, but they want to compare the performance of each hyperparameter combination across folds. What is the most effective way to log and visualize the per-combination and per-fold metrics in MLflow?

Accepted Answer

Create a separate MLflow run for each hyperparameter combination, and within each run, log the average metric across folds as well as individual fold metrics using distinct metric names.. The MLflow UI organizes metrics and parameters by run, making runs the natural unit for comparing hyperparameter combinations. By creating a separate run for each combination and logging both aggregate and per-fold metrics with distinct names, the data scientist can leverage the UI's comparison tools, such as parallel coordinates and scatter plots, to identify the best performing combination and understand fold-level variability.

Answer

Use MLflow's log_batch API to log all metrics and parameters for each combination in a single batch, without creating separate runs.

Answer

Log all metrics from all combinations into a single run, using metric names that encode the hyperparameters and fold numbers.

Answer

Use mlflow.log_metric with a step parameter for each fold, and log the hyperparameters as a JSON string in a single parameter.

Question 9

An ML engineer is orchestrating an end-to-end machine learning pipeline using Databricks Jobs. The pipeline consists of data preparation, distributed hyperparameter tuning with Hyperopt, and model registration. The engineer needs to pass the best-performing model's run ID from the Hyperopt task to the subsequent model registration task dynamically. Which mechanism should the engineer use in Databricks Jobs to achieve this?

Accepted Answer

Use dbutils.jobs.taskValues to set the run ID in the producer task and get the value in the consumer task.. Databricks Jobs task values allow tasks to pass small amounts of data, such as strings or numeric metrics, downstream to other tasks within the same workflow. Using dbutils.jobs.taskValues.set inside the Hyperopt notebook and dbutils.jobs.taskValues.get in the registration notebook enables seamless dynamic orchestration without relying on external storage solutions.

Answer

Store the run ID in a shared Delta table inside the workspace and query it from the downstream task.

Answer

Write the run ID to a text file in DBFS root and read it back in the downstream task.

Answer

Pass the run ID as a static parameter defined in the JSON job configuration before execution begins.

Question 10

Which TWO factors should be considered when choosing between Batch Inference and Real-time Inference for a model in Databricks?

Accepted Answer

The required latency for the model predictions to reach the end user.. Choosing between batch and real-time inference depends on latency requirements and data availability. Batch is suited for high-volume, non-time-sensitive tasks, whereas real-time is necessary for low-latency, event-driven predictions. Understanding these trade-offs is essential for ML architects to design cost-effective and performant serving strategies that align with business needs, as they dictate the infrastructure configuration, monitoring complexity, and the overall deployment strategy for the model lifecycle.

Answer

The memory limit of the driver node during the model training phase.

Answer

The number of features used in the machine learning model.

Answer

The programming language used to write the ML model training code.

Question 11

A data scientist is training a scikit-learn model on a large dataset using Databricks. They want to speed up hyperparameter tuning by running trials in parallel across a cluster. Which Databricks tool should they use?

Accepted Answer

Hyperopt with SparkTrials. Hyperopt with SparkTrials is the Databricks-recommended tool for parallel hyperparameter tuning. It distributes trials across Spark executors, leverages MLflow for logging, and supports advanced search algorithms. The other options either do not parallelize tuning or are not designed for custom parallel search.

Answer

MLflow Tracking with nested runs

Answer

Databricks AutoML

Answer

Pandas UDFs

Question 12

When configuring a Databricks Workflow to automate a machine learning pipeline, which TWO actions are necessary to ensure the pipeline is robust and manageable?

Accepted Answer

Use Databricks Workflow tasks to modularize data preparation, training, and evaluation.. Robust ML workflows require modularity and error handling. Defining distinct tasks for data preparation, model training, and evaluation allows for granular retries and easier debugging. Furthermore, enabling notifications ensures that failures are addressed promptly. By using job parameters and dependencies, you create a declarative pipeline that is consistent across environments, reducing the risk of manual configuration errors during deployment cycles and improving overall reliability of the production machine learning system.

Answer

Embed all data processing and training logic into a single, massive notebook cell.

Answer

Hardcode all file paths and cluster configuration settings directly in the code.

Answer

Run all pipeline stages on the same shared interactive cluster to save costs.

Question 13

Which TWO of the following are primary benefits of using the Databricks Feature Store for machine learning workflows?

Accepted Answer

Eliminating training-serving skew by using the same feature definitions for both training and inference.. The Databricks Feature Store facilitates feature reuse across teams and prevents training-serving skew. By centralizing features, organizations ensure that the exact data used during model training is consistently applied during real-time inference. This eliminates the need for redundant feature engineering pipelines and provides a unified lineage, which is essential for auditability and ensuring that models perform reliably across different production environments.

Answer

Providing a platform for distributed model training using Spark MLlib exclusively.

Answer

Automating the deployment of machine learning models to production environments.

Answer

Directly replacing the need for SQL-based data warehousing in Databricks.

Question 14

A data science team is preparing to deploy a custom machine learning model to Databricks Model Serving. Which TWO steps are required to ensure the model can successfully load and serve predictions using MLflow? Choose 2 answers.

Accepted Answer

Log the model using mlflow.sklearn.log_model() or equivalent flavor with a defined input example and model signature.. Deploying models to Databricks Model Serving requires logging the model with a valid signature and environment specification, and registering it to the Unity Catalog or Workspace Model Registry. These steps ensure the serving infrastructure can automatically provision the correct scoring environment, handle incoming JSON payloads, and validate data types against expected schemas.

Answer

Manually install all Python dependencies on every serving endpoint worker node via a custom startup script.

Answer

Export the model artifact into a raw Hadoop Distributed File System directory without using the MLflow format.

Answer

Configure Apache Spark streaming jobs to continuously push payload batches into the model serving container endpoint.

Question 15

A data scientist wants to speed up the process of finding the optimal hyperparameters for a machine learning model. Which Databricks-supported library is optimized for distributed hyperparameter tuning?

Accepted Answer

Hyperopt. Hyperopt is a widely used Python library for distributed hyperparameter optimization. In Databricks, it integrates seamlessly with Spark, allowing the distribution of trial runs across a cluster of nodes. This dramatically reduces the time required to search through large hyperparameter spaces, enabling data scientists to train more complex models and achieve better performance in a fraction of the time compared to serial tuning.

Answer

Pandas

Answer

Matplotlib

Answer

MLflow Tracking

Question 16

You are building a pipeline in Databricks and need to ensure that a training job only runs after the upstream data preparation job has successfully completed. Which Databricks feature should you use?

Accepted Answer

Use Databricks Workflows task dependencies.. Databricks Workflows allows for the creation of complex pipelines using task dependencies. By defining a dependency between the data preparation task and the training task, the orchestrator ensures the correct execution order. This dependency management is foundational for building reliable end-to-end ML pipelines, ensuring that models are always trained on processed, valid data and preventing failures caused by missing or stale input data during the automated training process.

Answer

Use the mlflow.end_run() command at the end of the data prep script.

Answer

Use Delta Live Tables (DLT) expectations to trigger the job.

Answer

Use a cron schedule with a 30-minute delay for the training job.

Question 17

A data scientist is using Databricks AutoML to train a classification model on a dataset with a highly imbalanced target variable. They want to ensure the model evaluation focuses on the minority class. Which evaluation metric should they prioritize when interpreting AutoML results?

Accepted Answer

F1 score. F1 score balances precision and recall, making it suitable for imbalanced classification where the minority class is of interest. AUC and accuracy can be misleading, and MSE is for regression. Prioritizing F1 ensures the model captures the minority class effectively.

Answer

Mean squared error (MSE)

Answer

Accuracy

Answer

Area under the ROC curve (AUC)

Question 18

A machine learning engineer is using Databricks Feature Store to create a feature table for a model that predicts customer churn. The feature table includes customer demographics and transaction history. The engineer wants to ensure that the model can access the latest feature values during online inference. What should the engineer do?

Accepted Answer

Publish the feature table to an online store using the Databricks Feature Store API.. To serve features for online inference, the feature table must be published to an online store. Databricks Feature Store provides APIs to publish feature tables to low-latency databases, enabling the model to retrieve the latest feature values in real time. This is a key step when deploying models that require online features, ensuring consistency between training and serving.

Answer

Enable change data feed on the feature table to capture updates.

Answer

Set up a Databricks SQL warehouse to query the feature table.

Answer

Use MLflow to log the feature table as a model artifact.

Question 19

A data scientist has registered a model in the Databricks Model Registry. They want to transition the model from 'Staging' to 'Production' but need to ensure that only specific users can perform this transition. Which Databricks feature should they use to enforce this access control?

Accepted Answer

Model Registry permissions with stage-level access control.. The Model Registry provides granular permissions at the model level, including the ability to restrict stage transitions. By configuring stage-level access control, administrators can ensure that only authorized users can move a model to Production. Other features like cluster ACLs, secret scopes, and volume permissions do not govern model registry operations.

Answer

Cluster access control lists (ACLs).

Answer

Unity Catalog volume permissions.

Answer

Workspace-level secret scopes.

Question 20

A data scientist is using MLflow on Databricks to track experiments. After running several training jobs, they notice that the run metrics are recorded but the model artifact is missing when they view the run details. They logged the model using the default MLflow API. What is the most likely cause?

Accepted Answer

The model artifact was logged outside of the active MLflow run context, so it was not associated with the run.. The model artifact is missing because log_model was called outside an active MLflow run, so it was not linked to the run. Metrics were logged within the run, explaining their presence. To ensure artifacts are associated, all logging calls must occur inside a run context created by mlflow.start_run(). This is a common oversight when refactoring code or using helper functions.

Answer

The model was too large to be stored in the MLflow artifact store, so it was automatically skipped.

Answer

The model was logged using mlflow.sklearn.log_model, but the run was not ended, so artifacts are not persisted.

Answer

The MLflow tracking server was not configured to log artifacts, so only metrics were recorded.