Databricks-ML-Assoc Model Development Practice Test 8 — 15 Questions

Question 1

A data scientist is training a linear regression model using scikit-learn on Databricks. They want to track the model's hyperparameters, such as fit_intercept and normalize, in MLflow. Which MLflow API call should they use to log these hyperparameters?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

mlflow.log_param. To track hyperparameters in MLflow, the appropriate API is mlflow.log_param, which records key-value pairs that define the model's configuration. This allows the parameters to be visible in the MLflow UI and searchable via the API. Logging metrics, artifacts, or tags would not correctly capture hyperparameters and would hinder experiment comparison.

Answer

mlflow.log_artifact

Answer

mlflow.log_metric

Answer

mlflow.set_tag

Question 2

A data scientist is preparing a dataset for training a model on Databricks. They want to split the data into training and testing sets, and they need to ensure that the split is reproducible across different runs. Which PySpark method should they use to split the DataFrame with a fixed random seed?

Accepted Answer

df.randomSplit([0.8, 0.2], seed=42). randomSplit is the PySpark method designed to split a DataFrame into multiple parts based on weights, and it accepts a seed for reproducibility. This allows the data scientist to obtain consistent training and testing sets across runs. Other methods like sample, repartition, or cache do not provide a reproducible train-test split.

Answer

df.repartition(2)

Answer

df.sample(fraction=0.8, seed=42)

Answer

df.cache()

Question 3

A data scientist is developing a model on Databricks and wants to use MLflow to track experiments. They create a new experiment using mlflow.create_experiment('my_experiment') and then run mlflow.start_run(). However, when they log parameters and metrics, they notice that the run is not associated with 'my_experiment' but with the default experiment. What is the most likely reason?

Accepted Answer

The mlflow.create_experiment() function only creates the experiment but does not set it as the active experiment; they must call mlflow.set_experiment() to use it.. Creating an experiment with mlflow.create_experiment() does not automatically set it as the active experiment for subsequent runs. You must explicitly call mlflow.set_experiment() to designate the experiment for logging. Otherwise, runs default to the notebook's experiment or the default experiment. This separation allows flexible experiment management but requires an extra step.

Answer

The experiment was created in a different workspace, so the run cannot be associated with it.

Answer

The mlflow.start_run() function requires an experiment_id argument to associate the run with a specific experiment; without it, the run goes to the default experiment.

Answer

The experiment name 'my_experiment' is invalid because it contains an underscore; MLflow experiment names must be alphanumeric.

Question 4

A data scientist is training a scikit-learn model on a Databricks cluster with autoscaling enabled. They observe that each epoch takes significantly longer than expected, and the Spark UI shows very low CPU utilization across the worker nodes. The dataset is small enough to fit in memory on a single node. What is the most likely cause of the slow training?

Accepted Answer

The model is being trained on the driver node only, and the worker nodes are idle because the training code does not distribute the workload.. The correct answer is that the model is being trained on the driver node only, leaving worker nodes idle. In Databricks, standard scikit-learn training runs on the driver unless you explicitly distribute it. The low CPU utilization on workers and slow epochs confirm that the workload is not parallelized. To speed up training, you could use Spark ML, Horovod, or parallelize hyperparameter tuning with Hyperopt.

Answer

The cluster is using a runtime version that does not support scikit-learn, forcing a fallback to single-threaded execution.

Answer

The cluster's autoscaling policy is aggressively terminating worker nodes, causing the training to restart.

Answer

The data is stored in DBFS, and reading it for each epoch introduces network latency that throttles training.

Question 5

A data scientist is using MLflow to track experiments. They notice that all runs from a particular notebook are being logged to the default experiment instead of the experiment they intended to use. They have already called mlflow.start_run() without specifying an experiment ID. What is the most likely cause?

Accepted Answer

The active MLflow experiment was not set using mlflow.set_experiment before starting the run.. MLflow logs runs to the active experiment, which is set via mlflow.set_experiment. If not set, runs go to the default experiment (ID 0). The start_run function does not automatically infer the intended experiment from the notebook context. To fix this, the data scientist should call mlflow.set_experiment with the desired experiment name or pass experiment_id to start_run.

Answer

The MLflow tracking server is misconfigured, causing runs to be redirected to the default experiment.

Answer

The MLflow client library version is outdated and does not support custom experiments.

Answer

The run was started with nested=True, which forces logging to the default experiment.

Question 6

A data scientist is using Databricks Feature Store to build a training set for a model that predicts customer churn. The feature table contains a column `customer_id` and several features, and the label is stored in a separate Delta table. The data scientist wants to ensure that the exact same feature values used during training are available at inference time. Which approach correctly uses Databricks Feature Store to create the training set?

Accepted Answer

Use `fs.create_training_set()` with the feature table and label DataFrame, specifying `customer_id` as the lookup key.. The `create_training_set` method is specifically designed to join feature tables with labels using a lookup key, preserving feature lineage and ensuring that the same feature computations are used during training and inference. This prevents training-serving skew and simplifies deployment.

Answer

Perform a manual join between the feature table and the label table using Spark, then log the resulting DataFrame as an MLflow artifact.

Answer

Export the feature table to a CSV file and merge it with the label data using pandas, then log the model with MLflow.

Answer

Use `fs.log_model()` with the feature table name and specify the label column; the method automatically creates the training set.

Question 7

A machine learning engineer is using MLflow to log a model built with XGBoost. They call mlflow.xgboost.log_model(xgb_model, 'model') and then attempt to load the model in a different environment using mlflow.pyfunc.load_model('runs:/<run_id>/model'). The load fails with an error about missing dependencies. Which action should they take to ensure the model can be loaded in the new environment?

Accepted Answer

They should include the XGBoost library in the conda environment when logging the model, either by passing a custom conda_env or by ensuring the library is installed in the current environment so MLflow can infer it.. When logging a model with MLflow, dependencies are recorded in a conda environment file. If a required library like XGBoost is missing from that file, loading in a new environment fails. Ensuring the library is installed during logging allows MLflow to capture it, or you can specify a custom conda environment. This guarantees the model can be reproduced.

Answer

They should log the model with the registered_model_name parameter to ensure dependencies are captured.

Answer

They should save the model in the ONNX format instead, because ONNX models have no dependencies.

Answer

They should use mlflow.xgboost.load_model() instead of mlflow.pyfunc.load_model(), because the latter does not support XGBoost models.

Question 8

A machine learning engineer is using MLflow to track experiments. They want to compare multiple runs and identify the run that produced the best model based on a custom metric called 'weighted_f1'. They have logged this metric using mlflow.log_metric. Which MLflow UI feature allows them to sort and filter runs by this metric to quickly find the best run?

Accepted Answer

The runs table with column sorting and the filter box using metric.weighted_f1.. The MLflow UI runs table provides a sortable and filterable view of all runs in an experiment. By sorting on the 'weighted_f1' metric column or using a filter expression, the engineer can instantly locate the top-performing run. This is the standard way to compare runs and select the best model.

Answer

The experiment notes field where you can manually record the best run ID.

Answer

The model registry's stage transitions, which automatically promote the run with the highest metric.

Answer

The artifact viewer, which displays metric plots and allows sorting by value.

Question 9

A machine learning engineer is using MLflow to log a model trained with XGBoost. They want to ensure that the model can be loaded and used for inference in a different environment without requiring the original training environment. Which MLflow feature allows the model to capture its dependencies and environment?

Accepted Answer

MLmodel file with conda_env and requirements.txt. The MLmodel file, along with conda_env and requirements.txt, captures the dependencies required to load and run the model. When logging a model, MLflow automatically generates these files based on the current environment. This enables the model to be deployed in a different environment by recreating the necessary dependencies, ensuring consistent behavior.

Answer

Run ID

Answer

Model signature

Answer

Model version

Question 10

A machine learning engineer is using MLflow to track experiments in Databricks. They want to record the hyperparameters used for each run so that they can compare runs later. Which MLflow method should they use to log a single hyperparameter?

Accepted Answer

`mlflow.log_param()`. `mlflow.log_param()` is the correct method to record hyperparameters for an MLflow run. It stores key-value pairs that are displayed in the experiment UI and can be used to filter and compare runs, which is essential for hyperparameter tuning.

Answer

`mlflow.log_metric()`

Answer

`mlflow.set_tag()`

Answer

`mlflow.log_artifact()`

Question 11

A data scientist is using MLflow to log a model on Databricks. They want to ensure that the model can be loaded and used for inference in a different environment. Which two of the following are necessary components that must be included when logging the model to guarantee portability? (Choose two.)

Accepted Answer

The conda environment file (conda.yaml) that lists all dependencies.. For a model to be portable across environments, it must include a signature to define input/output schemas and a conda environment file to capture dependencies. These ensure that the model can be correctly loaded and executed elsewhere. Other elements like training data, run ID, or input examples are not required for the model to function.

Answer

The MLflow run ID where the model was logged.

Answer

The model's input example, which provides a sample input for testing.

Answer

The training dataset used to fit the model.

Question 12

A machine learning engineer is using MLflow on Databricks to track experiments for a fraud detection model. They notice that runs from two different team members are being logged into the same experiment, but the engineer wants to ensure that all runs from the current notebook session are automatically associated with a specific experiment. Which MLflow API call should the engineer use to set the active experiment for the current session?

Accepted Answer

mlflow.set_experiment("/Shared/fraud-detection"). The correct API is mlflow.set_experiment, which sets the active experiment for the current session. This ensures that all subsequent runs are logged to the specified experiment without needing to pass the experiment ID to each run. It is the standard way to organize runs by project or team in Databricks.

Answer

mlflow.start_run(experiment_id="12345")

Answer

mlflow.set_tracking_uri("databricks")

Answer

mlflow.create_experiment("/Shared/fraud-detection")

Question 13

A machine learning engineer is tuning a scikit-learn GradientBoostingClassifier on Databricks. They want to run 40 hyperparameter combinations, each trained on the full dataset, while keeping the driver free of model training work and collecting all results in a single MLflow parent run. Which approach should they use?

Accepted Answer

Use Hyperopt with the default SparkTrials and set max_evals to 40.. Distributing many single-node model fits requires a mechanism that schedules each trial as cluster work rather than driver-local computation. SparkTrials in Hyperopt does exactly this, and with MLflow autologging each trial becomes a nested run beneath one parent run, satisfying both the parallelism and the consolidated tracking requirements in a single, supported pattern.

Answer

Use scikit-learn's GridSearchCV with n_jobs=-1 inside a single notebook cell.

Answer

Use Hyperopt with Trials and wrap the objective function in a Pandas UDF.

Answer

Launch 40 separate notebooks with dbutils.notebook.run and merge their runs afterward.

Question 14

A data scientist is training a gradient boosting model using Spark MLlib on a large dataset in Databricks. They notice that the model's performance on a validation set is significantly worse than on the training set, and they suspect overfitting. They want to use MLflow to track hyperparameters and metrics to diagnose the issue. Which combination of MLflow logging practices will best help them identify overfitting across multiple runs?

Accepted Answer

Log training and validation metrics separately for each run, and log hyperparameters such as maxDepth and minInstancesPerNode.. To diagnose overfitting, it is essential to compare training and validation performance across runs. Logging both metrics separately, along with relevant hyperparameters that control model complexity, allows the data scientist to visualize the gap and tune accordingly. MLflow's tracking and comparison features then make it straightforward to identify runs where validation performance lags.

Answer

Log only the training metric and the model artifact, and rely on the MLflow UI to automatically compute validation metrics.

Answer

Log hyperparameters and the final model, but log metrics only at the end of training to reduce clutter.

Answer

Use MLflow autologging for Spark MLlib, which automatically logs training and validation metrics and hyperparameters.

Question 15

A data scientist is evaluating feature importance for a tree-based model trained on Databricks. They want to understand which features contribute most to the model's predictions. Which TWO methods are appropriate for extracting feature importance from a scikit-learn Random Forest model? (Choose two.)

Accepted Answer

Access the feature_importances_ attribute of the fitted model.. The feature_importances_ attribute of a fitted Random Forest provides a quick, impurity-based measure of global feature importance. SHAP values offer a more detailed, model-agnostic approach that attributes each feature's contribution to individual predictions, which can be aggregated for global importance. Both are valid methods for understanding feature influence. The other options do not yield feature importance: predict_proba returns probabilities, decision function coefficients are for linear models, and get_params returns hyperparameters.

Answer

Examine the coefficients of the model's decision function.

Answer

Use the model's get_params method to retrieve feature importance.

Answer

Use the model's predict_proba method to derive feature importance.