Databricks-ML-Pro Model Development Practice Test 4 — 15 Questions

Question 1

A data scientist has registered a model in the MLflow Model Registry and wants to transition it to the Staging stage for pre-production testing. They have been assigned the `Can Manage` permission on the registered model. Which action can they perform?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Transition the model version to Staging and add a description.. `Can Manage` permission on a registered model grants the ability to transition model versions to any stage, including Staging, and to edit metadata such as descriptions. It does not automatically grant approval rights for Production if an approval workflow is configured. For the stated goal of pre-production testing, transitioning to Staging is fully within scope.

Answer

Only view the model version details without making changes.

Answer

Approve a request to transition the model to Production.

Answer

Transition the model version to Production and delete the registered model.

Question 2

A data scientist is comparing several MLflow runs for a fraud detection model. They want to quickly identify the run that achieved the highest area under the ROC curve (AUC) across all experiments in the workspace, without manually inspecting each run. Which MLflow feature should they use?

Accepted Answer

Use the MLflow search_runs API with an order_by clause on the AUC metric and a filter across experiments.. The MLflow search_runs API supports filtering and ordering across multiple experiments when you supply a list of experiment IDs. By ordering on the AUC metric descending, the data scientist gets the best run immediately. The UI comparison view is limited to one experiment, and the Model Registry only covers registered versions, so neither meets the cross-experiment requirement.

Answer

Use the MLflow UI's experiment comparison view and sort runs by the AUC metric.

Answer

Use the Databricks Model Registry to list all model versions and compare their AUC values.

Answer

Use mlflow.get_run() in a loop over all run IDs obtained from the Databricks Jobs API.

Question 3

A machine learning engineer is using MLflow to track experiments for a computer vision model. They want to log a confusion matrix as an artifact and also log the per-class precision and recall as metrics. They also want to compare these metrics across multiple runs in the MLflow UI. Which combination of MLflow logging APIs should they use?

Accepted Answer

Log the confusion matrix as an artifact using `mlflow.log_artifact()` with a PNG file, and log per-class precision and recall using `mlflow.log_metrics()` with metric names like `precision_class_0`.. The most effective approach is to log the confusion matrix as a PNG artifact for visual inspection and log per-class metrics with `mlflow.log_metrics()` for quantitative comparison in the MLflow UI. This leverages the UI's artifact viewer for images and its metrics comparison charts, enabling efficient analysis across runs.

Answer

Log the confusion matrix as a table using `mlflow.log_table()`, and log per-class precision and recall using `mlflow.log_metrics()` with metric names like `precision_class_0`.

Answer

Log the confusion matrix as a JSON artifact using `mlflow.log_dict()`, and log per-class precision and recall using `mlflow.log_metrics()` with metric names like `precision_class_0`.

Answer

Log the confusion matrix as a figure using `mlflow.log_figure()`, and log per-class precision and recall using `mlflow.log_metrics()` with metric names like `precision_class_0`.

Question 4

A data scientist is using MLflow to track experiments on Databricks. They want to compare multiple runs and identify the one with the lowest validation loss. Which MLflow feature should they use?

Accepted Answer

The MLflow Tracking UI's chart view, which allows plotting metrics across runs.. The MLflow Tracking UI's chart view is specifically designed for comparing runs visually. It allows plotting metrics such as validation loss across multiple runs, making it easy to spot the best run. Other features like the Model Registry or Projects serve different purposes and do not offer this direct comparison.

Answer

The MLflow Model Registry's stage transitions, which show the performance of models in different stages.

Answer

The `mlflow.search_runs` API, which returns a pandas DataFrame of runs that can be sorted by metrics.

Answer

The MLflow Projects feature, which packages code and parameters for reproducibility.

Question 5

A machine learning engineer is using MLflow to track experiments for a scikit-learn model. They want to ensure that every run automatically logs the model's signature, input examples, and environment dependencies to enable reproducibility and deployment. Which MLflow feature should they enable?

Accepted Answer

`mlflow.autolog()`. `mlflow.autolog()` is the unified autologging entry point that automatically instruments supported libraries, including scikit-learn. It logs parameters, metrics, model artifacts, and, importantly, model signatures, input examples, and environment details when available. This satisfies the need for consistent, automatic capture of reproducibility and deployment metadata. Manual logging or tracking URI configuration alone would not provide the same breadth of automatic logging.

Answer

`mlflow.set_tracking_uri()`

Answer

`mlflow.sklearn.autolog()`

Answer

`mlflow.log_model()` with `signature` and `input_example` arguments

Question 6

A machine learning engineer is developing a scikit-learn model in a Databricks notebook and wants to ensure that every training run's parameters, metrics, and the resulting model artifact are automatically captured in MLflow, including the model's input/output schema. They call `mlflow.sklearn.autolog()` at the start of the notebook. After several runs, they notice that the model signature is not being logged. Which action should they take to ensure the signature is captured?

Accepted Answer

Call `mlflow.sklearn.autolog(log_input_examples=True)` before training, ensuring that a sample of input data is available to infer the signature.. Autologging for scikit-learn in Databricks can infer and log the model signature when input examples are captured. Enabling `log_input_examples=True` ensures that a sample of the training data is available for signature inference. The other options either reference unsupported parameters or rely on manual logging that bypasses autologging. The correct approach is to configure autologging to capture input examples, which then allows MLflow to automatically log the signature.

Answer

Set the MLflow configuration `mlflow.sklearn.log_model_kwargs` to include `signature=True` before calling `mlflow.sklearn.autolog()`.

Answer

After training, manually call `mlflow.sklearn.log_model(model, 'model', signature=infer_signature(X_train, y_train))` to log the signature separately.

Answer

Use `mlflow.sklearn.autolog(disable=False)` and then call `mlflow.log_dict` with the model signature as a dictionary.

Question 7

A machine learning engineer is using Hyperopt with SparkTrials on a Databricks cluster to tune a scikit-learn model. They notice that each trial logs its own MLflow run, but the parent run does not capture the best hyperparameters or the best model. They want the best model to be automatically logged and easily retrievable. What should they do?

Accepted Answer

Use the MLflow autologging integration for Hyperopt, which automatically logs the best parameters and model to the parent run.. The MLflow Hyperopt autologging integration automatically creates a parent run and nested child runs for each trial. It logs the best parameters and the best model to the parent run, making the best result easily retrievable. Manually wrapping fmin or setting tracking URIs does not provide this automatic capture, and the best_run argument does not exist.

Answer

Set the environment variable MLFLOW_TRACKING_URI to the Databricks workspace URL before calling fmin.

Answer

Call fmin with the argument best_run=True, which instructs Hyperopt to log the best model to the parent run.

Answer

Use mlflow.start_run() with nested=True around the fmin call and manually log the best model after fmin returns.

Question 8

A data scientist is using Spark MLlib on Databricks to build a recommendation model with the ALS algorithm. They need to tune the hyperparameters `rank`, `regParam`, and `maxIter` to minimize RMSE on a validation set. They want to perform a systematic search that runs trials in parallel on the existing cluster and automatically logs each trial's parameters and metrics to MLflow. Which approach should they use?

Accepted Answer

Use `ParamGridBuilder` and `CrossValidator` from Spark MLlib, setting the evaluator to `RegressionEvaluator` with metricName "rmse".. Spark MLlib provides native tools for distributed hyperparameter tuning. Using `ParamGridBuilder` to define the search space and `CrossValidator` to evaluate combinations with a `RegressionEvaluator` ensures systematic k-fold cross-validation, parallel execution across the cluster, and automatic MLflow logging when run within an MLflow experiment. This is the recommended approach for tuning Spark ML estimators on Databricks.

Answer

Use `Hyperopt` with the `SparkTrials` backend and define an objective function that trains an ALS model and returns the RMSE.

Answer

Use `TrainValidationSplit` with a `ParamGridBuilder` and a `RegressionEvaluator` to evaluate the model.

Answer

Use MLflow's `mlflow.spark.autolog()` and manually loop over parameter combinations, logging metrics with `mlflow.log_metric`.

Question 9

A data scientist is using MLflow tracking on Databricks to log experiments for a computer vision model. They want to ensure that all relevant information for reproducibility is captured. Which TWO of the following should they log explicitly to enable full reproducibility? (Choose two.)

Accepted Answer

The random seed used for data splitting and model initialization.. To reproduce a machine learning experiment, it is essential to capture all sources of randomness and environment dependencies. The random seed ensures deterministic behavior, while the framework version captures the software environment. Together, they allow another data scientist to rerun the experiment and obtain the same results, assuming the same data and code. Other details like source code and hardware are helpful but not as universally critical.

Answer

The exact command line arguments passed to the training script.

Answer

The full source code of the training script as an artifact.

Answer

The hardware specifications of the cluster used for training.

Question 10

A data scientist is developing a model using Databricks AutoML. They want to ensure that the best model is automatically registered in the MLflow Model Registry and that the model can be easily deployed for batch inference. After running AutoML, they notice that the best model is not registered. Which configuration should they check to enable automatic registration?

Accepted Answer

The `registered_model_name` parameter in the AutoML experiment configuration must be set to a valid model name.. Databricks AutoML automatically registers the best model if the `registered_model_name` parameter is provided when configuring the experiment. This parameter tells AutoML to register the best model under the specified name in the MLflow Model Registry. Without it, the model remains only as an MLflow artifact. Therefore, checking and setting this parameter is the correct action to enable automatic registration.

Answer

The model must be manually registered after AutoML completes by using `mlflow.register_model` with the best run ID.

Answer

The `enable_registration` flag in the AutoML `run` method must be set to `True`.

Answer

The `mlflow.autolog()` function must be called before starting the AutoML run to enable registration.

Question 11

A data scientist is developing a model on Databricks and wants to ensure that the model's input schema is enforced during inference to prevent errors from malformed data. Which TWO actions should they take when logging the model to MLflow? (Choose two.)

Accepted Answer

Use `mlflow.pyfunc.log_model()` with a custom Python model that validates input against a predefined schema.. To enforce input schema during inference, the model must carry schema metadata and ideally include validation logic. Providing a signature via `mlflow.log_model()` defines the expected schema, and MLflow serving can use it for basic validation. For stricter enforcement, a custom PyFunc model can validate inputs against a schema before prediction. Together, these actions ensure malformed data is caught, reducing inference errors.

Answer

Register the model in the MLflow Model Registry and add a description of the schema.

Answer

Log an `input_example` to `mlflow.log_model()` to illustrate the expected data format.

Answer

Enable autologging with `mlflow.autolog()` to automatically capture the schema from training data.

Question 12

A data scientist is using Hyperopt with SparkTrials on a Databricks cluster to tune a gradient boosting model. They notice that the tuning process is slow and many trials are being pruned early, but the best model found has poor performance. They want to improve the tuning efficiency and final model quality. Which adjustment to the Hyperopt configuration is most likely to achieve this?

Accepted Answer

Set the `pruner` parameter in `fmin` to `hyperopt.pruners.NoPruner()` to disable pruning, and increase the `max_evals` to allow more trials to complete.. Early pruning in Hyperopt can terminate trials before they have a chance to fully evaluate, leading to suboptimal models. Disabling pruning with `NoPruner()` ensures each trial completes, and increasing `max_evals` allows a more thorough search. While this may increase tuning time, it directly improves the likelihood of finding a better model. The other options either do not address pruning or introduce changes that could further degrade performance.

Answer

Use `hyperopt.tpe.suggest` instead of `hyperopt.rand.suggest` for the search algorithm, and set `parallelism` to 1 to avoid conflicts.

Answer

Increase the `max_evals` parameter to allow more trials, and set `parallelism` to a higher value to use more Spark executors.

Answer

Adjust the `early_stopping_rounds` parameter in the gradient boosting model to a larger value, and use `Hyperopt`'s `SparkTrials` with `timeout` set to a higher value.

Question 13

A machine learning engineer is developing a scikit-learn model on Databricks and wants to ensure that the model's input schema is enforced during scoring. They log the model using `mlflow.sklearn.log_model` with the `signature` parameter. Which statement accurately describes the behavior when the model is later served via MLflow Model Serving?

Accepted Answer

The signature enables MLflow to perform input validation, rejecting requests that do not match the expected schema, including column names and data types.. Providing a signature when logging a model captures the expected input schema. MLflow Model Serving uses this signature to validate incoming requests, ensuring that the data matches the expected column names and types. This validation prevents scoring errors due to schema mismatches and is a best practice for production deployments. The signature does not perform automatic type conversion, so requests with incorrect types are rejected.

Answer

The signature is used only for documentation and does not affect input validation during serving.

Answer

MLflow Model Serving will reject requests with missing or extra columns, but it does not enforce data types.

Answer

The signature allows MLflow to automatically convert incoming data to the expected types, so type mismatches are silently corrected.

Question 14

A machine learning engineer is using MLflow to track experiments for a deep learning model on Databricks. They want to ensure that the model training runs are reproducible. Which TWO practices should they follow to maximize reproducibility? (Choose two.)

Accepted Answer

Log the exact versions of all Python libraries and the Databricks Runtime version as tags or parameters.. To maximize reproducibility, the engineer should log random seeds and library/runtime versions. Random seeds ensure that stochastic processes like weight initialization and data shuffling are repeatable. Library and runtime versions ensure the computational environment is consistent, as different versions can alter results. Together, these practices allow exact replication of training runs, which is essential for debugging and validation.

Answer

Use `mlflow.pytorch.autolog()` to automatically log all hyperparameters, metrics, and model artifacts.

Answer

Use MLflow Projects to package the code and dependencies, and log the Git commit hash as a tag.

Answer

Store the training dataset in DBFS and log its path as a parameter, assuming the data will not change.

Question 15

A data scientist is using MLflow to log a custom PyFunc model that wraps a pre-processing step and a trained XGBoost model. They want to ensure that the model can be loaded and used for batch inference on a Spark DataFrame without errors. Which approach should they take when defining the PyFunc model's predict method?

Accepted Answer

Accept a pandas DataFrame or Spark DataFrame as input and return the same type, using the appropriate APIs for each.. For a PyFunc model to work with both pandas and Spark DataFrames, the predict method should accept either type and return the corresponding type. This allows the model to be used for batch inference on Spark without costly conversions. The other options either force conversions that do not scale or assume automatic conversions that MLflow does not perform.

Answer

Define the predict method to accept only NumPy arrays, as MLflow automatically converts Spark DataFrames to NumPy arrays.

Answer

Use pandas DataFrames for input and output, and convert Spark DataFrames to pandas before calling predict.

Answer

Use the mlflow.pyfunc.PythonModel class and override the predict method to return a Spark DataFrame always, regardless of input type.