Databricks-ML-Pro Model Development Practice Test 9 — 15 Questions

Question 1

A data scientist is using MLflow to track experiments on Databricks. They want to record the value of a hyperparameter named 'learning_rate' for a run. Which MLflow function should they use?

Accepted Answer

mlflow.log_param("learning_rate", 0.01). mlflow.log_param() is specifically designed to log hyperparameters as key-value pairs. It enables filtering and comparison of runs based on parameter values in the MLflow UI. The other functions log metrics, tags, or artifacts, which are not suitable for hyperparameters. Using the correct function ensures that the 'learning_rate' is properly recorded and can be used for experiment analysis.

Answer

mlflow.log_artifact("learning_rate", 0.01)

Answer

mlflow.log_metric("learning_rate", 0.01)

Answer

mlflow.set_tag("learning_rate", 0.01)

Question 2

A data scientist is using MLflow on Databricks to train a model with a custom training loop. They want to log the model so that it can be loaded later with `mlflow.pyfunc.load_model()` and used for batch inference. The model artifacts include a Python class and a configuration file. Which approach should they use to log the model?

Accepted Answer

Use `mlflow.pyfunc.log_model()` with a custom PythonModel class that defines `predict()` and `load_context()`.. To log a custom Python model that can be loaded as a PyFunc, you must define a PythonModel subclass and pass it to mlflow.pyfunc.log_model. This provides the load_context and predict methods that MLflow uses at inference time. Other flavors are tied to specific libraries and cannot handle arbitrary Python classes.

Answer

Use `mlflow.sklearn.log_model()` with the `signature` parameter.

Answer

Use `mlflow.tensorflow.log_model()` with `saved_model=True`.

Answer

Use `mlflow.log_artifact()` to save the Python class and configuration file, then log the model with `mlflow.pyfunc.log_model()` without a PythonModel.

Question 3

A data scientist is using MLflow on Databricks to track a series of experiments. They want to compare the performance of different runs and identify the best model based on a custom metric called "f1_score". Which MLflow feature should they use to efficiently compare and rank these runs?

Accepted Answer

MLflow Experiments page in the Databricks workspace, using the metric column and sorting.. The MLflow Experiments page in Databricks provides a built-in, user-friendly interface to view all runs, display metrics, and sort by any metric. This allows quick identification of the best run based on f1_score. Other options either require unnecessary coding, are not designed for run comparison, or use unsupported methods.

Answer

MLflow Model Registry, by registering each run's model and comparing versions.

Answer

Databricks SQL, by creating a dashboard that queries the MLflow tracking database directly.

Answer

MLflow Tracking API, by writing a custom script to query the REST API and compute rankings.

Question 4

An ML engineer is using MLflow to track a deep learning experiment with PyTorch on Databricks. They want to capture the model's architecture, optimizer state, and training metrics, and later reproduce the exact training run. They call `mlflow.pytorch.autolog()` before training. After several epochs, they notice that metrics are logged but the model signature is missing, and the logged model cannot be loaded for inference without specifying the input example. What should they do to ensure the model is properly logged with a signature?

Accepted Answer

Explicitly log the model using `mlflow.pytorch.log_model()` with the `signature` argument, computed from a sample input using `mlflow.models.infer_signature()`.. Autologging for PyTorch does not automatically infer a model signature unless an input example is provided. To ensure a signature, explicitly log the model with mlflow.pytorch.log_model and pass a signature generated by mlflow.models.infer_signature using sample input. This enables proper model loading and inference without manual input specification.

Answer

Use `mlflow.pytorch.save_model()` instead of `log_model()` to automatically include a signature.

Answer

Provide an input example to `mlflow.pytorch.autolog()` via the `log_every_n_step` parameter.

Answer

Set the environment variable `MLFLOW_LOG_MODEL_SIGNATURE` to `true` before training.

Question 5

A machine learning engineer is preparing a model for deployment using Databricks Model Serving. They need to ensure that the model's input schema is enforced and that the model can be served with a specific version. Which TWO actions should they perform? (Choose two.)

Accepted Answer

Log the model with an input example using mlflow.sklearn.log_model(..., input_example=...). Logging the model with an input example captures the input schema, which Model Serving uses to validate requests. Specifying the model version when creating the serving endpoint ensures the correct version is deployed. Together, these actions enforce schema and control versioning. The other options manage lifecycle, log artifacts, or scale resources, but do not directly achieve the stated requirements.

Answer

Enable autoscaling on the serving endpoint to handle varying load.

Answer

Register the model in the MLflow Model Registry and transition it to Production.

Answer

Use mlflow.log_artifact() to save a JSON schema file alongside the model.

Question 6

A machine learning engineer is training a model using MLflow on Databricks and wants to ensure that the model's input schema is captured and enforced during inference. They are using the `mlflow.pyfunc` flavor. Which action should they take to enable schema enforcement?

Accepted Answer

Log the model with the `signature` parameter, specifying the input and output schema.. The correct way to enable schema enforcement is to log the model with a signature. This signature is stored with the model and used by MLflow's serving components to validate incoming data. Tags and artifacts do not provide runtime enforcement, and autologging does not enforce schemas during inference.

Answer

Save the schema as a separate artifact and load it manually in the inference code.

Answer

Use `mlflow.set_tag` to record the schema as a JSON string in the run's tags.

Answer

Enable autologging for the specific framework, which automatically captures and enforces the schema.

Question 7

A machine learning engineer is using Hyperopt with SparkTrials to tune a scikit-learn model on a Databricks cluster. They set max_evals=100 and parallelism=4. After the run, they notice that some trials report a loss of NaN and that the best model selected by Hyperopt has poor performance. What is the most likely reason for the NaN losses?

Accepted Answer

The objective function returns a NaN when the model fails to converge, and Hyperopt treats NaN as a valid loss, potentially selecting a failed trial as best.. Hyperopt's fmin function minimizes the loss returned by the objective. If the objective returns NaN, Hyperopt may not handle it gracefully; NaN comparisons are always false, so the best loss may not update correctly, and a failed trial could be inadvertently selected. The correct approach is to ensure the objective returns a finite value, such as a large number, when the model fails to train. This prevents NaN from polluting the search.

Answer

SparkTrials distributes trials across workers, and the NaN is caused by a serialization error when returning the loss from the worker to the driver.

Answer

The parallelism parameter is set too high, causing race conditions in the shared search space that corrupt the loss values.

Answer

The model's hyperparameters include a regularization parameter that, when set too high, causes the coefficients to become NaN, and Hyperopt does not handle this.

Question 8

A data scientist is using MLflow to track experiments on Databricks. They want to record the value of a hyperparameter named 'learning_rate' for a run. Which MLflow function should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The model artifact was logged outside of an active MLflow run.. MLflow only logs artifacts when there is an active run. Calling mlflow.sklearn.log_model() outside a run context results in the artifact not being associated with any run, so it appears missing. Other options like size limits or overwrites are less likely. Ensuring the logging call is inside a with mlflow.start_run() block or after mlflow.start_run() will correctly attach the model artifact to the run.

Answer

The model artifact was overwritten by a subsequent run with the same name.

Answer

The model artifact was too large and exceeded the maximum artifact size.

Answer

The model artifact was logged to a different experiment than the one being viewed.

Question 9

A data scientist is using MLflow to log a custom PyTorch model on Databricks. They want to ensure that the model can be loaded and used for inference without requiring the original training code. Which MLflow feature should they use to package the model with its dependencies?

Accepted Answer

`mlflow.pytorch.log_model` with the `pip_requirements` parameter to specify dependencies.. Using `mlflow.pytorch.log_model` with `pip_requirements` packages the model with its dependencies, creating a self-contained artifact that can be loaded and served without the original training code. Other methods either require manual reconstruction or do not capture dependencies automatically, making them less reliable for deployment.

Answer

`mlflow.pytorch.save_model` to save the model in PyTorch's native format, then log the directory.

Answer

`mlflow.log_artifact` to save the model's state dict and a requirements.txt file.

Answer

`mlflow.pyfunc.log_model` with a custom Python function that loads the PyTorch model.

Question 10

A data scientist is using MLflow to track experiments on Databricks. They want to record the value of a hyperparameter named 'learning_rate' for a run. Which MLflow function should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use `mlflow.log_metric` within the training loop.. The correct method is to use `mlflow.log_metric` within the training loop. This function is designed for logging metrics and supports step-wise logging, which is ideal for tracking custom metrics over epochs. It integrates with MLflow's UI for visualization and comparison.

Answer

Use `mlflow.set_tag` to record the metric values.

Answer

Use `mlflow.log_param` to log the metric values.

Answer

Use `mlflow.log_artifact` to save a file containing the metrics.

Question 11

A data scientist is training a scikit-learn model on Databricks and wants to capture the best hyperparameters found during a hyperparameter sweep. They are using MLflow Tracking with nested runs. Which approach correctly records the best parameters and metrics in the parent run?

Accepted Answer

Log the best parameters and metrics directly in the parent run using `mlflow.log_params()` and `mlflow.log_metrics()` after the sweep completes.. In MLflow, nested runs allow you to organize hyperparameter sweeps. The parent run acts as a container, and child runs log individual trials. To capture the best parameters and metrics in the parent, you must explicitly log them after the sweep. MLflow does not auto-propagate, so manual logging in the parent run is required.

Answer

Set the parent run's status to 'FINISHED' and then use `mlflow.log_artifact()` to attach a JSON file containing the best parameters.

Answer

Use `mlflow.log_params()` inside each child run and rely on MLflow to automatically propagate the best parameters to the parent run.

Answer

Use `mlflow.start_run(nested=True)` for the parent run and `mlflow.start_run()` for child runs, then log the best parameters in the parent run.

Question 12

A machine learning team is using MLflow on Databricks to manage experiments. They want to ensure that their model training runs are reproducible and that they can compare different runs effectively. Which TWO practices should they follow? (Choose two.)

Accepted Answer

Log the evaluation metrics for each run using `mlflow.log_metrics()`.. Logging hyperparameters and evaluation metrics are fundamental for reproducibility and comparison. Hyperparameters record the configuration, while metrics provide quantitative performance. Together, they enable filtering, sorting, and selecting the best runs. Other practices like using a single run, storing full datasets, or date-based naming do not support effective experiment tracking.

Answer

Set the experiment name to the current date to avoid confusion.

Answer

Store the training dataset in the MLflow run's artifact repository for every run.

Answer

Use a single MLflow run for all experiments to simplify tracking.

Question 13

A machine learning engineer is using MLflow to track experiments on Databricks. They want to ensure that the model's input schema is enforced during inference to prevent errors from malformed data. Which MLflow feature should they use when logging the model?

Accepted Answer

Log the model with `mlflow.pyfunc.log_model()` and provide the `signature` parameter with a `ModelSignature` object.. MLflow model signatures define the expected input and output types. When a model is logged with a signature, MLflow can validate input data during inference, ensuring it matches the schema. This prevents errors from malformed data and improves reliability. The `signature` parameter is the correct way to enforce schema.

Answer

Use `mlflow.register_model()` with a schema definition in the model version description.

Answer

Log the model with `mlflow.pyfunc.log_model()` and set the `pip_requirements` parameter to include a schema validation library.

Answer

Log the model with `mlflow.sklearn.log_model()` and set `input_example` to a sample DataFrame.

Question 14

A data scientist is using Hyperopt with SparkTrials on a Databricks cluster to tune an XGBoost classifier. After several trials, they notice that each trial runs on a single executor and the overall tuning job takes much longer than expected. They want to speed up hyperparameter tuning without changing the search space. Which adjustment is most likely to improve performance?

Accepted Answer

Increase the number of Spark executors and set `parallelism` in `SparkTrials` to a value greater than 1.. SparkTrials parallelizes hyperparameter trials across Spark executors when `parallelism` is set above 1. The data scientist observed that each trial used a single executor, indicating that parallelism was effectively 1 or the cluster lacked executors. By adding executors and increasing parallelism, multiple trials run simultaneously, cutting total tuning time. The search space and model logic remain unchanged, so this is the direct fix.

Answer

Enable autoscaling on the cluster and set `parallelism` to 1 in `SparkTrials`.

Answer

Increase the `max_evals` parameter in the `fmin` call to run more trials.

Answer

Switch from `SparkTrials` to `Trials` and run the search on the driver node only.

Question 15

An ML engineer is training a model on Databricks using MLflow and wants to ensure that the training process is deterministic across runs. They set the random seed for NumPy, Python, and the machine learning framework. However, they observe that the model's performance varies slightly between runs on the same data and cluster configuration. Which factor is most likely causing the non-determinism?

Accepted Answer

Non-deterministic operations in the machine learning framework, such as GPU-accelerated training with cuDNN, which may not be fully deterministic even with seeds set.. Non-determinism in deep learning on GPUs often arises from cuDNN's non-deterministic algorithms. Even with seeds set, operations like convolutions may produce slightly different results due to atomic operations or algorithm selection. To achieve determinism, you must set framework-specific flags to enforce deterministic algorithms and disable benchmarking. Other factors like autologging or tracking server configuration do not affect model training randomness.

Answer

The cluster's autoscaling feature causing different numbers of executors for each run.

Answer

The use of `mlflow.autolog()` which introduces randomness in logging.

Answer

The MLflow tracking server not being configured with a persistent backend store.