Databricks-ML-Pro Model Development Practice Test 10 — 15 Questions

Question 1

A data scientist is training a model with scikit-learn on Databricks and wants to track the experiment using MLflow. They call mlflow.start_run() and then train the model. After training, they call mlflow.log_param() and mlflow.log_metric(), but later find that the run is not visible in the MLflow experiment UI. What is the most likely reason?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

They did not set the MLflow tracking URI or experiment, so the run was logged to a different experiment or the default location.. The most common reason for not seeing a run in the expected experiment is that the tracking URI or experiment was not set correctly. MLflow defaults to a specific experiment, and if the user is looking at a different one, the run appears missing. Explicitly setting the experiment with mlflow.set_experiment() or the tracking URI ensures the run is logged to the intended location. Other options would typically cause errors or still show the run.

Answer

They forgot to call mlflow.end_run() to close the run, so it remains in a running state and is not displayed.

Answer

They called mlflow.log_param() and mlflow.log_metric() outside of the active run context, so the data was discarded.

Answer

The model training did not produce any metrics, so MLflow skipped creating the run.

Question 2

A data scientist is developing a scikit-learn model on Databricks and wants to log the model artifact to MLflow so that it can later be deployed for online inference. They call mlflow.sklearn.log_model() without providing a signature. What is the primary consequence of omitting the model signature?

Accepted Answer

The model will be logged, but MLflow cannot infer the input and output schema for deployment, potentially causing issues in serving or scoring.. Providing a signature when logging a model captures the expected input and output schema, which is essential for reliable deployment and scoring. Without it, MLflow cannot validate incoming data against the model's expectations, potentially leading to errors in serving or batch inference. The model still logs successfully, but the lack of schema metadata can cause integration problems.

Answer

MLflow will not log the model at all because a signature is mandatory for scikit-learn models.

Answer

The model will be logged, but MLflow Model Serving will reject requests because it cannot validate the input schema.

Answer

MLflow will automatically generate a signature by inspecting the training data, so omitting it has no effect.

Question 3

A machine learning engineer is using MLflow to log a model trained with a custom algorithm. They want to ensure that the model can be served with a specific input schema and that the schema is enforced during inference. Which MLflow feature should they use?

Accepted Answer

Model signature with input and output schema. The model signature is the MLflow feature that captures the expected input and output schema. When a model with a signature is served, MLflow validates incoming data against the schema, rejecting mismatches. This ensures that the model receives data in the correct format and types. Other features like tags, stages, and projects serve different purposes in the lifecycle.

Answer

MLflow run tags

Answer

Model registry stage transitions

Answer

MLflow projects with a conda environment

Question 4

A data scientist is using Databricks Feature Store to build a training set for a fraud detection model. They define a feature table with a primary key of `transaction_id` and a timestamp key of `event_ts`. When creating the training set with `create_training_set`, they specify `lookup_key=['transaction_id']`. The resulting training set contains features from multiple feature tables. Which statement describes how point-in-time correctness is ensured during this operation?

Accepted Answer

The timestamp key is automatically used to join features as of the timestamp of each label event, preventing data leakage from future feature values.. Point-in-time correctness in Databricks Feature Store is achieved by time-series joins using the timestamp key. When creating a training set, the system matches each label row with feature values whose timestamps are less than or equal to the label timestamp. This ensures that only historical, non-leaking features are used. The primary key alone is insufficient; the timestamp key is essential for temporal alignment.

Answer

The `create_training_set` function automatically sorts features by their creation date and selects the latest values before the label timestamp.

Answer

The primary key alone guarantees point-in-time correctness because it uniquely identifies each transaction and its associated features.

Answer

Point-in-time correctness is enforced only if the feature tables are registered with a `timestamp` column that matches the label DataFrame's index.

Question 5

A data scientist is developing a model on Databricks and wants to use MLflow to compare multiple runs. They need to quickly identify the run with the lowest validation loss. Which MLflow UI feature allows them to sort and filter runs based on metrics?

Accepted Answer

The runs table in the MLflow experiment UI, where you can sort by metrics and apply filters.. The MLflow experiment UI includes a runs table that lists all runs with their metrics. You can sort by any metric, such as validation loss, to find the lowest value. Filters can also be applied to focus on specific runs. This is the standard way to compare runs visually.

Answer

The artifacts tab of a run, where you can view logged metric files and manually sort them.

Answer

The notebook's output cell, where you can print a DataFrame of runs and sort it.

Answer

The model registry page, where you can compare model versions by their metrics.

Question 6

A machine learning engineer is using MLflow to log a custom PyTorch model. They define a custom pyfunc class that inherits from mlflow.pyfunc.PythonModel and implements predict(). After logging the model with mlflow.pyfunc.log_model(), they load it with mlflow.pyfunc.load_model() and call predict() with a pandas DataFrame. The prediction fails with an error about missing context. What is the most likely cause?

Accepted Answer

The predict() method signature must include a context parameter as the first argument, but the implementation omitted it.. MLflow's PythonModel.predict() method is defined as predict(self, context, model_input). The context argument is always passed by MLflow when calling predict. If the custom class defines predict(self, model_input), Python will raise a TypeError about missing arguments when MLflow attempts to call it with both context and model_input. The fix is to include context in the method signature, even if it is not used.

Answer

The model was logged without specifying the conda_env, so the context cannot be reconstructed during loading.

Answer

The pandas DataFrame passed to predict() must be converted to a NumPy array first, otherwise the context is not passed.

Answer

The custom pyfunc class must inherit from mlflow.pyfunc.PythonModel and also implement load_context(), which is missing.

Question 7

A machine learning engineer is using Hyperopt with SparkTrials on a Databricks cluster to tune a gradient boosting model. They notice that the tuning job is running slowly because each trial trains on the full dataset, and they want to speed up the search without sacrificing final model quality. Which approach is most appropriate?

Accepted Answer

Use a smaller subset of the training data for each trial and then retrain the best model on the full dataset.. Using a data subset for hyperparameter tuning accelerates the search by reducing training time per trial, enabling more configurations to be evaluated. Once the best hyperparameters are found, retraining on the full dataset ensures the final model benefits from all available data. This balances efficiency and quality, especially when full-dataset training is expensive.

Answer

Increase the parallelism parameter in SparkTrials to run more trials concurrently.

Answer

Reduce the number of trials and increase the max_evals parameter to compensate.

Answer

Switch from SparkTrials to Trials to avoid distributed overhead.

Question 8

A machine learning engineer is using MLflow to track experiments on Databricks. They want to compare multiple runs of a scikit-learn model and automatically log the best model to the Model Registry. They use `mlflow.sklearn.autolog()` and then call `mlflow.sklearn.log_model` with `registered_model_name`. However, they notice that the model version in the registry does not include the signature or input example. Which action should they take to ensure the signature and input example are logged?

Accepted Answer

Pass a `signature` and `input_example` to `mlflow.sklearn.log_model` explicitly, because autolog does not capture them for scikit-learn models.. Autolog for scikit-learn logs parameters, metrics, and the model artifact but does not automatically capture a model signature or input example. To include these, the engineer must explicitly pass `signature` and `input_example` to `mlflow.sklearn.log_model`. This is essential for model serving and validation, as the signature defines the expected input schema and the input example provides a sample for testing.

Answer

Register the model without a signature, then manually update the model version metadata in the Model Registry UI to add the signature and input example.

Answer

Set the environment variable `MLFLOW_LOG_MODEL_SIGNATURE` to `true` before training, which enables automatic signature logging for all models.

Answer

Use `mlflow.models.infer_signature` on the training data and then call `mlflow.sklearn.log_model` with the inferred signature, but input example cannot be logged for scikit-learn.

Question 9

A machine learning engineer is using MLflow to log a model. They want to include custom preprocessing logic that is not part of the model's native library. Which MLflow model flavor should they use to package the model with custom code?

Accepted Answer

`mlflow.pyfunc`. The `mlflow.pyfunc` flavor is designed for custom Python models. It allows you to define a class with a `predict` method that can include any preprocessing, inference, and postprocessing steps. This makes it ideal for packaging models with custom logic that is not supported by native flavors.

Answer

`mlflow.sklearn`

Answer

`mlflow.lightgbm`

Answer

`mlflow.tensorflow`

Question 10

A machine learning engineer is developing a model on Databricks and wants to ensure that the model's input schema is enforced during inference. They are using MLflow to log the model. What should they do?

Accepted Answer

Log the model with `mlflow.pyfunc.log_model` and provide a `signature` that includes the input schema.. MLflow's model signature defines the expected input and output schema. When a model is logged with a signature, MLflow can validate input data during inference, ensuring that the schema is enforced. This is the standard and most effective way to enforce input schema, as it leverages built-in functionality without custom code.

Answer

Log the model with `mlflow.pyfunc.log_model` and include a custom `predict` method that validates the input schema.

Answer

Log the model with `mlflow.sklearn.log_model` and set the `input_example` parameter to a sample of the training data.

Answer

Log the model with `mlflow.sklearn.log_model` and set the `registered_model_name` parameter to register the model in the Model Registry.

Question 11

A data scientist is using MLflow to track a training run. They want to log a dictionary of hyperparameters and a list of evaluation metrics that are computed at the end of each epoch. Which MLflow API calls should they use to log these items?

Accepted Answer

Use `mlflow.log_params()` for the hyperparameters and `mlflow.log_metrics()` for the metrics, calling the latter once per epoch with the epoch number as the step.. The correct approach is to use `mlflow.log_params()` for the hyperparameter dictionary and `mlflow.log_metrics()` for the metrics dictionary, specifying the epoch as the step. This leverages MLflow's batch logging APIs and ensures that metrics are recorded as time-series data, which the UI can plot over steps. It also keeps parameters organized in the parameters section.

Answer

Use `mlflow.log_param()` for each hyperparameter individually and `mlflow.log_metric()` for each metric, calling the latter once per epoch with the epoch number as the step.

Answer

Use `mlflow.log_artifact()` to log a JSON file containing the hyperparameters and metrics, and then parse it in the MLflow UI.

Answer

Use `mlflow.set_tag()` to log the hyperparameters and `mlflow.log_metric()` for the metrics, calling the latter once per epoch with the epoch number as the step.

Question 12

A data scientist is using MLflow to log a model that includes a custom preprocessing step. They want to ensure that the preprocessing is applied consistently during both training and inference. Which approach should they take?

Accepted Answer

Create a custom `PythonModel` that includes both the preprocessing and the model, and log it with `mlflow.pyfunc.log_model`.. Creating a custom `PythonModel` that integrates preprocessing and prediction ensures that the entire pipeline is encapsulated in a single MLflow model. When logged, this model applies preprocessing automatically during inference, guaranteeing consistency. This is the most robust approach when preprocessing involves custom logic not easily represented in a standard pipeline.

Answer

Use `mlflow.sklearn.log_model` and include the preprocessing steps in a scikit-learn `Pipeline` object.

Answer

Log the preprocessing code as a separate artifact and manually apply it before calling the model during inference.

Answer

Log the model with `mlflow.pyfunc.log_model` and provide the preprocessing function as a separate file in the `artifacts` parameter.

Question 13

A data scientist is using the Databricks Feature Store to build a training set for a fraud-detection model. The feature table is created with a primary key of `customer_id` and a timestamp key of `transaction_ts`. When calling `create_training_set`, the scientist wants to ensure that each label row receives exactly the most recent feature value available at or before the label's timestamp. Which argument must be supplied to `create_training_set` to enforce this point-in-time behavior?

Accepted Answer

Set `timestamp_lookup_key` to the label DataFrame's event timestamp column.. Point-in-time correctness in the Databricks Feature Store is achieved by telling the training-set builder which label column holds the event timestamp. That column is passed as `timestamp_lookup_key`, and the Feature Store then joins only feature rows whose timestamp key precedes or equals it. Without this argument, the join can pull the latest feature value regardless of time, introducing label leakage that inflates offline metrics and misleads deployment decisions.

Answer

Set `label` to the timestamp column of the label DataFrame.

Answer

Set `exclude_columns` to the timestamp key so the join ignores time.

Answer

Set `feature_names` to the timestamp key of the feature table.

Question 14

A machine learning engineer is preparing to deploy a model to production using MLflow Model Registry. They want to ensure that the model can be easily served and that its dependencies are correctly captured. Which TWO actions should they take when logging the model to guarantee that the serving environment can recreate the necessary Python environment? (Choose two.)

Accepted Answer

Log the model using `mlflow.sklearn.log_model()` with the `pip_requirements` parameter set to a list of pip requirement strings.. To ensure the serving environment can recreate dependencies, you must explicitly specify them when logging the model. Using the `conda_env` parameter with a Conda YAML file or the `pip_requirements` parameter with a list of pip requirements are the two supported ways to capture dependencies. Both methods result in MLflow saving the necessary environment files with the model, which are then used during serving to install the correct packages.

Answer

Log the model using `mlflow.sklearn.log_model()` with the `code_path` parameter to include custom transformation code, which also captures all dependencies.

Answer

Log the model using `mlflow.sklearn.log_model()` with the `signature` parameter to infer the input schema and automatically generate the environment.

Answer

Log the model using `mlflow.sklearn.log_model()` with the `registered_model_name` parameter to register the model, which automatically captures the environment.

Question 15

A machine learning engineer is training a model using scikit-learn on Databricks and wants to track the model's hyperparameters, metrics, and artifacts automatically without adding explicit logging calls. Which MLflow feature should they use?

Accepted Answer

mlflow.sklearn.autolog(). mlflow.sklearn.autolog() is the correct choice because it automatically logs scikit-learn model parameters, metrics, and artifacts. The other options are either low-level APIs that require manual logging or functions that only set experiment context or log individual artifacts, none of which provide automatic tracking.

Answer

mlflow.tracking.MlflowClient()

Answer

mlflow.set_experiment()

Answer

mlflow.log_artifact()