Databricks-ML-Assoc Model Development Practice Test 3 — 15 Questions

Question 1

A data scientist is training a scikit-learn model on Databricks and wants to log the trained model to MLflow so it can be loaded later for batch inference. They call mlflow.sklearn.log_model(model, 'model') but later find that when they load the model with mlflow.pyfunc.load_model, the predictions fail because the input DataFrame columns are in a different order than during training. Which MLflow feature should they have configured when logging the model to prevent this issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Set the model signature by passing the signature argument to log_model, derived from the training data.. A model signature is the MLflow mechanism that records the expected input and output schema for a model. When provided at logging time, it enables MLflow to validate and reorder incoming data during inference, ensuring column order matches what the model expects. Without it, pyfunc loads the model without schema awareness, leading to prediction errors when input structure varies.

Answer

Register the model in the MLflow Model Registry and assign it a stage such as Production.

Answer

Log the training DataFrame as an artifact using mlflow.log_artifact and load it alongside the model.

Answer

Enable autologging by calling mlflow.sklearn.autolog() before training the model.

Question 2

A data scientist is using MLflow to track experiments for a deep learning model in Databricks. They want to ensure that all relevant information is captured for reproducibility. Which TWO of the following should they log to achieve this? (Choose two.)

Accepted Answer

Hyperparameters. Hyperparameters and evaluation metrics are core to MLflow tracking. Hyperparameters define the model configuration, and metrics quantify performance. Together, they allow data scientists to compare runs and reproduce the best model. Other options like notebook revision history or cluster configuration are not standard MLflow logging elements for reproducibility.

Answer

The random seed used for data splitting

Answer

The cluster's auto-scaling configuration

Answer

The notebook's revision history

Question 3

A data scientist is developing a scikit-learn model with hyperparameter tuning using Hyperopt on a Databricks cluster. They notice that the tuning run launches many Spark jobs and the cluster spends considerable time on scheduling overhead. They want to reduce the overhead while still evaluating all hyperparameter combinations. Which approach should they use?

Accepted Answer

Use SparkTrials with parallelism set to the number of Spark task slots, allowing concurrent trials.. Hyperopt's SparkTrials integrates with Spark to run multiple trials concurrently, reducing the overhead of sequential job launches. By setting parallelism to match available task slots, the cluster utilizes resources efficiently. This is the intended pattern for distributed hyperparameter tuning on Databricks.

Answer

Replace Hyperopt with a manual grid search loop using scikit-learn's GridSearchCV.

Answer

Increase the Spark driver memory and set spark.sql.shuffle.partitions to a high value.

Answer

Switch to a single-node cluster with more memory to avoid Spark job scheduling.

Question 4

A machine learning engineer is using MLflow Tracking on Databricks to log experiments for a model training project. They want to compare runs across multiple notebooks and ensure that all runs from a specific project are grouped under the same experiment. Currently, runs are being logged to different experiments depending on the notebook. What is the most reliable way to ensure all runs are logged to a single, consistent experiment?

Accepted Answer

Call mlflow.set_experiment('/Shared/project_experiment') at the beginning of each notebook before starting any runs.. The MLflow Tracking API provides mlflow.set_experiment to designate the active experiment for the current session. When called at the start of each notebook with the same experiment path, it ensures all runs from that notebook are logged to the specified experiment, regardless of the default. This is the most reliable and idiomatic method for consistent experiment grouping across multiple notebooks.

Answer

Set the environment variable MLFLOW_EXPERIMENT_NAME to the desired experiment name in each notebook.

Answer

Log all runs with the same run name using mlflow.start_run(run_name='project_run').

Answer

Use mlflow.start_run(experiment_id='12345') for each run, specifying the same experiment ID.

Question 5

A machine learning engineer is tuning a scikit-learn RandomForestClassifier on Databricks. They use Hyperopt with the fmin function and set max_evals=20. After the run completes, they notice that only 15 trials appear in the MLflow experiment. They confirm that no trials failed and that the search space is valid. What is the most likely explanation for the missing trials?

Accepted Answer

SparkTrials was used, and some trials were stopped early or failed to log due to parallel execution.. When using SparkTrials for distributed Hyperopt, trials execute in parallel on Spark workers. If a trial is stopped early or if logging from executors is not synchronized, it may not be recorded as an MLflow run. This explains why fewer runs than max_evals appear. The other options describe features that do not exist or are not applicable in this context.

Answer

The MLflow experiment was created with a retention policy that automatically deletes older runs.

Answer

Hyperopt deduplicates identical parameter combinations, and some trials evaluated the same point.

Answer

The max_evals parameter was overridden by the default value of 10 in the Hyperopt library.

Question 6

A machine learning engineer is using Databricks AutoML to train a classification model on a dataset with a highly imbalanced target class. They want AutoML to optimize for a specific metric that handles imbalance well, such as area under the precision-recall curve (PR AUC). Which parameter should they set in the AutoML API call to specify this evaluation metric?

Accepted Answer

primary_metric="prauc". Databricks AutoML uses the primary_metric parameter to determine which metric to optimize during trials. For imbalanced classification, "prauc" is a supported option that focuses on precision-recall trade-offs. Other parameter names like eval_metric, metric, or loss are not part of the AutoML API and would not work. Correctly setting primary_metric ensures the best model is selected according to the desired evaluation criterion.

Answer

metric="prauc"

Answer

eval_metric="prauc"

Answer

loss="prauc"

Question 7

A machine learning engineer is using MLflow to track experiments on Databricks. They want to compare runs from two different experiments that used different preprocessing steps. They notice that the run metrics are not directly comparable because the preprocessing code was not logged. Which MLflow feature should they use to capture the preprocessing code and ensure reproducibility?

Accepted Answer

MLflow Projects with a conda.yaml environment and entry points.. The correct answer is MLflow Projects with a conda.yaml environment and entry points. MLflow Projects package code, dependencies, and entry points, enabling full reproducibility. By defining the environment and entry points, you ensure that preprocessing and training code are versioned and can be rerun exactly. Other options like logging artifacts or tags are manual and do not capture the complete code context automatically.

Answer

mlflow.log_param to record the preprocessing parameters.

Answer

mlflow.log_artifact to save the preprocessing script as an artifact.

Answer

mlflow.set_tag to add a tag with the preprocessing version.

Question 8

A data scientist is training a scikit-learn model on Databricks and wants to log the model to MLflow with a signature that includes both input and output schema. They also want to ensure that the model can be deployed for batch inference. Which combination of MLflow methods should they use to log the model with the signature?

Accepted Answer

Use `mlflow.sklearn.log_model()` with the `signature` parameter, providing an `infer_signature` object created from sample input and output DataFrames.. To log a scikit-learn model with both input and output signature, you must use `mlflow.sklearn.log_model()` and pass a signature object created by `infer_signature`, which can derive the schema from example DataFrames. This ensures the model's expected inputs and outputs are documented for deployment.

Answer

Use `mlflow.log_artifact()` to save the model pickle file and a separate JSON file containing the signature.

Answer

Use `mlflow.register_model()` after logging the model without a signature; the signature can be added later via the Model Registry UI.

Answer

Use `mlflow.sklearn.log_model()` with the `input_example` parameter only; MLflow will automatically infer and log the signature.

Question 9

A data scientist is using MLflow to track experiments on Databricks. They want to log a custom Python function as a model using mlflow.pyfunc.log_model. The function requires a non-standard library that is not part of the Databricks Runtime. They need to ensure the model can be loaded and served in a different environment. Which two steps must they take to ensure the model's dependencies are correctly captured and the model can be loaded? (Choose two.)

Accepted Answer

Use the conda_env parameter to specify a custom Conda environment that includes the non-standard library.. To ensure a custom pyfunc model with a non-standard library can be loaded elsewhere, the dependencies must be explicitly captured. This can be done by specifying a Conda environment via the conda_env parameter or by including a requirements.txt file in the model artifacts. Both methods allow MLflow to recreate the environment. Registration and signature are important but do not address dependency capture.

Answer

Use the pip_requirements parameter to specify a list of pip packages, including the non-standard library.

Answer

Set the model's signature using the signature parameter to include the input and output schema.

Answer

Log the model with the registered_model_name parameter to automatically register it in the Model Registry.

Question 10

A data scientist is training a scikit-learn model on Databricks and wants to log the trained model to MLflow so it can be loaded later for batch inference. They call mlflow.sklearn.log_model(model, 'model') but later find that when they load the model with mlflow.pyfunc.load_model, the predictions fail because the input DataFrame columns are in a different order than during training. Which MLflow feature should they have configured when logging the model to prevent this issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

An MLflow ModelSignature object created using infer_signature with sample input and output. To capture a model signature, you must create an MLflow ModelSignature object, typically by calling infer_signature on sample input and output data, and pass it to the signature parameter of log_model. This signature is then stored with the model and used for validation during serving.

Answer

A sample input DataFrame passed to the signature parameter

Answer

The conda_env parameter specifying the environment dependencies

Answer

The input_example parameter with a representative DataFrame

Question 11

A machine learning engineer is preparing to deploy a model trained with scikit-learn on Databricks. They want to ensure the model can be reliably loaded and served in a different environment, such as a staging cluster, without manual dependency installation. They plan to use MLflow to package the model. Which TWO actions should they take to ensure the model's environment dependencies are captured and reproducible? (Choose two.)

Accepted Answer

Log the model using mlflow.sklearn.log_model, which automatically captures the current conda environment and pip dependencies.. To ensure reproducible dependencies, the model must be logged with an environment specification. mlflow.sklearn.log_model automatically captures the current environment, while providing a custom conda_env file allows precise control over package versions. These two actions together guarantee that the model can be loaded in a different environment with the correct dependencies installed.

Answer

Register the model in the MLflow Model Registry and set its stage to Staging, which automatically installs dependencies on serving clusters.

Answer

Use mlflow.pyfunc.log_model with a custom Python model wrapper that imports all necessary libraries at runtime.

Answer

Enable autologging with mlflow.sklearn.autolog() before training, which automatically logs the model with its dependencies.

Question 12

A team is preparing a scikit-learn model for low-latency batch scoring on Databricks and wants to log it with MLflow so the correct environment is captured. Which TWO practices should they follow when logging the model? (Choose two.)

Accepted Answer

Set the registered_model_name argument so the model is versioned in the Model Registry on each log.. A deployable MLflow model needs its native flavor so the environment can be reconstructed, and a signature so the expected schema is explicit. Registering the model adds versioning and lineage for governed use. Logging a bare pickle or relying on ambient cluster libraries leaves the artifact non-reproducible, while a custom pyfunc wrapper adds no value when the native flavor already suffices.

Answer

Pin exact package versions in the notebook and rely on the cluster's installed libraries at load time.

Answer

Call mlflow.log_artifact on the pickled model file and rely on the tracking run to restore dependencies.

Answer

Log the model with mlflow.pyfunc.log_model and provide a custom Python function that calls the sklearn predict method.

Question 13

A data scientist is using MLflow to log a model trained with scikit-learn. They want to ensure that the model can be served as a REST API endpoint in Databricks. They call mlflow.sklearn.log_model(model, 'model') but omit the signature parameter. What is the primary consequence of omitting the model signature when logging the model?

Accepted Answer

Databricks model serving will reject the model deployment because it requires a signature to validate input data.. Databricks model serving requires a model signature to deploy a model as a REST API endpoint. The signature specifies the input and output schema, enabling the serving infrastructure to validate requests and generate API documentation. Without it, deployment fails. While a signature is not required for local inference or registration, it is essential for serving.

Answer

The model cannot be registered in the MLflow Model Registry because a signature is mandatory for registration.

Answer

The model will not be able to make predictions when loaded, because the signature is required for inference.

Answer

The model's dependencies will not be captured, leading to environment inconsistencies during serving.

Question 14

A data scientist is using MLflow to log a model and wants to ensure that the model can be reproduced and deployed reliably. They are considering various components to log. Which TWO of the following are essential to capture for reproducibility and deployment? (Choose two.)

Accepted Answer

The model signature to define input and output schema. The model signature and conda environment are essential for reliable deployment. The signature ensures correct input/output handling, and the environment ensures dependency reproducibility. Without them, deployment may fail or produce inconsistent results.

Answer

The source code of the training notebook

Answer

The model's performance metrics on the test set

Answer

The training dataset as an artifact

Question 15

A data scientist is using MLflow to log a model built with the XGBoost library. They call mlflow.xgboost.log_model with the model object and an artifact path. After logging, they want to load the model for batch inference using mlflow.pyfunc.load_model. However, they encounter an error stating that the model flavor 'xgboost' is not supported. What is the most likely cause?

Accepted Answer

The XGBoost library is not installed in the environment where the model is being loaded.. The error occurs because the XGBoost library is not installed in the environment where the model is being loaded. MLflow's pyfunc loading relies on the underlying flavor's library to deserialize the model. Without XGBoost, the xgboost flavor cannot be loaded. Ensuring the library is installed resolves the issue.

Answer

The model was logged to a run that has been deleted, so the artifacts are no longer available.

Answer

The model was logged with the wrong flavor; they should have used mlflow.sklearn.log_model instead.

Answer

The model was logged without a signature, so pyfunc cannot determine the input schema.