Databricks-ML-Pro Model Development Practice Test 7 — 15 Questions

Question 1

A team notices that their model performance is significantly lower in production than in training. They suspect 'data drift' in the feature inputs. Which Databricks capability should be used to monitor this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Using Databricks Lakehouse Monitoring to track inference data distributions.. Databricks Lakehouse Monitoring provides automated insights into data quality and drift by comparing production inference data against training baselines. This is essential for detecting the performance decay caused by concept or data drift. By identifying these issues early, teams can trigger automated retraining pipelines, maintaining model accuracy over time and ensuring the business value of the deployed model is sustained.

Answer

Manually querying the Delta table after every batch inference.

Answer

Enabling 'Audit Logs' to track individual inference request parameters.

Answer

Increasing the number of features in the model to cover more data.

Question 2

You are training a model on Databricks using MLflow. You need to log a custom model flavor to ensure it can be loaded in an environment without the original training code. Which approach is best practice?

Accepted Answer

Create a Python class inheriting from PythonModel and log it via mlflow.pyfunc.log_model.. Using the mlflow.pyfunc.log_model method allows you to encapsulate model artifacts, environment dependencies, and custom inference logic into a single, portable object. This is critical for Databricks model deployment because it decouples the inference environment from the training notebook, ensuring reproducibility across different clusters or serving endpoints while maintaining a standardized interface for prediction.

Answer

Serialize the model object using pickle and upload it to DBFS.

Answer

Use mlflow.sklearn.log_model with a hardcoded path to the local file system.

Answer

Register the model directly using the Model Registry without logging it to an experiment run.

Question 3

You are performing hyperparameter tuning using Hyperopt on Databricks. Which TWO configurations must be defined to ensure optimal performance and result tracking?

Accepted Answer

Use SparkTrials to distribute the hyperparameter search across the cluster.. Using SparkTrials allows Hyperopt to distribute trial execution across multiple cluster nodes, significantly accelerating grid or random search. Integrating MLflow within the objective function ensures that each trial's parameters, metrics, and artifacts are captured, enabling users to analyze the tuning history and select the best model based on validated performance metrics.

Answer

Call mlflow.end_run() manually at the end of every trial function.

Answer

Wrap the objective function in an MLflow autologging block.

Answer

Set the search space to use a linear scale for all numerical hyperparameters.

Question 4

You are developing a machine learning pipeline where you need to perform feature engineering on a large dataset using Spark, then train a model using Scikit-Learn. Which workflow is most efficient?

Accepted Answer

Perform feature engineering using Spark transformations, then collect to Pandas for training.. The most efficient workflow involves leveraging Spark for distributed data transformation and feature engineering, followed by collecting the processed data into a Pandas DataFrame for Scikit-Learn training. This approach uses the strengths of both frameworks, ensuring that computationally expensive transformations are parallelized across the cluster, while Scikit-Learn handles the specific modeling logic that is not natively distributed.

Answer

Convert the raw Spark DataFrame to Pandas immediately, then perform feature engineering.

Answer

Use a custom UDF to execute Scikit-Learn code on every partition of the Spark DataFrame.

Answer

Rewrite all feature engineering logic using only Scikit-Learn transformers.

Question 5

You are preparing a model for deployment in a production Databricks environment. Which THREE steps should be included in your model development pipeline to ensure model quality and traceability?

Accepted Answer

Define an input schema using the signature parameter in log_model.. A production-ready pipeline requires rigorous validation, tracking, and documentation. Using MLflow to log parameters/metrics ensures reproducibility; setting input schemas allows for automatic validation during serving; and registering models with stage transitions (Staging/Production) ensures that only validated models are exposed to production endpoints, preventing accidental deployment of broken code.

Answer

Log the model with all temporary files generated during the training process.

Answer

Hardcode the model version number directly into the application code.

Question 6

You need to perform cross-validation on a large dataset while ensuring that your model remains performant. What is the Databricks-recommended approach?

Accepted Answer

Use a Spark-based cross-validator or distribute folds across the cluster.. Leveraging Spark's distributed nature for cross-validation allows you to train folds in parallel, maximizing cluster utilization. By using libraries like Spark MLlib or distributed cross-validation wrappers, you can scale the evaluation process to datasets that would otherwise be impossible to handle on a single node. This ensures robust model evaluation while respecting the operational constraints of large-scale distributed data processing.

Answer

Collect all data to the driver node and use Scikit-Learn's cross_val_score.

Answer

Perform cross-validation only on a small, sampled subset of the data.

Answer

Hardcode the fold split indices to ensure reproducibility.

Question 7

Refer to the exhibit. You are loading a model from the registry. What does the 'models:/MyModel/1' URI specifically represent?

Accepted Answer

A unique identifier for the registered model name and its specific version.. The URI format 'models:/ModelName/Version' is the standard way to reference registered models within MLflow. It points to a specific, versioned artifact stored in the registry, ensuring that the inference code consistently uses the exact model state that was approved. This abstraction allows developers to change model versions without modifying the underlying inference application, promoting a stable production environment.

Answer

A direct pointer to the local file system path of the model artifact.

Answer

An alias for the most recently trained model in the current experiment.

Answer

A reference to the raw source code used to train the model.

Question 8

When logging a model that requires custom libraries (e.g., a specific version of a non-standard package), how do you ensure the environment is reproducible on the serving endpoint?

Accepted Answer

Provide an explicit conda.yaml or requirements file during the log_model call.. Including a conda.yaml file or an MLflow requirements file ensures that the exact environment dependencies are captured alongside the model. When the model is deployed to a serving endpoint, Databricks uses this metadata to recreate the identical software stack. This prevents the 'it works on my machine' problem, ensuring that inference logic is executed in a consistent, predictable environment across all deployment stages.

Answer

Install the dependencies manually on the serving cluster after deployment.

Answer

Include the library binaries directly in the model artifact folder.

Answer

Rely on the serving cluster's default environment libraries.

Question 9

You are developing an MLflow project and want to ensure that your code is reusable. What is the benefit of defining an MLproject file?

Accepted Answer

It packages code and dependencies for reliable, reproducible execution.. An MLproject file defines the entry points, dependencies, and environment for an MLflow project, effectively packaging the code as a self-contained unit. This is essential for reproducibility, as it allows others to execute the project with a single command, automatically setting up the environment and executing the code exactly as the author intended, regardless of the individual's specific machine environment.

Answer

It automatically generates documentation for the training code.

Answer

It prevents the model from being registered in the Model Registry.

Answer

It allows the code to run directly on the Databricks SQL Warehouse.

Question 10

When logging a model to the MLflow Model Registry, what is the primary benefit of using a registered model name rather than just the model URI?

Accepted Answer

It enables seamless staging and production management without code changes.. Registered model names provide a level of indirection that allows you to manage model lifecycle stages such as 'Staging', 'Production', and 'Archived'. By referencing a model name, you can update the underlying model version without requiring code changes in your inference applications. This promotes operational stability, as teams can transition models through an automated CI/CD pipeline while maintaining consistent endpoints for downstream consumers of the model.

Answer

It automatically increases the model training speed during the next iteration.

Answer

It forces the model to be saved in a specific proprietary cloud format.

Answer

It ensures that the model is automatically encrypted using hardware security modules.

Question 11

A machine learning engineer needs to track hyperparameter tuning experiments in Databricks using MLflow. Which approach best ensures that model training runs are associated with the correct code version and environment settings?

Accepted Answer

Run the training notebook from a Databricks Repo and use the mlflow.tracking.fluent API to track experiments.. Integrating MLflow with Git projects via Databricks Repos allows for automatic logging of the git commit hash. This practice is crucial for reproducibility, as it enables data scientists to map specific model performance metrics back to the exact codebase state used during development, ensuring auditability and consistency across development, staging, and production environments.

Answer

Manually log the git commit hash as a parameter in every mlflow.log_param call within the training script.

Answer

Utilize mlflow.set_tracking_uri with a local file system path for all distributed training nodes.

Answer

Hardcode the environment configuration inside the model training loop using environment variables.

Question 12

Refer to the exhibit. A data scientist is logging a Scikit-Learn model to the MLflow Model Registry. Which benefit does providing the `signature` and `input_example` offer during the deployment phase?

Accepted Answer

It enables MLflow to enforce data types during inference, preventing schema mismatch errors.. Providing a model signature and input example defines the expected data schema, allowing MLflow to perform type validation during inference. This is vital for production systems, as it prevents runtime errors caused by malformed inputs and enables Databricks to automatically generate deployment documentation and test payloads, significantly reducing the debugging time when deploying models to Model Serving endpoints.

Answer

It automatically scales the cluster size based on the input example size.

Answer

It allows the model to be trained in distributed mode using PySpark.

Answer

It automatically converts the model into an ONNX format for faster inference.

Question 13

A machine learning team is using `mlflow.autolog()` to track experiments. They notice that certain custom metrics are not being captured. What is the most effective way to address this?

Accepted Answer

Implement a custom callback in the training loop that uses mlflow.log_metric for the specific metrics.. While autologging captures standard metrics like accuracy, custom metrics require explicit logging calls using `mlflow.log_metric`. This is important because business-specific KPIs, such as profit margin or specific domain-based error rates, often require custom logic that framework-specific autologgers cannot infer automatically. Explicit logging ensures comprehensive experiment visibility and allows for precise model selection based on business goals rather than just technical performance.

Answer

Disable autologging and manually log every single parameter and metric.

Answer

Increase the logging frequency of the autologger via the configuration file.

Answer

Create a custom Python class that inherits from the MLflow Autologger base class.

Question 14

When training a model in Databricks, which storage layer should you prioritize for training data to ensure maximum throughput and compatibility with Feature Store?

Accepted Answer

Delta tables.. Delta Lake is the recommended storage format for training data in Databricks. It provides ACID transactions, schema enforcement, and efficient data versioning, which are essential for reproducible machine learning. Using Delta tables allows Feature Store to perform time-travel queries, ensuring that the data used for training is consistent with the data used for inference, which minimizes the risk of data leakage.

Answer

CSV files stored in DBFS.

Answer

Local temporary storage on the driver node.

Answer

JSON files stored in an unmanaged external mount.

Question 15

Which practice is most effective for managing dependencies to ensure consistent model training and inference results across different Databricks clusters?

Accepted Answer

Include a requirements.txt file with the notebook or project and specify it in the MLflow model logging process.. Using environment management files (e.g., requirements.txt or conda.yaml) ensures that the exact library versions used during development are replicated in the training and serving environments. This consistency is critical for preventing 'works on my machine' issues, where discrepancies in library versions lead to different numerical outputs or runtime crashes during inference, thus maintaining the reliability and reproducibility of the machine learning pipeline.

Answer

Manually install libraries on each cluster node before starting the job.

Answer

Rely on the default Databricks Runtime library set for all projects.

Answer

Use broad version constraints like '>=1.0' in the environment configuration.