Databricks-ML-Pro ML Ops Practice Test 11 — 15 Questions

Question 1

You are monitoring a model served on Databricks Model Serving. You need to detect data drift in the incoming requests without delaying predictions. Which approach should you use?

Accepted Answer

Enable inference logging on the endpoint and analyze the logged requests asynchronously using a Databricks job that computes drift metrics.. Inference logging on Model Serving captures request and response data to a Delta table without adding latency to the prediction path. An asynchronous job can then analyze this logged data to compute drift metrics, enabling detection without impacting performance. Synchronous or in-model computations are impractical due to latency and single-request limitations.

Answer

Add a pre-processing step in the model's predict function that computes drift metrics on each request.

Answer

Configure the endpoint to call an external monitoring service synchronously before returning predictions.

Answer

Use Databricks SQL dashboards to query the model's training data and compare it to live predictions in real time.

Question 2

A machine learning team is using MLflow Model Registry to manage a model that is deployed to a production endpoint. They need to implement a CI/CD pipeline that automatically transitions a model version from 'Staging' to 'Production' only after it passes a set of validation tests. Which MLflow feature allows them to trigger the transition based on test results?

Accepted Answer

MLflow Model Registry stage transitions via the REST API. The MLflow Model Registry REST API allows programmatic stage transitions. In a CI/CD pipeline, after validation tests pass, you can call the API to move the model version from Staging to Production. This integrates seamlessly with automation tools. Webhooks are for notifications, not for initiating transitions, and other options lack the direct capability to change stages based on test results.

Answer

MLflow Projects with conditional steps

Answer

Databricks Jobs with task dependencies

Answer

MLflow Model Registry webhooks

Question 3

A data scientist is using MLflow Tracking to log experiments for a model that predicts customer lifetime value. They want to compare runs across multiple experiments and identify the best performing run based on a custom metric called 'rmse'. Which MLflow feature should they use?

Accepted Answer

MLflow Tracking UI and API. MLflow Tracking provides both a UI and an API to log, query, and compare runs. The UI allows visual comparison of metrics like 'rmse' across experiments, and the API supports programmatic search and filtering. This makes it the appropriate feature for identifying the best performing run based on a custom metric.

Answer

MLflow Models

Answer

MLflow Model Registry

Answer

MLflow Projects

Question 4

A fraud detection model is served via a Databricks Model Serving endpoint. The team wants to capture the incoming request payloads and the model's predictions to a Delta table for monitoring and future retraining. Which approach is most appropriate?

Accepted Answer

Enable inference tables on the model serving endpoint to automatically log requests and responses to a Delta table.. Databricks Model Serving provides inference tables as a built-in mechanism to log request and response payloads to a Delta table. This satisfies monitoring and retraining data capture without custom code. The other options either add latency, require unsupported dependencies in the serving environment, or confuse training-time autologging with serving-time logging.

Answer

Modify the model's predict method to write each request and response to a Delta table using the Spark connector.

Answer

Configure the endpoint to send logs to a cloud storage bucket and schedule a job to load them into Delta.

Answer

Use MLflow autologging on the serving endpoint to capture inference requests as MLflow runs.

Question 5

A machine learning engineer is responsible for monitoring a production model deployed to Databricks Model Serving. The model predicts customer churn and is served via a REST endpoint. The engineer needs to detect data drift and model performance degradation over time. Which TWO actions should the engineer take to enable effective monitoring? (Choose two.)

Accepted Answer

Enable inference logging on the model serving endpoint to capture request and response payloads.. Effective monitoring of a production model requires capturing inference data and analyzing it for drift and performance. Enabling inference logging provides the raw data, while scheduling a job to compute drift metrics against the training set enables detection of data drift. Together, these actions allow the engineer to identify when the model's input distribution diverges from training, which is a key signal of degradation.

Answer

Enable automatic model retraining triggered by any change in the input data schema.

Answer

Use the model's training accuracy as a proxy for production performance and set up alerts based on that metric.

Answer

Configure the endpoint to use a smaller instance type to reduce cost, as monitoring does not require additional resources.

Question 6

You are implementing a CI/CD pipeline for a machine learning model on Databricks. The pipeline must automatically run unit tests, train the model, and deploy it to a staging endpoint. Which TWO practices should you follow to ensure the pipeline is reproducible and reliable? (Choose two.)

Accepted Answer

Pin all library dependencies to specific versions in a requirements file or conda environment specification.. For a reproducible and reliable CI/CD pipeline, code should be version-controlled and checked out consistently, and dependencies should be pinned to specific versions. These practices ensure that the pipeline runs the same code in the same environment every time. Autoscaling and manual promotion do not address reproducibility, and using latest libraries can introduce instability.

Answer

Use the latest version of all libraries to ensure compatibility and security patches.

Answer

Configure the pipeline to run on a cluster with autoscaling enabled to handle variable workloads.

Answer

Manually promote the model to staging after reviewing the test results to ensure quality.

Question 7

An ML engineering team maintains a feature table in Databricks Feature Store that is populated by a nightly batch job. The same features feed both an offline training pipeline and a Model Serving endpoint. After a schema change adds two columns to the source Delta table, the endpoint begins returning errors during online lookup. The team confirms the offline training pipeline still works. Which change most likely restores the endpoint?

Accepted Answer

Publish the updated feature table to the online store so the online table schema matches the offline table, then redeploy the model that consumes those features.. Feature Store maintains two representations: the offline Delta table for training and an online table for low-latency serving. Schema changes flow to the offline table automatically but must be republished to refresh the online store. Because the endpoint queries the online store, its stale schema causes lookup errors even though training still succeeds. Republishing and redeploying restores alignment.

Answer

Add the new columns to the model signature and register a new model version, leaving the online store untouched.

Answer

Recreate the serving endpoint with a larger workload size so the added columns can be cached in memory during lookups.

Answer

Switch the endpoint to read features directly from the offline Delta table instead of the online store.

Question 8

You need to ensure that a model deployed to a Databricks Model Serving endpoint can be rolled back quickly if it starts performing poorly. Which feature should you use?

Accepted Answer

Enable multiple model versions on the endpoint and use traffic splitting to shift traffic back to a previous version.. Model Serving endpoints can host multiple model versions and use traffic splitting to route requests. If a new version underperforms, you can instantly shift traffic back to a previous version by adjusting the split, achieving a fast rollback without redeployment. Other methods are manual, slow, or do not directly control the serving endpoint.

Answer

Use MLflow Model Registry to transition the model version back to Staging, which automatically reverts the serving endpoint.

Answer

Delete the current model version from the registry and register the previous version as a new version.

Answer

Configure the endpoint to use a webhook that triggers a rollback job when performance metrics drop.

Question 9

A data scientist wants to automate the retraining of a model whenever new data arrives in a Delta table. They need to orchestrate a multi-step workflow that includes data validation, feature engineering, model training, and deployment. Which Databricks feature should they use?

Accepted Answer

Databricks Jobs with a multi-task workflow. Databricks Jobs with multi-task workflows is the built-in orchestration service that allows you to define dependencies, schedule triggers, and monitor execution of complex ML pipelines. It can be triggered by file arrival events, making it ideal for retraining when new data lands. Other options like MLflow Projects, Delta Live Tables, and Repos serve different purposes and lack native orchestration capabilities.

Answer

Databricks Repos

Answer

MLflow Projects

Answer

Delta Live Tables

Question 10

A machine learning engineer is setting up a CI/CD pipeline for a model deployed to Databricks Model Serving. The pipeline must automatically update the serving endpoint when a new model version is registered in the MLflow Model Registry and passes a validation job. Which Databricks feature should the engineer use to trigger the update?

Accepted Answer

MLflow Model Registry webhooks that call a Databricks job to validate and update the endpoint.. MLflow Model Registry webhooks enable event-driven automation. By configuring a webhook on MODEL_VERSION_CREATED, a Databricks job can be triggered to validate the new version and then update the Model Serving endpoint via the REST API. This integrates validation gating and deployment in a CI/CD pipeline.

Answer

Databricks Model Serving's built-in auto-update feature, which automatically deploys the latest model version from the registry.

Answer

A Databricks notebook with a while loop that continuously checks the Model Registry for new versions and updates the endpoint.

Answer

Databricks Jobs with a schedule trigger that polls the Model Registry every hour for new versions.

Question 11

A team is using Databricks Feature Store to manage features for a real-time model served via Databricks Model Serving. They need to ensure that the online feature values used at inference time are consistent with the training data. Which TWO practices should they implement? (Choose two.)

Accepted Answer

Use point-in-time lookups when creating the training dataset to avoid leaking future data.. Consistency between online and offline features is achieved by using the same pipeline to publish to both stores and by using point-in-time lookups during training. These practices ensure that the transformations and temporal alignment match between training and inference. Manual copying or direct Delta queries do not provide the required consistency or performance.

Answer

Log the model with the feature store's training set specification to enable automatic feature lookup at serving time.

Answer

Store all features in a single Delta table and query it directly from the model serving endpoint.

Answer

Manually copy feature values from the offline store to the online store on a daily basis using a notebook.

Question 12

A team runs a weekly retraining job that produces a new model version in Unity Catalog. Their production endpoint is currently serving version 4. They want to promote version 5 with zero downtime and the ability to roll back instantly if error rates rise. Which approach best meets these requirements?

Accepted Answer

Configure the serving endpoint to serve version 5 and rely on the platform's rolling update so traffic shifts gradually while the previous configuration remains available for rollback.. Model Serving supports rolling updates that keep the endpoint live while a new model version loads, and the previous configuration remains restorable. Repointing the endpoint to the newer version and letting the rolling update run achieves zero-downtime promotion with a straightforward rollback. Deleting the old version or duplicating endpoints introduces risk or cost without improving the outcome.

Answer

Delete version 4 from the registry, deploy version 5, and recreate the endpoint so it picks up the new version automatically.

Answer

Set the endpoint's served entities to version 5 with a traffic percentage of zero, then raise traffic to one hundred percent after validation.

Answer

Create a second endpoint for version 5, run both endpoints in parallel indefinitely, and split traffic manually at the load balancer.

Question 13

You are building a CI/CD pipeline that must promote an MLflow model version from Staging to Production in Databricks only after automated validation. The pipeline runs in a service principal context. Which two actions are required to implement this safely and repeatably? (Choose two.)

Accepted Answer

Grant the service principal CAN_MANAGE_PRODUCTION (or CAN_MANAGE) on the registered model so it can transition versions into Production.. Safe automated promotion needs two things: permission for the automation identity to change stages, and a programmatic call to perform the transition after validation. Granting the service principal CAN_MANAGE_PRODUCTION or CAN_MANAGE satisfies the first, and invoking the transition-stage API after validation satisfies the second. Archiving policies, artifact storage changes, and human tokens are neither required nor advisable for this pipeline.

Answer

Enable automatic model version archiving so that older Production versions are removed before the new version is promoted.

Answer

Require the service principal to use a personal access token tied to a human user so that audit logs show a named approver.

Answer

Store the model artifacts in a Unity Catalog volume and reference the volume path when registering the model version.

Question 14

Your team uses MLflow Projects to package training code. A colleague runs the project on a Databricks cluster and it fails with a dependency conflict because the cluster has an older version of a library than the project's conda environment specifies. What is the most reliable way to ensure the project uses its declared dependencies without modifying the shared cluster?

Accepted Answer

Use the MLflow Projects CLI with the --conda flag or run via mlflow run, which creates an isolated conda environment from the project's conda.yaml.. MLflow Projects are designed to encapsulate dependencies in a conda.yaml, and running them with conda enabled creates an isolated environment that installs the specified versions. This avoids conflicts with shared cluster libraries and preserves reproducibility. Flags controlling experiment names, global installs, or notebook conversions do not provide the same isolation or reliability.

Answer

Run the MLflow project with the --experiment-name flag to isolate dependencies per experiment.

Answer

Install the required library version globally on the cluster and restart it before running the project.

Answer

Convert the project to a Databricks notebook and use %pip install at the top of the notebook.

Question 15

A machine learning engineer is using MLflow to track experiments and has logged a model with a signature. They now want to register this model in the MLflow Model Registry and promote it to Production. Which MLflow API call should they use to add the model to the registry?

Accepted Answer

mlflow.register_model(model_uri, name). The mlflow.register_model function is designed to take an existing model URI and add it as a new version to the Model Registry. It is the standard way to register a model after logging. Other options either create empty models or log new models, which do not fit the scenario of promoting an already logged model.

Answer

mlflow.log_model(model, artifact_path, registered_model_name=name)

Answer

mlflow.pyfunc.save_model(path, loader_module, data_path, registered_model_name=name)

Answer

mlflow.create_registered_model(name)