Databricks-ML-Pro ML Ops Practice Test 10 — 15 Questions

Question 1

A financial institution is using Databricks to build and deploy a credit risk model. The model must comply with regulations that require full auditability of the model's lineage, including data sources, transformations, and training parameters. The team uses MLflow for tracking and the Feature Store for feature management. Which TWO of the following practices are essential to meet the auditability requirements? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use Databricks Feature Store to create feature tables and log the training set with the model, including feature lookup keys.. To meet auditability requirements, it is essential to automatically capture data and feature lineage. Logging data versions and feature table versions in MLflow provides a record of the exact inputs used. Using the Feature Store to log the training set with the model ensures that feature lineage is preserved. Together, these practices create a transparent and verifiable chain from data to model, which is necessary for regulatory compliance.

Answer

Enable MLflow autologging for all supported frameworks to capture parameters and metrics automatically.

Answer

Manually document the data sources and transformations in a separate spreadsheet for each model version.

Answer

Store all training code in a Git repository and commit after each run.

Question 2

A fraud detection model was deployed to a Databricks Model Serving endpoint. Over the past week, the endpoint's p99 latency has increased from 45 ms to 320 ms, but the model's predictions remain accurate. The serving logs show that each request now includes a larger JSON payload with additional transaction metadata. What is the most likely cause of the degradation?

Accepted Answer

The model's feature vector has grown, and the scoring function is performing expensive feature transformations on the raw payload before inference.. The increased payload size likely triggers additional feature engineering or parsing inside the model's scoring logic, raising per-request CPU time. Since accuracy is unchanged, the model itself is fine; the bottleneck is the preprocessing step. The other explanations either contradict the payload-size correlation or would impact accuracy or availability.

Answer

The serving endpoint is hitting the maximum number of concurrent requests and queuing requests, which adds wait time.

Answer

The model artifact is being re-downloaded from the MLflow Model Registry on every request due to a misconfigured artifact cache.

Answer

The model's underlying compute cluster has been scaled down, reducing available CPU resources for inference.

Question 3

A machine learning engineer is using Databricks Model Serving to deploy a model that requires a custom Python library not available in the default environment. The engineer wants to ensure the endpoint uses the exact library version and that the deployment is reproducible. Which approach should the engineer take?

Accepted Answer

Package the custom library as a Python wheel and include it in the model's conda environment or requirements file when logging the model with MLflow.. To ensure a custom library is available and reproducible in Databricks Model Serving, it must be included as a dependency when logging the MLflow model. Packaging the library as a wheel and adding it to the conda environment or requirements file ensures that Model Serving installs the exact version during deployment. This is the supported and recommended practice.

Answer

Install the custom library on all driver and worker nodes of the cluster used for model training, then deploy the model to Model Serving.

Answer

Use the Databricks REST API to upload the library to the Model Serving endpoint after deployment, then restart the endpoint.

Answer

Create an init script that installs the custom library on the Model Serving cluster, and attach it to the endpoint configuration.

Question 4

A data scientist has trained a model and logged it with MLflow. They now need to deploy it as a real-time endpoint on Databricks Model Serving. The model requires a custom Python library that is not available in the default environment. What is the correct way to ensure the library is available when serving the model?

Accepted Answer

Include the library in the model's conda environment or requirements file when logging the model with MLflow.. The correct way is to include the custom library in the model's conda environment or requirements file when logging with MLflow. Databricks Model Serving uses this specification to build the serving environment, ensuring the library is available. Other methods like cluster-level installations or runtime scripts do not apply to serverless serving endpoints and can introduce reliability issues.

Answer

Install the library on the driver node of the cluster used for serving, and restart the cluster.

Answer

Use a custom container image for the model endpoint and install the library at runtime via a startup script.

Answer

Add the library to the Databricks workspace's global init script so it is available to all clusters.

Question 5

A team is preparing to promote a new model version to production in the MLflow Model Registry. They must ensure the model can be served with a consistent environment across staging and production and that dependency drift is detected before promotion. Which TWO practices should they follow? (Choose two.)

Accepted Answer

Log the model with its conda environment and requirements files so the exact dependency versions are captured as part of the model version.. Capturing the exact dependency environment with the model version and then validating that environment against the target serving environment before promotion together ensure consistent serving and catch drift early. Floating to latest versions, mismatching runtimes, or relying on bare pickles all break reproducibility or hide drift.

Answer

Store only the model's pickle file and reconstruct dependencies manually on each environment, since pickle serialization is environment-independent.

Answer

Rely on the serving endpoint to install the latest available versions of each library at deployment time so the environment stays current.

Answer

Pin the serving cluster's runtime to a different Databricks Runtime version than training used, to validate cross-version compatibility during staging.

Question 6

Your team uses MLflow Model Registry. A model version currently in Production has a critical flaw and must be rolled back to a previous version. The previous version is in the Archived stage. What is the most operationally sound approach to restore service quickly while preserving the audit trail?

Accepted Answer

Transition the flawed Production version to Archived, then transition the previous version from Archived to Production.. The correct approach is to archive the problematic Production version and promote the prior known-good version back to Production. This restores service using an already-validated artifact while maintaining full stage-transition history for audit and rollback traceability. Deleting versions, creating new models, or bypassing the registry all sacrifice governance or introduce unnecessary operational risk.

Answer

Create a new registered model with the previous version's artifacts and point the serving endpoint to it.

Answer

Delete the flawed Production model version and re-register the previous version as a new model version.

Answer

Leave the flawed version in Production and instead update the serving endpoint to load the previous version by its run ID.

Question 7

A fraud detection model is served on a Databricks Model Serving endpoint. You notice that predictions for the same input vector differ between two consecutive requests within seconds, and there is no feature store or external cache involved. The model was logged with a fixed random seed and deterministic inference code. Which action should you take first to diagnose the inconsistency?

Accepted Answer

Inspect the endpoint's request logs to confirm the payloads are identical and check whether the model version serving traffic changed between requests.. When a deterministic model returns different outputs for the same input, the fastest path to the cause is to verify the two requests were truly identical and to confirm which model version handled each. A version transition during a rollout, or a payload difference such as field ordering or missing values, explains the symptom without any model retraining. Retraining, scaling, or changing flavors are premature and do not produce diagnostic evidence.

Answer

Convert the model to a different flavor such as ONNX and re-register it, because the original flavor cannot guarantee deterministic inference.

Answer

Increase the endpoint's concurrency and enable autoscaling, since differing predictions indicate the replicas are running different code.

Answer

Retrain the model with a different random seed and redeploy, because seed instability is the likely cause of varying predictions.

Question 8

A data scientist has developed a scikit-learn model and wants to deploy it as a REST API endpoint on Databricks Model Serving. They have logged the model with MLflow and registered it in the MLflow Model Registry. The model requires a specific Python library that is not pre-installed in the Databricks Model Serving environment. What should the data scientist do to ensure the library is available when the model is served?

Accepted Answer

Add the library to the model's conda environment and log the model with that environment.. Databricks Model Serving builds the serving environment based on the conda environment recorded with the MLflow model. To include a custom library, you must add it to that conda environment before logging the model. When the model is deployed, the serving infrastructure will create an environment with the specified dependencies. Other methods like init scripts or Spark config do not apply to the serverless serving environment.

Answer

Package the library as a wheel file and upload it to DBFS, then reference it in the serving endpoint configuration.

Answer

Modify the cluster's Spark configuration to include the library path.

Answer

Install the library on all cluster nodes using an init script.

Question 9

You are designing a CI/CD pipeline for a machine learning model on Databricks. The pipeline must automatically retrain the model when new data arrives, validate it, and deploy it to a serving endpoint if it passes. Which two components are essential to achieve this? (Choose two.)

Accepted Answer

A Databricks Model Serving endpoint that can be updated with the new model version via the MLflow API.. A Databricks Job provides the orchestration to run training and validation when new data arrives, and a Model Serving endpoint is required to deploy the validated model for real-time predictions. Together, they form the backbone of an automated retraining and deployment pipeline. MLflow Tracking, feature stores, and Repos are valuable but not strictly essential for the pipeline to function.

Answer

MLflow Tracking to log parameters, metrics, and artifacts for each run.

Answer

A Databricks Repos integration to version control notebooks and manage code changes.

Answer

A feature store that automatically ingests new data and updates features in real time.

Question 10

You are monitoring a production model deployed on Databricks Model Serving. You notice that the model's predictions have gradually become less accurate over time, likely due to data drift. You need to implement a solution that automatically detects drift and triggers retraining. Which Databricks feature should you use to monitor the model's input data and performance?

Accepted Answer

Databricks Lakehouse Monitoring for model quality and data drift, configured on the inference table that logs requests and responses.. Databricks Lakehouse Monitoring is the appropriate feature to monitor model input data and performance for drift. It can be configured on inference tables to automatically compute drift metrics and alert when thresholds are exceeded, enabling timely retraining. Other options lack the automated drift detection and integration with production workflows required for this scenario.

Answer

Databricks SQL dashboards that query the inference table and display accuracy metrics.

Answer

MLflow Tracking to log prediction results and manually compare distributions over time.

Answer

Unity Catalog lineage to track data transformations and identify changes in input data sources.

Question 11

A team is using Databricks Feature Store to manage features for a real-time fraud detection model. They need to ensure that the features used during training are consistent with those served at inference time. Which two actions should they take to achieve this? (Choose two.)

Accepted Answer

Publish the model with the Feature Store, so that the model automatically looks up the latest feature values from the online store at inference time.. Using FeatureStoreClient.create_training_set ensures point-in-time correctness during training, and publishing the model with Feature Store ensures that inference uses the same feature lookups from the online store. Together, these actions guarantee that features are consistent between training and serving, which is critical for real-time fraud detection.

Answer

Manually copy the feature values from the offline store to the online store before each inference request to ensure freshness.

Answer

Implement a custom UDF in the model to compute features on the fly from raw data, bypassing the Feature Store entirely.

Answer

Train the model using a separate notebook that reads directly from the online store to simulate inference-time feature retrieval.

Question 12

An ML engineer is instrumenting a Databricks job that trains and evaluates several models. They want each model's metrics, parameters, and artifacts grouped so that a downstream automated promotion step can compare candidates within the same experiment. Which MLflow practice best supports programmatic comparison across runs in this job?

Accepted Answer

Log all candidates into a single experiment, tagging each run consistently so the promotion step can filter and rank runs by metric within that experiment.. Grouping candidate runs in one experiment lets MLflow's search API rank and filter them by metric and tag in a single query, which is what an automated promotion step needs. Splitting experiments, overloading registry descriptions, or duplicating metrics elsewhere all break that integrated, queryable comparison.

Answer

Log all candidates as versions of a single registered model and compare them by reading each version's description field for metric values.

Answer

Write metrics to a Delta table outside MLflow and have the promotion step read that table, using MLflow only to store the model artifacts.

Answer

Create a separate experiment for each model candidate so that runs are isolated and cannot interfere with one another.

Question 13

A data science team uses MLflow Tracking with a remote tracking server backed by a Databricks-hosted MySQL instance for the backend store and an Azure Data Lake Storage Gen2 path for artifacts. A model-training notebook writes metrics and a model artifact, then calls mlflow.register_model to promote the run into Unity Catalog. Reviewers report that the run's metrics appear in the experiment UI, but the model version in the registry cannot be loaded by the deployment job. The deployment job fails when it tries to download the artifact. Which action most directly resolves the deployment failure?

Accepted Answer

Grant the deployment job's service principal READ privileges on the ADLS Gen2 artifact location, or configure a storage credential and external location that the deployment job can access.. MLflow separates metadata from artifacts: the backend store holds run and metric records, while the artifact store holds the model files. A deployment job that can read experiment metadata still needs its own credentials to the artifact location. Aligning the service principal's access, or exposing the path through Unity Catalog storage credentials and external locations, is what lets the artifact download succeed.

Answer

Re-run the training notebook with mlflow.set_tracking_uri pointing at a local file path so the artifact is written to the driver's disk.

Answer

Register the model again using the run ID instead of the artifact URI so the registry stores the artifact inline in the backend database.

Answer

Increase the --model-serve-timeout on the deployment cluster so the job has more time to fetch the artifact from the tracking server.

Question 14

You are responsible for deploying a machine learning model to a Databricks Model Serving endpoint. The model must be updated frequently with new versions. You want to ensure that the endpoint remains available during updates and that traffic can be shifted gradually to the new version. Which deployment strategy should you use?

Accepted Answer

Canary deployment by configuring the serving endpoint with multiple model versions and specifying a traffic split percentage.. Databricks Model Serving allows you to configure multiple model versions on a single endpoint and set traffic split percentages. This enables canary deployments, where you can gradually shift traffic to a new version while monitoring performance, ensuring high availability and easy rollback. Other strategies either cause downtime or are not natively supported.

Answer

Rolling update by stopping the endpoint, updating the model, and restarting the endpoint.

Answer

Shadow deployment by duplicating requests to a new endpoint without affecting production traffic.

Answer

Blue-green deployment by creating a new endpoint and switching traffic via DNS.

Question 15

You are responsible for monitoring a critical model deployed to Databricks Model Serving. You need to detect data drift and model performance degradation. Which TWO of the following actions should you take? (Choose two.)

Accepted Answer

Set up a Databricks SQL dashboard that queries the inference table and compares feature distributions to a baseline.. Inference tables capture the necessary request and response data for monitoring, while Databricks SQL dashboards enable analysis and alerting on that data. Together, they provide a robust solution for detecting data drift and model performance degradation. Other actions, such as changing instance types or isolating workspaces, do not address the monitoring requirements.

Answer

Deploy the model to a different workspace to isolate production traffic.

Answer

Use MLflow to log the model's training metrics and compare them to production metrics manually.

Answer

Configure the endpoint to use a smaller instance type to reduce cost.