Databricks-ML-Pro Model Deployment Practice Test 6 — 15 Questions

Question 1

An ML engineer is updating a model serving endpoint to use a new model version. They want to gradually shift traffic from the old version to the new version to monitor performance before full rollout. Which feature of Databricks Model Serving should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure traffic splitting on the existing endpoint by specifying a percentage of traffic to route to each served model version.. Databricks Model Serving allows configuring traffic splitting on a single endpoint by assigning a percentage of traffic to each served model version. This enables gradual rollout, allowing the team to monitor the new version's performance while still serving the old version. It is the native and recommended approach for safe model updates.

Answer

Create a new endpoint for the new model version and use a load balancer to distribute traffic.

Answer

Use the 'Champion' and 'Challenger' aliases in Unity Catalog to automatically route traffic based on model performance.

Answer

Enable A/B testing by deploying the new model to a separate endpoint and using Databricks SQL to compare metrics.

Question 2

A data scientist is deploying a model to Databricks Model Serving. The model was trained using a scikit-learn pipeline that includes a custom transformer. The custom transformer is defined in a Python module that is not part of the model artifact. What should the data scientist do to ensure the model can be served successfully?

Accepted Answer

Log the model with the custom transformer code included in the model's artifacts using MLflow's custom logging.. To deploy a model with custom code, you must include that code in the model's artifacts when logging with MLflow. Using the `code_path` parameter in `mlflow.pyfunc.log_model` allows you to specify additional code files that will be packaged with the model. Databricks Model Serving then loads this code, ensuring the custom transformer is available during inference. This is the standard method for handling custom logic in served models.

Answer

Include the custom transformer code in the model's conda environment as a pip dependency.

Answer

Re-train the model without the custom transformer, as Model Serving does not support custom code.

Answer

Convert the custom transformer to a built-in scikit-learn transformer before logging.

Question 3

A team has deployed a model to Databricks Model Serving and enabled inference tables. They notice that the inference table contains request and response payloads but no ground truth labels. They want to automatically join ground truth labels for monitoring. What should they do?

Accepted Answer

Configure a Databricks SQL query that joins the inference table with a ground truth Delta table and schedule it to refresh a monitoring metric.. Databricks Model Serving inference tables capture request and response payloads but do not include ground truth labels. To monitor model quality, you must join the inference table with a separate ground truth table. This is typically done by scheduling a query or pipeline that performs the join and computes metrics, which can then be used for model monitoring.

Answer

Use the MLflow Model Registry webhook to trigger a job that updates the inference table with ground truth labels.

Answer

Set the 'log_inputs' and 'log_outputs' flags to true in the endpoint configuration, which will automatically include ground truth labels.

Answer

Enable 'auto_capture_config' with 'ground_truth_column' set to the label column name in the inference table.

Question 4

An ML engineer has registered a scikit-learn model in the Unity Catalog Model Registry and wants to serve it as a real-time endpoint using Databricks Model Serving. The model's MLflow signature expects a JSON payload with an array of records. The engineer creates a serving endpoint with a workload size of Medium and configures the served entity to use the latest model version. Which additional configuration is required to enable automatic payload logging to a Delta table for monitoring?

Accepted Answer

Enable inference tables on the endpoint by specifying a Unity Catalog table location.. Automatic payload logging in Databricks Model Serving is achieved through inference tables, which require specifying a Unity Catalog table location when creating or updating the endpoint. This captures request and response data for monitoring and debugging. Other logging mechanisms like delivery logs or MLflow environment variables do not record inference payloads.

Answer

Set the environment variable MLFLOW_ENABLE_SYSTEM_METRICS_LOGGING to true in the model's conda environment.

Answer

Attach a delivery log to the endpoint and specify a Delta table path.

Answer

Configure the model signature to include a 'log_payloads' parameter set to true.

Question 5

A data scientist has deployed a model to a Databricks Model Serving endpoint. The endpoint is configured with scale-to-zero enabled and a workload size of Small. After a period of inactivity, the endpoint scales down to zero. A client application sends a request to the endpoint after this idle period. What happens to the first request?

Accepted Answer

The request is queued until the endpoint scales up, then processed with increased latency.. With scale-to-zero, the endpoint reduces replicas to zero after inactivity. When a new request arrives, the system must provision resources and load the model, causing the request to be queued and served with higher latency. This is expected behavior and not an error.

Answer

The request fails with a 503 Service Unavailable error because the endpoint is offline.

Answer

The request is immediately served by a cold-start replica with no added latency.

Answer

The request is routed to a fallback model version that is always kept warm.

Question 6

A data scientist has trained a model and registered it in Unity Catalog. They now need to deploy it for real-time inference with automatic scaling and a REST API endpoint. Which Databricks feature should they use?

Accepted Answer

Databricks Model Serving. Databricks Model Serving is the purpose-built feature for deploying models as real-time REST endpoints with automatic scaling. It handles infrastructure, scaling, and monitoring, allowing data scientists to focus on model development. Other options are either for automation, batch processing, or data governance, and do not provide the required real-time serving capabilities.

Answer

Unity Catalog functions

Answer

Databricks Jobs with a serving cluster

Answer

MLflow Model Registry webhooks

Question 7

A company has deployed a model to a Databricks Model Serving endpoint. The model's predictions must be logged to a Delta table for monitoring and auditing. The ML engineer wants to enable inference logging without modifying the model's code. Which approach achieves this with minimal effort?

Accepted Answer

Enable inference logging on the serving endpoint by specifying a Delta table path in the endpoint configuration.. Inference logging is a built-in feature of Databricks Model Serving that automatically logs request and response data to a Delta table. It can be enabled via the endpoint configuration without altering the model code. Other options require code changes, custom jobs, or misuse of MLflow tracking, and do not provide the same seamless auditing capability.

Answer

Use a Databricks job to periodically query the endpoint's logs via the REST API and insert them into a Delta table.

Answer

Wrap the model's predict method to write inputs and outputs to a Delta table using Spark, then redeploy the model.

Answer

Configure the model to log its predictions to MLflow, then enable MLflow tracking for the serving endpoint.

Question 8

An ML engineer is deploying a model to Databricks Model Serving and needs to enable automatic scaling based on traffic. The model has variable inference latency and the team wants to optimize cost while maintaining performance. Which TWO configurations are required to achieve this? (Choose two.)

Accepted Answer

Enable auto-scaling by specifying a target concurrency per replica.. To enable automatic scaling for a Databricks Model Serving endpoint, you must define the minimum and maximum replica counts and set a target concurrency per replica. These settings allow the serving infrastructure to adjust the number of replicas in response to traffic, ensuring performance while controlling cost. Other options either do not directly enable scaling or are not applicable to Model Serving.

Answer

Deploy the model using a GPU-enabled workload type to reduce latency.

Answer

Set the endpoint to use a dedicated cluster with autoscaling enabled.

Answer

Set the scale-to-zero option to true so the endpoint can shut down when idle.

Question 9

An ML engineer is deploying a scikit-learn model to a Databricks Model Serving endpoint. The model expects a single feature vector of 10 float values per request. The endpoint must return predictions in under 100 ms. Which approach should the engineer use to minimize per-request overhead?

Accepted Answer

Log the model using the native scikit-learn flavor and deploy it with a workload size that provides sufficient CPU.. For low-latency single-vector inference, minimizing framework overhead is critical. The native scikit-learn flavor avoids the extra Python layer that pyfunc introduces, and a suitable workload size provides the CPU needed to execute the model quickly. Autoscaling and request batching address throughput rather than per-request latency, so they do not help meet the strict latency target.

Answer

Log the model with a signature that specifies a tensor input and enable the endpoint's request batching.

Answer

Enable autoscaling on the endpoint and send requests with a batch size of 1.

Answer

Use MLflow's pyfunc flavor with a custom predict method that processes one row at a time.

Question 10

An ML engineer is deploying a model to Databricks Model Serving that uses a custom Python function as a pre-processing step. The function relies on a global variable defined in a separate module. After deployment, the endpoint returns errors indicating the global variable is not defined. The engineer confirmed the module is included in the model's conda environment. What is the most likely cause?

Accepted Answer

The global variable is not serialized with the model because MLflow only saves the model's predict method and its direct dependencies.. The error arises because MLflow's serialization does not capture global variables from external modules unless they are explicitly included in the model's code. When the model is loaded for serving, the pre-processing function may reference a global variable that was not saved, resulting in an undefined variable error. To fix this, the engineer should ensure all necessary state is encapsulated within the model or use `code_path` to include the module and initialize variables properly.

Answer

Databricks Model Serving runs each inference in a separate process, so global variables are not shared across requests.

Answer

The model was logged with `mlflow.sklearn.log_model` instead of `mlflow.pyfunc.log_model`, so custom pre-processing code is ignored.

Answer

The conda environment does not include the module because MLflow only captures packages installed via pip, not local modules.

Question 11

A data scientist has deployed a model to Databricks Model Serving and wants to monitor its performance over time. They need to track prediction drift and data quality issues. Which Databricks feature should they use to automatically capture inference logs and compute metrics?

Accepted Answer

Enable inference tables on the serving endpoint.. Inference tables are a Databricks feature that automatically logs request and response payloads from a Model Serving endpoint into a Delta table. This data can then be used with Lakehouse Monitoring to track prediction drift, data quality, and model performance. Other options do not provide automatic, scalable logging of production inference data.

Answer

Set up a Databricks job to periodically query the endpoint.

Answer

Use the model's signature to validate incoming data.

Answer

Configure MLflow tracking to log model predictions.

Question 12

A fraud detection model is deployed to a Databricks Model Serving endpoint. The team wants to test a new model version without affecting existing predictions. They need to send a copy of live traffic to the new version and log its predictions for comparison, while the current version continues to serve all responses. Which feature should they use?

Accepted Answer

Enable the endpoint's shadow mode by specifying the new model version as a shadow.. Shadow mode on Databricks Model Serving allows a new model version to receive a copy of live traffic while the primary version continues to serve all responses. This enables safe evaluation of the new model's predictions without impacting users. Traffic splitting would affect some responses, and external load balancing is not native and adds complexity. Therefore, enabling shadow mode is the correct approach.

Answer

Use MLflow's pyfunc flavor to log both models and select the active one at request time.

Answer

Deploy the new model version to a separate endpoint and use a load balancer to mirror requests.

Answer

Configure the endpoint with a traffic split of 50/50 between the current and new model versions.

Question 13

A data science team is using Databricks Model Serving to deploy a model that must process sensitive data. They need to ensure that all inference requests are logged for auditing purposes, including the input data and predictions. Which approach should they take?

Accepted Answer

Enable inference logging on the Model Serving endpoint and configure a Delta table to store the logs.. Inference logging in Databricks Model Serving is the built-in feature for capturing request and response data to a Delta table. It is designed for auditing, monitoring, and compliance, providing a scalable and managed solution. Other methods like custom logging or MLflow Tracking are not suited for production-scale inference logging and may introduce reliability or performance issues.

Answer

Use MLflow Tracking to log each inference request as a run in an experiment.

Answer

Set up a Databricks Job that periodically queries the endpoint and logs the responses.

Answer

Configure the model to write logs to a file in DBFS using a custom Python logger.

Question 14

A team has deployed a model to a Databricks Model Serving endpoint. They want to monitor the endpoint's performance and detect data drift over time. Which Databricks feature should they use to automatically track inference data and compute drift metrics?

Accepted Answer

Inference tables. Inference tables are a Databricks feature that automatically logs the input and output of a model serving endpoint to a Delta table. This data can then be used to compute drift metrics, monitor model performance, and trigger alerts. It is the built-in solution for capturing inference data for monitoring purposes.

Answer

MLflow tracking server

Answer

Model serving endpoint logs

Answer

Databricks SQL dashboards

Question 15

An ML engineer is updating a production model serving endpoint to use a new model version. The endpoint currently serves version 1 with the 'Champion' alias. The engineer wants to test version 2 with a small percentage of live traffic before full rollout. Which deployment strategy should they use in Databricks Model Serving?

Accepted Answer

Configure traffic splitting on the existing endpoint to route a percentage to version 2.. Databricks Model Serving allows you to serve multiple model versions on a single endpoint and split traffic between them by specifying percentages. This enables a canary deployment where a small fraction of requests go to the new version, allowing real-world testing before a full rollout. Other options either cause a full cutover, require external tools, or do not use live traffic.

Answer

Create a new endpoint for version 2 and use a load balancer to split traffic.

Answer

Update the 'Champion' alias to point to version 2 and monitor performance.

Answer

Deploy version 2 to a staging endpoint and compare metrics offline.