Databricks-ML-Pro ML Ops Practice Test 3 — 15 Questions

Question 1

You are using Databricks Feature Store to serve features for a real-time model deployed on Databricks Model Serving. The feature table is updated every 15 minutes with new streaming data, but the online feature store is refreshed only once daily. Your model's predictions have become stale and inaccurate within an hour of the daily refresh. Which action should you take to minimize prediction latency while ensuring the online store reflects the latest feature values?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Ensure the feature table is created with an online store and that the streaming writes to the feature table automatically propagate to the online store.. The online feature store in Databricks is designed to serve features with low latency and is automatically updated when the underlying feature table receives new data, provided the table is configured with an online store. If updates are only happening daily, the online store may not be enabled or the streaming writes are not propagating. Enabling and correctly configuring the online store ensures that the latest feature values are available for real-time inference without manual intervention.

Answer

Switch the model serving endpoint to read features directly from the offline feature table using a JDBC connection at inference time.

Answer

Increase the size of the online store cluster to handle the increased load from more frequent updates.

Answer

Configure the feature table to publish to the online store more frequently, such as every 15 minutes, using a scheduled job that calls publish_table with a higher frequency.

Question 2

Your team runs a nightly Databricks job that retrains a demand-forecasting model and registers a new version in the MLflow Model Registry. A separate production job loads the model by stage. After a recent promotion, the production job started failing with a 'model not found' error, even though the version appears in the registry. The production job uses a service principal with 'Can Read' permission on the registered model. What is the most likely cause?

Accepted Answer

The service principal lacks 'Can Read' permission on the model version's underlying artifact location, so the production job cannot download the artifact even though it can see the registry entry.. The symptom—a registered version that is visible but cannot be loaded by a service principal with only model-level read—points to artifact-level access. In Unity Catalog, model artifacts reside in governed storage, and registry permissions do not automatically propagate to those files. Granting the service principal read access on the containing schema or volume resolves the failure. The other options describe plausible but inconsistent causes.

Answer

The production job is caching an old MLflow client that does not support the registry API version used by the new model.

Answer

The model version was archived immediately after promotion, so the production job cannot resolve the stage.

Answer

The new model version was registered to a different registered model name due to a typo in the training job's registry URI.

Question 3

A machine learning engineer is configuring a Databricks job to run a hyperparameter tuning workflow using Hyperopt. The job must run on a cluster that automatically scales and uses spot instances to reduce cost, while ensuring that the tuning process can recover from spot instance interruptions. Which configuration should they use?

Accepted Answer

Enable autoscaling and spot instances with fallback to on-demand, and use MLflow tracking with a persistent run ID to resume tuning.. The optimal configuration combines autoscaling, spot instances with on-demand fallback, and MLflow tracking with a persistent run ID. This allows the cluster to scale with workload, reduces cost via spot instances, and ensures that if a spot instance is reclaimed, the tuning process can resume using the logged trials in MLflow. Other options either lack cost efficiency or resilience.

Answer

Use a fixed-size cluster with only spot instances and rely on Hyperopt's built-in checkpointing to a local disk.

Answer

Configure the job to use a single-node cluster with autoscaling disabled to avoid spot instance interruptions.

Answer

Use a job cluster with autoscaling and on-demand instances only, and log trials to a Delta table for recovery.

Question 4

You are building a Databricks job that trains a model and logs it to MLflow. The job must run on a schedule and you need to ensure that every run is associated with a unique experiment and that the model artifacts are stored in a central location for governance. Which combination of MLflow APIs and Databricks features should you use?

Accepted Answer

Use mlflow.set_experiment() with a Unity Catalog volume path for artifacts, start a run with mlflow.start_run(), and log the model with mlflow.log_model(). Schedule the job using Databricks Jobs with a service principal that has write access to the volume.. The correct approach leverages MLflow's experiment tracking and model logging within a Databricks job, while using Unity Catalog volumes for artifact storage to ensure centralized governance, access control, and lineage. Scheduling with a service principal provides consistent permissions and auditability. This combination meets the requirements for reproducibility and governance in an ML Ops pipeline.

Answer

Use mlflow.create_experiment() to create a new experiment for each run, log the model with mlflow.log_artifact(), and store artifacts in the driver's local filesystem. Schedule the job with a new cluster each time.

Answer

Use mlflow.set_tracking_uri() to point to an external MLflow server, log the model with mlflow.log_model(), and use Databricks Repos to version the training code. Store artifacts in the external server's default location.

Answer

Use mlflow.set_experiment() to set the experiment path, start a run with mlflow.start_run(), and log the model with mlflow.log_model(). Configure the job to run on a cluster with a DBFS mount for artifacts.

Question 5

A data scientist has registered a model in the MLflow Model Registry and wants to automate the promotion of model versions to the Production stage only after they pass a validation suite. Which mechanism should be used to enforce this gate?

Accepted Answer

Use MLflow webhooks to trigger a Databricks job that runs validation tests and then calls the MLflow API to transition the model version.. MLflow webhooks enable event-driven automation: when a new model version is registered, a webhook can trigger a Databricks job that runs validation tests and, if successful, transitions the version to Production via the MLflow API. This creates a reliable, auditable gate. Polling or manual transitions are less efficient and do not enforce the gate as strictly.

Answer

Use MLflow Model Registry stage transitions directly from the training notebook, conditioned on a validation metric.

Answer

Use a Databricks Job that polls the MLflow Model Registry for new versions and runs validation tests, then transitions the version using the MLflow API.

Answer

Use a Databricks Job with a notebook that runs validation tests and then manually transitions the model version via the MLflow UI.

Question 6

You are using Databricks Model Serving to deploy a model that requires a custom Python library not available in the default environment. You have packaged the library as a wheel file and included it in the MLflow model's conda environment. After deployment, the endpoint fails with an ImportError for that library. What is the most likely reason?

Accepted Answer

The library was installed in the model's environment, but the model code imports it under a different name or the library's dependencies are not included.. When a custom library is included in the conda environment but the model fails with ImportError, the issue is usually that the library's dependencies are not fully specified or the import name differs from the package name. The conda environment must include all transitive dependencies, and the model code must import the correct module. Verifying the environment and testing locally can help identify the missing piece.

Answer

The wheel file was not uploaded to a location accessible by the serving infrastructure.

Answer

The model's conda environment specifies the library, but the serving environment does not have the necessary system dependencies to build or install it.

Answer

The model was logged with a different Python version than the one used by the serving endpoint, causing binary incompatibility.

Question 7

You are preparing to promote a model version to 'Production' in the MLflow Model Registry on Databricks. Your organization requires that the promotion be gated by automated checks. Which TWO actions should you take to enforce a robust promotion gate? (Choose two.)

Accepted Answer

Use the MLflow Model Registry webhook to trigger a validation job that transitions the version only after tests pass.. A robust automated gate requires both a signal that validation passed and a mechanism that acts on it. Tags record the validation outcome in a queryable way, while registry webhooks can trigger the validation job and conditionally perform the stage transition. Together they enforce that no version reaches 'Production' without passing automated tests. Permission controls and metrics do not block promotion automatically.

Answer

Restrict 'Can Manage' permission on the registered model to a small group and rely on manual review before promotion.

Answer

Configure the model version to be immutable after registration so no further changes can be made.

Answer

Enable automatic stage transitions based on the model's training accuracy metric logged during the run.

Question 8

You are using Databricks Repos to manage ML project code and need to ensure that a training job always uses the latest version of the code from a specific branch. The job is scheduled to run daily. Which configuration should you use to guarantee the job uses the most recent commit on that branch?

Accepted Answer

Configure the job to use a Git reference with the branch name and set the job to always pull the latest commit.. Databricks Jobs support Git integration where you can specify a branch, tag, or commit. Using a branch name ensures the job checks out the latest commit on that branch at each run. This is the simplest and most reliable way to guarantee the job uses the most recent code without manual updates.

Answer

Clone the repository into a Databricks Repo and set the job to use the local path, relying on manual pulls to update the code.

Answer

Set the job to use a specific commit hash to ensure reproducibility, and update the hash daily via a script.

Answer

Use a Databricks job with a notebook that runs a git pull command at the start of each execution.

Question 9

A data scientist at a retail company has trained a scikit-learn model using MLflow Tracking and logged the model with the default signature. When they attempt to register the model to Unity Catalog using mlflow.register_model(), the registration fails with an error about missing required metadata. The workspace has Unity Catalog enabled and the target catalog and schema already exist. What is the most likely cause?

Accepted Answer

The model signature was not logged, which is required when registering models to Unity Catalog.. When registering a model to Unity Catalog, MLflow requires a model signature to be logged with the model. The signature defines the input and output schema, enabling Unity Catalog to enforce schema validation and governance. Without it, registration fails. The scenario indicates the model was logged with the default signature, but the error suggests the signature is missing or invalid, so ensuring a proper signature is logged resolves the issue.

Answer

The target catalog or schema does not have the `CREATE MODEL` privilege granted to the user.

Answer

The model was logged without the `mlflow.pyfunc` flavor, which is required for Unity Catalog registration.

Answer

The MLflow run was not logged to an experiment backed by Unity Catalog, so the model lacks a Unity Catalog model signature.

Question 10

A data scientist has trained a scikit-learn model and logged it with MLflow using the default flavor. They now need to register this model in the MLflow Model Registry and later serve it on a Databricks Model Serving endpoint. Which MLflow model flavor should they use to ensure the model can be loaded and served without custom code?

Accepted Answer

mlflow.pyfunc. The pyfunc flavor offers a standardized Python function interface that Databricks Model Serving can load and call directly. It abstracts the underlying framework, so scikit-learn models are served without custom wrappers. Other flavors are framework-specific or require conversion, adding steps that the scenario explicitly wants to avoid.

Answer

mlflow.tensorflow

Answer

mlflow.onnx

Answer

mlflow.sklearn

Question 11

A data scientist notices that a model registered in MLflow Model Registry shows a different signature than the one logged during training. The model was trained in a notebook that used a pandas DataFrame with a column of type int64, but the registered signature shows long. What is the most likely explanation?

Accepted Answer

MLflow automatically converts pandas int64 to long during signature inference because the underlying pyfunc representation uses Spark types.. MLflow's signature inference maps pandas int64 to the MLflow long type to maintain a framework-neutral representation compatible with Spark and other runtimes. This mapping is intentional and documented, so seeing long instead of int64 in the registered signature is expected. The other explanations incorrectly attribute the change to caching, manual overrides, or user error.

Answer

The signature was manually overridden during registration using the MLflow UI, which defaults integer columns to long.

Answer

The model was registered from a run that used a different DataFrame schema, and the registry cached the older signature.

Answer

The training notebook used a Spark DataFrame, which always produces long types, and the data scientist misremembered the input type.

Question 12

A data scientist wants to promote a model version to Production in MLflow Model Registry but the transition is blocked. The registered model has stage transitions enabled and the user has CAN_EDIT permission on the model. Which is the most likely reason the transition fails?

Accepted Answer

The workspace has a model registry webhook or a stage transition request that requires approval before the version can enter Production.. With CAN_EDIT on the registered model, the user has sufficient permission to change stages. The remaining common blocker is a governance gate: a pending stage transition request awaiting approval, or a registry webhook that intercepts the transition. These mechanisms are designed to prevent unapproved promotion to Production, so they are the most likely cause when permissions are already correct.

Answer

The user needs CAN_MANAGE_STAGING or CAN_MANAGE_PRODUCTION permissions rather than CAN_EDIT to change stages.

Answer

The model version's run was deleted, so the version can no longer be transitioned between stages.

Answer

The model version is in the Archived stage and must be moved to Staging before Production.

Question 13

A data science team is using MLflow Model Registry to manage a model that is retrained weekly. They want to ensure that the Production stage always points to the latest approved model version. The team uses a CI/CD pipeline that registers a new model version and transitions it to Staging after validation. What should the pipeline do next to automatically promote the model to Production while ensuring that the previous Production version is archived?

Accepted Answer

Transition the new version to Production; MLflow automatically archives the previous Production version.. MLflow Model Registry enforces a single Production version per model. When a new version is transitioned to Production, the existing Production version is automatically archived. This built-in behavior simplifies CI/CD pipelines: after validation, the pipeline only needs to transition the new version to Production, and the old version is archived without explicit action.

Answer

Call the MLflow API to transition the new version to Production and simultaneously transition the old version to Archived.

Answer

Delete the previous Production version and then transition the new version to Production.

Answer

Manually archive the old version first, then transition the new version to Production.

Question 14

You are using MLflow Model Registry to manage a model that is deployed to a Databricks Model Serving endpoint. You have transitioned a new model version to the 'Production' stage. You want the serving endpoint to automatically update to serve this new version without manual intervention. What should you do?

Accepted Answer

Configure the serving endpoint to reference the model by stage, e.g., 'models:/my_model/Production'.. By configuring the Model Serving endpoint to reference the model by stage (e.g., 'Production'), any version transitioned to that stage is automatically served. This eliminates the need for manual updates or custom automation, providing a seamless continuous deployment pipeline. Other methods, such as jobs or webhooks, require additional setup and are not the native automatic solution.

Answer

Manually update the endpoint to point to the new model version each time a new version is promoted.

Answer

Set up a Databricks Job that periodically checks for new model versions and updates the endpoint via the REST API.

Answer

Use MLflow webhooks to trigger a notebook that updates the serving endpoint when a model version transitions to 'Production'.

Question 15

A data science team is transitioning from manual model training to automated pipelines in Databricks. They require a mechanism to track model lineage, versions, and stage transitions programmatically. Which Databricks component best satisfies this requirement?

Accepted Answer

MLflow Model Registry. MLflow Model Registry provides a centralized model store, set of APIs, and UI for managing the full lifecycle of MLflow models. It enables versioning, model stage transitions, and lineage tracking, which are critical for MLOps maturity. By using the Model Registry, teams ensure reproducibility and governance, allowing for seamless promotion of models from staging to production while maintaining an audit trail of changes and deployment history.

Answer

Databricks Jobs Scheduler

Answer

Databricks Repos

Answer

Delta Lake