Databricks-ML-Assoc Practice Test 20 — 15 Questions

Question 1

A data scientist is using MLflow to track experiments on Databricks. They notice that the metrics logged during a run are not appearing in the MLflow UI. The run is part of an experiment with many runs. What is the most likely cause for the missing metrics?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The metrics were logged to a different experiment than the one currently selected in the MLflow UI.. Metrics are associated with a specific run, and runs belong to an experiment. If the run was created under a different experiment than the one selected in the MLflow UI, its metrics will not be visible. This often happens when the experiment is changed mid-session or when using a different tracking URI. Checking the experiment ID of the run resolves the issue.

Answer

The metrics were logged with a step value that is not monotonically increasing, causing MLflow to ignore them.

Answer

The metrics were logged as strings instead of numeric values, so MLflow silently drops them.

Answer

The metrics were logged using mlflow.log_metric but the run was not properly ended, so they are still buffered.

Question 2

Which Databricks feature should be used to provide a managed, secure, and scalable endpoint for real-time inference of models logged in the Model Registry?

Accepted Answer

Databricks Model Serving endpoints.. Databricks Model Serving provides a fully managed, low-latency API endpoint for model inference. It abstracts away the infrastructure requirements, enabling horizontal scaling and high availability. It is the standard service for deploying models in production, as it integrates directly with the Model Registry, allowing teams to push production-ready models to endpoints with minimal configuration and secure access controls.

Answer

Databricks Jobs running the model in a scheduled notebook.

Answer

A standard interactive cluster running a Flask server.

Answer

Delta Live Tables pipelines.

Question 3

What is the primary role of an 'MLflow Signature' during the model deployment phase?

Accepted Answer

To define the input and output schema for validation. The MLflow signature acts as a contract between the model and the application that consumes it. By defining expected input types and shapes, the signature enables automatic validation of incoming payloads before they reach the model. This prevents runtime errors, such as type mismatches or missing features, and ensures that the serving endpoint remains robust and reliable when receiving production traffic from various sources.

Answer

To encrypt the model artifacts for security

Answer

To determine the optimal number of cluster nodes

Answer

To compress the model for faster deployments

Question 4

Refer to the exhibit. What is the effect of using the 'registered_model_name' parameter in the 'log_model' function?

Accepted Answer

It registers a new version of the model in the MLflow Model Registry.. Including 'registered_model_name' automatically registers the model in the MLflow Model Registry as part of the logging process. This simplifies the deployment pipeline by eliminating the need to manually promote the model later. It creates a new version entry directly in the registry, ensuring that the model is immediately available for staging or production assignment, which is a best practice for streamlined machine learning operations.

Answer

It restricts access to the model to the current user only.

Answer

It automatically promotes the model to the 'Production' stage.

Answer

It forces the model to be saved in a specific S3 bucket.

Question 5

A data scientist is monitoring model drift in Databricks. Which TWO approaches are recommended to detect performance degradation in a production model?

Accepted Answer

Regularly compare current prediction metrics against the baseline training metrics.. Monitoring model drift is essential to ensure the continued accuracy of production models. Comparing current model performance metrics against a baseline established during training allows for proactive retraining. Similarly, tracking the distribution of input features (data drift) helps identify changes in data patterns that may undermine the model's reliability. Combining these methods ensures a comprehensive view of the model's health and readiness for redeployment or adjustment.

Answer

Delete the original training data to force the model to learn new patterns.

Answer

Automate model retraining to run every hour, regardless of drift.

Answer

Manually inspect every prediction in the production logs for errors.

Question 6

Which THREE factors should be considered when choosing the 'workload size' (e.g., Small, Medium, Large) for a Databricks Model Serving endpoint?

Accepted Answer

The memory requirements of the model artifact. Selecting the correct workload size is a trade-off between latency requirements, compute costs, and the model's resource footprint. Larger models with high parameter counts often require 'Large' configurations to prevent memory exhaustion or high tail latency. Conversely, smaller models may perform adequately on 'Small' instances, which are cost-effective. Monitoring the endpoint's resource utilization metrics is key to right-sizing the deployment and ensuring optimal performance within the budget constraints.

Answer

The total number of users in the workspace

Answer

The color scheme of the MLflow UI

Question 7

What is the primary function of the 'Model Signatures' in MLflow?

Accepted Answer

To define the input/output schema for the model.. Model Signatures define the expected data types and structure for the inputs and outputs of a machine learning model. This metadata is crucial for data validation and automated infrastructure testing in Databricks. By enforcing this schema, you ensure that the serving layer can validate incoming requests against the model's expectations, which significantly reduces runtime errors and improves the overall reliability of production machine learning applications within the workspace.

Answer

To act as a digital watermark for security.

Answer

To encrypt the model artifacts.

Answer

To identify the author of the model.

Question 8

A data scientist is training a Random Forest model on a 500GB dataset using Databricks. They notice the training process is slow and memory-intensive on a single worker node. Which approach should they take to optimize the training process?

Accepted Answer

Use the Spark MLlib Random Forest implementation which natively supports distributed training.. Utilizing Spark MLlib's distributed training capabilities allows the Random Forest algorithm to partition data across the cluster nodes rather than relying on a single executor. This significantly reduces the memory overhead per node and parallelizes the tree-building process. Understanding this mechanism is vital for scaling ML workflows on Databricks, as it shifts the bottleneck from local memory constraints to cluster-wide compute resources, facilitating the processing of large-scale datasets efficiently.

Answer

Increase the driver node memory to accommodate the full dataset during training.

Answer

Convert the dataset to a single massive CSV file stored on DBFS to improve I/O speeds.

Answer

Enable local caching for all input features to minimize data shuffling.

Question 9

A data scientist needs to deploy a model to Databricks Model Serving. Which component is strictly required to be logged in MLflow to enable the 'Model Serving' feature?

Accepted Answer

The model signature defining input and output schema. Databricks Model Serving requires a model artifact to be logged with a signature. The signature defines the expected input schema, which allows the serving endpoint to validate incoming requests. Without a signature, the model cannot be correctly parsed by the serving infrastructure, preventing the deployment from starting. This ensures that the production inference endpoint operates with expected data formats, maintaining reliability in downstream applications.

Answer

The model's training accuracy metrics

Answer

A Unity Catalog registered function

Answer

A dedicated high-concurrency cluster

Question 10

Which Databricks component should be used to track parameters, code versions, metrics, and output files when running machine learning experiments?

Accepted Answer

MLflow Tracking. MLflow Tracking is the primary component for logging the inputs and outputs of machine learning runs. It allows data scientists to organize experiments and compare performance across different iterations. By centralizing this information, teams can ensure reproducibility and visibility into the model development process, which is foundational for maintaining high-quality machine learning workflows and facilitating collaboration within the Databricks environment.

Answer

Databricks Model Registry

Answer

Databricks Feature Store

Answer

Unity Catalog

Question 11

When using Databricks Model Serving, what is the primary benefit of using a 'Provisioned Throughput' endpoint over a 'Serverless' endpoint?

Accepted Answer

It provides guaranteed performance and lower latency for heavy workloads.. Provisioned Throughput is designed for high-performance use cases requiring guaranteed performance, such as low-latency requirements or high-throughput scenarios that cannot tolerate fluctuations in resource availability. By reserving dedicated capacity, it provides the predictable performance needed for enterprise-level applications, ensuring that critical models meet their SLAs despite external load variations. This is a crucial choice for models that require consistent performance regardless of traffic volume patterns.

Answer

It is always the cheapest option for low-traffic models.

Answer

It eliminates the need to provide a model signature.

Answer

It is the only way to deploy non-Python models.

Question 12

A machine learning team is using Databricks Jobs to orchestrate a multi-step ML pipeline: data ingestion, feature engineering, model training, and batch inference. They need to ensure that if the model training step fails, the batch inference step does not run, and that the entire pipeline can be retried from the failed step. Which Databricks Jobs feature should they use to achieve this?

Accepted Answer

Configure each task with a dependency on the previous task and set the job to repair runs on failure.. Databricks Jobs allows defining tasks with dependencies, so the inference task will only run if the training task succeeds. The repair run feature enables retrying a failed run from the failed task, reusing successful task results. This combination provides both conditional execution and efficient recovery.

Answer

Use a single notebook with all steps and rely on the notebook's built-in error handling to skip inference on failure.

Answer

Set up a continuous job that always runs all tasks, and use conditional logic in the inference task to check if training succeeded.

Answer

Use Databricks Jobs with a cluster that has autoscaling enabled and set the inference task to run only if the training task logs a success metric.

Question 13

Which of the following describes the purpose of a 'Validation Set' in the model development cycle?

Accepted Answer

To help tune hyperparameters and avoid overfitting.. A validation set is used during training to tune hyperparameters and select the best model configuration. By evaluating on unseen data during the training phase, data scientists can prevent overfitting—where the model memorizes the training data rather than learning patterns. This is a crucial step in the ML workflow, as it ensures the model will generalize well to new data after it is deployed to production.

Answer

To serve as the final test set for reporting final accuracy.

Answer

To increase the training dataset size.

Answer

To train the model parameters directly.

Question 14

A data scientist is performing hyperparameter tuning for a scikit-learn model using MLflow on a Databricks cluster. They want to parallelize the tuning to reduce total runtime. Which Databricks-supported method allows them to run multiple trials concurrently with minimal code changes?

Accepted Answer

Use Hyperopt with `SparkTrials`.. Hyperopt's `SparkTrials` is the Databricks-recommended way to parallelize hyperparameter tuning. It distributes trials across Spark executors, integrates with MLflow for automatic logging, and requires only a small change to the tuning code. Other options either do not provide parallelism or require manual orchestration, making them less efficient.

Answer

Use Databricks Jobs to schedule multiple notebook runs with different parameters.

Answer

Use MLflow's `mlflow.sklearn.autolog()` with a single run.

Answer

Use MLflow Projects to package the training code and run it via `mlflow run`.

Question 15

An ML engineer is using MLflow Tracking to compare multiple runs of a hyperparameter tuning experiment. The engineer wants to quickly identify the run that achieved the best validation accuracy and then promote that run's model to the Model Registry. Which MLflow feature allows the engineer to view and compare runs in a centralized UI?

Accepted Answer

MLflow Tracking UI. The MLflow Tracking UI is designed to log and compare runs, displaying metrics, parameters, and artifacts in a sortable table. It allows the engineer to identify the best run and then register its model to the Model Registry with a few clicks, streamlining the workflow.

Answer

MLflow Model Registry

Answer

MLflow Projects

Answer

MLflow Models