Databricks-ML-Assoc Model Development Practice Test 4 — 15 Questions

Question 1

A machine learning engineer is using MLflow to track experiments in Databricks. They want to compare multiple runs and identify the best performing model based on a custom metric called 'f1_score'. They also want to ensure that the best model is automatically registered to the MLflow Model Registry with a stage transition to 'Staging'. Which TWO of the following actions are required to achieve this? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Log the 'f1_score' metric using mlflow.log_metric() during each run.. To automatically identify and register the best model based on a custom metric, you must first log the metric during each run so it can be compared. Then, you need to programmatically search for the run with the highest metric value and register its model artifact to the Model Registry, optionally transitioning it to a stage. Logging the metric and using the API to register the best run are the essential steps.

Answer

Set the model's stage to 'Staging' during logging by passing stage='Staging' to mlflow.sklearn.log_model().

Answer

Create a webhook that triggers on model registration to transition the model to 'Staging'.

Answer

Use the MLflow UI to compare runs and manually register the best model to the Model Registry.

Question 2

A data scientist is using MLflow to track experiments for a model they are developing. They want to compare the performance of different runs and quickly identify the best one based on a custom metric called 'f1_score'. Which MLflow feature allows them to filter and sort runs by this metric in the Databricks workspace?

Accepted Answer

The MLflow experiment UI's runs table, where they can sort by the 'f1_score' metric column. The MLflow experiment UI includes a runs table that allows sorting and filtering by any logged metric. This is the most straightforward way to compare runs and identify the best model based on a custom metric like f1_score, without writing additional code.

Answer

The mlflow.search_runs API, which returns a pandas DataFrame that can be sorted programmatically

Answer

The Databricks notebook's display function, which can render a table of runs from a query

Answer

The Model Registry's model versions page, where metrics are displayed for each version

Question 3

A machine learning engineer is training a distributed XGBoost classifier on a large Delta table in Databricks using the xgboost.spark.SparkXGBClassifier estimator within a Spark ML Pipeline. The engineer wants every booster round's evaluation metric to appear as a separate MLflow metric step, and also wants the fitted pipeline to be reproducible when loaded later in a different notebook. Which two MLflow integrations should the engineer configure to meet both requirements?

Accepted Answer

Set mlflow.xgboost.autolog() before fitting the pipeline, which captures per-round evaluation metrics and also logs the fitted SparkXGBClassifierModel as an MLflow model artifact.. The XGBoost autologging integration is designed for both the native xgboost API and the xgboost.spark estimators on Databricks. It attaches to the booster's callback system so every evaluation round becomes a stepped MLflow metric, and it logs the fitted estimator with XGBoost flavor metadata so a later notebook can reload and reproduce predictions. Spark-native autologging paths do not cover the booster's callback metrics.

Answer

Call mlflow.spark.autolog() before fitting the pipeline, because SparkXGBClassifier is a Spark ML estimator and Spark autologging records the booster's evaluation history and serializes the fitted pipeline.

Answer

Wrap the fit call inside mlflow.start_run() and manually call mlflow.log_metric() inside an xgboost callback, then persist the fitted pipeline with mlflow.spark.log_model(), which will store the booster in the Spark flavor.

Answer

Enable mlflow.pyspark.ml.autolog() before fitting, which instruments the Spark ML Pipeline stages and automatically records booster evaluation rounds and the fitted pipeline artifact.

Question 4

A data scientist is building a pipeline using MLflow. Which TWO of the following tasks should be performed to ensure experiment reproducibility and auditability?

Accepted Answer

Log the git commit hash using mlflow.set_tag().. Reproducibility in machine learning requires strict tracking of code versions, dependencies, and parameters. By logging the source code version (git hash) and the specific Python environment (conda.yaml or requirements.txt), the scientist ensures that any collaborator can recreate the exact training state later. These practices are fundamental to the Databricks ML lifecycle, ensuring that models can be retrained or audited for compliance during the deployment phase.

Answer

Manually copy the training data into the MLflow model folder.

Answer

Disable the automatic logging feature to save memory.

Answer

Use a global variable for all experiment parameters.

Question 5

When performing hyperparameter tuning using Hyperopt with SparkTrials on Databricks, what is the primary advantage of using SparkTrials over the standard Trials object?

Accepted Answer

SparkTrials allows for the distribution of training jobs across multiple workers.. SparkTrials enables the parallel execution of hyperparameter trials across the Spark cluster. By distributing the training tasks, it significantly reduces the time required to complete large grid or random searches. Understanding this distinction is essential for optimizing the development lifecycle, as it prevents the bottleneck of sequential execution on a single node, which is unfeasible for complex models or large datasets common in professional enterprise scenarios.

Answer

SparkTrials automatically selects the best hyperparameters.

Answer

SparkTrials provides built-in visualization of the parameter space.

Answer

SparkTrials forces the use of a GPU-enabled cluster.

Question 6

A model has been trained on a dataset containing categorical features with high cardinality. Which technique is most effective for preparing these features for a linear model in a Databricks environment?

Accepted Answer

Use Target Encoding to map categories to their mean target value.. Linear models require numeric inputs and are sensitive to the scale and representation of categorical variables. One-hot encoding high-cardinality features results in sparse, high-dimensional matrices that degrade performance. Target encoding or using specialized embeddings is more efficient. Choosing the right encoding strategy is a critical model development decision that directly impacts training speed, convergence, and the overall predictive accuracy of the final model deployed into production.

Answer

One-hot encode all categorical variables.

Answer

Drop the categorical features entirely.

Answer

Convert all features into a single string representation.

Question 7

Which Databricks feature allows data scientists to automatically track parameters, metrics, and models during the training process?

Accepted Answer

MLflow Tracking.. MLflow Tracking is the core component of the Databricks machine learning platform for monitoring experiments. It provides a centralized API to log parameters and metrics, enabling teams to compare different versions of models side-by-side. Mastering this feature is foundational for the exam, as it is the primary mechanism for managing the iterative nature of model development and ensuring that research is structured and reproducible.

Answer

Delta Lake.

Answer

Databricks Jobs.

Answer

Unity Catalog.

Question 8

Refer to the exhibit. Why is including the `signature` and `input_example` in the `log_model` call considered a professional best practice?

Accepted Answer

It enables schema validation and improves model serving usability.. Providing a signature and input example allows MLflow to define the expected schema for the model. This metadata facilitates automatic type validation and allows the model serving infrastructure to generate accurate API documentation. This is critical for downstream consumers who need to integrate the model, as it prevents runtime integration errors and clarifies the interface expectations before the model is even deployed into production.

Answer

It forces the model to run on a GPU during inference.

Answer

It speeds up the model training process.

Answer

It reduces the storage size of the model artifact.

Question 9

A data scientist is using Databricks to train a deep learning model. They need to monitor training loss and accuracy in real-time. Which tool is best suited for this task?

Accepted Answer

MLflow log_metric API.. MLflow's logging API allows users to log metrics at each step of the training loop. By calling `mlflow.log_metric()` during the epochs of a deep learning model, the scientist can visualize the training progress in the MLflow UI. This real-time feedback loop is essential for detecting issues like vanishing gradients or overfitting early in the development lifecycle, preventing wasted compute hours on non-converging or poor-performing models.

Answer

Databricks File System (DBFS) logs.

Answer

Spark UI metrics tab.

Answer

Unity Catalog lineage.

Question 10

Which of the following describes the purpose of a 'Validation Set' in the model development cycle?

Accepted Answer

To help tune hyperparameters and avoid overfitting.. A validation set is used during training to tune hyperparameters and select the best model configuration. By evaluating on unseen data during the training phase, data scientists can prevent overfitting—where the model memorizes the training data rather than learning patterns. This is a crucial step in the ML workflow, as it ensures the model will generalize well to new data after it is deployed to production.

Answer

To serve as the final test set for reporting final accuracy.

Answer

To increase the training dataset size.

Answer

To train the model parameters directly.

Question 11

A team is transitioning their model development from a single notebook to a production-grade ML pipeline. Which Databricks feature should they use to manage and coordinate this end-to-end process?

Accepted Answer

Databricks Workflows.. Databricks Workflows allows for the creation of multi-task pipelines that can run notebooks, Python scripts, or JARs in a defined order. This is essential for professional ML development, where data preprocessing, training, and evaluation need to happen sequentially. Moving from ad-hoc notebooks to controlled workflows improves reliability, allows for dependency management, and enables scheduling, which is critical for continuous integration and delivery of machine learning models.

Answer

Databricks Feature Store.

Answer

MLflow Model Registry.

Answer

Databricks SQL.

Question 12

When logging a model in Databricks, why is it recommended to specify the `pip_requirements` or `conda_env` explicitly instead of relying on the environment's current state?

Accepted Answer

It ensures the model remains portable and reproducible.. Relying on the current notebook environment is risky because it captures all installed packages, not just those required for the model. Specifying dependencies explicitly ensures that the environment is minimal and reproducible, which is vital for preventing environment conflicts in production. This practice ensures that the model can be deployed successfully in clean, isolated environments, a core requirement for stable production model serving and CI/CD pipelines.

Answer

It forces the model to use the latest version of all libraries.

Answer

It significantly reduces the model artifact's memory usage.

Answer

It allows the model to run without any Python libraries.

Question 13

A data scientist is training a Random Forest model on a 500GB dataset using Databricks. They notice the training process is slow and memory-intensive on a single worker node. Which approach should they take to optimize the training process?

Accepted Answer

Use the Spark MLlib Random Forest implementation which natively supports distributed training.. Utilizing Spark MLlib's distributed training capabilities allows the Random Forest algorithm to partition data across the cluster nodes rather than relying on a single executor. This significantly reduces the memory overhead per node and parallelizes the tree-building process. Understanding this mechanism is vital for scaling ML workflows on Databricks, as it shifts the bottleneck from local memory constraints to cluster-wide compute resources, facilitating the processing of large-scale datasets efficiently.

Answer

Increase the driver node memory to accommodate the full dataset during training.

Answer

Convert the dataset to a single massive CSV file stored on DBFS to improve I/O speeds.

Answer

Enable local caching for all input features to minimize data shuffling.

Question 14

Which TWO of the following practices are recommended when performing feature engineering on Databricks using Feature Store to ensure consistency between training and inference?

Accepted Answer

Register feature tables in the Databricks Feature Store with unique primary keys.. Feature Store ensures that the exact same transformation logic used during model training is accessible during low-latency inference. By utilizing the Feature Store's unified interface, data scientists prevent training-serving skew, which is a common cause of poor model performance in production. This practice promotes reproducibility and streamlines the deployment pipeline by decoupling data preparation from model training code, ensuring that production features are calculated reliably using the same definitions as training features.

Answer

Hardcode all feature transformation logic directly into the model training notebook.

Answer

Calculate features in real-time within the model inference endpoint only.

Answer

Avoid using time-based features to ensure the model remains simple.

Question 15

When logging a model using MLflow in Databricks, which component is required to capture the environment dependencies so that the model can be accurately reproduced on a different cluster?

Accepted Answer

A conda.yaml or requirements.txt file specifying library versions.. Capturing environment dependencies is essential for reproducibility, as models often rely on specific library versions. MLflow uses the conda.yaml or requirements.txt files to snapshot the Python environment during the log_model process. This ensures that when the model is loaded in a deployment environment, it functions exactly as it did during development, preventing runtime errors caused by version mismatches or missing dependencies in the target production environment.

Answer

The raw input dataset in Parquet format.

Answer

The cluster's Spark configuration JSON string.

Answer

The raw model object without additional metadata.