Databricks Machine Learning practice questions

It denies users the ability to delete experiments.

Why it fails: The 'effect' key is set to 'allow', which explicitly grants permission to perform the listed action. The policy name 'deny' is just a label or description and does not override the operational logic defined within the JSON structure. Therefore, the policy effectively permits the deletion action for authorized users.

It restricts the policy only to the 'mlflow' namespace.

Why it fails: While it references 'mlflow:DeleteExperiment', the policy's primary impact is on the permission to delete experiments, not the namespace itself. The policy effectively authorizes the deletion action within the scope defined by the policy engine, regardless of how the namespace is structured or accessed in the workspace.

The policy is invalid and will be rejected by Databricks.

Why it fails: The JSON syntax is valid for policy definition. While the naming might be confusing, the engine will parse the key-value pairs according to standard policy logic. Therefore, the policy will be accepted and applied, potentially creating unintended access if the administrator did not intend to grant deletion permissions.

Model training loss metrics

Why it fails: Training loss metrics are recorded as part of the run's metrics in MLflow, not as part of the model signature. The signature is strictly concerned with the interface (input/output) of the model, not the performance history or the accuracy achieved during the training process on the validation set.

Model hyperparameters

Why it fails: Hyperparameters are logged as parameters during the MLflow tracking process. They describe how the model was trained but do not describe the model's interface for runtime prediction. Including them in the signature would be technically incorrect, as they are not part of the inference request/response cycle.

Automated online feature sync via Databricks Jobs

Why it fails: Automated online synchronization replicates feature table data from offline Delta tables to low-latency online stores like DynamoDB or Cosmos DB. While necessary for real-time serving, synchronization does not resolve historical point-in-time join logic required during offline training dataset creation.

Feature table time-travel queries using Delta Lake versions

Why it fails: Delta time travel allows querying specific table versions or timestamps based on file modification history. However, it does not perform row-level temporal joins across observation timestamps, which is required when merging multiple feature tables with varying update frequencies.

Feature Store point-in-time training set creation API

Why it fails: Incorrect option mapping because the correct API automatically merges observation data frames with feature tables using timestamp keys, preventing future information from leaking into historical training records and preserving model generalization validity.

Re-calculating features manually in the inference code.

Why it fails: Manually re-calculating features in the inference script defeats the purpose of the Feature Store and introduces significant risk. Differences in code implementation or library versions between the training and inference environments often lead to training-serving skew, resulting in degraded model performance and inaccurate predictions for live users.

Always storing features as raw parquet files.

Why it fails: Storing features strictly as raw parquet files ignores the capabilities of the Online Store. Real-time inference requires a low-latency database, such as DynamoDB or CosmosDB, to serve features. Parquet files are optimized for offline processing and batch jobs, not the sub-millisecond response times required for production serving applications.

Ignoring the time-series nature of data.

Why it fails: Ignoring time-series data leads to significant data leakage during training. Feature Store supports point-in-time joins, ensuring that models only use data that was available at the time of observation. Failing to account for this will result in overly optimistic model validation metrics that do not translate to production.

Hard-code all hyperparameters within the training loop.

Why it fails: Hard-coding parameters prevents the optimization algorithm from varying them. Hyperopt must be able to inject values into the training function dynamically. Static values result in a single training run rather than an iterative search, failing to leverage the power of automated hyperparameter tuning for improved model performance.

Restrict training to a single CPU core only.

Why it fails: Restricting training to a single core negates the benefits of using a Databricks cluster. Hyperparameter tuning is intended to maximize hardware utilization. By limiting resources, you create a bottleneck that defeats the purpose of distributed computing and significantly extends the time needed to identify the best model.

Daily batch processing capabilities for bulk updates.

Why it fails: While batch processing is important for offline training, it is not a requirement for the Online Store. The Online Store is designed for point-lookups rather than bulk processing. Using it for daily batch jobs would be inefficient and potentially interfere with the performance of real-time inference lookups for users.

Direct integration with local CSV storage.

Why it fails: Integrating directly with local CSV files is impractical and insecure for production-grade Online Stores. Such stores typically utilize managed cloud databases like DynamoDB or CosmosDB to ensure reliability, scalability, and security. CSV-based storage lacks the required performance and management features necessary for enterprise-level real-time model serving and feature lookups.

The model is too large for the network.

Why it fails: While network bandwidth can affect latency, it does not typically cause performance inconsistency in the model logic itself. If the model is too large, the error would be consistent (e.g., timeouts or persistent high latency), not intermittent or varying. The issue likely lies in environment or dependency mismatches.

The input data has too many missing values.

Why it fails: While missing values can affect model output, they do not explain performance inconsistencies unless the data pipeline is fundamentally flawed. Inconsistency usually points to a software-level difference between environments, such as library versions, rather than data quality issues, which would typically produce consistently poor results across the board.

The model was trained on a GPU cluster.

Why it fails: Training on a GPU cluster does not inherently cause inconsistent performance. If the serving endpoint is configured correctly to handle the model's requirements, it will perform consistently. The inconsistency is far more likely to stem from library version mismatches or differences in how the inference environment is set up.

Can Run

Why it fails: The 'Can Run' permission typically allows a user to trigger the notebook execution, which is helpful but does not explicitly describe the 'view-only' restriction requested. Other permissions might provide more access than necessary, so choosing the most restrictive option that satisfies the requirement is the best practice for security.

Can Manage

Why it fails: The 'Can Manage' permission gives the user full control, including the ability to change permissions and delete the notebook. This is far beyond the requirement of simply viewing results and would create a significant security risk by allowing unauthorized users to modify or destroy vital project assets.

Owner

Why it fails: An owner has full control over the notebook. This permission level is inappropriate for someone who should only view the results. Giving ownership to another user transfers control, which is the opposite of the restricted sharing and governance required for secure model development and project management within Databricks.

Manually copy model artifacts to a persistent S3/DBFS bucket

Why it fails: Manually copying artifacts is unnecessary and error-prone. Databricks automatically manages model artifact storage when you log a model using MLflow. The system handles versioning and persistence internally, ensuring that the Serving endpoint can reliably retrieve the correct model files without requiring manual file management by the user.

Disable authentication on the serving endpoint

Why it fails: Disabling authentication is a severe security risk and is not a valid or recommended step for production deployment. Databricks Model Serving endpoints are protected by standard Databricks access controls, and security best practices dictate that authentication must always be enabled to protect sensitive model data and endpoints.

Convert the model to a flat JSON file

Why it fails: Models are deployed as serialized objects, such as Python pickles or MLflow flavor formats, not flat JSON files. Converting a model to JSON would strip away the necessary logic and structural information required for the serving engine to perform inference, rendering the model unusable for prediction tasks.

Change the model URI to `models:/prod_ml.churn.final_model/3` and rely on Unity Catalog permissions to block access to all later versions.

Why it fails: Pinning the numeric version stops alias drift, but Unity Catalog model permissions are granted at the registered-model level, not per version. You cannot easily grant EXECUTE on version 3 while denying version 4. The permissions rationale is therefore invalid, even though the version pin itself would work.

Keep the numeric version URI `models:/prod_ml.churn.final_model/3` and require an explicit code or configuration change to move the job to a newly approved version.

Why it fails: This pins the version and prevents alias-driven changes, but it does not satisfy the requirement that promotion be an explicit, controlled action tied to approval. Relying on manual code edits is error-prone and lacks the auditable promotion step the scenario demands. It is close but not the strongest governed answer.

Replace the model URI with `models:/prod_ml.churn.final_model@champion` so the alias resolves consistently in every environment.

Why it fails: Using the `@champion` alias makes the job follow whichever version currently holds the alias, which is exactly the uncontrolled promotion behavior the engineer wants to prevent. An alias is a movable pointer, so re-pointing it to version 4 would silently change the scoring model. This does not guarantee version 3 is used.

The label dataset does not have a timestamp column, so Feature Store defaulted to using the latest features.

Why it fails: The scenario states that the feature table has a timestamp column, but it does not specify whether the label dataset has one. For point-in-time correctness, the label dataset must also have a timestamp column. However, the most likely cause is that the timestamp column was not specified in the create_training_set call, not that it is missing from the label dataset.

The feature table was not registered with a primary key, so Feature Store could not perform time travel.

Why it fails: Feature Store requires a primary key for feature tables, but the issue described is about timestamp handling. Even with a primary key, if the timestamp is not used in the training set creation, point-in-time correctness is not enforced. The primary key alone does not ensure temporal alignment.

The feature table was created as a Delta table without enabling change data feed, so historical values are not available.

Why it fails: Feature Store uses Delta Lake's time travel capabilities, which do not require change data feed. Change data feed is for capturing row-level changes, not for point-in-time lookups. Enabling it would not affect the training set creation in this context.

Enable autoscaling on the served model and configure the endpoint to use the 'Production' stage; new versions in Production will be automatically picked up.

Why it fails: While autoscaling is supported, MLflow Model Serving does not automatically update endpoints when a new model version is transitioned to a stage. The endpoint must be explicitly updated to point to the new version. Therefore, this approach will not achieve zero-downtime updates.

Use the Databricks Model Serving UI to create an endpoint with autoscaling enabled and specify the model version; for updates, create a new endpoint and switch traffic using a load balancer.

Why it fails: Creating a new endpoint and switching traffic via a load balancer is not a native feature of Databricks Model Serving. The service supports updating an existing endpoint to a new model version with zero downtime, so this manual approach is unnecessary and complex.

Configure the endpoint to serve the 'Staging' stage and manually promote to 'Production' after testing; autoscaling is not available for stage-based endpoints.

Why it fails: Autoscaling is available for all served models regardless of stage. Manually promoting to Production does not automatically update the endpoint; you must explicitly update the endpoint configuration. This option incorrectly states that autoscaling is unavailable for stage-based endpoints.

Enable change data feed on the Delta table.

Why it fails: Enabling change data feed is not a requirement for creating a feature table. While change data feed can be useful for tracking changes, Databricks Feature Store does not require it. Feature tables are Delta tables with additional metadata. The primary requirements are specifying keys and a timestamp column. Change data feed is a separate Delta Lake feature that can be enabled independently but is not mandatory here.

Publish the feature table to an online store.

Why it fails: Publishing to an online store is an optional step for enabling online serving. It is not required for creating a feature table itself. Feature tables can be created and used for training without an online store. The online store is only needed when serving features in real-time. Thus, this is not a required step for creating the feature table, although it may be needed later depending on the use case.

Partition the feature table by the primary key.

Why it fails: Partitioning by the primary key is not a requirement. In fact, partitioning by a high-cardinality primary key is generally discouraged because it can lead to many small files and performance issues. Feature Store does not mandate partitioning; it is an optimization choice. The essential requirements are the primary key and timestamp key, not partitioning. So this option is incorrect.

In the AutoML experiment setup, set the 'evaluation_metric' parameter to 'roc_auc'.

Why it fails: There is no 'evaluation_metric' parameter in Databricks AutoML configuration. The correct parameter is 'primary_metric'. Using 'evaluation_metric' would likely cause an error or be ignored. This option is a distractor because it sounds like a plausible parameter name, but it is not part of the AutoML API. The primary metric is the one used for model selection.

In the AutoML experiment setup, set the 'optimization_metric' parameter to 'roc_auc'.

Why it fails: 'optimization_metric' is not a recognized parameter in Databricks AutoML. The correct parameter is 'primary_metric'. Although 'optimization_metric' might seem logical, it is not part of the API. Using it would not change the metric. This option is a distractor because it sounds like it could control the optimization objective, but the actual parameter name is different.

In the AutoML experiment setup, set the 'target_metric' parameter to 'roc_auc'.

Why it fails: 'target_metric' is not a valid parameter for Databricks AutoML. The correct parameter is 'primary_metric'. While 'target_metric' might be intuitive, it is not used in AutoML. This option is incorrect because it references a non-existent configuration option. The primary metric is the term used in Databricks AutoML to specify the metric for optimization.

Increase the number of Spark executors to distribute the load.

Why it fails: Increasing executors can help with distributed computation, but out-of-memory errors on the driver suggest that the driver is collecting too much data or running trials that consume excessive memory. Adding executors does not alleviate driver memory pressure. The issue is likely due to trials running on the driver or large results being collected.

Switch to using Trials instead of SparkTrials to run trials sequentially.

Why it fails: Using Trials runs hyperparameter tuning sequentially on the driver, which can actually increase driver memory usage because all trials run in the same process. It does not distribute the load and may exacerbate memory issues. SparkTrials is designed for distributed tuning, and the problem is likely due to too many concurrent trials overwhelming the driver.

Increase the driver memory by selecting a larger driver instance type for the cluster.

Why it fails: Increasing driver memory can help, but it does not address the root cause of excessive concurrency. It may be a temporary fix but is not as efficient as reducing concurrent trials. Moreover, it increases cost. The better approach is to tune the concurrency to match available driver resources.

Source code version control commit hash

Why it fails: While logging the commit hash can aid reproducibility, it is not strictly required for every experiment. MLflow automatically logs the source notebook or script path and Git commit if available, but it is not a manual log requirement. The question asks for what the engineer should log; hyperparameters and metrics are the core items. The commit hash is optional and often captured automatically.

The model's learned weights

Why it fails: Logging the model's learned weights is typically done by logging the model artifact itself, not as a separate item. While it is important, it is not one of the two primary things to log for reproducibility in this context. Hyperparameters and metrics are more fundamental. Logging the model is a separate step. So this option is not among the two correct choices.

The entire training dataset

Why it fails: Logging the entire training dataset is impractical and not recommended due to size and privacy concerns. MLflow can log dataset metadata (e.g., path, version) but not the full data. For reproducibility, it's better to log the data version or path. Thus, this is not a correct choice. The engineer should log hyperparameters and metrics instead.

Increase the number of trials in AutoML

Why it fails: Increasing the number of trials may explore more models and hyperparameters, but it does not change the optimization objective. Without setting the primary metric to recall, more trials will still optimize for the default metric (e.g., F1). Thus, it does not ensure the model is optimized for recall; it only increases computational cost.

Manually adjust class weights in the training code

Why it fails: AutoML automates model training and does not expose manual class weight adjustments in the standard configuration. While class weights can affect recall, AutoML's interface does not provide a direct setting for them. The recommended approach is to set the primary metric to recall, which AutoML uses to guide model selection and hyperparameter tuning.

Use the default primary metric (F1 score)

Why it fails: The default primary metric for binary classification in AutoML is typically F1 score, which balances precision and recall. While F1 may be a reasonable default, it does not specifically optimize for recall. If recall is the key business metric, using the default would not guarantee the best recall performance, as models with higher precision might be favored.

scale_to_zero_enabled

Why it fails: scale_to_zero_enabled allows the endpoint to scale down to zero when there is no traffic, which reduces cost but may introduce cold-start latency. It does not control auto-scaling based on load; it only enables scaling to zero. For handling variable traffic with auto-scaling, this parameter alone is insufficient because it does not define the scaling range.

workload_size

Why it fails: workload_size is a parameter that indicates the expected workload size (Small, Medium, Large) and influences the initial provisioning of the endpoint. It does not provide dynamic auto-scaling based on real-time traffic. It is used at creation to set a baseline but does not define scaling limits or enable automatic adjustment.

instance_type

Why it fails: instance_type specifies the size of the compute instances used for serving, such as Standard_DS3_v2. It determines the resources per instance but does not control the number of instances or auto-scaling behavior. While choosing an appropriate instance type is important for performance, it does not enable auto-scaling to handle variable traffic.

Configure a Databricks SQL warehouse to serve features via JDBC for online inference.

Why it fails: A Databricks SQL warehouse is designed for analytical queries, not low-latency online feature serving. JDBC connections to a SQL warehouse would introduce too much latency for real-time inference. Online stores are specialized key-value stores optimized for fast lookups, not SQL warehouses.

Register the feature table in the MLflow Model Registry to enable online serving.

Why it fails: The MLflow Model Registry manages model versions and their metadata, not feature tables. Registering a feature table there does not publish it to an online store. Online serving of features is handled by the Feature Store's online store publishing mechanism, not the Model Registry.

Create a separate Delta table for online features and manually copy data using a scheduled job.

Why it fails: Manually copying data to a separate Delta table does not provide low-latency online serving. The online store integration requires a supported online database, not another Delta table. Manual copying would also lack the automatic synchronization and low-latency access needed for real-time inference.

Ensure the custom function is defined in the same notebook used for training and re-log the model with the function included as a code dependency.

Why it fails: Defining the function in the training notebook is insufficient because MLflow does not automatically capture notebook-defined functions as model dependencies. The function must be explicitly included in the model's code directory or logged as part of the model. Without proper inclusion, the serving endpoint cannot locate the function. This option does not address the missing code dependency and would result in the same error.

Add the custom function to the model's conda environment YAML file and re-log the model.

Why it fails: Conda environment YAML files specify Python package dependencies, not custom Python code. While adding the function as a package might work if properly packaged, simply listing it in the YAML does not embed the function code into the model artifact. The model serving endpoint requires the custom function to be available in the model's code path, which is not achieved by modifying the environment file alone. This approach fails because the function's source code is not included.

Log the custom function as an artifact and update the model's signature to reference the artifact path.

Why it fails: Logging the function as an artifact and referencing it in the signature does not make the function executable within the model's environment. The signature describes input/output schema, not code dependencies. The serving endpoint expects the function to be importable or defined in the model's code. This approach misinterprets the purpose of artifacts and signatures, and would not resolve the missing function error.