Select Two (Multi-Select) Questions

Hardcode the data path to an external S3 bucket within the model object.

Why it fails: Hardcoding paths is a dangerous practice that limits portability and security. Models should remain agnostic of infrastructure, relying on data being passed into the inference endpoint. Hardcoding also creates security vulnerabilities by embedding sensitive connection strings or bucket paths directly into the serialised model artifact.

Include the entire training dataset in the model's metadata.

Why it fails: Model metadata is intended for lightweight configuration and logging, not for storing massive training datasets. Including large datasets within the metadata object will significantly degrade performance, bloat artifact size, and likely cause failures during the saving and loading process due to memory limits in the MLflow tracking service.

Log the model with the feature store's training set specification to enable automatic feature lookup at serving time.

Why it fails: While logging with the training set specification is a good practice for feature lookup, it alone does not ensure consistency if the online store is not updated correctly. The question asks for practices that ensure consistency; this option is about enabling lookup, not about maintaining data parity. It is part of the solution but not sufficient by itself, and the stem asks for two practices, with A and B being the core ones.

Store all features in a single Delta table and query it directly from the model serving endpoint.

Why it fails: Querying a Delta table directly from a serving endpoint is not supported for low-latency lookups and would introduce high latency. Feature Store online stores are optimized for real-time access. This approach would also bypass the consistency mechanisms of Feature Store, risking skew and performance issues.

Manually copy feature values from the offline store to the online store on a daily basis using a notebook.

Why it fails: Manual copying is error-prone and can introduce delays or mismatches. It does not guarantee that the same transformations are applied, leading to training-serving skew. Automated pipelines are required to maintain consistency. This practice is not recommended and would likely cause discrepancies between online and offline features.

MLflow Tracking to log parameters, metrics, and artifacts for each run.

Why it fails: MLflow Tracking is important for experiment management and reproducibility, but it is not strictly essential for the CI/CD pipeline to function. The pipeline can retrain, validate, and deploy without logging to MLflow, although it would lack traceability. The core automation and deployment are handled by other components, so this is not one of the two essential components.

A Databricks Repos integration to version control notebooks and manage code changes.

Why it fails: Version control is a best practice for collaboration and reproducibility, but it is not essential for the automated retraining and deployment pipeline to operate. The pipeline can run from notebooks stored in the workspace without Repos. While Repos improves development workflows, it is not a core component that enables the automated CI/CD process.

A feature store that automatically ingests new data and updates features in real time.

Why it fails: A feature store can be beneficial for managing features consistently, but it is not essential for the CI/CD pipeline itself. The pipeline can read raw data and compute features on the fly. While a feature store adds value, it is an additional component rather than a core requirement for retraining, validation, and deployment automation.

Use random seeds for all ML libraries.

Why it fails: While random seeds are necessary for reproducibility of individual runs, they do not manage lineage or environment configurations. Relying solely on seeds is insufficient for MLOps, as it ignores the dependency management and code-versioning aspects required to track how a specific model was created.

Store all training data as CSV files in the workspace.

Why it fails: Storing training data as CSVs in the workspace is not scalable or secure. MLOps best practices suggest using Delta tables for versioning data through time travel, as CSVs do not provide the transactional integrity or efficient lineage tracking required for enterprise-scale machine learning operations.

Manually copy artifacts to a shared folder.

Why it fails: Manual file management is error-prone and lacks auditability. It bypasses the MLflow Model Registry's built-in tracking and versioning mechanisms, rendering lineage tracing impossible. MLOps requires automated, centralized storage solutions to ensure that artifacts are managed consistently throughout their lifecycle without human intervention or oversight.

Deploy the model to a different workspace to isolate production traffic.

Why it fails: Deploying to a different workspace does not help monitor drift or performance; it simply changes the environment. It could complicate monitoring by splitting data across workspaces. The goal is to centralize monitoring data and analysis, not isolate it. This action does not contribute to detecting data drift or model degradation.

Use MLflow to log the model's training metrics and compare them to production metrics manually.

Why it fails: MLflow tracks training metrics, but it does not automatically capture production metrics. Manually comparing training metrics to production is not a scalable or real-time monitoring solution. While MLflow can be part of a monitoring strategy, it lacks the automatic data capture needed for drift detection. Inference tables and dashboards provide a more direct and automated approach.

Configure the endpoint to use a smaller instance type to reduce cost.

Why it fails: Using a smaller instance type may reduce cost but does not aid in detecting data drift or model performance degradation. In fact, it could negatively impact latency and throughput. Monitoring drift and performance requires data collection and analysis, not changes to compute resources. This action is unrelated to the monitoring goals.

Enable automatic model retraining triggered by any change in the input data schema.

Why it fails: Automatic retraining on schema changes alone is not a monitoring action and can be risky; schema changes may not indicate performance degradation. Monitoring should first detect drift and performance issues, and then retraining can be considered as a remediation step. This option does not address the need to detect drift and degradation, and could lead to unnecessary retraining without proper validation.

Use the model's training accuracy as a proxy for production performance and set up alerts based on that metric.

Why it fails: Training accuracy is not a reliable indicator of production performance because it does not account for data drift or changes in the underlying data distribution. Relying on it can mask degradation. Monitoring should focus on production data and, when labels are available, on live performance metrics. Using training accuracy as a proxy would fail to detect issues such as concept drift or feature drift that occur after deployment.

Configure the endpoint to use a smaller instance type to reduce cost, as monitoring does not require additional resources.

Why it fails: Reducing instance size does not aid monitoring and may actually hinder performance if the endpoint becomes overloaded. Monitoring requires capturing and storing inference data, which can be resource-intensive. The goal is to enable observability, not to cut costs at the expense of data collection. A smaller instance could lead to increased latency or failed requests, undermining the monitoring effort.

Deploy two separate endpoints and use an external load balancer to distribute traffic between them.

Why it fails: While external load balancing can distribute traffic, it is not a native Databricks Model Serving feature and adds complexity. It also does not provide integrated monitoring and traffic management within Databricks. The recommended approach is to use a single endpoint with multiple served entities and traffic configuration.

Enable automatic canary deployment by setting a flag in the model's MLflow metadata.

Why it fails: There is no automatic canary deployment flag in MLflow metadata. Canary releases require manual configuration of traffic splits and multiple served entities. Relying on such a flag would not work because the feature does not exist; you must explicitly configure the endpoint and traffic settings.

Configure the endpoint to use a single served entity and rely on the model's internal logic to route requests based on a random seed.

Why it fails: Using a single served entity with internal routing logic does not provide a true canary release because both versions would need to be packaged into one model, which is not standard. It also lacks the ability to monitor and adjust traffic splits at the endpoint level. This approach is not supported for canary deployments in Model Serving.

Enable MLflow autologging for all supported frameworks to capture parameters and metrics automatically.

Why it fails: Autologging captures parameters, metrics, and models for supported frameworks, but it does not automatically log data versions or feature table versions. While it helps with reproducibility of code and hyperparameters, it lacks the data lineage component required for auditability. Relying solely on autologging would miss critical information about the data used.

Manually document the data sources and transformations in a separate spreadsheet for each model version.

Why it fails: Manual documentation is error-prone, not scalable, and cannot be automatically linked to the model. It does not provide the verifiable lineage that auditors require. Automated logging of data and feature versions within MLflow and the Feature Store is the reliable way to ensure auditability. Spreadsheets are not integrated with the ML lifecycle.

Store all training code in a Git repository and commit after each run.

Why it fails: Version control of code is good practice, but it does not capture data lineage or feature versions. Auditors need to know which data and features were used, not just the code. While Git integration in Databricks can log the commit hash to MLflow, it is insufficient alone for full auditability of data and feature lineage.

Manually review the model's performance metrics in a notebook before allowing the pipeline to proceed.

Why it fails: Manual review introduces delays and human error, contradicting the goal of an automated CI/CD pipeline. While manual review can be useful for high-risk models, the requirement is for automated retraining and promotion based on thresholds. Relying on manual steps reduces the efficiency and reliability of the pipeline.

Store the model artifacts in a Git repository alongside the code to ensure versioning.

Why it fails: Git is not designed for large binary artifacts like model files; it can bloat the repository and slow down operations. MLflow Model Registry or Unity Catalog volumes are better suited for model versioning and storage. Using Git for artifacts can lead to performance issues and does not provide the same governance features.

Configure the pipeline to directly overwrite the production model endpoint with the newly trained model without validation to reduce latency.

Why it fails: Skipping validation risks deploying underperforming or faulty models, which can cause business impact. Automated deployment should always include validation steps to ensure quality. Overwriting without validation bypasses safety checks and is not a best practice for ML Ops, as it can lead to silent failures in production.

Store only the model's pickle file and reconstruct dependencies manually on each environment, since pickle serialization is environment-independent.

Why it fails: Pickle files are not environment-independent; they depend on compatible library versions and Python versions, and manual reconstruction is error-prone. Storing only the pickle discards the recorded environment that makes serving reproducible, so this approach neither ensures consistency nor helps detect dependency drift between staging and production.

Rely on the serving endpoint to install the latest available versions of each library at deployment time so the environment stays current.

Why it fails: Installing the latest library versions at deployment time defeats reproducibility, because the environment changes independently of the model version and can break inference. It also makes drift undetectable, since there is no fixed reference to compare against. Consistency requires capturing and honoring the recorded dependency versions, not floating to newest.

Pin the serving cluster's runtime to a different Databricks Runtime version than training used, to validate cross-version compatibility during staging.

Why it fails: Deliberately using a different runtime than training is a compatibility test, not a consistency practice, and it increases the chance of dependency drift between staging and production. The requirement is a consistent environment across both environments, so introducing a runtime mismatch undermines the goal rather than supporting it.

Enable automatic model version archiving so that older Production versions are removed before the new version is promoted.

Why it fails: Archiving older versions is a lifecycle policy choice, not a requirement for promotion. A new version can be transitioned to Production while previous versions remain in Production or are archived separately. Automatic archiving is not a feature that must be enabled for the transition to succeed, and enabling it could remove versions that auditors or rollback procedures still need. It is therefore not one of the required actions.

Require the service principal to use a personal access token tied to a human user so that audit logs show a named approver.

Why it fails: Service principals should authenticate with their own credentials or OAuth, not a human user's personal access token. Using a human token undermines the purpose of automation, creates a security risk if the user leaves, and produces misleading audit logs. It also does not grant any additional capability needed for the transition. This is an anti-pattern and not a required action for safe, repeatable promotion.

Store the model artifacts in a Unity Catalog volume and reference the volume path when registering the model version.

Why it fails: Model artifacts are stored in the run's artifact location and referenced by the model version automatically. Pointing registration at a Unity Catalog volume path is not necessary for stage transitions and does not affect whether the service principal can promote a version. While Unity Catalog can govern model access, it is not a required step to implement Staging-to-Production promotion in this pipeline, so it is not a correct choice.

Manually copy the feature values from the offline store to the online store before each inference request to ensure freshness.

Why it fails: Manual copying is error-prone and not scalable. Databricks Feature Store automatically syncs features from the offline store to the online store when you publish feature tables. Manual intervention would introduce latency and potential inconsistencies, defeating the purpose of the Feature Store.

Implement a custom UDF in the model to compute features on the fly from raw data, bypassing the Feature Store entirely.

Why it fails: Bypassing the Feature Store means you lose the consistency guarantees it provides. Custom UDFs might not replicate the exact transformations used during training, leading to training-serving skew. While on-the-fly computation can be useful, it does not ensure consistency and is not the purpose of the Feature Store.

Train the model using a separate notebook that reads directly from the online store to simulate inference-time feature retrieval.

Why it fails: The online store is designed for low-latency serving and may not contain historical data needed for training. Reading from the online store for training would not provide point-in-time correctness and could lead to data leakage or missing features. Training should use the offline store with point-in-time joins.

Use the latest version of all libraries to ensure compatibility and security patches.

Why it fails: Using the latest versions can introduce breaking changes and make the pipeline non-reproducible. A model trained with one version may not perform the same with another. For reliable CI/CD, dependencies should be pinned to avoid unexpected behavior. While security patches are important, they should be applied deliberately and tested, not automatically in a pipeline.

Configure the pipeline to run on a cluster with autoscaling enabled to handle variable workloads.

Why it fails: Autoscaling helps with performance and cost, but it does not directly contribute to reproducibility or reliability of the pipeline logic. The cluster size may vary, but the code and environment remain the same. While useful, it is not a core practice for ensuring reproducible and reliable ML pipelines in the context of CI/CD.

Manually promote the model to staging after reviewing the test results to ensure quality.

Why it fails: Manual promotion introduces human intervention and reduces automation, which can lead to inconsistencies and delays. For a reliable CI/CD pipeline, promotion should be automated based on predefined criteria, such as passing tests and meeting performance thresholds. Manual steps are error-prone and not reproducible.

Enable scale-to-zero to reduce costs during idle periods.

Why it fails: Scale-to-zero reduces cost by scaling down to zero when idle, but it introduces cold start latency when traffic resumes. For handling sudden spikes without dropping requests, scale-to-zero is counterproductive because it may cause initial requests to queue or time out during scale-up.

Adjust the model's batch size in the predict function to process more requests per inference.

Why it fails: Batch size affects the model's inference throughput but does not control the endpoint's auto-scaling behavior. While optimizing batch size can improve efficiency, it does not directly influence how many replicas are provisioned in response to traffic.

Set the endpoint's workload size to 'Large' to increase per-replica capacity.

Why it fails: Workload size determines the compute resources (CPU, memory) allocated per replica, which can affect the number of concurrent requests a single replica can handle. However, it does not directly control the scaling parameters like minimum and maximum replicas, which are the primary levers for auto-scaling.

Grant the production service principal MANAGE on the registered model so it can update versions as needed.

Why it fails: Privileges govern who may alter or administer the model; they do not record provenance. Giving the service principal MANAGE expands its authority without creating any link to training data or code, and it may even weaken governance by broadening write access. Permissions and traceability are separate concerns.

Increase the model version's stage description with a free-text summary of the business purpose and the approving manager's name.

Why it fails: A textual description documents intent and approval but carries no machine-verifiable pointer to data or code. An auditor needs to reproduce or inspect the exact inputs, and prose cannot substitute for a recorded dataset version and commit. Descriptions are useful metadata but insufficient on their own for provenance.

Configure the serving endpoint to log every request and response payload to a Delta table for later inspection.

Why it fails: Request and response logging supports monitoring and drift analysis, but it captures inference traffic rather than training provenance. It says nothing about which dataset or code revision trained the model. While valuable for observability, it does not establish the lineage an auditor demands between the production version and its training inputs.

Scale-to-zero and autoscaling cannot be enabled simultaneously on the same endpoint.

Why it fails: Scale-to-zero and autoscaling can be enabled together. Scale-to-zero is essentially a special case of autoscaling where the minimum replica count is zero. When both are enabled, the endpoint scales down to zero during idle periods and scales up based on demand, combining cost savings with elasticity.

Autoscaling scales based on CPU utilization of the serving containers, not on request concurrency.

Why it fails: Databricks Model Serving autoscaling primarily uses request concurrency (or queue depth) as the metric for scaling decisions, not CPU utilization. This is because inference workloads are often I/O or GPU-bound, and concurrency better reflects the need for additional replicas. Scaling on CPU alone could be inefficient.

Scale-to-zero is only available for GPU workload types, not for CPU workload types.

Why it fails: Scale-to-zero is available for both CPU and GPU workload types in Databricks Model Serving. It is a general feature that helps reduce costs when the endpoint is idle. The choice of workload type does not restrict the ability to enable scale-to-zero.

Deploy the model using a GPU-enabled workload type to reduce latency.

Why it fails: GPU acceleration can improve inference speed but does not enable automatic scaling. The question asks for configurations to achieve autoscaling, not to reduce latency. Using a GPU workload type is orthogonal to scaling and may increase cost without addressing the scaling requirement. It is not a required configuration for autoscaling.

Set the endpoint to use a dedicated cluster with autoscaling enabled.

Why it fails: Databricks Model Serving abstracts the underlying compute and manages scaling automatically; you do not configure a dedicated cluster with autoscaling. The serving infrastructure handles resource allocation based on the replica settings. Specifying a dedicated cluster is not a supported configuration for Model Serving endpoints.

Set the scale-to-zero option to true so the endpoint can shut down when idle.

Why it fails: Scale-to-zero is not supported for all workload types and can introduce cold-start latency. While it reduces cost during idle periods, it does not directly address scaling based on traffic fluctuations. The question focuses on automatic scaling under load, not idle shutdown. Enabling scale-to-zero alone would not provide the necessary scaling behavior for variable traffic.

Set the experiment name to the current date to avoid confusion.

Why it fails: Naming experiments by date may help organize runs chronologically, but it does not contribute to reproducibility or comparison. Experiment names should be descriptive of the project or model. Date-based names can become ambiguous and do not capture the purpose. The key practices for reproducibility are logging parameters and metrics, not naming conventions.

Store the training dataset in the MLflow run's artifact repository for every run.

Why it fails: Storing the full dataset as an artifact for every run is inefficient and often impractical due to size. MLflow is designed to track metadata and models, not large datasets. Instead, log a reference to the data (e.g., a Delta table version or path) to maintain reproducibility without duplicating storage. This practice avoids bloating the artifact store and keeps runs lightweight.

Use a single MLflow run for all experiments to simplify tracking.

Why it fails: Using a single run for multiple experiments mixes parameters and metrics, making it impossible to attribute results to specific configurations. MLflow runs are designed to represent individual executions. Combining them defeats the purpose of tracking and comparison. Each experiment should have its own run to maintain isolation and enable effective analysis.

Enable autoscaling on the serving endpoint to handle varying load.

Why it fails: Autoscaling adjusts the number of replicas based on traffic but does not enforce input schema or specify model version. It is an operational feature for scalability and cost management. While useful for production, it does not address the requirements of schema enforcement or version selection. Therefore, it is not a correct action for this scenario.

Register the model in the MLflow Model Registry and transition it to Production.

Why it fails: Registering and transitioning a model to Production manages its lifecycle stage but does not enforce input schema. Model Serving can serve models from any stage, but schema enforcement comes from the logged model signature. This action is about versioning and stage management, not about validating input data. Therefore, it does not meet the requirement for schema enforcement.

Use mlflow.log_artifact() to save a JSON schema file alongside the model.

Why it fails: While you can log a JSON schema as an artifact, Model Serving does not automatically use it to enforce input schema. The schema must be part of the model's signature, typically provided via input_example or signature during logging. Logging a separate artifact does not integrate with the serving layer's validation. Thus, this action does not enforce the input schema for the deployed model.

Log the model using `mlflow.sklearn.log_model()` with the `code_path` parameter to include custom transformation code, which also captures all dependencies.

Why it fails: The `code_path` parameter allows you to include additional code files with the model, which is useful for custom transformations. However, it does not automatically capture Python package dependencies. You still need to specify dependencies via Conda or pip requirements. Including code without dependencies may lead to failures if the code imports packages not present in the serving environment.

Log the model using `mlflow.sklearn.log_model()` with the `signature` parameter to infer the input schema and automatically generate the environment.

Why it fails: The `signature` parameter defines the input and output schema of the model, which is useful for validation and serving, but it does not capture Python dependencies. MLflow does not generate the environment from the signature; it requires explicit dependency specification or infers from the current environment. This action alone would not guarantee the serving environment can recreate dependencies.

Log the model using `mlflow.sklearn.log_model()` with the `registered_model_name` parameter to register the model, which automatically captures the environment.

Why it fails: The `registered_model_name` parameter registers the model in the Model Registry, but it does not capture dependencies. Registration is about versioning and lifecycle management, not environment capture. MLflow still relies on the environment files (Conda or pip) that are logged with the model artifacts. This action alone does not ensure dependency capture.