Refer to the Exhibit Practice Questions

All items are stored in the same local directory on the driver node.

Why it fails: MLflow artifacts are typically stored in the configured MLflow tracking server location, which is usually an S3 bucket or Azure Blob Storage in Databricks. The driver node's local storage is ephemeral and not suitable for persistent storage of models or large training artifacts across cluster restarts.

The log_artifact command uploads the weights directly to the Model Registry.

Why it fails: The 'log_artifact' command adds files to the run's metadata, not the Model Registry. The Model Registry is a separate entity that tracks model versions and lifecycle stages. Artifacts are associated with a run, whereas a registry entry points to a specific URI where the model is stored.

The log_model function is equivalent to copying the file directly to DBFS.

Why it fails: The 'log_model' function does much more than a file copy; it performs serialization, environment capturing, and metadata generation. It creates a complete package that includes dependencies required for inference. Copying a raw file to DBFS would lack the necessary MLflow metadata required for seamless deployment and model versioning tracking.

It automatically scales the cluster size based on the input example size.

Why it fails: Input examples are used for validation and documentation, not for resource orchestration. Cluster scaling is managed by the serverless or classic compute configuration, which responds to request load rather than the specific structure or size of the model's input data provided during the registration process.

It allows the model to be trained in distributed mode using PySpark.

Why it fails: The model signature is a metadata object stored with the model artifact and has no influence on the training process. Distributed training is controlled by the training code, such as using Spark MLlib or Horovod, and is independent of the signature defined for the final registered model.

It automatically converts the model into an ONNX format for faster inference.

Why it fails: Providing a signature does not trigger model conversion. Converting a model to ONNX requires explicit use of conversion libraries and is a separate process from logging a model to the MLflow registry. Signature definition is strictly for metadata and runtime validation, not for architectural format transformation.

Blue/Green deployment; the primary risk is the high cost of running two identical clusters simultaneously.

Why it fails: Blue/Green deployment typically involves a full switch from one environment to another rather than a percentage-based split. Furthermore, the use of scale-to-zero in the config contradicts the risk of high idle costs, as the infrastructure will shut down when not actively processing the split traffic.

A/B testing; the primary risk is that the workload size 'Small' is insufficient for 90% of the traffic.

Why it fails: While this could be used for A/B testing, the workload size is a standard setting that can often handle substantial traffic depending on the model complexity. The more immediate and technical risk in this specific JSON configuration is the interaction between the low traffic percentage and the scale-to-zero functionality.

Shadow deployment; the primary risk is that v2 will interfere with the predictions returned by v1.

Why it fails: In a shadow deployment, the new model receives traffic but its results are not returned to the end user. The config shown explicitly routes 10% of actual traffic to v2, meaning those users will receive v2's predictions, which confirms this is an active Canary deployment rather than a Shadow one.

The model was registered, but the cluster lacks permissions to read it.

Why it fails: Permission errors typically result in an Access Denied or Forbidden exception, not a Resource Does Not Exist error. This specific error message confirms the registry cannot locate the requested object, suggesting a synchronization issue or a misconfigured reference to the model name or version.

The model serving endpoint is already occupied by another model version.

Why it fails: Model serving endpoints can be updated regardless of existing models. If the endpoint were occupied, you would receive a conflict error or the system would overwrite the configuration, not an error stating that a specific registry version does not exist in the database.

The training cluster ran out of memory during the model artifact upload.

Why it fails: An out-of-memory error would result in a cluster failure or a connection timeout error. It would not manifest as a specific REST exception indicating that a named model version is missing from the registry, which is a logic-level error rather than a resource-level error.

The 'entity_version' is set to a legacy version that uses more expensive hardware.

Why it fails: Hardware is determined by the workload size (which is not shown but defaults to Small), not by the model version number. Version 5 and Version 1 would run on the same underlying instance type unless the workload_size parameter itself was modified between deployments.

The endpoint is using a GPU-accelerated instance by default for all Unity Catalog models.

Why it fails: Databricks Model Serving does not default to GPU instances; users must explicitly select a GPU workload type. The exhibit does not show a GPU type being requested, so the cost is likely coming from the continuous running of standard CPU-based instances instead.

Multiple versions of the model are being served simultaneously, doubling the cost.

Why it fails: The 'served_entities' list in the exhibit contains only one object (entity_version 5). Therefore, only one model version is active. If multiple versions were being served, they would each appear as separate entries in the 'served_entities' array within the JSON configuration.

It executes regardless of the failure.

Why it fails: The dependency configuration explicitly requires 'train_model' to complete successfully. The scheduler enforces this logic; it will not trigger the evaluation notebook if the upstream training notebook reports a non-zero exit code or fails to complete, as the downstream task depends on the successful output of the former.

It attempts to run using the last successful model artifact.

Why it fails: Databricks Jobs do not automatically fallback to previous artifacts unless specifically configured with complex logic. By default, the task simply fails or skips. Assuming it would automatically pick up a previous version is dangerous as it would lead to evaluating stale models against current training data.

It retries the 'train_model' task automatically until success.

Why it fails: Retries must be explicitly defined in the task configuration. The provided JSON does not include a retry policy. Without a defined retry count, the system treats a single failure as a terminal state for the task, meaning the dependent task will remain blocked and the job will report failure.

mlflow.get_metric('accuracy', run_id='550e8400-e29b-41d4-a716-446655440000')

Why it fails: The MLflow API does not have a top-level `get_metric` function that accepts a run ID directly in this manner. Metrics are nested within the run data object, and the tracking client must be instantiated to query the backend store for the specific run's results.

mlflow.search_runs(run_ids=['550e8400-e29b-41d4-a716-446655440000'])['accuracy']

Why it fails: The `search_runs` function returns a pandas DataFrame, not a direct dictionary access. Accessing a column named 'accuracy' directly from the list returned by the search function will raise a KeyError, as the metrics are stored within the 'data' structure of the run entity.

mlflow.load_metric('accuracy', '550e8400-e29b-41d4-a716-446655440000')

Why it fails: There is no `load_metric` function in the MLflow Python API. Metrics are fundamentally part of the run metadata and are retrieved via the `MlflowClient` or the `mlflow.get_run` interface, which provides a structured way to access parameters, tags, and performance metrics for specific experiments.

It scales the number of model versions deployed in the registry.

Why it fails: The auto-scale setting controls the compute capacity (number of workers) for the endpoint, not the number of model versions. The Model Registry handles versions independently of the serving infrastructure's scaling settings. Confusing these two components leads to misunderstandings about how endpoint costs and performance are managed.

It automatically re-trains the model when request volume drops.

Why it fails: There is no correlation between auto-scaling infrastructure and model retraining. Retraining is a data-heavy, compute-intensive task that should be triggered by performance metrics or data drift, not by the number of inference requests being served. This would be a highly illogical and resource-wasting operational pattern.

It forces the endpoint to only use 10 workers permanently.

Why it fails: If auto-scaling is enabled, the worker count is dynamic, not permanent. The 'max_workers' setting defines the upper limit, but the system will scale up and down within the defined range based on traffic. Setting a fixed worker count would defeat the purpose of auto-scaling.

MLflow overwrites the previous value with the latest value.

Why it fails: MLflow preserves all logged values for a given metric key. Overwriting would destroy the training history, preventing the analysis of convergence. Maintaining the series of values is essential for visualizing training progress and debugging model performance throughout the entire training process in a professional ML experiment lifecycle.

The run will throw an error because the metric key is not unique.

Why it fails: Metric keys are not required to be unique within a single run. MLflow expects multiple logs for the same key to track training progress. Throwing an error would disrupt the training workflow, which is the opposite of the purpose of a professional tracking tool designed for iterative development processes.

MLflow averages the values and stores the mean.

Why it fails: Averaging the values would hide the training trajectory, making it impossible to see if the model is learning correctly or fluctuating. Storing the full sequence is necessary for transparency and proper evaluation, as it allows users to see the actual progress of the metric as the model is optimized.

It enables automatic hyperparameter tuning during the next training iteration.

Why it fails: The model signature defines data types for input and output, but it does not contain logic related to hyperparameter tuning. Tuning is managed by algorithms like Hyperopt or automated ML tools, which rely on defined objective functions and search spaces, independent of the model's data schema definition.

It automatically encrypts the model weights for secure storage in the registry.

Why it fails: Signatures are metadata objects used for schema validation; they do not impact the encryption status of model artifacts. Model security is managed via the platform's underlying storage encryption settings and Unity Catalog access controls, which are separate from the metadata defined in the MLflow model configuration files.

It forces the model to be converted into a serialized format like ONNX.

Why it fails: A signature is a metadata schema representation and does not trigger format conversion processes. The underlying model format remains whatever was defined during the saving process, regardless of whether a signature is provided. Conversion to ONNX or other formats requires explicit user-driven actions or library-specific conversion utilities.

A direct pointer to the local file system path of the model artifact.

Why it fails: The 'models:/' prefix is a special protocol identifier for the MLflow Model Registry, not a local file system path. It instructs MLflow to query the registry service to locate and fetch the model, rather than attempting to read from a directory on the local machine where the code executes.

An alias for the most recently trained model in the current experiment.

Why it fails: This is a fixed reference to a version number, not a pointer to the 'most recent' model. The most recent model would be accessed via a different convention, such as using 'latest' or by querying the experiment directly, and this specific URI does not dynamically update when new runs occur.

A reference to the raw source code used to train the model.

Why it fails: The model URI points to the serialized model artifact and its associated metadata, not the raw training code. While the artifact might contain information about the code, the URI itself is an access mechanism for the model object used for inference, not a repository for source code files.

The model version is not set to 'PRODUCTION' in the metadata.

Why it fails: The status 'READY' refers to the registration state, not the lifecycle stage. While the stage might be set to 'PRODUCTION', the error is related to loading the artifact from the provided path, not the lifecycle status itself, which is a logic check, not an IO error.

The conda_env file is missing from the artifact storage.

Why it fails: If the file were missing, the error would specifically indicate a 'FileNotFoundException' for that file. The problem described is a general failure to load the model artifact, pointing towards the addressing method (the path) being incorrect rather than a missing specific configuration file.

The run_id 'd7a8e9f0' has been archived and is no longer accessible.

Why it fails: Archiving a run would typically trigger a specific 'AccessDenied' or 'RunNotFound' error. The issue here is related to the loading process of the artifact itself, implying the model exists but the method of referencing it is flawed, which is common with incorrect path usage.

The model must be registered to the model registry before logging.

Why it fails: Registration is a separate step that occurs after logging. While registration is required for deployment, it does not fix the issue of a non-fitted model. An unfitted model will still fail to produce predictions, regardless of whether it is registered in the registry or not.

The model must be wrapped in an mlflow.pyfunc.PythonModel class.

Why it fails: While creating a custom pyfunc is useful for complex logic, it is not required for standard Scikit-Learn models. Scikit-learn has a native MLflow flavor that handles serialization automatically. The issue here is the lack of training, not the choice of the model wrapper or framework integration.

The code must explicitly define the input signature in log_model.

Why it fails: A signature is recommended for validation but is not strictly required to enable the 'predict' method. A model can technically function without a signature, though it is not a production best practice. The fundamental failure in this scenario is that the model has no learned logic to execute.

Update the config to replace 'model_version': '5' with 'model_version': '6' in the current served_models list.

Why it fails: Replacing the version performs an immediate, full cutover rather than a traffic split. This approach provides no way to compare performance between versions or gradually transition users, which is the primary goal of A/B testing. It lacks the safety mechanisms required for validating model performance in production.

Create two separate endpoints, one for version 5 and one for version 6, then split traffic using a load balancer.

Why it fails: Managing two distinct endpoints increases complexity, cost, and management overhead. Databricks natively supports traffic splitting within a single endpoint, which simplifies orchestration, monitoring, and request routing. Using an external load balancer is redundant and introduces unnecessary latency and points of failure outside of the integrated Databricks platform.

Delete the current endpoint and recreate it with a new configuration that includes only version 6.

Why it fails: Deleting and recreating the endpoint causes unnecessary downtime for the model. Traffic splitting should be handled via configuration updates on the existing endpoint to maintain service continuity. Recreating the endpoint is inefficient and disrupts the persistent logs and monitoring data gathered for the currently deployed model version.

client.update_model_version(name='revenue_forecast', version=5, new_stage='Production')

Why it fails: The `update_model_version` method is intended for modifying metadata like descriptions or tags, not for lifecycle stage transitions. Using the incorrect method will result in an AttributeError or a failure to update the stage, as the API structure is designed to enforce specific transition workflows.

mlflow.register_model(model_uri='revenue_forecast/5', stage='Production')

Why it fails: The `register_model` function is used for creating a new version of a model, not for moving an existing version between stages. Attempting to use this for promotion will incorrectly create a duplicate entry or fail because the model version already exists in the registry.

client.set_model_stage(model='revenue_forecast', ver=5, to='Production')

Why it fails: The method `set_model_stage` does not exist in the MLflow client library. The library uses `transition_model_version_stage` to handle these operations. Using non-existent methods will cause the script to crash, highlighting the importance of knowing the specific SDK signatures for Databricks ML development.