Refer to the Exhibit Practice Questions

The model is too large for the GPU memory.

Why it fails: A memory issue would manifest as an OOM (Out of Memory) error, not a network timeout related to a connection to an external vector store. This error clearly identifies a communication failure between the model service and the database, which is a network or configuration layer issue.

The model weights are corrupted.

Why it fails: Corrupted weights would cause errors during the model loading phase or during the inference computation itself, such as segmentation faults or arithmetic errors. It would not typically present as a connection timeout to an external dependency, as the model service would have already initialized before attempting a query.

The input prompt is too long for the model.

Why it fails: Prompt length limits result in model-specific errors, typically token limits or context window errors. They do not trigger network timeout errors regarding external service connections. This error is specific to the infrastructure communication path, not the content of the request being processed by the generative model.

The endpoint will be permanently disabled.

Why it fails: The endpoint is not disabled; it is simply scaled down to zero resources when idle. It remains active and ready to accept traffic, but the infrastructure resources are only provisioned on demand. This is a standard and well-supported feature for optimizing the financial efficiency of Databricks Model Serving.

The endpoint will have higher latency than a fixed instance.

Why it fails: Cold start latency only affects the first request after the endpoint has scaled down. It does not mean the overall latency of the model is higher during normal operation. Once instances are running, performance is identical to a fixed-size deployment, so this is not a general performance increase.

The endpoint will be restricted to CPU usage only.

Why it fails: The choice of GPU instance type is independent of the auto-scaling policy. Setting min_instances to zero does not change the underlying hardware or the model's ability to utilize GPU acceleration once it is provisioned. The configuration merely controls the scaling behavior, not the hardware capabilities of the endpoint.

Set the traffic_percentage of version 6 to 10 and keep version 5 at 100.

Why it fails: Traffic percentages must sum to 100%. Setting version 6 to 10 and version 5 to 100 results in a total of 110%, which is invalid for traffic routing configurations. The sum must always be exactly 100 to ensure predictable and correct distribution across the served model routes.

Remove version 5 and set version 6 to 10.

Why it fails: Removing version 5 while setting version 6 to 10 will result in 90% of traffic having nowhere to go, causing service errors. Canary deployments require that all traffic be accounted for across the defined routes. The sum of traffic percentages must always equal 100 for successful request routing.

Create a new endpoint entirely for version 6 and switch the DNS record.

Why it fails: Creating a new endpoint for a canary deployment is an unnecessarily complex and disruptive approach. Databricks Model Serving supports weighted traffic routing within a single endpoint. This feature is designed specifically to handle canary deployments and A/B testing natively without needing to manage external DNS records.

The endpoint will error because total traffic must equal 100% for each model version individually.

Why it fails: The configuration is valid as long as the total traffic distribution across all routes sums to 100%. In this case, 90% plus 10% equals 100%, which is a valid traffic policy. The system will correctly route requests according to these defined weights without errors.

The endpoint will only route traffic to churn_v1 after churn_v2 reaches capacity.

Why it fails: Traffic splitting in Databricks Model Serving is weight-based, not capacity-based. The system will continue to distribute requests according to the defined percentages regardless of the underlying load, provided the instances are healthy. It does not act as a failover mechanism based on capacity thresholds.

The endpoint will round-robin requests between v1 and v2 regardless of the weights.

Why it fails: Weights define the probability distribution for incoming requests. A round-robin approach would imply a 50/50 split, which contradicts the explicit 90/10 configuration provided in the exhibit. The routing engine respects the percentages defined in the traffic_config to ensure the desired distribution is maintained over time.

It increases the throughput of the model by optimizing memory allocation during inference.

Why it fails: Scaling to zero has no impact on memory allocation or model throughput. It is strictly an infrastructure management configuration designed to reduce costs by powering down resources during idle periods. Performance optimization is typically handled by adjusting the workload size or configuring model-specific hardware requirements instead.

It ensures that the model always stays warm to minimize latency for every user request.

Why it fails: Scaling to zero actually does the opposite of keeping a model warm. By powering down resources when idle, the first request after an idle period will experience a 'cold start' latency penalty while the serving infrastructure initializes. This configuration prioritizes cost savings over immediate request-time responsiveness.

It enables high availability by automatically replicating the model across multiple availability zones.

Why it fails: Scale-to-zero is an infrastructure lifecycle setting and does not relate to multi-zone replication or high availability configurations. High availability in Databricks Model Serving is achieved through built-in platform features and configuration of multiple replicas, rather than through the power-management logic of scaling to zero during idle periods.

The user lacks the SELECT privilege on the schema object.

Why it fails: The SELECT privilege is not applicable to schema objects in Unity Catalog; permissions for schemas are governed by the USAGE privilege. The failure occurs because the user lacks USAGE on the schema, not because they are missing SELECT permissions on the schema container itself.

The user requires the MODIFY privilege to perform joins.

Why it fails: A join operation is a read-only activity that only requires the SELECT privilege on the underlying tables. The MODIFY privilege is required for DML operations like INSERT, UPDATE, or DELETE, and it is not a requirement for performing complex read operations or joining datasets.

The user needs to be an owner of the tables to perform joins.

Why it fails: Table ownership is not required for users to query tables. As long as the user has been granted the necessary SELECT and USAGE privileges via the Databricks SQL GRANT command, they can access and join tables regardless of who originally created or currently owns the table objects.

The catalog owner needs to refresh the table metadata.

Why it fails: Refreshing metadata does not propagate permissions to new objects. The issue is not related to metadata freshness but rather the fundamental way Unity Catalog handles object identity; each re-creation generates a distinct object that inherits no permissions from its predecessor.

The analyst group needs to be re-added to the schema permissions.

Why it fails: While schema permissions remain, table-specific permissions are lost. Re-adding the group to the schema would not restore the specific SELECT privilege on the new table object, because that privilege is tied to the specific table object that was deleted and replaced.

The user who recreated the table is not the catalog owner.

Why it fails: Ownership does not solve the underlying issue of permission binding. Even if the catalog owner recreated the table, the previous SELECT permissions for the analyst group would remain lost. Permissions are object-specific, not attached to the user who performed the DDL operation.

To increase compute capacity for the primary model.

Why it fails: Traffic splitting does not manage compute capacity. The number of instances and the resources allocated to each model version are determined by the scaling policy and workload types, not the traffic distribution settings, which simply control the routing of user requests between the available versions.

To load balance between two different cloud regions.

Why it fails: The 'served_model_name' refers to specific versions of a model deployed on the same serving endpoint. Traffic routing within a serving endpoint is local to the service and does not involve multi-region load balancing, which is a separate infrastructure-level configuration that is not managed through this endpoint routing policy.

To force all traffic to model-v1 when model-v2 errors.

Why it fails: This configuration represents a static split, not an automatic failover policy. If 'model-v2' returns errors, the endpoint will continue to route 10% of traffic to it. The system does not possess built-in intelligence to detect model-level errors and automatically redirect traffic back to the primary model.

The model was trained on too much data.

Why it fails: Training on large datasets typically improves model performance. It is highly unlikely that having more training data would directly cause a significant drop in relevance scores unless the model suffered from severe overfitting to irrelevant patterns.

The inference cluster has too much compute capacity.

Why it fails: Excessive compute capacity might lead to higher costs, but it does not affect the semantic quality or relevance of the model's responses. Compute resources are independent of the logic determining how relevant an answer is to a prompt.

The faithfulness score increased too much.

Why it fails: Faithfulness and relevance are distinct metrics. An increase in faithfulness means the model is better at sticking to context; this should theoretically aid relevance, not harm it. A high faithfulness score is generally a positive performance indicator.

Upgrade the model serving endpoint to a higher capacity cluster size.

Why it fails: Increasing cluster capacity only affects compute resource limits, not security or authorization policies. The error clearly indicates an authorization failure, meaning that compute resources are irrelevant until the identity used for deployment is granted the required permissions to access the specified model registry object.

Delete the model serving endpoint and recreate it using a personal access token.

Why it fails: Using a personal access token is discouraged for automated CI/CD deployments as tokens are linked to individual users and expire. Recreating the endpoint does not address the underlying permission issue, as the service principal will continue to lack access to the model regardless of how the endpoint is created.

Restart the Databricks workspace to force an update of the internal permission cache.

Why it fails: Permission errors in Unity Catalog are evaluated in real-time by the access control service. Restarting the workspace does not clear or refresh permissions. The issue is a missing grant, which requires a deliberate administrative action to update the security policy associated with the specific service principal account.

The temperature is too low for the model to generate full sentences.

Why it fails: Temperature controls the randomness of the token selection, not the length of the output. While a very low temperature can make the model repetitive, it does not cause it to truncate its response. The truncation is directly tied to the token limit constraint provided in the configuration.

The top_p parameter is too high, leading to early termination.

Why it fails: Top-p sampling controls the diversity of the generated tokens by considering only the top probability mass. It has no effect on the maximum number of tokens allowed. Truncation mid-sentence is a hallmark of reaching the maximum token limit, not a result of token selection sampling strategies.

The model version is incompatible with the specified parameters.

Why it fails: These parameters are standard across most generative models served on Databricks. While individual models might behave differently, the truncation behavior is consistent with hitting a hard token limit. There is no evidence in the exhibit to suggest that these parameters are incompatible with the Mistral-7B model.

The vector index is offline.

Why it fails: The model signature error is related to the code or the model registration process, not the status of the vector index. While an offline index would cause retrieval errors at runtime, it would not trigger a model signature validation error during the logging or registration process.

The user has insufficient Unity Catalog permissions.

Why it fails: Insufficient permissions would typically result in an 'Access Denied' or '403 Forbidden' error when trying to write to the MLflow Registry or workspace. A signature error specifically points to a structural mismatch in the model object being logged, not to environmental access issues.

The LLM endpoint is overloaded.

Why it fails: Overload would manifest as a timeout or a '503 Service Unavailable' error during runtime. Signature validation happens at the development/deployment phase, before the application is even serving requests. It is a metadata definition issue, not a runtime capacity issue.

Increase the 'workload_size' in the model configuration.

Why it fails: Workload size configurations relate to resource allocation and scaling, not to security or access control. Changing the workload size will not grant the necessary permissions required to access the endpoint, and the error will persist regardless of how many resources are allocated to the running model service.

Update the workspace token expiration policy.

Why it fails: Token expiration policies control the lifespan of credentials, not the granular access control of serving endpoints. Even with a valid, non-expired token, the user must still be explicitly granted the 'Can query' permission on the target endpoint. Modifying token settings will not address the underlying authorization gap here.

Restart the serving endpoint to refresh security policies.

Why it fails: Restarting the endpoint does not modify security policies. Permissions in Databricks are governed by the identity-based access control system and are applied in real-time. Simply restarting the infrastructure without changing the underlying permissions configuration will not resolve the lack of access for the principal attempting to query the model.

mlflow.log_artifact(local_path="chain.py")

Why it fails: Logging a file as an artifact is not the same as logging it as a model. While the code might be saved, it cannot be loaded using the 'pyfunc.load_model' interface, which expects a specific model structure and metadata to enable serving and inference capabilities in Databricks.

mlflow.register_model("path/to/chain")

Why it fails: Registering a model is an action performed after the model is already logged. You cannot register a model that hasn't been created and saved through a logging function. This approach skips the critical step of defining the model's signature and environment, leading to deployment failures.

mlflow.sklearn.log_model(my_chain_object)

Why it fails: MLflow's sklearn flavor is intended for Scikit-Learn models, not for generic RAG chains or custom LLM pipelines. Using the wrong flavor will result in serialization errors or an inability to properly invoke the model, as the underlying expectations for input and output formats will not align.

The model version is incompatible with the CPU workload type.

Why it fails: CPU workload types are compatible with most standard model formats. If there were an incompatibility, the endpoint would likely fail to deploy entirely rather than exhibiting latency on initial requests, making this explanation inconsistent with the observed behavior of slow response times following a period of inactivity.

The model name is not registered in the Unity Catalog.

Why it fails: If the model name were unregistered, the endpoint deployment would fail during the creation process, and the endpoint would remain in a 'Failed' state. This does not explain why the endpoint provides responses after a delay, as the error would be systemic and prevent any successful inference request.

The workload type 'CPU' is incorrectly specified as a string.

Why it fails: The 'workload_type' field requires a string input representing the compute architecture. Providing 'CPU' is a valid configuration value in the API schema. Therefore, the syntax is correct, and the latency issue is strictly related to the resource management behavior defined by the scaling policy, not the syntax.