Troubleshooting Scenario Questions

Fairness

Why wrong: Fairness is fundamentally about preventing algorithmic bias and ensuring equitable treatment across groups defined by protected attributes like race, gender, or age. In the scenario of a self-driving car navigating an unexpected physical environment, the issue is not whether the system treats certain demographic groups disparately, but whether it can perceive and react safely to novel environmental conditions. Therefore, while fairness is a critical responsible AI principle, it does not govern the system's resilience to physical world anomalies.

Privacy and security

Why wrong: Privacy and security concern the protection of data and the AI system from unauthorized access, misuse, or cyberattacks. These principles address issues like data encryption, secure model deployment, and defense against adversarial input manipulation. A self-driving car navigating an unexpected physical environment may implicate security if we are talking about malicious sensor spoofing, but the principle itself is about safeguarding information assets and digital integrity, not about how the system copes with environmental changes like rain, construction zones, or road debris. Since the test concerns the navigation system's operational stability, privacy and security are not the governing principle.

Transparency

Why wrong: Transparency is about clearly disclosing how an AI system works, its capabilities, limitations, and the rationale behind its decisions, so that stakeholders can understand and audit it. It might involve generating explanations for why a car stopped or turned, but it does not dictate how the system reacts to an unexpected physical environment. Transparency does not improve the actual robustness of the navigation system; it merely provides visibility into its behavior. Therefore, while transparency is an important property for trust and accountability, it is not the principle being tested when the car encounters a novel environmental condition.

Fairness

Why wrong: Fairness pertains to ensuring AI systems do not produce biased or discriminatory outcomes across groups defined by attributes such as race, gender, or age. The question's scenario contains no indication of differential treatment across demographic groups; instead, it describes a uniform failure triggered by weather conditions. Therefore, fairness is not the primary principle implicated by the autonomous vehicle's inadequate performance.

Privacy and security

Why wrong: Privacy and security focus on safeguarding sensitive data and protecting AI systems from malicious attacks, such as adversarial inputs or unauthorized access. A self-driving car failing in adverse weather is an unintended functional deficiency, not a data breach or cyberattack. Thus, while security is important for autonomous systems, it does not explain the described weather-related reliability problem.

Transparency

Why wrong: Transparency in AI refers to the degree to which an AI system's decision-making process is visible and explainable to stakeholders, such as developers or regulators. The self-driving car scenario centers on an operational failure during adverse weather, which is a matter of robust performance rather than a lack of interpretability. While transparent models may help debug failures, the core violation here is not opacity but unsafe, unreliable behavior.

Accuracy

Why wrong: Accuracy is the ratio of all correct predictions (true positives plus true negatives) to total predictions, and it becomes dangerously misleading when classes are imbalanced. With a fraud rate of just 1%, a model that classifies every transaction as legitimate still achieves 99% accuracy, yet catches none of the fraud. Because accuracy credits correct predictions on the dominant class, it can hide a complete lack of fraud detection and should not be used to judge such a model.

Precision

Why wrong: Precision is calculated as true positives divided by true positives plus false positives (TP/(TP+FP)), meaning it measures the purity of the fraud claims: how many flagged frauds are actually fraudulent. A model can achieve very high precision by flagging only a few obvious fraud cases while ignoring the majority of fraudulent transactions, since missed frauds (false negatives) do not affect precision. Thus precision tells you about the trustworthiness of the alerts, but it cannot show how many real fraud cases the model failed to catch.

F1 score

Why wrong: The F1 score is the harmonic mean of precision and recall, giving equal weight to both in a single number, but this balance often obscures the exact issue at hand. In fraud detection, a model with high precision and only moderate recall can have a reasonably high F1 score even though it is still missing many fraudulent transactions. Because only recall measures the false-negative rate directly, relying on F1 can mask the model's inability to catch fraud; the most direct signal of detection failure is recall.

Completions API

Why wrong: The older Completions API is designed for single-turn, free-form text continuation from a standalone prompt. It does not manage a conversation history or track roles, so to simulate multi-turn dialogue you would have to manually concatenate every past user and assistant message into one giant string. That approach quickly grows unwieldy, loses clear context boundaries, and is why Chat Completions—with its structured message list—is the appropriate API for an assistant.

Embeddings API

Why wrong: The Embeddings API is fundamentally different: it transforms input text into a high-dimensional numerical vector that encodes semantic meaning. These vectors are used to measure similarity between texts, powering search, recommendations, or clustering, but they do not generate language or produce conversational replies. Calling this API would not return a coherent assistant response, so it is unsuitable for powering a chat assistant.

Fine-tuning

Why wrong: Fine-tuning is a training-time process, not an inference-time API. It adjusts a pre-trained model's weights using a custom dataset to specialize its behavior for specific tasks or domain language. After fine-tuning, you still need a dedicated inference endpoint such as the Chat Completions API to actually run conversations; fine-tuning itself cannot accept a user message and produce a reply, so it is the wrong choice for this production assistant.

Precision

Why wrong: Precision is the ratio of true positive predictions to all positive predictions (TP/(TP+FP)). When the model predicts no positives, both TP and FP are zero, so precision is undefined (0/0) and cannot be calculated. Even if one arbitrarily defines precision as 0, it only reflects that no positive prediction was made, not that actual disease cases were missed, so it fails to reveal the model's recall failure.

F1 score

Why wrong: F1 score is the harmonic mean of precision and recall, given by 2·precision·recall/(precision+recall). In this scenario, precision is undefined (0/0) and recall is 0, making F1 either indeterminate or 0 if precision is forced to 0. As a single aggregate, F1 obscures the distinct roles of precision and recall; here the core problem is zero recall, which recall alone communicates clearly and unambiguously.

Mean absolute error

Why wrong: Mean absolute error (MAE) computes the average absolute difference between predicted and actual continuous values, typically for regression models. Binary classification outcomes (e.g., disease present/absent) are categorical, not continuous, so MAE is not a meaningful performance metric in this context. For classification, one should use confusion-matrix metrics like recall, precision, or F1, which directly assess prediction correctness for each class.

Precision for the default class

Why wrong: Precision = TP/(TP+FP). With no default predictions, TP = 0 and FP = 0, so the denominator is zero and precision is undefined (0/0), not simply zero. It cannot be computed or compared, whereas recall is well-defined and directly exposes that no defaults are found.

F1 score for the default class

Why wrong: F1 = 2 * (precision * recall) / (precision + recall). Since recall is 0, the numerator is 0; even if precision is treated as undefined, F1 collapses to 0 because no true positives were retrieved. However, recall alone already pinpoints the root cause (no default detection), while F1 combines precision and recall and obscures the specific failure mode.

Accuracy

Why wrong: Accuracy = (TP+TN)/(total). Here all non-defaults are predicted correctly (TN = 98% of data) and all defaults are missed, so accuracy = 0.98, matching the dominant class. This high value masks the fact that the model is useless for default detection, highlighting why accuracy is misleading under class imbalance.