Hard Difficulty Questions

Object Detection

Why wrong: Object detection identifies objects and their bounding boxes (rectangular regions), but does not assign a class to every pixel, which is required for precise boundary measurement.

Image Classification

Why wrong: Image classification assigns a label to the entire image, not to individual pixels. It cannot provide pixel-level boundaries for tumors.

Optical Character Recognition

Why wrong: OCR extracts text from images, not relevant for analyzing tumor boundaries in MRI scans.

Set frequency penalty to 1

Why wrong: Frequency penalty reduces repetition of tokens but does not eliminate randomness; outputs may still vary.

Set top_p to 1

Why wrong: Top_p controls nucleus sampling; setting it to 1 includes all tokens, which still allows random choices and does not guarantee determinism.

Set max_tokens to the expected output length

Why wrong: Max_tokens limits the length of the output but does not affect the randomness of token selection.

Underfitting

Why wrong: Underfitting would cause poor performance on both training and test sets, which is not the case here.

High bias

Why wrong: High bias is associated with underfitting, where the model is too simple to capture patterns. This model shows high variance (overfitting).

Data drift

Why wrong: Data drift refers to a gradual change in data distribution over time, but the issue here is immediate poor generalization to a different city, not a temporal change.

Custom Vision image classification

Why wrong: Image classification only labels the entire image (e.g., defective or not) and does not indicate where the defect is located.

Azure Face API

Why wrong: Face API is designed for detecting and analyzing human faces, not for general defect detection.

Optical Character Recognition (OCR)

Why wrong: OCR is used to extract printed or handwritten text from images, not for detecting defects.

Image Classification

Why wrong: Image Classification labels the whole image (e.g., 'contains nodule'), but does not localize or outline the nodule.

Object Detection

Why wrong: Object Detection draws bounding boxes around objects, not the pixel-level boundaries needed for precise outlines.

Optical Character Recognition (OCR)

Why wrong: OCR extracts text from images, not relevant for detecting or outlining anatomical structures.

Key Phrase Extraction

Why wrong: Key Phrase Extraction identifies important terms in text but does not have a specialized medical model; it yields general phrases, not structured medical entities.

Named Entity Recognition (NER)

Why wrong: Standard NER recognizes general entities like people, organizations, and locations, but it is not trained to extract medical-specific entities like diseases or medications.

Custom Text Classification

Why wrong: Custom Text Classification would require the user to provide labeled training data and train a model, which is not a prebuilt capability aimed at medical entities.

Image classification

Why wrong: Image classification assigns a label to the whole image, with no object localization or pixel masks.

Object detection

Why wrong: Object detection provides bounding boxes around objects but not pixel-level segmentation masks for individual instances.

Semantic segmentation

Why wrong: Semantic segmentation labels each pixel with a class (e.g., car, road) but does not distinguish between different instances of the same class.

The tradeoff between model accuracy and inference speed

Why wrong: Accuracy-speed tradeoff is a deployment concern — bias-variance is a fundamental ML concept about underfitting vs. overfitting.

The tradeoff between training data quantity and model quality

Why wrong: Data quantity effects are a dataset management concern — bias-variance describes error sources at any data size.

The difference in fairness metrics between biased and unbiased model versions

Why wrong: Fairness metrics relate to demographic bias — bias-variance tradeoff is a statistical concept about model complexity and error sources.

Fairness

Why wrong: Fairness is about avoiding bias and ensuring equitable outcomes; while important, the core issue here is explainability, not bias.

Reliability & Safety

Why wrong: Reliability & Safety ensures the system operates correctly and safely under various conditions, not specifically about providing explanations.

Privacy & Security

Why wrong: Privacy & Security protects personal data from unauthorized access or misuse; the issue here is about explainability, not data protection.

Sentiment analysis and key phrase extraction

Why wrong: Sentiment analysis detects positive/negative tone, and key phrase extraction finds important words — neither assigns department categories nor extracts specific custom entities.

Language detection and translation

Why wrong: Language detection identifies the language of the email and translation changes it to another language; these do not route or extract custom data.

Named entity recognition (NER) and summarization

Why wrong: Pre-built NER extracts general entities (e.g., people, places) but not custom fields like account numbers; summarization condenses text without extracting specific data or classifying.

R-squared

Why wrong: R-squared measures goodness-of-fit (variance explained), not the size of errors, and does not penalize outliers more.

Mean Absolute Percentage Error (MAPE)

Why wrong: MAPE expresses errors as percentages, but it still treats all errors equally; large dollar errors may have small percentage errors if the house is expensive.

F1 score

Why wrong: F1 score is used for classification models (precision/recall), not regression.

A

Why wrong: Fairness is about preventing bias and ensuring equitable treatment; the kiosk's lack of text alternatives is an accessibility issue, not a fairness issue in the bias sense.

B

Why wrong: Reliability and Safety concern the system performing accurately and safely; while relevant, the primary concern here is excluding users with disabilities.

D

Why wrong: Transparency is about providing clear information on how the AI works and its limitations; this scenario is about the lack of alternative interaction methods, not about explaining decisions.

Fairness

Why wrong: Fairness is about ensuring AI systems do not discriminate against groups. While related, the core issue here is that the system is not accessible to speakers of other languages, which is more directly about inclusiveness.

Reliability and safety

Why wrong: Reliability and safety are about the system performing correctly and without causing harm. The language limitation does not directly affect reliability or safety.

Transparency

Why wrong: Transparency involves informing users that they are interacting with an AI and understanding how decisions are made. The language limitation is not a transparency issue.

Remove the offending proxy feature 'years of continuous employment' from the training data.

Why wrong: While removing one proxy feature may help, other proxy features might still encode the bias, making this solution incomplete.

Train a separate model for each gender group to ensure equal outcomes.

Why wrong: Training separate models can lead to separate and potentially unequal outcomes, and it does not align with the fairness principle of treating all groups equitably.

Collect more training data from underrepresented groups.

Why wrong: Collecting more data can help reduce bias in the long term, but it does not address the existing biased outcomes from the current model.

Hold-out validation

Why wrong: Hold-out validation splits the data into a training set and a fixed validation set, reducing the amount of data available for training, which is not ideal when data is scarce.

Data augmentation

Why wrong: Data augmentation artificially increases the size of the training dataset by creating modified copies, but it does not provide a direct method for performance estimation.

Principal Component Analysis (PCA)

Why wrong: PCA is a dimensionality reduction technique used to reduce the number of features, not to evaluate model performance or estimate generalization error.

Inclusiveness

Why wrong: Inclusiveness is about designing for all users, but the primary violation here is unfair outcomes due to training bias, which is directly addressed by Fairness.

Reliability and Safety

Why wrong: Reliability and Safety focus on system performance and avoiding harm, not specifically on bias in decision making.

Privacy and Security

Why wrong: Privacy and Security protect user data and system integrity, not directly related to biased outcomes.

Fine-tuning the model on a curated dataset

Why wrong: Fine-tuning would retrain the model, which contradicts the requirement of not retraining the model. It also requires additional labeled data and compute resources.

Reducing the temperature parameter to zero

Why wrong: Temperature controls the randomness of token selection; setting it to zero makes outputs deterministic but does not address the knowledge or biases embedded in the model, so stereotypes may still appear.

Increasing the maximum output length

Why wrong: Increasing max output length allows the model to generate longer text, which could actually increase the chance of including harmful stereotypes rather than preventing them.

Pre-training data cleaning

Why wrong: Pre-training data cleaning removes harmful examples from the training data, but it does not filter content generated by the already-trained model during inference.

Prompt engineering with safety instructions

Why wrong: Prompt engineering instructs the model to avoid harmful content, but the model may still occasionally generate such content, and it does not guarantee post-generation filtering.

Model fine-tuning on safe examples

Why wrong: Fine-tuning the model on safe examples reduces the likelihood of harmful outputs, but it does not provide a real-time filtering mechanism for every generated response.

Accuracy

Why wrong: Accuracy is misleading in imbalanced datasets because a model that never predicts fraud can still show high accuracy.

Precision

Why wrong: Precision measures how many of the predicted fraud cases are actually correct, but it doesn't show how many real fraud cases were missed.

F1 score

Why wrong: F1 score combines precision and recall; while it gives a balanced view, recall alone is the most direct metric to expose the model's inability to catch fraud cases.

Accuracy

Why wrong: Accuracy is not informative for imbalanced datasets because it can be high even if the model fails to detect any fraudulent transactions.

Mean Squared Error

Why wrong: Mean Squared Error is a metric for regression tasks, not for classification evaluation.

R-squared

Why wrong: R-squared is a regression metric that indicates the proportion of variance explained; it is not suitable for classification evaluation.