Hard Difficulty Questions

Only image resolution and file size — higher resolution means better quality

Why wrong: Assessing image generation solely via resolution and file size conflates technical properties with perceptual quality. A high-resolution image can still contain severe artefacts, object-level errors, or poor composition, while a low-resolution image might be highly coherent (or, conversely, a file size can be large due to noise without meaningfully better content). Quality evaluation must consider aesthetic appeal, semantic fidelity to the prompt, absence of distortions, and scene coherence—none of which are implied by pixel dimensions or storage footprint.

Simply asking the model what score it gives its own output

Why wrong: Asking the generator to rate its own output is fundamentally unreliable because the model has no external reference for what 'good' looks like; its internal scoring is typically based on learned priors that may not align with human perceptual judgment. It cannot detect its own failure modes, such as duplicated limbs or semantic contradictions, and it may be miscalibrated or biased toward its own generation distribution. Independent, objective measurements—like FID for realism and CLIP for prompt alignment—plus human raters are needed to mitigate these blind spots.

Counting the number of objects correctly included vs. missing from the prompt

Why wrong: Counting how many objects in the prompt appear in the generated image exercises only a coarse sense of prompt adherence. It fails to assess whether those objects are realistically rendered, correctly positioned, or visually coherent with the scene, and it ignores aesthetic quality, artefact presence, and safety risks. A generated image may include all requested objects yet still be low quality due to garbled textures or unnatural composition, so this approach cannot serve as a standalone quality metric.

Checking whether a document's references (citations) are correctly formatted

Why wrong: This is a document-level formatting task that verifies whether citations comply with a bibliographic style guide (e.g., APA, IEEE, MLA), using regular expressions or metadata matching rather than natural language understanding. Coreference resolution, by contrast, is a linguistic analysis that identifies whether two textual mentions—such as "she" and "Dr. Emily Carter"—point to the same real-world entity. Since citation checking does not involve resolving anaphoric references, this option is incorrect.

Resolving conflicts when multiple languages are mixed in the same document

Why wrong: Mixing languages in a document is a code-switching phenomenon, which involves language identification, language segmentation, and cross-lingual token alignment. Coreference resolution, in contrast, focuses on whether two expressions share the same referent regardless of surface form, and it does not resolve conflicts that arise from alternating grammatical systems. Code-switching is therefore an orthogonal NLP problem, making this option incorrect.

Matching database foreign keys to their referenced primary keys

Why wrong: Matching foreign keys to primary keys is a relational database referential-integrity constraint that ensures every foreign key value points to an existing primary key in another table. Coreference resolution, however, is a natural language processing technique applied to unstructured text, not to structured table columns. It resolves linguistic mentions like "it" to "server," rather than establishing key relationships, so this option is incorrect.

Feature engineering

Why wrong: Feature engineering creates or transforms predictor variables to better capture underlying patterns in the data. While this can improve model accuracy, it does not constrain the model's hypothesis space or add a penalty for coefficient magnitude, so it does not directly counteract overfitting caused by excessively complex models. Overfitting from too many or overly flexible features would remain unless the feature set is aggressively simplified, which is not the primary mechanism of regularization.

Cross-validation

Why wrong: Cross-validation is a resampling procedure used to estimate how well a model generalizes to unseen data by partitioning the dataset into training and validation folds. It does not change the model's learning algorithm or impose any constraint on the coefficients, so it cannot prevent overfitting itself. Rather, it helps detect overfitting by revealing a gap between training and validation performance, but the model complexity remains untouched and will still overfit if left unregularized.

Hyperparameter tuning

Why wrong: Hyperparameter tuning systematically searches for the best configuration of settings like learning rate, tree depth, or number of layers. While it can indirectly reduce overfitting if a less complex hyperparameter setting is chosen, it does not specifically penalize coefficient size or add a structural constraint to the objective function. The correct, direct remedy for overfitting in a regression model is regularization, which modifies the loss function itself rather than merely selecting among pre-defined model configurations.

Object detection

Why wrong: Object detection can detect people in frames, but it does not track their movement across frames or count them entering/leaving specific zones.

Optical Character Recognition (OCR)

Why wrong: OCR extracts text from images and is unrelated to detecting people or tracking movement.

Semantic segmentation

Why wrong: Semantic segmentation classifies each pixel into categories (e.g., person, floor), but it does not track individuals or count zone entries over time.

When an AI model is licensed for use by two different organisations simultaneously

Why wrong: A scenario where two organisations separately license the same AI model is strictly a commercial and intellectual property matter, involving agreements, pricing, and contractual terms. It does not introduce a new ethical dimension about the capabilities themselves; whether one or one thousand organisations use the model, the underlying risk of misuse remains identical. Dual-use is not about the number of licensees or the breadth of deployment, but about the intrinsic potential of the model's functionality to be applied in both constructive and destructive contexts.

Deploying the same AI model for both training and inference to reduce costs

Why wrong: Deploying the same model for both training and inference is not technically standard, since training requires iteratively updating weights while inference requires a fixed, stable model; in practice, one may reuse a pre-trained model for a different task via transfer learning or fine-tuning, which is a cost-saving efficiency technique. This relates to resource optimisation and model reuse, not to the potential for a beneficial capability to be turned to malicious ends. Dual-use is about the inherent misuse potential of a model's function, not about reusing the model across operational phases.

Combining two AI models to achieve better results than either model alone

Why wrong: Combining two AI models, such as via bagging, boosting, or stacking, is a well-known model ensembling technique used to enhance predictive performance by reducing variance or bias (e.g., random forests, gradient boosting). This is a purely machine-learning-centric optimization strategy, not a characteristic of how the technology might be repurposed for harm. The notion of dual-use specifically refers to the same fundamental AI capability being simultaneously beneficial and exploitable, whereas ensembling addresses accuracy, not ethical risk.

Prebuilt Named Entity Recognition (NER)

Why wrong: Prebuilt Named Entity Recognition recognizes a fixed, general-purpose set of entities such as persons, organizations, locations, dates, and quantities. It is not trainable, so it cannot be customized to learn a domain-specific schema such as 'governing law', 'non-compete scope', or 'payment milestone'. While it may detect a party name in a contract, it cannot reliably extract the unique legal fields the law firm requires, making it unsuitable for this task.

Key Phrase Extraction

Why wrong: Key phrase extraction returns the most statistically salient phrases in a document, based on how frequently and significantly they appear, but assigns no semantic labels to them. It does not understand whether a phrase is a party, a clause type, or an effective date, and it produces an unordered list rather than a structured entity-response. For automated extraction of specific information, this method lacks the precision and schema alignment needed.

Text Summarization

Why wrong: Text summarization condenses a long contract into a shorter narrative, keeping the main ideas but in continuous prose. It does not return labeled, structured fields; a summarized passage might mention a clause but never mark it as a discrete entity like 'GoverningLawClause'. For automatic extraction of specific information into a database or case-management system, you need a classification and extraction model, not a paraphrasing tool.

Image Analysis (describe image and detect objects)

Why wrong: Image Analysis for describe image and object detection identifies and labels visual content (e.g., "car", "license plate") and creates natural-language captions, but its output is object classes and bounding boxes rather than transcribed text. It cannot reliably isolate and extract the actual alphanumeric characters stamped on a plate; that requires a dedicated OCR engine. Without reading the characters, the company would not know the plate number.

Face API

Why wrong: Face API is designed exclusively for detecting and analyzing human faces, including face landmarks, attributes like age and emotion, and person identification/verification. It has no object detection for vehicles and no text extraction capability for license plates. Feeding it a parking-lot image will either find no human face or ignore the plate region entirely, so it is entirely unsuitable for this task.

Custom Vision (object detection)

Why wrong: Custom Vision object detection can be trained to draw bounding boxes around license plates and classify them as "license plate" objects, but it does not read or output the characters inside those boxes. The model's output is a categorical label plus location coordinates and confidence, not a text string. To obtain the actual plate characters from a detected region, you must add an OCR step, so Custom Vision alone cannot fulfill the requirement.

Reliability and Safety

Why wrong: Reliability and Safety concern whether the AI system performs its intended function consistently and without causing physical, financial, or operational harm due to errors, crashes, or unexpected behavior. While a biased lending model could be seen as unreliable, the core issue here is not a malfunction or safety failure but rather the learned discriminatory patterns from training data. The system may be highly reliable in terms of uptime and accuracy, yet still produce unfair results, which is a distinct problem.

Privacy and Security

Why wrong: Privacy and Security involve protecting personal and financial data from unauthorized access, leakage, or misuse through encryption, access controls, and compliance with data protection regulations. The loan recommendation bias described is not a data breach or confidentiality failure; the system may fully protect user data yet still render unfair decisions based on biased historical labels. Therefore, while privacy and security are critical, they address a different risk than the fairness violation at issue here.

Inclusiveness

Why wrong: Inclusiveness focuses on designing AI systems that are accessible and usable by people of all abilities, backgrounds, and languages, such as supporting screen readers, providing multilingual interfaces, or accommodating diverse user needs. The bias in loan recommendations does not stem from a lack of accessibility or exclusion from use, but rather from disparate treatment of protected classes in the decision-making logic. Thus, improving inclusiveness would not directly remedy the discriminatory outcomes described.

Fairness

Why wrong: A fairness violation would require evidence that the model's screening decisions systematically disadvantage protected groups, such as by rejecting candidates of a particular ethnicity at a higher rate. The scenario instead centers on the operator's inability to articulate why a specific applicant was rejected, which is an explainability concern. Without access to the decision logic, one cannot even determine whether bias exists, so fairness is not the immediate principle breached.

Accountability

Why wrong: Accountability is about establishing which person or team owns the model and its outcomes, including maintaining audit trails and taking corrective action. Although an AI developer or HR manager should ultimately be answerable for the resume screener, the described failure is a lack of explanation for the model's judgment, not a lack of ownership. Thus, accountability is relevant to governance but does not pinpoint the transparency deficiency at issue.

Privacy and security

Why wrong: Privacy and security govern how resume data is collected, stored, accessed, and protected from breaches or unauthorized use. Nothing in the scenario suggests that candidate information was exposed, misused, or handled without consent, so a data-protection violation is not present. The central problem remains the inability to explain a decision, which is unrelated to confidentiality or infrastructure safeguards.

Azure billing documents showing the monthly cost of running a model in production

Why wrong: Model cards are not financial artifacts; Azure invoices and cost reports are operational usage records generated by Azure Cost Management. A model card instead answers questions like 'What was this model trained on?', 'Which demographic groups perform worse?', and 'What caveats should deployment teams know?' Runtime cost, reserved capacity, and monthly spend are useful for budgeting but say nothing about a model's fairness, robustness, or intended use, which are the core disclosures in a model card.

Technical specification sheets for AI hardware accelerators used in model training

Why wrong: Hardware specification sheets describe physical components such as GPU memory, thermal design power, or tensor-core throughput, but they do not describe a trained model's behavior. Model cards are not device datasheets; they document the model's training recipe, validation methodology, known biases, and deployment risks. AI accelerator specifications are relevant when choosing compute for training or inference, yet they neither disclose model limitations nor enable the kind of responsible-AI scrutiny that model cards provide.

Playing cards used in gamification of AI training to motivate data labellers

Why wrong: The term model card has a precise meaning in responsible AI, and it has nothing to do with card games or gamified labeling work. Model cards are concise, standardized documents published by model developers to explain a model's intended tasks, training data, evaluation metrics, known biases, and failure modes. Gamified data-labeling incentives such as points or badges might improve labeling efficiency, but they do not serve the transparency and accountability purpose that model cards are designed for.

Fairness

Why wrong: Fairness in responsible AI is concerned with ensuring outcomes and decisions do not systematically disadvantage protected groups; it is evaluated through group-level metrics such as demographic parity or equalized odds. Re-identifying individuals in a medical dataset is a privacy breach that violates confidentiality, not a fairness failure. Even a model that treats all demographic groups equitably can still expose patient identities, so this option misidentifies the ethical principle at stake.

Transparency

Why wrong: Transparency refers to clearly communicating an AI system's capabilities, limitations, and decision-making logic to stakeholders, ensuring users understand when and how AI is used. It does not speak to whether underlying data can be re-identified; a fully transparent system could still leak patient identities. This option is wrong because the failure here is data protection, not a lack of explainability or openness about the system's behavior.

Accountability

Why wrong: Accountability in responsible AI concerns establishing clear human ownership, governance structures, and audit trails for how an AI system is developed and deployed. While a loss of patient privacy might eventually lead to accountability issues, the immediate harm described is unauthorized re-identification, which is a security and privacy failure rather than an absence of role ownership. Thus this principle does not match the specific violation in the scenario.

Choosing between model accuracy and computational cost

Why wrong: Choosing between model accuracy and computational cost is a practical engineering decision about resources, hardware, and operational budgets, not a statistical property of estimators. You can often improve accuracy by adding parameters or ensembling, but that changes bias and variance in ways unrelated to an accuracy-cost curve. The bias-variance tradeoff describes how model error changes with complexity, irrespective of price, energy, or run-time costs.

Deciding whether to use biased training data or unbiased test data

Why wrong: This statement conflates data bias/fairness with the statistical bias-variance tradeoff. In ML, bias-variance decomposition concerns model error due to simplifying assumptions (bias) and sensitivity to training sample fluctuations (variance). Biased training data introducing demographic skew is a dataset/fairness problem, while unbiased test data concerns representative evaluation, not model complexity or generalization error sources.

The tradeoff between training speed and model size

Why wrong: Training speed versus model size is a computational resource tradeoff involving memory footprint, throughput, and infrastructure cost, not generalization error. Although a larger model may have lower bias or higher variance, the speed/size tension addresses engineering limits such as latency and deployment constraints. Bias-variance is specifically about how model capacity affects error on unseen data, independent of wall-clock training time.

Object detection

Why wrong: Object detection localizes parts by drawing a rectangular bounding box and a class label around each detected item, but it does not estimate the object's actual boundary at the pixel level. For a robotic arm, a bounding box can contain large empty regions or portions of neighboring parts, so the computed center of the box may not correspond to the part's true centroid or stable grasping region. Without fine contour information, the gripper risks missing the part or colliding with adjacent objects.

Image classification

Why wrong: Image classification assigns a single class label to the entire input image, effectively answering 'what is the dominant category in this scene?' while discarding spatial location. In a bin containing multiple parts, the model might output 'part' as the top label but cannot identify each individual object or its pixel coordinates, so the robot has no way to target a specific part. It also cannot distinguish between two different part types present simultaneously.

Optical Character Recognition

Why wrong: Optical Character Recognition (OCR) is designed to detect and decode text glyphs, such as part numbers or serial numbers, converting pixel patterns of characters into machine-readable strings. It provides no measure of the part's geometric extent, surface, or pose, which are the properties needed to position a gripper. Even if the bin has a label, OCR tells the robot which part it is looking at, not where or how to pick it up.

A: Accuracy

Why wrong: Accuracy simply measures the proportion of correct predictions overall. In credit card fraud detection, fraudulent transactions are extremely rare—often under 0.1% of all transactions—so a model that never predicts fraud can appear 99.9% accurate while completely failing at its primary task. Because accuracy gives equal weight to false positives and false negatives, it hides the severe business cost of missed fraud and is therefore a poor performance measure for this use case.

B: Precision

Why wrong: Precision quantifies how many of the transactions flagged as fraudulent are actually fraudulent, so a high-precision model rarely harasses legitimate customers. However, achieving high precision usually requires the model to be very conservative and only flag cases with strong evidence, which allows many subtle or novel fraud patterns to go undetected. For fraud detection, optimizing only precision would sacrifice the model's ability to uncover true fraud, leaving the bank exposed to losses.

C: Recall

Why wrong: Recall measures the proportion of actual fraudulent transactions that the model successfully identifies, making it critical for catching as much fraud as possible. But a pure recall-focused approach drives the model to flag a large number of borderline legitimate transactions as suspicious, frustrating customers and flooding the bank's fraud-review queue. That excessive false-positive burden is a real operational cost, so recall in isolation is not a suitable evaluation metric.

Image Classification

Why wrong: Image classification assigns a single global label to the entire CT scan or slice, such as “nodule present” or “nodule absent.” It provides no spatial coordinates, no region outline, and no pixel mask, so it cannot identify where a nodule is located or delineate its boundary. Outlining requires a model that makes predictions at the pixel level, which image classification fundamentally lacks.

Object Detection

Why wrong: Object detection improves on image classification by drawing a bounding box around each nodule, giving the approximate location and size. However, a bounding box is a rectangular region that inevitably includes adjacent healthy tissue and cannot follow the irregular, spiculated contours of a real nodule. Because the task asks for precise outlines rather than coarse rectangle coordinates, object detection is insufficient for exact boundary delineation.

Optical Character Recognition (OCR)

Why wrong: OCR is designed to recognize and digitize printed or handwritten text characters from images, such as extracting patient identifiers from scanned forms. CT scans are medical images that contain anatomical structures rather than text, so there are no characters or strings for OCR to interpret. Therefore, OCR has no role in identifying or outlining lung nodules and is irrelevant to this imaging task.

Prebuilt entity recognition and key phrase extraction

Why wrong: Prebuilt entity recognition extracts common entities (e.g., people, places) but may not capture specialized legal entities like case numbers. Key phrase extraction identifies important phrases but does not provide structured entity extraction suitable for redaction or custom entity types.

Sentiment analysis and language detection

Why wrong: Sentiment analysis assigns a positive/negative/neutral score to text, while language detection simply identifies which language the document is written in. Neither operation extracts or redacts any text spans, and neither attempts to identify legal entities or personal data. Using these capabilities would leave all sensitive information and case-specific details untouched.

PII detection only

Why wrong: PII detection alone only recognizes a fixed set of personal information types, such as SSNs, emails, and physical addresses. It cannot be taught to recognize legal-specific entities like case numbers, docket IDs, or judicial officer names, because those do not belong to any predefined PII category. The firm's extraction requirement therefore remains unmet without a custom trained model.

Use the pre-built Computer Vision object detection API directly.

Why wrong: The pre-built Computer Vision object detection API is a fixed, general-purpose model trained on common object categories such as people, vehicles, and household animals; it does not include a penguin class and cannot be retrained or fine-tuned by users. Attempting to use it on drone imagery would likely yield no detections or spurious low-confidence boxes for penguins, because the model has no learned visual representation of that specific species. Thus, a custom object detection model is required to recognize and accurately count penguins.

Use the Computer Vision Image Analysis API with the 'dense captioning' feature.

Why wrong: Dense captioning in the Computer Vision Image Analysis API annotates image regions with natural-language descriptions (e.g., "a bird in the snow") and coarse bounding boxes, but it does not return class-verified detections with confidence scores suitable for counting. Its underlying model is trained for general scene understanding and lacks a penguin class, so it may miss, merge, or misdescribe individual birds. Therefore, while it can hint at content, it is not an object detection or counting mechanism.

Train a Custom Vision image classification model with the labeled images.

Why wrong: Custom Vision image classification trains a model to assign one global label to the entire input image, such as "penguin" or "no penguin," but it produces no bounding boxes or localization information. Even with accurately labeled penguin images, the model cannot determine where each penguin is or how many distinct individuals occupy the frame. Counting individuals requires an object detection model outputting separate bounding boxes and per-instance confidences, making classification fundamentally insufficient for this task.

Accuracy

Why wrong: Accuracy simply divides the number of correct predictions by the total number of samples, treating every sample equally. In a dataset with 95% negative-class samples, a trivial model that predicts the majority class for every row achieves 95% accuracy while correctly identifying zero positive cases, which is catastrophically misleading. This metric is blind to class prevalence and error type, making it unsuitable for imbalanced classification scenarios where minority-class recall is critical.

F1_score

Why wrong: F1_score is the harmonic mean of precision and recall, giving equal conceptual weight to false positives and false negatives. However, it is threshold-dependent: the value changes depending on the chosen classification cutoff, whereas AutoML needs a primary metric that can fairly compare models without manual threshold tuning. For imbalanced data, F1 also requires you to specify which class is the positive class, and aggregate versions like macro-F1 or weighted-F1 do not have the same prevalence-aware statistical guarantees as AUC_weighted in AutoML's model-sweeping process.

Log_loss

Why wrong: Log_loss measures the cross-entropy between predicted probability distributions and true labels, heavily penalizing confident but wrong predictions. While it is sensitive to probability calibration, it does not inherently account for class prevalence: each sample contributes equally to the loss by default, so the minority class's influence is diluted when it is underrepresented. For highly imbalanced data, log_loss can therefore be dominated by the majority class's probability errors, and it is more appropriate for calibration evaluation than as a primary metric for model selection in AutoML.

Increase the 'temperature' parameter to 0.0

Why wrong: Temperature is a sampling parameter that controls the randomness of token selection; setting it to 0 makes the output deterministic (greedy decoding). It has no impact on the model's knowledge source or its ability to access external information. Even with temperature at 0, the model will still produce answers from its pretrained parameters and may include facts not present in the provided documents.

Use the system message to instruct the model to only use provided documents

Why wrong: System messages set the conversational behavior and can request that the model base answers on provided context, but they are not a hard guarantee. The underlying transformer model has no built-in enforcement mechanism for such instructions; without retrieval grounding, the model may fall back on its training data, especially if the provided documents lack the specific answer. System messages are advisory, not a technical constraint.

Set the 'max_tokens' parameter to a low value

Why wrong: The max_tokens parameter controls the maximum number of tokens (words/subwords) in the generated response, effectively capping output length. It does not influence what information the model uses to formulate the answer; the model still draws on its pretrained knowledge from public internet data. A low value may truncate the answer mid-sentence or force overly terse responses, but it cannot confine the model to the provided documents.

Precision

Why wrong: Precision = TP/(TP+FP) measures the proportion of predicted fraud cases that are truly fraudulent. It emphasizes minimizing false positives, not false negatives. A model could achieve very high precision by flagging only a few highly suspicious transactions, yet still miss many real frauds, which directly contradicts the goal of catching all fraudulent activity.

Accuracy

Why wrong: Accuracy = (TP+TN)/Total provides an overall percentage of correct predictions but is heavily skewed in imbalanced datasets where the majority class (legitimate transactions) dominates. For example, a model that simply predicts 'not fraud' for every transaction could achieve extremely high accuracy while flagging zero frauds, making it misleading when the real priority is catching all rare positive cases.

Specificity

Why wrong: Specificity = TN/(TN+FP) measures the true negative rate, i.e., how accurately the model identifies legitimate transactions. It focuses on minimizing false positives, not false negatives. In fact, a model that never flags anything would have perfect specificity (100%) but would miss all frauds, so it is unsuitable for the goal of capturing fraudulent transactions.