AI0-001 Hard Questions — 25 Difficult Questions

Question 1

A company uses a neural network for fraud detection. The dataset has 99% legitimate, 1% fraudulent. The model achieves 99% accuracy but fails to detect most frauds. Which metric should they focus on?

Accepted Answer

Recall. Recall (sensitivity) measures the proportion of actual positives correctly identified. In this fraud detection scenario with 99% legitimate and 1% fraudulent transactions, a 99% accuracy can be achieved by simply predicting all transactions as legitimate, which yields 0% recall for the fraud class. Focusing on recall ensures the model captures the majority of fraudulent cases, addressing the critical failure to detect fraud despite high accuracy.

Answer

Precision

Answer

F1-score

Answer

AUC-ROC

Question 2

A data science team is deploying a real-time fraud detection model on edge devices in retail stores. The model must infer under 10ms and fit within 50MB memory. Which combination of techniques should the team apply?

Accepted Answer

Model quantization to INT8 and pruning of low-weight connections. Quantization reduces model precision (e.g., FP32 to INT8) to shrink memory and speed up inference, while pruning removes redundant parameters. Distillation can further compress. These are standard for edge deployment.

Answer

Model parallelism and distributed inference

Answer

Increase batch size and use FP16 precision

Answer

Train a larger model and use distillation to transfer knowledge

Question 3

A financial institution is building a fraud detection system using a supervised learning model. The dataset is highly imbalanced with 99.9% legitimate transactions and 0.1% fraudulent ones. Which approach would be MOST effective to train the model to detect fraud?

Accepted Answer

Use SMOTE to generate synthetic fraudulent transactions. SMOTE (Synthetic Minority Oversampling Technique) is the most effective approach because it generates synthetic fraudulent transactions by interpolating between existing minority class samples, thereby balancing the dataset without losing information. This allows the model to learn decision boundaries for fraud detection more effectively than simple undersampling or metric adjustments, especially given the extreme 99.9% vs 0.1% imbalance.

Answer

Train the model using accuracy as the performance metric

Answer

Undersample the legitimate transactions to match the number of fraudulent ones

Answer

Increase the regularization strength in the model

Question 4

A self-driving car company is testing an AI model for pedestrian detection. During simulation, the model fails to detect pedestrians in low-light conditions. The safety team wants to improve robustness without retraining the entire model from scratch. Which approach is most appropriate?

Accepted Answer

Apply data augmentation techniques to simulate low-light conditions in the training dataset.. Data augmentation techniques, such as adjusting brightness, contrast, and adding noise, can synthetically create low-light training examples from existing data. This improves the model's robustness to low-light conditions without requiring a full retraining from scratch, as it directly addresses the distribution shift in the input data.

Answer

Replace the convolutional layers with transformer layers to improve attention.

Answer

Use adversarial training to add imperceptible perturbations to training images.

Answer

Increase the model's depth by adding more convolutional layers.

Question 5

A data scientist is training a model to detect fraudulent transactions. To protect customer privacy, the team wants to ensure that the model does not inadvertently memorize and reveal sensitive information about individuals in the training set. Which technique should be applied during training?

Accepted Answer

Differential privacy. Differential privacy is the correct technique because it adds calibrated noise to the training process or output, ensuring that the model cannot infer whether any specific individual's data was included in the training set. This directly addresses the goal of preventing memorization and leakage of sensitive information while still allowing the model to learn useful patterns for fraud detection.

Answer

Federated learning

Answer

Homomorphic encryption

Answer

Model quantization

Question 6

An organization wants to implement an AI ethics board. Which composition best ensures independence and expertise?

Accepted Answer

Mix of internal stakeholders and external ethicists. An AI ethics board must combine internal stakeholders (who understand organizational context, data flows, and operational constraints) with external ethicists (who provide independent, unbiased perspectives and specialized knowledge of ethical frameworks like IEEE Ethically Aligned Design or the EU AI Act). This composition ensures the board can evaluate AI systems for bias, fairness, and transparency without being dominated by business or technical interests, which is critical for maintaining trust and regulatory compliance.

Answer

All members from the legal department

Answer

IT department head and data scientists

Answer

Only senior executives from the company

Question 7

A social media company uses an AI content moderation system to filter hate speech. The system uses a natural language processing model trained on user reports. Recently, the model's false positive rate has increased, blocking legitimate posts. An internal audit reveals that a coordinated group of users has been falsely reporting harmless posts, causing the model to learn incorrect patterns. The company needs to address the attack and restore accuracy. The engineering team can modify the training pipeline. What is the most effective first step?

Accepted Answer

Redesign the training pipeline to incorporate a reputation system for reporting users. The root cause is a coordinated attack where malicious users exploit the reporting mechanism to poison the training data. Incorporating a reputation system into the training pipeline allows the model to weigh or filter user reports based on the trustworthiness of the reporting user, directly mitigating the impact of false reports without discarding legitimate feedback. This addresses the adversarial behavior at the source, restoring accuracy by ensuring the model learns from reliable signals.

Answer

Increase the weight of non-reported posts to counteract the reported posts' influence

Answer

Apply adversarial training to make the model robust to crafted inputs

Answer

Retrain the model on a dataset that excludes all user-reported posts

Question 8

A data scientist notices the model overfits. Which change to the exhibit's configuration would most likely reduce overfitting?

Accepted Answer

Add L2 regularization to dense layers. Adding L2 regularization to dense layers penalizes large weights by adding a squared magnitude term to the loss function, which forces the model to learn simpler patterns and reduces overfitting. This directly addresses the core issue of the model memorizing noise in the training data.

Answer

Remove dropout layers

Answer

Increase learning rate to 0.01

Answer

Increase units in the first dense layer to 512

Question 9

A data scientist notices that a model's performance on the training set is excellent, but validation accuracy is poor. The team used the same dataset for feature engineering and model selection. What is the MOST likely cause?

Accepted Answer

Train/test leakage caused by using the same data for feature engineering and model selection. Reusing the same data for feature engineering and model selection creates data leakage from validation set into training decisions, leading to overfitting and poor generalization.

Answer

The dataset is too small for the model complexity

Answer

The learning rate is too high

Answer

The model is overfitting due to high variance

Question 10

A company uses an AI model to screen job applicants. A disparate impact analysis reveals that the model's rejection rate for a protected group is significantly higher than for others. Which THREE actions should the company take to address this?

Accepted Answer

Revisit training data for historical bias and consider reweighting. Revisiting the training data for historical bias and applying reweighting directly addresses the root cause of disparate impact. If the training data contains biased labels or skewed representation of the protected group, the model will learn and amplify those biases. Reweighting adjusts the loss function to give more importance to underrepresented or disadvantaged groups, helping to equalize error rates across groups.

Answer

Ignore the disparity because the model is accurate overall

Answer

Remove all demographic attributes from the dataset

Question 11

Which TWO deployment strategies allow for testing a new model version before fully rolling it out?

Accepted Answer

Canary deployment. Canary deployment is correct because it routes a small percentage of live traffic to the new model version while the majority continues using the stable version. This allows real-world validation of the new model's performance and error rates under production load before a full rollout, minimizing blast radius if issues arise.

Answer

Shadow deployment

Answer

Direct cutover

Answer

Blue-Green deployment

Question 12

A team has trained a large transformer model that achieves 95% accuracy but requires 8 GB of GPU memory for inference. They need to deploy it on edge devices with only 2 GB of memory and minimal accuracy loss. Which combination of techniques should they apply?

Accepted Answer

Apply INT8 quantization, pruning, and model distillation. The team needs to reduce the model's memory footprint from 8 GB to under 2 GB while preserving accuracy. INT8 quantization reduces memory by 4x (from 32-bit floats to 8-bit integers), weight pruning removes redundant connections, and model distillation trains a smaller student model to mimic the teacher, collectively achieving the required compression with minimal accuracy loss.

Answer

Use model distillation to create a smaller student model

Answer

Apply INT8 quantization and weight pruning only

Answer

Use FP16 precision and increase batch size

Question 13

Which THREE factors are most critical to consider when designing a continuous integration/continuous deployment (CI/CD) pipeline for machine learning?

Accepted Answer

Data quality and schema validation. Data quality and schema validation (A) are critical because ML pipelines are highly sensitive to data drift, missing values, and schema mismatches that can silently degrade model performance. Without automated validation at the CI stage, bad data can pass through and corrupt model training or inference, leading to unreliable outputs in production.

Answer

A/B testing framework for comparing models

Answer

Automated unit testing of application code

Question 14

An e-commerce company deploys a recommendation model that must serve predictions with sub-100 ms latency for millions of users during peak hours. The model is a large neural network. Which architecture is most suitable?

Accepted Answer

Use a distributed system with load balancers and model replicas.. Distributing the model across multiple servers with load balancers and replicas allows horizontal scaling to handle millions of concurrent users while maintaining sub-100 ms latency. This architecture provides fault tolerance and can dynamically adjust to peak traffic loads, which is essential for real-time inference with large neural networks.

Answer

Batch process predictions every hour.

Answer

Deploy the model on a single powerful GPU server.

Answer

Use serverless functions with auto-scaling.

Question 15

A data scientist is training a convolutional neural network (CNN) for object detection. The training loss decreases rapidly but then plateaus at a high value, and the validation loss starts increasing. Which action should the scientist take to improve the model?

Accepted Answer

Reduce the model complexity. The training loss decreasing rapidly then plateauing at a high value while validation loss increases is classic overfitting. Reducing model complexity (Option C) directly addresses overfitting by decreasing the number of parameters or applying regularization (e.g., dropout, L2), which forces the network to learn more generalizable features rather than memorizing noise in the training data.

Answer

Increase the learning rate

Answer

Increase the number of epochs

Answer

Add more convolutional layers

Question 16

A financial institution wants to deploy an AI system that automates loan approvals. Under the EU AI Act, this system would be classified as high-risk. Which of the following is a MANDATORY requirement for high-risk AI systems before market placement?

Accepted Answer

Establishing a risk management system throughout the system lifecycle. The EU AI Act requires a conformity assessment (often including third-party involvement) for high-risk AI systems to ensure compliance with requirements such as data governance, transparency, and human oversight. The other options are either voluntary, not required, or apply to lower-risk tiers.

Answer

Obtaining a CE marking of conformity through self-assessment only

Answer

Acquiring liability insurance for AI-related harms

Answer

Publishing the training dataset in an open repository

Question 17

A data pipeline ingests streaming data from IoT sensors. The current batch processing pipeline causes stale predictions. Which architecture change is most appropriate?

Accepted Answer

Implement Apache Kafka and stream processing. Implementing Apache Kafka and stream processing enables real-time data ingestion and processing, reducing latency and stale predictions. Option A (larger batch interval) increases staleness. Option B (micro-batch with Spark) still uses batch processing with small intervals, not true streaming. Option C (HDFS) is for storage, not real-time processing.

Answer

Use a larger batch interval

Answer

Revert to micro-batch processing with Apache Spark

Answer

Store raw data in Hadoop HDFS

Question 18

A large enterprise is developing an internal LLM-powered assistant that can access the internet and execute code. To mitigate risks from excessive agency (e.g., the model performing unauthorized actions), which THREE security measures should be implemented?

Accepted Answer

Require human-in-the-loop approval for code execution and write operations. Requiring human-in-the-loop approval for code execution and write operations directly enforces a control over the model's agency, preventing it from performing unauthorized actions such as modifying files or executing arbitrary commands. This measure ensures that any action with side effects is vetted by a human operator, mitigating the risk of excessive agency where the LLM could autonomously cause harm.

Answer

Deploy monitoring for anomalous input patterns

Answer

Apply output filtering to block sensitive data in responses

Question 19

An attacker repeatedly queries a public LLM API with carefully crafted inputs to reconstruct the model's architecture and approximate weights. This is an example of which attack?

Accepted Answer

Model extraction. Model extraction attacks involve querying a public API with carefully crafted inputs to reconstruct a target model's architecture and approximate weights. By analyzing the outputs (e.g., logits or probabilities), an attacker can train a substitute model that mimics the original, enabling offline exploitation or competitive intelligence. This directly matches the scenario described.

Answer

Data poisoning

Answer

Membership inference

Answer

Model inversion

Question 20

A healthcare company is developing a predictive model to identify patients at risk of readmission within 30 days. The data engineering team has built a pipeline that collects data from multiple sources, including electronic health records (EHR), lab results, and wearable device data. During initial testing, the model's performance is poor, with high false positives. Upon investigation, the team discovers that the data contains significant temporal misalignment: lab results are timestamped when ordered, not when collected; wearable data is aggregated hourly; and EHR data has inconsistent update frequencies. The data pipeline currently joins all features on the patient ID without aligning timestamps. The data volume is large, and processing time is a concern. Which action should the data engineering team take to most effectively address the issue and improve model performance?

Accepted Answer

Implement a window-based feature aggregation (e.g., 6-hour windows) and align all features to the same time windows before joining.. Implementing a window-based feature aggregation with consistent time windows (e.g., 6-hour or 12-hour) and aligning all data to those windows before joining ensures temporal consistency and reduces noise. This approach addresses the root cause of misalignment while managing data volume through aggregation. Simply discarding data or padding with zeros loses valuable information. Using an interpolation algorithm may introduce unrealistic values for irregularly sampled data. Leaving the pipeline as-is and tuning the model does not fix the data quality issue.

Answer

Discard all records where timestamps do not match exactly across sources, and only use records with perfect alignment.

Answer

Leave the pipeline unchanged and instead adjust the model's classification threshold to reduce false positives.

Answer

Use a data imputation algorithm to fill in missing timestamps and then join on the nearest timestamp.