Hard Difficulty Questions

Precision

Why wrong: Precision focuses on false positives, but the main issue is missing actual frauds.

F1-score

Why wrong: F1-score balances precision and recall, but recall is the primary concern.

AUC-ROC

Why wrong: AUC-ROC is a general measure but does not directly address the failure to detect frauds.

Model parallelism and distributed inference

Why wrong: Model parallelism is for multi-device setups; edge devices are typically single-device, adding complexity without meeting constraints.

Increase batch size and use FP16 precision

Why wrong: Increasing batch size is not suitable for real-time single inference; FP16 may not reduce memory enough for 50MB limit.

Train a larger model and use distillation to transfer knowledge

Why wrong: Training a larger model first and then distilling is a valid approach, but the question asks for techniques applied to the deployed model; quantization and pruning are more direct.

Train the model using accuracy as the performance metric

Why wrong: Accuracy as a metric treats all correct predictions equally, so a model that classifies every transaction as legitimate achieves 99.9% accuracy without detecting any fraud. This fails because the 0.1% fraudulent class is the target of detection, and accuracy is blind to class distribution. It is tempting because accuracy is the default metric for balanced datasets, where misclassification costs are symmetric and each class is equally represented.

Undersample the legitimate transactions to match the number of fraudulent ones

Why wrong: Undersampling loses valuable data and may degrade model performance.

Increase the regularization strength in the model

Why wrong: Regularization does not address class imbalance.

Replace the convolutional layers with transformer layers to improve attention.

Why wrong: Changing architecture typically requires full retraining and may not address low-light specifically.

Use adversarial training to add imperceptible perturbations to training images.

Why wrong: Adversarial training improves resistance to malicious perturbations but not necessarily to natural low-light conditions.

Increase the model's depth by adding more convolutional layers.

Why wrong: Increasing depth changes architecture and requires retraining; it may not directly address low-light performance.

Federated learning

Why wrong: Federated learning trains across decentralized data but doesn't inherently prevent memorization; it still risks leakage from gradients.

Homomorphic encryption

Why wrong: Homomorphic encryption allows computation on encrypted data but doesn't address memorization during training.

Model quantization

Why wrong: Quantization reduces model size and speeds up inference but does not provide privacy guarantees.

All members from the legal department

Why wrong: Legal focuses on compliance, not broader ethics.

IT department head and data scientists

Why wrong: Lacks ethical training and independence.

Only senior executives from the company

Why wrong: Executives may have conflicts of interest and lack ethics expertise.

Increase the weight of non-reported posts to counteract the reported posts' influence

Why wrong: Weighting may not fully correct the poisoned patterns.

Apply adversarial training to make the model robust to crafted inputs

Why wrong: Adversarial training targets input perturbations, not training data poisoning.

Retrain the model on a dataset that excludes all user-reported posts

Why wrong: Excluding all reported posts loses valuable training data and may not eliminate poisoning.

Remove dropout layers

Why wrong: Removing dropout would remove a regularization technique and likely increase overfitting.

Increase learning rate to 0.01

Why wrong: Increasing the learning rate may cause the model to converge too quickly or oscillate, often worsening overfitting.

Increase units in the first dense layer to 512

Why wrong: Increasing units increases model capacity, which typically worsens overfitting.

The dataset is too small for the model complexity

Why wrong: Small data can cause overfitting, but the described workflow strongly indicates leakage as the primary issue.

The learning rate is too high

Why wrong: High learning rate would cause divergence, not large train-validation gap after convergence.

The model is overfitting due to high variance

Why wrong: Overfitting is a symptom; the root cause here is leakage from improper data splitting practices.

Ignore the disparity because the model is accurate overall

Why wrong: Ignoring disparity may violate anti-discrimination laws.

Remove all demographic attributes from the dataset

Why wrong: Removing attributes may not eliminate bias due to proxy variables, and can hinder fairness analysis.

Shadow deployment

Why wrong: Shadow deployment duplicates traffic to the new version but does not serve user requests, so it does not test real behavior.

Direct cutover

Why wrong: Direct cutover replaces the old version instantly with no testing period.

Blue-Green deployment

Why wrong: Blue-Green switches all traffic at once, not gradual testing.

Use model distillation to create a smaller student model

Why wrong: Distillation alone may not achieve the required memory reduction if the student model is still large; combining with quantization and pruning is more effective.

Apply INT8 quantization and weight pruning only

Why wrong: Quantization and pruning help, but may not be sufficient alone to reduce memory from 8 GB to 2 GB without significant accuracy loss; distillation is often needed.

Use FP16 precision and increase batch size

Why wrong: FP16 reduces memory somewhat, but increasing batch size increases memory usage, which is counterproductive.

A/B testing framework for comparing models

Why wrong: A/B testing is part of deployment, not CI/CD pipeline design.

Automated unit testing of application code

Why wrong: Unit testing is important but not specific to ML pipelines.

Batch process predictions every hour.

Why wrong: Batch processing cannot provide real-time sub-100 ms latency.

Deploy the model on a single powerful GPU server.

Why wrong: A single server is a bottleneck and cannot scale to millions of concurrent users.

Use serverless functions with auto-scaling.

Why wrong: Serverless functions may have cold start latency exceeding the 100 ms target.

Increase the learning rate

Why wrong: Increasing the learning rate may cause the loss to oscillate or diverge.

Increase the number of epochs

Why wrong: More epochs would likely increase overfitting further.

Add more convolutional layers

Why wrong: Adding more layers increases complexity and would likely worsen overfitting.

Obtaining a CE marking of conformity through self-assessment only

Why wrong: High-risk systems typically require third-party conformity assessment unless a harmonised standard covers all aspects.

Acquiring liability insurance for AI-related harms

Why wrong: Insurance is not a mandatory requirement of the EU AI Act.

Publishing the training dataset in an open repository

Why wrong: The Act does not require public release of training data; data governance is required but confidentiality may be maintained.

Use a larger batch interval

Why wrong: Larger batch intervals would increase, not decrease, staleness.

Revert to micro-batch processing with Apache Spark

Why wrong: Micro-batch processing still incurs latency and may not meet real-time requirements.

Store raw data in Hadoop HDFS

Why wrong: HDFS is for distributed storage; it does not speed up processing.

Deploy monitoring for anomalous input patterns

Why wrong: Monitoring detects attacks but does not prevent the model from performing excessive actions; it is a detective control, not a preventative one.

Apply output filtering to block sensitive data in responses

Why wrong: Output filtering prevents data leakage but does not directly mitigate the risk of the model taking unauthorized actions (excessive agency).

Data poisoning

Why wrong: Data poisoning corrupts training data, not inference queries.

Membership inference

Why wrong: Membership inference determines whether a specific data record was part of the training set, not the model's architecture or weights. The attacker here reconstructs structural parameters via repeated queries, which is model extraction. It is tempting because both attacks exploit query access to a model, but membership inference targets individual record presence, not the model's internal configuration.

Model inversion

Why wrong: Model inversion reconstructs training data, not model parameters.

Discard all records where timestamps do not match exactly across sources, and only use records with perfect alignment.

Why wrong: This would discard most of the data, reducing sample size and likely causing bias.

Leave the pipeline unchanged and instead adjust the model's classification threshold to reduce false positives.

Why wrong: Tuning the threshold does not fix the underlying data quality issue; model performance will remain suboptimal.

Use a data imputation algorithm to fill in missing timestamps and then join on the nearest timestamp.

Why wrong: Imputation may introduce unrealistic values for timestamps, and nearest-neighbor join could still cause misalignment.