Hard Difficulty Questions

It typically has lower inference latency.

Why wrong: Latency depends on implementation, not architecture choice.

It handles novel scenarios better due to joint training.

Why wrong: End-to-end learning may generalize differently, but modular allows targeted improvements.

It requires less engineering effort.

Why wrong: Modular systems often require more engineering for interfaces.

Amazon S3

Why wrong: S3 is object storage, not a vector database.

Snowflake

Why wrong: Snowflake is a data warehouse, not optimized for vector search.

Kubeflow Pipelines component

Why wrong: Kubeflow Pipelines orchestrates ML workflows, not autoscaling.

Cluster Autoscaler

Why wrong: Cluster Autoscaler adds or removes nodes, not pods.

Vertical Pod Autoscaler (VPA)

Why wrong: VPA adjusts resource requests/limits, not the number of pods.

Amazon S3

Why wrong: S3 is object storage, not optimised for vector similarity search.

pgvector

Why wrong: pgvector is an extension for PostgreSQL; while capable, it may not provide the same low-latency performance as Pinecone at very large scale.

BigQuery

Why wrong: BigQuery is a data warehouse for analytics, not vector search.

Data augmentation

Why wrong: Data augmentation improves generalization but doesn't stop training.

L2 regularization

Why wrong: L2 reduces overfitting but doesn't stop training when validation loss increases.

Dropout

Why wrong: Dropout helps but doesn't address the increasing validation loss trend.

Replace the model with a decision tree for transparency

Why wrong: May not be feasible if deep learning is required.

Rely on the model's internal attention weights (if transformer-based)

Why wrong: Attention weights are not always reliable explanations.

Calculate global feature importance using permutation importance

Why wrong: Global importance doesn't explain individual decisions.

Univariate selection using chi-squared test.

Why wrong: Univariate selection may pick many false positives due to multiple comparisons.

Using all features with L2 regularization.

Why wrong: L2 regularization shrinks coefficients but does not eliminate features, still risking overfitting.

Principal Component Analysis (PCA) without feature selection.

Why wrong: PCA reduces dimensions but the components may still overfit; also loses interpretability.

The policy allows access from a wide private IP range

Why wrong: While the policy allows a wide private IP range, it is still restricted to corporate networks, making this less significant than unlimited inference requests.

The policy does not require multi-factor authentication

Why wrong: Multi-factor authentication is explicitly required in the policy, so this is not a weakness.

The policy grants 'audit_log' access to data scientists

Why wrong: Granting audit_log access to data scientists is a minor access control issue but not as critical as model extraction risk.

Use a larger batch interval

Why wrong: Larger batch intervals would increase, not decrease, staleness.

Revert to micro-batch processing with Apache Spark

Why wrong: Micro-batch processing still incurs latency and may not meet real-time requirements.

Store raw data in Hadoop HDFS

Why wrong: HDFS is for distributed storage; it does not speed up processing.

Input data and model name only

Why wrong: Logging only input data and model name is insufficient because it lacks the model output and version, which are necessary for complete traceability.

Source code and training dataset hash

Why wrong: Logging source code and training dataset hash is not part of inference audit trails; inference logging requires inputs, outputs, and version.

Model outputs and confidence scores only

Why wrong: Logging only model outputs and confidence scores misses the input data and model version, making it impossible to trace back to specific decisions.

Retrain the model using only the most recent three months of data.

Why wrong: Retraining is not possible because original training data is unavailable; also, this action contradicts the constraint.

Collect new labeled data and perform transfer learning from the original model.

Why wrong: This requires new data and retraining, which is not feasible given the constraint of no retraining.

Decrease the decision threshold to a lower value, such as 0.3.

Why wrong: Lowering the threshold would classify more customers as churn, increasing false positives.

It requires the base models to be of different types

Why wrong: Bagging typically uses the same type of base model (e.g., all decision trees).

It reduces the error by combining weak learners

Why wrong: That describes boosting; bagging combines strong or unstable models to reduce variance.

Add L1 regularization

Why wrong: L1 regularization introduces sparsity but does not directly stabilize training variance from learning rate.

Increase the batch size

Why wrong: Increasing batch size can reduce variance but may not address plateauing; it could also slow convergence.

Increase the number of epochs

Why wrong: More epochs may not help if the model is already overfitting or stuck; it doesn't address variance.

Decrease the learning rate

Why wrong: Learning rate affects training speed, not directly overfitting.

Increase the training batch size

Why wrong: Larger batch sizes can lead to sharper minima and may not reduce overfitting.

Rebalance the training data using SMOTE and retrain the model

Why wrong: SMOTE retraining requires time and may not fully resolve bias if the model still learns proxy features.

Use adversarial debiasing during training to remove protected attribute correlations

Why wrong: Adversarial debiasing requires retraining, which contradicts the limited resources constraint.

Replace the model with a simpler logistic regression model to improve interpretability

Why wrong: Switching models is a major change and may degrade overall performance.

Membership inference

Why wrong: Membership inference only determines if a data point was in training, not reconstruct the data itself.

Data poisoning

Why wrong: Data poisoning involves corrupting training data, not reconstructing it.

Model extraction

Why wrong: Model extraction copies the model's behavior, not the training data.

Switch to an unsupervised learning approach to detect anomalies

Why wrong: Unsupervised may not be effective for known threat patterns.

Simplify the model to a logistic regression to reduce the attack surface

Why wrong: Simpler models may be less accurate and still vulnerable.

Add random noise to all input data to confuse evasion attempts

Why wrong: Random noise may degrade performance and is not targeted.

Model extraction

Why wrong: Model extraction aims to steal the model via queries, not to manipulate its outputs.

Prompt injection

Why wrong: Prompt injection targets LLMs via text prompts, not traditional ML models.

Data poisoning

Why wrong: Data poisoning corrupts training data, not inference-time inputs.

Randomly select 1,000 scans from the new hospital and have them re-labeled by the radiologist

Why wrong: Random selection may not focus on the most impactful corrections, and 1,000 may be insufficient.

Collect 20,000 more scans with the same semi-automated labeling process

Why wrong: Adding more noisy labels will not correct the underlying label noise and may worsen accuracy.

Reduce the CNN's number of layers and apply dropout to combat overfitting

Why wrong: This addresses overfitting but not label noise or domain shift; accuracy drop is likely due to distribution change.

ChurnPredict v1

Why wrong: ChurnPredict v1 uses an interpretable model and is compliant.

FraudDetect v4

Why wrong: FraudDetect v4 uses an interpretable model and is compliant.

LoanApproval v3

Why wrong: LoanApproval v3 uses an interpretable model and is compliant.