AI Implementation and Operations practice questions

Increase the temperature parameter to reduce randomness.

Why it fails: Temperature controls creativity, not factual correctness.

Collect more general training data and retrain the model from scratch.

Why it fails: Retraining from scratch is costly and may not address domain specificity.

Roll back to a previous version of the model that was more accurate.

Why it fails: Rolling back does not fix the domain knowledge gap.

Increase the learning rate.

Why it fails: Learning rate affects training speed, not generalization.

Add more layers to the neural network.

Why it fails: Adding layers increases capacity, likely worsening overfitting.

Increase the size of the training dataset.

Why it fails: More data can help but is not always the first action.

It guarantees the model's predictions are private.

Why it fails: Privacy is not the primary goal of adversarial debiasing.

It is simpler to implement than pre-processing techniques.

Why it fails: Adversarial training is complex and computationally intensive.

It ensures equal approval rates across all groups.

Why it fails: Adversarial debiasing aims for equal opportunity, not necessarily equal outcome.

Randomly replace some recommendations with popular items.

Why it fails: Random replacements can hurt accuracy and do not guarantee diversity.

Use only popularity-based recommendations.

Why it fails: Popularity recommendations are uniform and reduce diversity.

Increase the number of recommendations and use collaborative filtering.

Why it fails: Collaborative filtering may already be in use; more recommendations may not improve diversity.

Batch size.

Why it fails: Batch size affects gradient noise, not capacity directly.

Optimizer choice (e.g., SGD vs Adam).

Why it fails: Optimizer affects training dynamics, not capacity.

Learning rate.

Why it fails: Learning rate does not change model capacity.

A scheduled training job consuming GPU resources on the same node.

Why it fails: Training jobs are typically isolated from serving nodes.

A network outage between the model server and the client.

Why it fails: Network outage would cause connection errors, not latency increase.

A bug in the model's preprocessing code causing incorrect predictions.

Why it fails: Incorrect predictions do not directly cause latency spikes.

A required feature is missing from the production data pipeline.

Why it fails: No mention of missing features; all features are present.

The model was overfitted to the training data.

Why it fails: Training and test accuracy are close (0.87 vs 0.85), so overfitting is unlikely.

The model's confidence threshold needs to be adjusted.

Why it fails: Adjusting threshold changes precision-recall trade-off but does not fix the underlying drift.

Add a regularization term to penalize high credit scores.

Why it fails: Penalizing high credit scores is arbitrary and likely to degrade performance on the new prime borrower segment.

Deploy an ensemble of the original model and a neural network.

Why it fails: Ensembling may improve robustness but does not fix the distribution shift and increases operational complexity.

Reject all predictions where the confidence score is below 0.9.

Why it fails: This reduces false positives but does not adapt the model to the new data distribution; many correct predictions may be discarded.

Accuracy

Why it fails: Accuracy aggregates all four confusion-matrix cells, so it is dominated by the large true-negative count and does not isolate false positives. It would be the right summary when classes are balanced and overall correctness is the goal.

F1-score

Why it fails: F1-score combines precision and recall into a single harmonic mean, so it reflects the balance between false positives and false negatives rather than isolating false-positive avoidance. It is tempting because it summarises overall classification quality when both error types matter equally; here, the team needs precision alone, which measures FP against TP+FP.

Recall

Why it fails: Recall is TP/(TP+FN), measuring the capture of actual positives and ignoring false positives entirely, so it cannot answer this question. It is tempting because recall matters when missing positives is costly, such as screening for fraud or disease, but here the team needs precision, TP/(TP+FP).

The cloud instance does not have enough memory

Why it fails: The instance has double the local RAM, so raw capacity is not the constraint; the likely cause is data or batch size scaling with the cloud configuration, or memory-hungry defaults. Insufficient memory would be the answer if the instance genuinely had less RAM than required.

The training model is too large for the GPU memory

Why it fails: The error is a host-level OOM kill, meaning system RAM was exhausted, not GPU memory; GPU exhaustion surfaces as CUDA out-of-memory errors from the framework. GPU memory sizing is the right concern when a model's weights and activations exceed VRAM while host RAM remains sufficient.

The cloud instance ran out of disk space

Why it fails: Disk exhaustion produces write failures or no-space-left errors, not the kernel's OOM killer terminating a process. Capacity planning for datasets, checkpoints and container images is the correct focus when jobs fail during writes or image pulls rather than during memory allocation.

Increase the batch size for inference

Why it fails: Increasing batch size raises throughput per GPU but does not lower the cost of the 16GB footprint; latency typically grows as batches queue, violating the no-latency-increase constraint. It is tempting because batching amortises GPU overhead across requests, and it would be the right choice when throughput, not cost, is the bottleneck.

Add more GPU nodes to distribute the load

Why it fails: Adding GPU nodes scales capacity horizontally, multiplying GPU spend rather than reducing it, and leaves each replica's utilisation unchanged. Horizontal scaling is the right answer when demand exceeds current capacity and latency must be preserved under higher concurrent load.

Switch to CPU-based inference

Why it fails: CPU inference removes GPU cost but executes transformer matrix operations far slower, so latency rises substantially. CPU serving suits small models or latency-tolerant batch workloads, not a 7B model with an explicit requirement to hold latency steady.

Increase the complexity of the model to capture more patterns

Why it fails: Increasing model complexity does not address the measured selection-rate disparity; it risks amplifying existing biases by fitting noise rather than enforcing fairness constraints. This option is tempting because adding complexity can improve predictive accuracy on imbalanced data, which would be correct if the goal were reducing underfitting rather than mitigating demographic bias.

Add more training data from both groups

Why it fails: Adding more data without balancing may not reduce bias.

Remove the protected attribute from the model input

Why it fails: Dropping the protected attribute does not remove bias, since correlated proxy features such as postcode or occupation still encode group membership. Attribute removal is appropriate for privacy or compliance goals, not for correcting measured disparity in selection rates.

Automated code review and approval gates

Why it fails: Code review gates govern source changes; they do not cover the model lifecycle components a complete MLOps framework requires, such as data and model versioning, automated training, and deployment monitoring. It is tempting because CI/CD governance is genuinely part of MLOps, and would be correct for a software delivery pipeline question.

Real-time model monitoring dashboard

Why it fails: A monitoring dashboard observes deployed models; it does not build, package, or deploy them, so it cannot satisfy the pipeline's core stages. It is tempting because monitoring is genuinely part of MLOps maturity, and would be the right answer if the question asked about post-deployment observability rather than the essential framework components.

Promote version 2 from Staging to Production

Why it fails: Promoting version 2 from Staging re-registers it as the production artefact, which still requires a deployment step to serve it. It is tempting because registry promotion looks like an instant switch, but the fastest rollback redeploys the previously served stable version already built, avoiding any new promotion or build cycle.

Retrain the model with corrected data

Why it fails: Retraining produces a new model artefact, requiring training time and a fresh deployment before the bug is resolved. It is tempting because corrected data improves future accuracy, but the scenario demands the fastest rollback without redeploying from scratch, which retraining cannot satisfy; reverting to the prior stable version does.

Delete version 3 from the registry

Why it fails: Deleting version 3 removes the buggy artefact from the registry but leaves production still serving it, so the fault persists. It is tempting as cleanup, yet deletion is irreversible and does not restore service; the fastest rollback points production back at the previously stable registered version instead.

The model version is incompatible with the serving framework

Why it fails: A serving framework version mismatch typically produces a load or deserialisation error at deployment, not during inference on an already-deployed model. The error more likely reflects input tensor shape or preprocessing mismatch. Framework incompatibility is tempting because deployment errors often trace to versioning, but that would surface at model load, not per-inference.

The inference server does not support batch processing

Why it fails: Batch processing governs how many images are submitted per request, not whether a single image's tensor shape matches the model's expected input dimensions. It is tempting because inference servers do have batch-size limits, and exceeding them would be the correct diagnosis if the error mentioned batch dimensions rather than input shape.

The model is overfitting to a specific image size

Why it fails: Overfitting affects generalisation accuracy on unseen data, not the runtime shape mismatch raised at inference. It is tempting because overfitting is a common model-quality concern, and it would be the correct diagnosis if validation loss diverged from training loss rather than the server rejecting the input tensor's dimensions.

API response latency

Why it fails: Latency is unrelated to data distribution.

Area under the ROC curve (AUC)

Why it fails: AUC is also a lagging performance metric.

Model accuracy on a holdout validation set

Why it fails: Holdout accuracy only registers drift once labels arrive and predictions have already degraded, so detection is delayed. It is tempting because holdout accuracy is the conventional measure of generalisation during model development, where labelled data is available and retrospective performance is what matters.

Use a sliding window of the most recent data for training

Why it fails: A sliding window discards older observations entirely, so the model loses historical patterns and drifts toward recent noise. It suits stationary or rapidly shifting distributions, whereas incremental learning with replay is required when past knowledge must be retained.

Deploy an ensemble of models trained on different time periods

Why it fails: An ensemble of period-specific models averages stale and current predictions, so it never adapts weights to new trends and still risks forgetting when members are refreshed. Ensembling suits variance reduction across diverse learners, not sequential time-series adaptation where the model must update on incoming data.

Retrain the entire model from scratch every week

Why it fails: Weekly full retraining discards learned weights, so the model cannot retain prior knowledge and lags current trends between cycles. Retraining from scratch suits one-off major distribution shifts or architecture changes, not continuous streaming where incremental updates preserve earlier learning.

Document the disparity and proceed with deployment because accuracy is high

Why it fails: High overall accuracy does not excuse a sixfold false positive rate disparity; documenting and deploying leaves the harm in production. The first action is to investigate and mitigate the bias before deployment. Documentation alone would suffice only for a monitored, non-deployed experimental model with no affected decisions.

Replace the model with a simpler model that is less discriminatory

Why it fails: Simpler models may not capture complexity and could still be biased.

Adjust the decision threshold for the protected group to equalize false positive rates

Why it fails: Equalising false positive rates by shifting the threshold for one group is illegal disparate treatment and masks the underlying bias rather than removing it. Threshold adjustment is legitimate only when a single global operating point must be tuned to a business cost ratio, not to equalise outcomes across protected groups.

The model overfits to late-night user behavior

Why it fails: Overfitting to late-night behaviour would degrade performance persistently, not produce a sharp 8 AM drop that self-corrects by noon. Overfitting is diagnosed by train-versus-validation divergence, so it would fit a scenario showing poor generalisation on held-out data rather than a daily cyclical pattern.

The model suffers from catastrophic forgetting due to daily retraining

Why it fails: Catastrophic forgetting describes a model losing previously learned tasks after training on new ones, which is a sequential multi-task problem, not a daily cyclical dip. It would be the right diagnosis if retraining on new categories caused performance on older categories to collapse permanently.

The retraining pipeline has a bug that only affects morning predictions

Why it fails: A pipeline bug would corrupt predictions consistently after each retraining run, not selectively between 8 AM and noon. Bugs are diagnosed through pipeline logs and reproducibility checks; that explanation would fit a scenario where errors appear immediately after deployment and persist until the next retrain.

Retrain the model using the new schema and redeploy

Why it fails: Retraining on the new schema discards the diagnosis step; the missing age_group and added customer_segment must first be reconciled with the pipeline's expected feature contract. It tempts because retraining restores accuracy once data drift is confirmed, which would be right after the schema change is validated and mapped.

Revert to the previous week's model version that was performing well

Why it fails: Reverting the model version leaves the upstream schema change in place, so the pipeline still feeds mismatched features and degradation persists. It tempts because rollback is the standard first response to a bad deployment, correct when the regression originates in the model artefact rather than the input data.

Contact the data engineering team to revert the schema change

Why it fails: Reverting the schema is a temporary fix that leaves the model unable to consume the new 'customer_segment' feature and still missing 'age_group', so degradation recurs. Contacting data engineering suits planned schema governance, not adapting a live pipeline to legitimate upstream feature evolution.