AI Implementation and Operations practice questions

Increase the temperature parameter to reduce randomness.

Why wrong: Temperature controls creativity, not factual correctness.

Collect more general training data and retrain the model from scratch.

Why wrong: Retraining from scratch is costly and may not address domain specificity.

Roll back to a previous version of the model that was more accurate.

Why wrong: Rolling back does not fix the domain knowledge gap.

The synthetic data should include a wide variety of events, even if not realistic.

Why wrong: Unrealistic variety can confuse the model.

The synthetic data should be generated using an unsupervised generative model.

Why wrong: Unsupervised generation may not target rare events specifically.

The synthetic data should be as large as possible to cover all possibilities.

Why wrong: Quantity without quality can dilute the model's focus.

Increase the learning rate.

Why wrong: Learning rate affects training speed, not generalization.

Add more layers to the neural network.

Why wrong: Adding layers increases capacity, likely worsening overfitting.

Increase the size of the training dataset.

Why wrong: More data can help but is not always the first action.

It guarantees the model's predictions are private.

Why wrong: Privacy is not the primary goal of adversarial debiasing.

It is simpler to implement than pre-processing techniques.

Why wrong: Adversarial training is complex and computationally intensive.

It ensures equal approval rates across all groups.

Why wrong: Adversarial debiasing aims for equal opportunity, not necessarily equal outcome.

Use a more complex model to better fit the data.

Why wrong: More complex models can overfit and increase false positives.

Shorten the observation window to detect anomalies faster.

Why wrong: Shorter windows increase sensitivity to transient noise.

Increase the training data to include more normal patterns.

Why wrong: More normal data might not reduce false positives if threshold unchanged.

Model accuracy on a recent holdout set.

Why wrong: Accuracy drop indicates drift but does not isolate input drift.

F1 score on the training data.

Why wrong: F1 score is a performance metric, not a drift metric.

Root mean squared error (RMSE) on test data.

Why wrong: RMSE measures prediction error, not input drift.

Randomly replace some recommendations with popular items.

Why wrong: Random replacements can hurt accuracy and do not guarantee diversity.

Use only popularity-based recommendations.

Why wrong: Popularity recommendations are uniform and reduce diversity.

Increase the number of recommendations and use collaborative filtering.

Why wrong: Collaborative filtering may already be in use; more recommendations may not improve diversity.

Increase the number of training epochs to ensure convergence.

Why wrong: Epoch count affects the model, but does not guarantee reproducibility.

Use the same GPU hardware for both training and inference.

Why wrong: GPU hardware differences may cause numerical variations.

Use parallel data loading to speed up inference.

Why wrong: Parallelism can introduce non-deterministic ordering.

Open-source licensing of all models.

Why wrong: Open-source is not an ethical requirement.

Maximizing model accuracy to increase revenue.

Why wrong: Ethical compliance focuses on fairness, not profit.

Batch size.

Why wrong: Batch size affects gradient noise, not capacity directly.

Optimizer choice (e.g., SGD vs Adam).

Why wrong: Optimizer affects training dynamics, not capacity.

Learning rate.

Why wrong: Learning rate does not change model capacity.

Reduce number of epochs to 5.

Why wrong: Fewer epochs may underfit, but overfitting is already present.

Increase batch size to 64.

Why wrong: Larger batch size often leads to sharper minima, potentially more overfitting.

Increase learning rate to 0.01.

Why wrong: Higher learning rate may cause training to diverge.

A scheduled training job consuming GPU resources on the same node.

Why wrong: Training jobs are typically isolated from serving nodes.

A network outage between the model server and the client.

Why wrong: Network outage would cause connection errors, not latency increase.

A bug in the model's preprocessing code causing incorrect predictions.

Why wrong: Incorrect predictions do not directly cause latency spikes.

Increase the confidence threshold for predictions.

Why wrong: This adjusts the trade-off between precision and recall but does not address the underlying data drift.

Decrease the learning rate of the training algorithm.

Why wrong: Learning rate is a hyperparameter for training, not for inference; it does not affect deployed model performance.

Deploy an additional ensemble of models for redundancy.

Why wrong: Ensemble methods improve accuracy if models are diverse, but they do not fix drift without retraining.

Minimize data collection by not logging any user interactions.

Why wrong: While minimizing data is good, it prevents auditing and debugging; also, DSARs require proving data was not processed if not retained.

Anonymize all user data before logging interactions.

Why wrong: Anonymization may prevent re-identification, but DSARs require access to personal data; anonymized data is not personal data, but the organization must still be able to identify the user's data before anonymization.

Encrypt all chat logs and store them indefinitely for audit purposes.

Why wrong: Indefinite storage violates the storage limitation principle; encryption does not exempt from deletion requests.

Use a different tokenizer during fine-tuning.

Why wrong: Tokenizer choice affects vocabulary but not factual consistency.

Decrease the temperature parameter to 0.1 during inference.

Why wrong: Lower temperature makes output more deterministic but does not guarantee factual accuracy.

Set a maximum token limit of 50 for each summary.

Why wrong: Truncating output may omit important details but does not prevent hallucination.

Increase the batch size to process more events simultaneously.

Why wrong: Larger batches reduce per-event latency only if parallelism is improved; but for real-time streaming, batching adds delay.

Add more feature engineering steps to improve model accuracy.

Why wrong: Additional feature engineering increases pre-processing time, worsening latency.

Migrate from a decision tree ensemble to a deep neural network.

Why wrong: DNNs are typically slower than tree ensembles for tabular data.

Manually deploy the model to production after approval.

Why wrong: Manual steps break traceability and reproducibility.

Store only the final model artifact in a shared drive.

Why wrong: Without versioning data and code, the model cannot be reproduced.

Use a spreadsheet to record model version numbers.

Why wrong: Spreadsheets are not automated and prone to errors; version control tools are better.

Increase the size of the training dataset by collecting more unlabeled data.

Why wrong: More unlabeled data does not help without labels; it may even worsen drift if distribution differs.

Retrain the model immediately with the most recent data.

Why wrong: Retraining without diagnosis may waste resources; first understand the drift.

A required feature is missing from the production data pipeline.

Why wrong: No mention of missing features; all features are present.

The model was overfitted to the training data.

Why wrong: Training and test accuracy are close (0.87 vs 0.85), so overfitting is unlikely.

The model's confidence threshold needs to be adjusted.

Why wrong: Adjusting threshold changes precision-recall trade-off but does not fix the underlying drift.

Add a regularization term to penalize high credit scores.

Why wrong: Penalizing high credit scores is arbitrary and likely to degrade performance on the new prime borrower segment.

Deploy an ensemble of the original model and a neural network.

Why wrong: Ensembling may improve robustness but does not fix the distribution shift and increases operational complexity.

Reject all predictions where the confidence score is below 0.9.

Why wrong: This reduces false positives but does not adapt the model to the new data distribution; many correct predictions may be discarded.