AI0-001 AI Implementation and Operations — 30 Questions

Question 1

A company deployed a chatbot using a pre-trained language model. Users report that the chatbot provides incorrect answers to domain-specific questions. Which approach should the AI team prioritize to improve accuracy without retraining the entire model?

Accepted Answer

Fine-tune the model on a curated dataset of domain-specific conversations.. Fine-tuning on a curated domain-specific dataset is the most efficient way to improve accuracy for specialized queries without retraining the entire model. It adjusts the model's weights using a smaller, targeted dataset, preserving general language understanding while adapting to domain terminology and context.

Answer

Increase the temperature parameter to reduce randomness.

Answer

Collect more general training data and retrain the model from scratch.

Answer

Roll back to a previous version of the model that was more accurate.

Question 2

An AI system misclassifies rare but critical events. The team considers using synthetic data. Which consideration is MOST important for ensuring the synthetic data improves performance on real rare events?

Accepted Answer

The synthetic data should accurately represent the distribution and features of real rare events.. Option C is correct because synthetic data must faithfully replicate the distribution and feature space of real rare events to enable the model to learn meaningful decision boundaries. If the synthetic data does not capture the true underlying patterns—such as specific sensor readings or transaction anomalies—the model will fail to generalize to actual rare events, defeating the purpose of augmentation.

Answer

The synthetic data should include a wide variety of events, even if not realistic.

Answer

The synthetic data should be generated using an unsupervised generative model.

Answer

The synthetic data should be as large as possible to cover all possibilities.

Question 3

A data scientist trains a regression model and notices the training loss is low but validation loss is high. Which technique should be applied FIRST to address this issue?

Accepted Answer

Apply L1 or L2 regularization to the model.. The scenario describes overfitting, where the model memorizes the training data but fails to generalize to unseen data. Applying L1 or L2 regularization (Option D) is the correct first step because it adds a penalty to the loss function for large weights, discouraging complexity and reducing overfitting without requiring additional data or architectural changes.

Answer

Increase the learning rate.

Answer

Add more layers to the neural network.

Answer

Increase the size of the training dataset.

Question 4

A company deploys an AI model for loan approval. The model shows bias against a protected group. The team decides to use adversarial debiasing. What is the PRIMARY advantage of this approach?

Accepted Answer

It reduces bias while preserving predictive performance by learning representations that are invariant to sensitive attributes.. Adversarial debiasing is an in-processing technique that trains a primary model to predict the target (e.g., loan approval) while simultaneously training an adversary to predict the sensitive attribute from the model's learned representations. The primary model is penalized when the adversary succeeds, forcing it to learn representations that are invariant to the sensitive attribute. This reduces bias while preserving predictive performance because the model retains the ability to learn task-relevant patterns that are not correlated with the protected attribute.

Answer

It guarantees the model's predictions are private.

Answer

It is simpler to implement than pre-processing techniques.

Answer

It ensures equal approval rates across all groups.

Question 5

An AIOps platform monitors server metrics and triggers alerts. The team notices too many false positives. Which adjustment should be made to the anomaly detection model?

Accepted Answer

Raise the anomaly score threshold for triggering alerts.. Raising the anomaly score threshold (Option D) directly reduces false positives by requiring a higher deviation from normal behavior before an alert is triggered. In AIOps platforms, the anomaly score is a numeric value (e.g., 0–100) that quantifies how unusual a metric is; a higher threshold means only more extreme deviations generate alerts, filtering out minor fluctuations that were incorrectly flagged.

Answer

Use a more complex model to better fit the data.

Answer

Shorten the observation window to detect anomalies faster.

Answer

Increase the training data to include more normal patterns.

Question 6

A team deploys a machine learning model as a REST API. They want to monitor model drift. Which metric is MOST appropriate for detecting drift in the input data distribution?

Accepted Answer

Population stability index (PSI) comparing training and recent data.. Population stability index (PSI) is the most appropriate metric for detecting drift in input data distribution because it directly measures the shift between the training data distribution and the recent production data distribution. PSI is calculated by binning both distributions and computing the sum of (proportion in bin of recent data minus proportion in bin of training data) times the natural log of their ratio, making it sensitive to changes in feature distributions without requiring ground truth labels.

Answer

Model accuracy on a recent holdout set.

Answer

F1 score on the training data.

Answer

Root mean squared error (RMSE) on test data.

Question 7

A company uses an AI system to recommend products. The recommendation accuracy is high, but users complain about lack of diversity. Which strategy should the team adopt to improve diversity without significantly sacrificing accuracy?

Accepted Answer

Modify the loss function to include a term that penalizes overly similar recommendations.. Option D is correct because modifying the loss function to include a diversity penalty directly addresses the lack of recommendation diversity at the algorithmic level. By adding a regularization term that penalizes overly similar recommendations, the model learns to balance accuracy with variety, ensuring that the output set remains diverse without a significant drop in relevance. This approach is a standard technique in recommendation systems, often implemented via Determinantal Point Processes (DPPs) or diversity-aware loss functions.

Answer

Randomly replace some recommendations with popular items.

Answer

Use only popularity-based recommendations.

Answer

Increase the number of recommendations and use collaborative filtering.

Question 8

A machine learning engineer is deploying a model to production. Which TWO practices are essential for ensuring reproducibility of model predictions?

Accepted Answer

Version-control the model artifact (e.g., using MLflow or DVC).. Version-controlling the model artifact (D) is essential because it allows you to reproduce the exact model binary that generated a prediction, ensuring that any changes to the model code, hyperparameters, or training data do not silently alter outputs. Tools like MLflow or DVC store the model along with its metadata, enabling rollback and auditability in production.

Answer

Increase the number of training epochs to ensure convergence.

Answer

Use the same GPU hardware for both training and inference.

Answer

Use parallel data loading to speed up inference.

Question 9

An organization is implementing an AI governance framework. Which THREE components are essential for compliance with ethical AI standards?

Accepted Answer

Data privacy protection measures (e.g., differential privacy).. Options A, B, and D are correct because bias auditing, explainability, and privacy protection are core ethical requirements. Option C is wrong because maximizing revenue is a business goal, not an ethical requirement. Option E is wrong because open-source is not a mandatory component.

Answer

Open-source licensing of all models.

Answer

Maximizing model accuracy to increase revenue.

Question 10

A data scientist is tuning a deep learning model. Which TWO hyperparameters directly affect the model's capacity to overfit?

Accepted Answer

Number of layers in the network.. Option A is correct because increasing the number of layers increases the model's depth, which expands its representational capacity and allows it to learn more complex patterns, including noise, thereby directly increasing overfitting risk. Option D is correct because dropout is a regularization technique that randomly drops neurons during training; a low dropout rate (e.g., 0.0) removes this regularization, while a high rate (e.g., 0.5) reduces overfitting by preventing co-adaptation of neurons.

Answer

Batch size.

Answer

Optimizer choice (e.g., SGD vs Adam).

Answer

Learning rate.

Question 11

A team trained a ResNet-50 model with the configuration shown. The high training accuracy and lower validation accuracy suggest overfitting. Which change to the training configuration is MOST likely to reduce overfitting?

Accepted Answer

Add dropout layers after convolutional layers.. Adding dropout layers after convolutional layers is a regularization technique that randomly drops a fraction of neurons during training, which forces the network to learn more robust features and reduces overfitting. This directly addresses the symptom of high training accuracy with lower validation accuracy by preventing the model from relying too heavily on specific neurons.

Answer

Reduce number of epochs to 5.

Answer

Increase batch size to 64.

Answer

Increase learning rate to 0.01.

Question 12

An operations team sees the log entries above for a production ML model. What is the MOST likely root cause of the latency spike?

Accepted Answer

A memory leak in the model serving container causing gradual slowdown.. The log entries show a gradual increase in latency over time, which is characteristic of a memory leak in the model serving container. As memory consumption grows, garbage collection pauses become more frequent and longer, eventually causing request processing to slow down. This pattern is distinct from a sudden spike caused by resource contention or network issues.

Answer

A scheduled training job consuming GPU resources on the same node.

Answer

A network outage between the model server and the client.

Answer

A bug in the model's preprocessing code causing incorrect predictions.

Question 13

A company deploys a computer vision model for quality inspection on a manufacturing line. After deployment, the model's accuracy drops from 95% to 80% over two weeks. Which action is most likely to address this issue?

Accepted Answer

Retrain the model using recently collected production data.. Option B is correct because data drift is a common cause of performance degradation over time, and retraining with recent data realigns the model. Option A is wrong because increasing threshold may reduce false positives but does not address drift. Option C is wrong because adjusting learning rate is irrelevant for inference. Option D is wrong because adding redundant models increases complexity without solving drift.

Answer

Increase the confidence threshold for predictions.

Answer

Decrease the learning rate of the training algorithm.

Answer

Deploy an additional ensemble of models for redundancy.

Question 14

An organization is implementing an AI-powered chatbot for customer service. The chatbot must comply with GDPR and handle data subject access requests (DSARs). Which design approach best ensures compliance?

Accepted Answer

Implement an audit trail that logs interactions with a unique user identifier, and provide a mechanism to delete logs upon user request.. Option C is correct because GDPR requires that personal data be stored only as long as necessary and that data subjects have the right to erasure. By logging interactions with a unique user identifier and providing a deletion mechanism, the chatbot can fulfill DSARs while maintaining an audit trail for compliance monitoring. This approach balances operational needs with regulatory obligations.

Answer

Minimize data collection by not logging any user interactions.

Answer

Anonymize all user data before logging interactions.

Answer

Encrypt all chat logs and store them indefinitely for audit purposes.

Question 15

A data scientist fine-tunes a large language model for a legal document summarization task. After fine-tuning, the model performs well on test data but produces summaries that include hallucinated legal clauses. Which mitigation strategy is most effective?

Accepted Answer

Implement retrieval-augmented generation (RAG) to provide factual context.. Option A is correct because RAG grounds the model in retrieved documents, reducing hallucinations. Option B is wrong as temperature affects creativity but does not eliminate hallucinations. Option C is wrong because it reduces output length but may not prevent false content. Option D is wrong because it does not address the model's tendency to fabricate.

Answer

Use a different tokenizer during fine-tuning.

Answer

Decrease the temperature parameter to 0.1 during inference.

Answer

Set a maximum token limit of 50 for each summary.

Question 16

A team deploys a real-time fraud detection model on a streaming platform. The model must produce predictions within 100 milliseconds per event. Initial latency is 150 ms. Which optimization is most likely to meet the latency requirement?

Accepted Answer

Apply model quantization to reduce precision from FP32 to INT8.. Model quantization reduces the numerical precision of the model's weights and activations from FP32 to INT8, which decreases memory footprint and speeds up inference. This optimization directly addresses the 150 ms latency by enabling faster arithmetic operations on modern hardware, often cutting inference time by 2-4x, which can bring latency below the 100 ms requirement.

Answer

Increase the batch size to process more events simultaneously.

Answer

Add more feature engineering steps to improve model accuracy.

Answer

Migrate from a decision tree ensemble to a deep neural network.

Question 17

A DevOps team is deploying a machine learning model using a CI/CD pipeline. They want to ensure the model is reproducible and traceable. Which TWO practices should they implement?

Accepted Answer

Version the training dataset and code using Git and DVC.. Option A and D are correct because versioning data and code, and using containers ensure reproducibility. Option B is wrong because manual deployment is error-prone and not traceable. Option C is wrong as it does not track the model. Option E is wrong because manual tagging is inconsistent.

Answer

Manually deploy the model to production after approval.

Answer

Store only the final model artifact in a shared drive.

Answer

Use a spreadsheet to record model version numbers.

Question 18

An AI operations team is monitoring a deployed image classification model. They notice a gradual increase in prediction confidence but a drop in accuracy. Which THREE actions should they take to diagnose the issue?

Accepted Answer

Analyze the model's calibration curve to see if confidence scores align with actual accuracy.. Option A is correct because a calibration curve (reliability diagram) directly compares predicted confidence scores against actual accuracy. In this scenario, increasing confidence with dropping accuracy indicates miscalibration—the model is becoming overconfident. Analyzing the calibration curve reveals whether the confidence scores systematically deviate from true probabilities, which is the core diagnostic step for this specific symptom.

Answer

Increase the size of the training dataset by collecting more unlabeled data.

Answer

Retrain the model immediately with the most recent data.

Question 19

Based on the exhibit, what is the most likely cause of the accuracy drop?

Accepted Answer

Data drift in the 'income' feature has caused the model to become less accurate.. The exhibit shows a sudden and sustained drop in model accuracy coinciding with a shift in the distribution of the 'income' feature. This is a classic symptom of data drift, where the statistical properties of the input feature change over time, causing the model's learned patterns to no longer match the production data. Option B correctly identifies this as the most likely cause because the model was trained on a prior income distribution and is now encountering values outside that range.

Answer

A required feature is missing from the production data pipeline.

Answer

The model was overfitted to the training data.

Answer

The model's confidence threshold needs to be adjusted.

Question 20

You are an AI engineer at a financial services firm. The company has deployed a gradient boosting model to predict loan default risk. The model takes features such as credit score, debt-to-income ratio, loan amount, and employment length. In production, the model processes about 10,000 predictions per day with an average latency of 50ms. Recently, the accuracy has dropped from 92% to 85%. You also notice that the average credit score of applicants has increased significantly because the marketing team launched a campaign targeting prime borrowers. The model was originally trained on data from the past three years, which included a mix of prime and subprime borrowers. You need to restore model performance while minimizing downtime and retraining cost. Which action should you take first?

Accepted Answer

Retrain the model using the last three months of production data with labels.. The drop in accuracy is due to data drift—the production data now has a different distribution (higher credit scores) than the training data. Retraining on the most recent three months of production data with labels directly addresses this shift by adapting the model to the new population, and it minimizes downtime because it uses existing infrastructure and avoids complex architectural changes.

Answer

Add a regularization term to penalize high credit scores.

Answer

Deploy an ensemble of the original model and a neural network.

Answer

Reject all predictions where the confidence score is below 0.9.