AI0-001 AI Implementation and Operations Practice Test 2 — 15 Questions

Question 1

A company deploys an AI model for loan approval. The model shows bias against a protected group. The team decides to use adversarial debiasing. What is the PRIMARY advantage of this approach?

Accepted Answer

It reduces bias while preserving predictive performance by learning representations that are invariant to sensitive attributes.. Adversarial debiasing is an in-processing technique that trains a primary model to predict the target (e.g., loan approval) while simultaneously training an adversary to predict the sensitive attribute from the model's learned representations. The primary model is penalized when the adversary succeeds, forcing it to learn representations that are invariant to the sensitive attribute. This reduces bias while preserving predictive performance because the model retains the ability to learn task-relevant patterns that are not correlated with the protected attribute.

Answer

It guarantees the model's predictions are private.

Answer

It is simpler to implement than pre-processing techniques.

Answer

It ensures equal approval rates across all groups.

Question 2

An AIOps platform monitors server metrics and triggers alerts. The team notices too many false positives. Which adjustment should be made to the anomaly detection model?

Accepted Answer

Raise the anomaly score threshold for triggering alerts.. Raising the anomaly score threshold (Option D) directly reduces false positives by requiring a higher deviation from normal behavior before an alert is triggered. In AIOps platforms, the anomaly score is a numeric value (e.g., 0–100) that quantifies how unusual a metric is; a higher threshold means only more extreme deviations generate alerts, filtering out minor fluctuations that were incorrectly flagged.

Answer

Use a more complex model to better fit the data.

Answer

Shorten the observation window to detect anomalies faster.

Answer

Increase the training data to include more normal patterns.

Question 3

A team deploys a machine learning model as a REST API. They want to monitor model drift. Which metric is MOST appropriate for detecting drift in the input data distribution?

Accepted Answer

Population stability index (PSI) comparing training and recent data.. Population stability index (PSI) is the most appropriate metric for detecting drift in input data distribution because it directly measures the shift between the training data distribution and the recent production data distribution. PSI is calculated by binning both distributions and computing the sum of (proportion in bin of recent data minus proportion in bin of training data) times the natural log of their ratio, making it sensitive to changes in feature distributions without requiring ground truth labels.

Answer

Model accuracy on a recent holdout set.

Answer

F1 score on the training data.

Answer

Root mean squared error (RMSE) on test data.

Question 4

A company uses an AI system to recommend products. The recommendation accuracy is high, but users complain about lack of diversity. Which strategy should the team adopt to improve diversity without significantly sacrificing accuracy?

Accepted Answer

Modify the loss function to include a term that penalizes overly similar recommendations.. Modifying the loss function to include a diversity penalty directly addresses the lack of recommendation diversity at the algorithmic level. By adding a regularization term that penalizes overly similar recommendations, the model learns to balance accuracy with variety, ensuring that the output set remains diverse without a significant drop in relevance. This approach is a standard technique in recommendation systems, often implemented via Determinantal Point Processes (DPPs) or diversity-aware loss functions.

Answer

Randomly replace some recommendations with popular items.

Answer

Use only popularity-based recommendations.

Answer

Increase the number of recommendations and use collaborative filtering.

Question 5

A machine learning engineer is deploying a model to production. Which TWO practices are essential for ensuring reproducibility of model predictions?

Accepted Answer

Version-control the model artifact (e.g., using MLflow or DVC).. Version-controlling the model artifact (D) is essential because it allows you to reproduce the exact model binary that generated a prediction, ensuring that any changes to the model code, hyperparameters, or training data do not silently alter outputs. Tools like MLflow or DVC store the model along with its metadata, enabling rollback and auditability in production.

Answer

Increase the number of training epochs to ensure convergence.

Answer

Use the same GPU hardware for both training and inference.

Answer

Use parallel data loading to speed up inference.

Question 6

An organization is implementing an AI governance framework. Which THREE components are essential for compliance with ethical AI standards?

Accepted Answer

Data privacy protection measures (e.g., differential privacy).. Data privacy protection measures like differential privacy are essential for compliance with ethical AI standards because they ensure that individual data points cannot be re-identified from model outputs. Differential privacy works by adding calibrated noise to training data or query responses, providing mathematical guarantees against membership inference attacks. This directly addresses regulatory requirements such as GDPR and CCPA, making it a core component of any AI governance framework.

Answer

Open-source licensing of all models.

Answer

Maximizing model accuracy to increase revenue.

Question 7

A data scientist is tuning a deep learning model. Which TWO hyperparameters directly affect the model's capacity to overfit?

Accepted Answer

Number of layers in the network.. Increasing the number of layers increases the model's depth, which expands its representational capacity and allows it to learn more complex patterns, including noise, thereby directly increasing overfitting risk. Option D is correct because dropout is a regularization technique that randomly drops neurons during training; a low dropout rate (e.g., 0.0) removes this regularization, while a high rate (e.g., 0.5) reduces overfitting by preventing co-adaptation of neurons.

Answer

Batch size.

Answer

Optimizer choice (e.g., SGD vs Adam).

Answer

Learning rate.

Question 8

A team trained a ResNet-50 model with the configuration shown. The high training accuracy and lower validation accuracy suggest overfitting. Which change to the training configuration is MOST likely to reduce overfitting?

Accepted Answer

Add dropout layers after convolutional layers.. Adding dropout layers after convolutional layers is a regularization technique that randomly drops a fraction of neurons during training, which forces the network to learn more robust features and reduces overfitting. This directly addresses the symptom of high training accuracy with lower validation accuracy by preventing the model from relying too heavily on specific neurons.

Answer

Reduce number of epochs to 5.

Answer

Increase batch size to 64.

Answer

Increase learning rate to 0.01.

Question 9

An operations team sees the log entries above for a production ML model. What is the MOST likely root cause of the latency spike?

Accepted Answer

A memory leak in the model serving container causing gradual slowdown.. The log entries show a gradual increase in latency over time, which is characteristic of a memory leak in the model serving container. As memory consumption grows, garbage collection pauses become more frequent and longer, eventually causing request processing to slow down. This pattern is distinct from a sudden spike caused by resource contention or network issues.

Answer

A scheduled training job consuming GPU resources on the same node.

Answer

A network outage between the model server and the client.

Answer

A bug in the model's preprocessing code causing incorrect predictions.

Question 10

An organization is implementing an AI-powered chatbot for customer service. The chatbot must comply with GDPR and handle data subject access requests (DSARs). Which design approach best ensures compliance?

Accepted Answer

Implement an audit trail that logs interactions with a unique user identifier, and provide a mechanism to delete logs upon user request.. GDPR requires that personal data be stored only as long as necessary and that data subjects have the right to erasure. By logging interactions with a unique user identifier and providing a deletion mechanism, the chatbot can fulfill DSARs while maintaining an audit trail for compliance monitoring. This approach balances operational needs with regulatory obligations.

Answer

Minimize data collection by not logging any user interactions.

Answer

Anonymize all user data before logging interactions.

Answer

Encrypt all chat logs and store them indefinitely for audit purposes.

Question 11

A team deploys a real-time fraud detection model on a streaming platform. The model must produce predictions within 100 milliseconds per event. Initial latency is 150 ms. Which optimization is most likely to meet the latency requirement?

Accepted Answer

Apply model quantization to reduce precision from FP32 to INT8.. Model quantization reduces the numerical precision of the model's weights and activations from FP32 to INT8, which decreases memory footprint and speeds up inference. This optimization directly addresses the 150 ms latency by enabling faster arithmetic operations on modern hardware, often cutting inference time by 2-4x, which can bring latency below the 100 ms requirement.

Answer

Increase the batch size to process more events simultaneously.

Answer

Add more feature engineering steps to improve model accuracy.

Answer

Migrate from a decision tree ensemble to a deep neural network.

Question 12

A DevOps team is deploying a machine learning model using a CI/CD pipeline. They want to ensure the model is reproducible and traceable. Which TWO practices should they implement?

Accepted Answer

Version the training dataset and code using Git and DVC.. Versioning the training dataset and code using Git and DVC (Data Version Control) ensures that every model training run can be exactly reproduced by tracking changes to both the source code and the data. DVC stores metadata pointers in Git while the actual data is stored in a remote cache, enabling precise rollback and audit trails. This practice directly supports reproducibility and traceability, which are core requirements for MLOps and CI/CD pipelines.

Answer

Manually deploy the model to production after approval.

Answer

Store only the final model artifact in a shared drive.

Answer

Use a spreadsheet to record model version numbers.

Question 13

An AI operations team is monitoring a deployed image classification model. They notice a gradual increase in prediction confidence but a drop in accuracy. Which THREE actions should they take to diagnose the issue?

Accepted Answer

Analyze the model's calibration curve to see if confidence scores align with actual accuracy.. A calibration curve (reliability diagram) directly compares predicted confidence scores against actual accuracy. In this scenario, increasing confidence with dropping accuracy indicates miscalibration—the model is becoming overconfident. Analyzing the calibration curve reveals whether the confidence scores systematically deviate from true probabilities, which is the core diagnostic step for this specific symptom.

Answer

Increase the size of the training dataset by collecting more unlabeled data.

Answer

Retrain the model immediately with the most recent data.

Question 14

Based on the exhibit, what is the most likely cause of the accuracy drop?

Accepted Answer

Data drift in the 'income' feature has caused the model to become less accurate.. The exhibit shows a sudden and sustained drop in model accuracy coinciding with a shift in the distribution of the 'income' feature. This is a classic symptom of data drift, where the statistical properties of the input feature change over time, causing the model's learned patterns to no longer match the production data. Option B correctly identifies this as the most likely cause because the model was trained on a prior income distribution and is now encountering values outside that range.

Answer

A required feature is missing from the production data pipeline.

Answer

The model was overfitted to the training data.

Answer

The model's confidence threshold needs to be adjusted.

Question 15

You are an AI engineer at a financial services firm. The company has deployed a gradient boosting model to predict loan default risk. The model takes features such as credit score, debt-to-income ratio, loan amount, and employment length. In production, the model processes about 10,000 predictions per day with an average latency of 50ms. Recently, the accuracy has dropped from 92% to 85%. You also notice that the average credit score of applicants has increased significantly because the marketing team launched a campaign targeting prime borrowers. The model was originally trained on data from the past three years, which included a mix of prime and subprime borrowers. You need to restore model performance while minimizing downtime and retraining cost. Which action should you take first?

Accepted Answer

Retrain the model using the last three months of production data with labels.. The drop in accuracy is due to data drift—the production data now has a different distribution (higher credit scores) than the training data. Retraining on the most recent three months of production data with labels directly addresses this shift by adapting the model to the new population, and it minimizes downtime because it uses existing infrastructure and avoids complex architectural changes.

Answer

Add a regularization term to penalize high credit scores.

Answer

Deploy an ensemble of the original model and a neural network.

Answer

Reject all predictions where the confidence score is below 0.9.