AI0-001 AI Implementation and Operations Practice Test 5 — 15 Questions

Question 1

During model monitoring, a loan approval model shows disparate impact against a protected group. The model's overall accuracy is high, but the false positive rate for the protected group is 0.12 compared to 0.02 for other groups. Which action should the operations team take first?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Retrain the model with reweighted training data to minimize disparity. Retraining the model with reweighted training data directly addresses the root cause of disparate impact—biased historical data—by assigning higher weights to underrepresented groups during training. This technique, often implemented via cost-sensitive learning or sample reweighting, adjusts the model's internal decision boundaries to reduce false positive rate disparities without sacrificing overall accuracy. The operations team should first attempt to mitigate bias at the data level before considering threshold adjustments or model replacement, as reweighting preserves the model's learned patterns while promoting fairness.

Answer

Document the disparity and proceed with deployment because accuracy is high

Answer

Replace the model with a simpler model that is less discriminatory

Answer

Adjust the decision threshold for the protected group to equalize false positive rates

Question 2

A real-time recommendation system uses a model retrained daily. The operations team notices that click-through rate drops sharply at 8 AM each day and recovers by noon. The retraining job runs at midnight. What is the most likely cause?

Accepted Answer

There is data drift due to morning user patterns not seen in training. The sharp drop in click-through rate at 8 AM, followed by recovery by noon, strongly indicates data drift caused by a shift in user behavior patterns during morning hours. Since the model is retrained at midnight using data that predominantly captures late-night user behavior, it fails to generalize to the distinct morning user patterns (e.g., different browsing habits, content preferences). This is a classic example of temporal data drift where the training distribution does not match the inference distribution at specific times of day.

Answer

The model overfits to late-night user behavior

Answer

The model suffers from catastrophic forgetting due to daily retraining

Answer

The retraining pipeline has a bug that only affects morning predictions

Question 3

An organization uses a batch prediction pipeline that processes daily customer data to generate marketing recommendations. One month after deployment, the model's performance degrades significantly. The data pipeline logs show that the input data schema has changed — a new categorical feature 'customer_segment' has been added, and the existing feature 'age_group' is now missing. Which step should the operations team take first?

Accepted Answer

Update the data preprocessing pipeline to handle missing features and add the new feature. The immediate priority is to ensure the data preprocessing pipeline can handle the schema change without breaking. The pipeline must gracefully handle the missing 'age_group' feature (e.g., by imputing or dropping it) and incorporate the new 'customer_segment' feature before any model retraining or rollback. This prevents data drift from causing inference errors and maintains pipeline stability.

Answer

Retrain the model using the new schema and redeploy

Answer

Revert to the previous week's model version that was performing well

Answer

Contact the data engineering team to revert the schema change

Question 4

A model serving endpoint is tested using curl commands. Based on the exhibit, what is the most likely issue?

Accepted Answer

The model is experiencing intermittent high latency leading to timeouts. The exhibit shows that the first curl request succeeds (HTTP 200), but subsequent requests fail with 'curl: (28) Operation timed out' after the default timeout of 30 seconds. This pattern of intermittent success followed by timeouts is characteristic of a model experiencing high latency spikes, not a persistent server error or configuration issue. The server is reachable and the model responds correctly some of the time, ruling out deployment or malformed input issues.

Answer

The server is returning HTTP 500 errors

Answer

The input features are malformed

Answer

The model is not deployed on the server

Question 5

Refer to the exhibit. A machine learning pipeline configuration is shown. During a deployment, the model evaluation passes with accuracy 0.86 and precision 0.79. However, the pipeline proceeds to deploy. What is the most likely reason for this behavior?

Accepted Answer

The deployment only checks the accuracy threshold for rollback condition. The pipeline configuration shows a rollback condition that only checks the accuracy metric (accuracy < 0.85). Since the model achieved accuracy 0.86, which is above the threshold, the condition is not triggered, and the pipeline proceeds to deploy regardless of the precision value. The precision metric is not part of the rollback evaluation logic in this configuration.

Answer

The precision metric is not included in the evaluation script

Answer

The deployment target is set to staging instead of production

Answer

The operator manually overrode the threshold

Question 6

Refer to the exhibit. A batch inference job fails with the given logs. What is the most likely root cause of the failure?

Accepted Answer

The input data contains missing values that are not handled in preprocessing. The logs indicate a 'ValueError' or similar exception when the batch inference job attempts to process the input data. This error typically arises when the preprocessing pipeline encounters missing values (e.g., NaN or None) that it cannot handle, causing the job to fail. Option B is correct because missing values not handled in preprocessing are a common root cause for such failures, especially when the training data had no missing values but the inference data does.

Answer

The input data has values that exceed the model's expected range

Answer

The model was not trained to handle categorical features

Answer

The model version is outdated and incompatible with the current preprocessing pipeline

Question 7

Which TWO actions are most appropriate for managing model drift in a production AI system?

Accepted Answer

Periodically retrain the model on recent data. Periodically retraining the model on recent data is a fundamental strategy to combat model drift, ensuring the model adapts to changes in the underlying data distribution (e.g., concept drift or covariate shift). This aligns with MLOps best practices for maintaining model accuracy over time in production AI systems.

Answer

Freeze the model to prevent any changes

Answer

Roll back to a previous model version if performance degrades

Answer

Manually review all model predictions

Question 8

Which TWO techniques should be considered when optimizing a deep learning model for deployment on edge devices with limited computational resources?

Accepted Answer

Model quantization. Model quantization reduces the precision of the model's weights and activations (e.g., from 32-bit floating point to 8-bit integer), which significantly decreases memory footprint and computational latency. This makes it a primary technique for deploying deep learning models on edge devices with limited resources.

Answer

Apply adversarial training

Answer

Use a GPU for inference

Answer

Increase the number of layers

Question 9

Which THREE components are essential for implementing a successful MLOps pipeline for a continuously deployed AI system?

Accepted Answer

Model registry for version control and metadata management. A model registry (C) is essential for MLOps because it provides version control, metadata management, and lineage tracking for all trained models. This enables reproducibility, auditability, and seamless rollback in a continuously deployed AI system, ensuring that only validated models are promoted to production.

Answer

Manual approval gates for each deployment

Answer

Canary deployment strategy

Question 10

A company has developed a deep learning model for image classification. The team wants to deploy the model to production with high availability and scalability. Which approach should they use?

Accepted Answer

Use a containerized approach with Kubernetes.. Containerization with Kubernetes provides the orchestration, auto-scaling, and self-healing capabilities required for high availability and scalability in production. Kubernetes manages container lifecycles, distributes traffic across replicas via Services and Ingress controllers, and can automatically scale pods based on CPU/memory metrics or custom metrics, ensuring the deep learning model handles variable loads without downtime.

Answer

Run the model on a laptop during business hours.

Answer

Deploy the model as a monolithic application on a single server.

Answer

Embed the model directly into a mobile app.

Question 11

An AI operations team notices that the accuracy of a deployed fraud detection model has been declining over the past month. Which action should the team take to address this issue proactively?

Accepted Answer

Set up automated retraining pipeline triggered by performance degradation thresholds.. It establishes an automated retraining pipeline triggered by performance degradation thresholds, which aligns with MLOps best practices for maintaining model accuracy in production. This proactive approach ensures the model is retrained when its performance drops below a predefined metric (e.g., AUC or F1 score), without requiring manual intervention. It addresses concept drift, which is a common cause of declining accuracy in deployed fraud detection models.

Answer

Retrain the model with the most recent data immediately.

Answer

Manually update the model weights weekly.

Answer

Replace the model with a rule-based system.

Question 12

A financial institution needs to integrate an AI-based credit scoring model into an existing mainframe system that processes transactions in COBOL. The model is deployed as a REST API. What is the best strategy to ensure minimal disruption and maintain data integrity?

Accepted Answer

Use an API gateway with versioning and circuit breaker patterns.. An API gateway with versioning and circuit breaker patterns allows the COBOL mainframe to call the REST API without modifying its core transaction logic. The gateway handles protocol translation, rate limiting, and failover, ensuring minimal disruption to the existing mainframe system while maintaining data integrity through transactional consistency and graceful degradation.

Answer

Copy all transaction data to a cloud database for the model to access.

Answer

Rewrite the mainframe system in Java to directly call the model.

Answer

Install a GPU on the mainframe to run the model natively.

Question 13

During model training, the data science team discovers that many input features contain missing values. Which step should be taken to improve data quality?

Accepted Answer

Implement data validation checks to handle missing data appropriately (e.g., imputation).. Data validation checks, such as imputation (e.g., mean, median, or KNN imputation), directly address missing values by estimating plausible replacements based on the available data. This improves data quality and prevents bias or loss of information that could degrade model performance. In the context of AI implementation, handling missing data is a fundamental data preprocessing step to ensure robust model training.

Answer

Increase the model complexity to handle missing data.

Answer

Ignore missing values and train the model.

Answer

Remove all records with missing values.

Question 14

An e-commerce company deploys a recommendation model that must serve predictions with sub-100 ms latency for millions of users during peak hours. The model is a large neural network. Which architecture is most suitable?

Accepted Answer

Use a distributed system with load balancers and model replicas.. Distributing the model across multiple servers with load balancers and replicas allows horizontal scaling to handle millions of concurrent users while maintaining sub-100 ms latency. This architecture provides fault tolerance and can dynamically adjust to peak traffic loads, which is essential for real-time inference with large neural networks.

Answer

Batch process predictions every hour.

Answer

Deploy the model on a single powerful GPU server.

Answer

Use serverless functions with auto-scaling.

Question 15

A team of data scientists and engineers is working on multiple AI projects. They often struggle to reproduce experiments and manage model versions. Which tool or practice should they adopt?

Accepted Answer

Use an MLOps platform that provides version control, tracking, and reproducibility.. An MLOps platform (e.g., MLflow, Kubeflow, or Vertex AI) provides integrated version control for code, data, and models, along with experiment tracking and reproducibility. This directly addresses the team's struggle to reproduce experiments and manage model versions by automating lineage capture and enabling consistent environment recreation.

Answer

Document experiments in a shared Word document.

Answer

Share code via email attachments.

Answer

Keep all models in a shared network drive.