AI Implementation and Operations

A company deployed a chatbot using a pre-trained language model. Users report that the chatbot provides incorrect answers to domain-specific questions. Which approach should the AI team prioritize to improve accuracy without retraining the entire model?

An AI system misclassifies rare but critical events. The team considers using synthetic data. Which consideration is MOST important for ensuring the synthetic data improves performance on real rare events?

A data scientist trains a regression model and notices the training loss is low but validation loss is high. Which technique should be applied FIRST to address this issue?

A company deploys an AI model for loan approval. The model shows bias against a protected group. The team decides to use adversarial debiasing. What is the PRIMARY advantage of this approach?

An AIOps platform monitors server metrics and triggers alerts. The team notices too many false positives. Which adjustment should be made to the anomaly detection model?

A team deploys a machine learning model as a REST API. They want to monitor model drift. Which metric is MOST appropriate for detecting drift in the input data distribution?

A company uses an AI system to recommend products. The recommendation accuracy is high, but users complain about lack of diversity. Which strategy should the team adopt to improve diversity without significantly sacrificing accuracy?

A machine learning engineer is deploying a model to production. Which TWO practices are essential for ensuring reproducibility of model predictions?

An organization is implementing an AI governance framework. Which THREE components are essential for compliance with ethical AI standards?

A data scientist is tuning a deep learning model. Which TWO hyperparameters directly affect the model's capacity to overfit?

A team trained a ResNet-50 model with the configuration shown. The high training accuracy and lower validation accuracy suggest overfitting. Which change to the training configuration is MOST likely to reduce overfitting?

An operations team sees the log entries above for a production ML model. What is the MOST likely root cause of the latency spike?

A company deploys a computer vision model for quality inspection on a manufacturing line. After deployment, the model's accuracy drops from 95% to 80% over two weeks. Which action is most likely to address this issue?

An organization is implementing an AI-powered chatbot for customer service. The chatbot must comply with GDPR and handle data subject access requests (DSARs). Which design approach best ensures compliance?

A data scientist fine-tunes a large language model for a legal document summarization task. After fine-tuning, the model performs well on test data but produces summaries that include hallucinated legal clauses. Which mitigation strategy is most effective?

A team deploys a real-time fraud detection model on a streaming platform. The model must produce predictions within 100 milliseconds per event. Initial latency is 150 ms. Which optimization is most likely to meet the latency requirement?

A DevOps team is deploying a machine learning model using a CI/CD pipeline. They want to ensure the model is reproducible and traceable. Which TWO practices should they implement?

An AI operations team is monitoring a deployed image classification model. They notice a gradual increase in prediction confidence but a drop in accuracy. Which THREE actions should they take to diagnose the issue?

Based on the exhibit, what is the most likely cause of the accuracy drop?

You are an AI engineer at a financial services firm. The company has deployed a gradient boosting model to predict loan default risk. The model takes features such as credit score, debt-to-income ratio, loan amount, and employment length. In production, the model processes about 10,000 predictions per day with an average latency of 50ms. Recently, the accuracy has dropped from 92% to 85%. You also notice that the average credit score of applicants has increased significantly because the marketing team launched a campaign targeting prime borrowers. The model was originally trained on data from the past three years, which included a mix of prime and subprime borrowers. You need to restore model performance while minimizing downtime and retraining cost. Which action should you take first?

A company deployed a machine learning model on a cloud inference service. Users report high latency during peak hours. The model is deployed on a single instance. Which action should the team take to reduce latency without significant architectural changes?

A data science team uses Git for version control of model code and DVC for data versioning. They want to implement a model registry to track trained models, their hyperparameters, and performance metrics. Which tool is specifically designed for this purpose and integrates with the existing workflow?

An ML team monitors a production model using a dashboard that shows daily performance metrics. Over the past month, the model's accuracy has dropped from 92% to 87%, while the data distribution of input features has remained stable according to statistical tests. Which type of model drift is most likely occurring?

A company must deploy a new model version with zero downtime. The current model is served via a REST API on a Kubernetes cluster. Which deployment strategy should the team use to gradually shift traffic to the new version while monitoring for errors?

An operations team monitors a classification model in production. The confusion matrix for the model shows the following values: TP=1500, FN=500, FP=600, TN=2400. Which metric should the team calculate to assess the model's ability to avoid false positives?

An ML engineering team has a retraining pipeline that triggers automatically when model accuracy drops below a threshold. Recently, the model's accuracy has been fluctuating, causing frequent retraining and high compute costs. The team suspects the data distribution is changing slowly. Which approach should the team implement to reduce unnecessary retraining while maintaining model performance?

A company deploys an AI model via a REST API that handles sensitive customer data. To secure the endpoint, the security team requires that only authenticated and authorized applications can invoke the API. Which mechanism should be implemented?

A data scientist submits a model training job to a cloud ML platform. The job fails with an error: "Out of memory: Killed process." The training code is proven to work on the developer's local machine with 16GB RAM. The cloud instance has 32GB RAM. What is the most likely cause?

A company serves a large language model (LLM) on a Kubernetes cluster. The inference latency is acceptable but the cost is high due to GPU usage. The model is 7 billion parameters and requires 16GB GPU memory. The team wants to reduce cost without increasing latency. Which strategy should they implement?

A team monitors a production model for bias. They measure the selection rate for two demographic groups and find a significant difference. Which TWO actions should the team take to mitigate bias? (Choose two.)

An ML operations team needs to monitor a deployed model's performance. Which TWO metrics are most useful for detecting concept drift in a regression model? (Choose two.)

An organization wants to implement a robust MLOps pipeline. Which THREE components are essential for a complete MLOps framework? (Choose three.)

A model serving pod is failing with OOMKilled. What is the most likely cause?

An ML team uses the model registry above. After deploying version 3 to production, they discover it has a critical bug. What is the fastest way to roll back to a stable version without re-deploying from scratch?

A developer sees the above error during inference on a deployed image classification model. What is the most likely cause?

A data scientist is deploying a machine learning model to production. The model was trained on an imbalanced dataset. Which technique should be used during deployment to mitigate bias without retraining the model?

An AI system experiences degraded accuracy over time due to changes in user behavior. Which monitoring metric should be prioritized to detect this issue earliest?

A company uses a large language model (LLM) to generate customer support responses. They notice the model sometimes produces harmful outputs. Which implementation strategy best reduces this risk while maintaining performance?

During an AI model deployment, the operations team notices that inference requests are taking longer than expected. Which component is most likely causing the bottleneck?

A data engineering team is designing a pipeline to train a model on streaming data. The data arrives in a time-series format. Which approach should they use to ensure the model reflects current trends without catastrophic forgetting?

A financial institution deploys an AI credit scoring model. After six months, the model's performance drops significantly. Analysis shows that the relationship between features and labels has changed. Which term describes this phenomenon?

An organization is deploying an AI model on edge devices with limited computational resources. Which model optimization technique is most appropriate?

A company uses an AI model to predict equipment failures. The model outputs a probability of failure. To minimize false alarms, the operations team wants a high precision. Which deployment strategy should they implement?

A team is implementing an ML pipeline using a feature store. Which benefit does a feature store primarily provide in an AI operations context?

Which TWO actions should be taken to ensure an AI model complies with GDPR requirements when processing personal data?

Which THREE factors are most critical to consider when designing a continuous integration/continuous deployment (CI/CD) pipeline for machine learning?

An AI system is being implemented in a healthcare setting. Which TWO ethical considerations should be prioritized?

A company deploys a deep learning model for real-time image classification. After deployment, they notice high inference latency exceeding the 100ms SLA. Which action would most likely reduce latency without significantly impacting accuracy?

An AI system for fraud detection shows a gradual decline in precision over several weeks, though recall remains stable. Which type of model drift is most likely occurring?

A data science team uses a CI/CD pipeline for ML models. They need to ensure that each model version is traceable back to the exact training data and hyperparameters. Which practice should be implemented?

A batch inference pipeline fails intermittently with out-of-memory errors when processing large datasets. The pipeline uses pandas DataFrames and feeds a pre-trained model. Which change would most effectively reduce memory consumption?

A company wants to roll out a new recommendation model to production. They decide to run an A/B test where 10% of users see the new model and 90% see the old model. After one week, the new model shows a 5% improvement in click-through rate. What is the next best action?

During model monitoring, a loan approval model shows disparate impact against a protected group. The model's overall accuracy is high, but the false positive rate for the protected group is 0.12 compared to 0.02 for other groups. Which action should the operations team take first?

A real-time recommendation system uses a model retrained daily. The operations team notices that click-through rate drops sharply at 8 AM each day and recovers by noon. The retraining job runs at midnight. What is the most likely cause?

A CI/CD pipeline for a computer vision model uses canary deployment. After deploying a new version to 5% of traffic, the pipeline automatically rolls back due to a spike in error rate. The new model's inference time is 20% higher than the previous version. The operations team finds that the error is caused by timeout in the inference service. Which action should be taken to prevent future rollbacks?

An organization uses a batch prediction pipeline that processes daily customer data to generate marketing recommendations. One month after deployment, the model's performance degrades significantly. The data pipeline logs show that the input data schema has changed — a new categorical feature 'customer_segment' has been added, and the existing feature 'age_group' is now missing. Which step should the operations team take first?

A model serving endpoint is tested using curl commands. Based on the exhibit, what is the most likely issue?

Refer to the exhibit. A machine learning pipeline configuration is shown. During a deployment, the model evaluation passes with accuracy 0.86 and precision 0.79. However, the pipeline proceeds to deploy. What is the most likely reason for this behavior?

Refer to the exhibit. A batch inference job fails with the given logs. What is the most likely root cause of the failure?

Which TWO actions are most appropriate for managing model drift in a production AI system?

Which TWO techniques should be considered when optimizing a deep learning model for deployment on edge devices with limited computational resources?

Which THREE components are essential for implementing a successful MLOps pipeline for a continuously deployed AI system?

A company has developed a deep learning model for image classification. The team wants to deploy the model to production with high availability and scalability. Which approach should they use?

An AI operations team notices that the accuracy of a deployed fraud detection model has been declining over the past month. Which action should the team take to address this issue proactively?

A financial institution needs to integrate an AI-based credit scoring model into an existing mainframe system that processes transactions in COBOL. The model is deployed as a REST API. What is the best strategy to ensure minimal disruption and maintain data integrity?

During model training, the data science team discovers that many input features contain missing values. Which step should be taken to improve data quality?

A healthcare AI system that diagnoses medical images must provide explanations for its predictions to comply with regulatory requirements. Which technique should the team implement?

An e-commerce company deploys a recommendation model that must serve predictions with sub-100 ms latency for millions of users during peak hours. The model is a large neural network. Which architecture is most suitable?

A team of data scientists and engineers is working on multiple AI projects. They often struggle to reproduce experiments and manage model versions. Which tool or practice should they adopt?

An AI system used for hiring has been found to exhibit racial bias against certain candidates. Which step should the organization take to mitigate this?

A company operating in the EU must comply with GDPR. An AI model processes personal data for customer segmentation. Which of the following ensures compliance?

Based on the exhibit, what is the most likely cause of the pod failure and its solution?

Based on the exhibit, which action is permitted by this policy?

Based on the exhibit, which action is most likely to resolve the memory issue?

Which TWO of the following are best practices for monitoring AI models in production?

Which TWO deployment strategies allow for testing a new model version before fully rolling it out?

Which THREE components are essential in an MLOps pipeline?

A data science team deployed a model for real-time predictions. After two weeks, the model's accuracy dropped from 92% to 80%. The monitoring system shows no data drift in features, but the target variable distribution has shifted. Which approach should the team use to detect this issue?

A company is deploying a fraud detection model that must return predictions within 100ms to avoid transaction delays. The team is deciding between batch and real-time inference. Which factor most strongly supports a real-time inference architecture?

An MLOps team uses a CI/CD pipeline to automate model retraining. The pipeline triggers on new labeled data, runs feature engineering, retrains the model, evaluates against a holdout set, and deploys if metrics exceed thresholds. Recently, a retrained model passed validation but caused a 5% accuracy drop in production. Which improvement best prevents this?

Refer to the exhibit. The monitoring dashboard for a deployed churn prediction model shows a drift detected flag. However, the error rate and latency are within acceptable ranges. What is the most appropriate immediate action?

Refer to the exhibit. A team created an access policy for a fraud detection model endpoint. An intern reports being unable to access the model for testing. Reviewing the policy, what is the most likely cause?

A global retailer uses an AI model to forecast demand across thousands of stores. After deployment, the model's predictions become less accurate during holiday seasons. The training data included two years of holiday periods. What is the most effective operational strategy to handle this recurring seasonal drift?

An organization deploys an AI model on edge devices for real-time image classification. Which metric is most important to monitor for ensuring the device's operational health?

A healthcare company must deploy a diagnostic AI model that uses protected health information (PHI). To comply with HIPAA, the operations team needs to ensure data privacy during model inference. Which practice should be implemented?

An MLOps team automates model deployment with a CI/CD pipeline. A performance regression is detected after deploying a new model version. The team needs to automatically roll back to the previous version. Which approach best enables safe automated rollback?

Which TWO are best practices for deploying AI models in a containerized production environment? (Select TWO.)

Which TWO techniques are most effective for ensuring model explainability in a production loan approval AI system subject to regulatory review? (Select TWO.)

Which THREE are common pitfalls when operationalizing AI models? (Select THREE.)

A large financial services company deploys multiple AI models on a shared Kubernetes cluster with GPU nodes. The models serve real-time fraud detection and credit scoring. Recently, the operations team observed frequent out-of-memory (OOM) errors during peak hours, causing inference failures. The monitoring dashboards show GPU memory utilization averaging 90% during peak times, and pods are being evicted. The team has allocated 8GB per pod and the total cluster GPU memory is 32GB. The models require at least 4GB each, but the fraud detection model occasionally spikes to 7GB. Which course of action best resolves the OOM errors while maintaining high availability?

An e-commerce company uses a machine learning model to recommend products to users. The model is retrained weekly and deployed to production. For the past three weeks, the model's click-through rate (CTR) has been stable except on Mondays, when it drops by 15%. Analysis reveals that the training data is extracted on Sundays and includes only weekday behavior. On Mondays, user behavior shifts due to weekend browsing patterns not captured in the training data. The team wants to maintain a weekly retraining cadence but fix the Monday performance drop. Which solution best addresses the Monday CTR drop without changing the retraining frequency?

A startup has developed a natural language processing model for sentiment analysis. Their CI/CD pipeline includes a step that runs unit tests on the model's output format and a validation step that checks accuracy on a static test dataset. Recently, the pipeline often fails during the validation step, but the failures are inconsistent—sometimes the same model version passes, sometimes fails. The team suspects the test dataset is small and randomly sampled. They need a reliable validation process to deploy models with confidence. Which approach should the team implement?

A company deploys a machine learning model for fraud detection. After one month, the false positive rate has increased significantly. The model is retrained weekly on all historical data. What is the MOST effective immediate action?

A data scientist is monitoring a deployed image classification model. Which TWO actions are best practices for detecting model drift? (Choose 2.)

A deployed NLP sentiment analysis model experiences a sharp decline in accuracy on customer reviews. The team has verified the input data format and pipeline are correct. Which THREE actions should be taken to diagnose and remediate? (Choose 3.)

A large e-commerce company has deployed a real-time product recommendation system using a neural collaborative filtering model. The model was trained on six months of user click and purchase data. For the first three months after deployment, the click-through rate (CTR) improved by 15%. However, starting in the fourth month, CTR began decreasing steadily despite no changes to the system infrastructure or data pipeline. The product manager suspects model decay but the engineering team insists the model is static and should not degrade. The data science lead suggests investigating further. They have access to production logs, A/B testing framework, and historical model versions. What is the BEST course of action to diagnose and address the issue?

A financial institution uses a machine learning model to approve personal loans. The model was trained on historical data that includes applicant age, income, credit score, and loan amount. Compliance officers have received customer complaints suggesting the model may be discriminating against applicants over 60 years old. Initial analysis shows that the approval rate for applicants over 60 is 20 percentage points lower than for younger applicants with similar credit profiles. The data science team has been asked to investigate and remediate any bias. They have access to the training data, model coefficients, and can retrain or modify the model. What is the FIRST step the team should take?

A small business launched a customer support chatbot powered by a pre-trained language model. The chatbot was fine-tuned on a dataset of past support tickets. For the first week, it performed well, accurately answering 85% of queries. After a routine software update that included a new version of the underlying language model library, the chatbot's accuracy dropped to 60% and it began giving nonsensical responses to some questions. The update did not change any code or configuration specific to the chatbot. The business has a backup of the previous environment. What is the MOST appropriate immediate action?

A healthcare AI startup has developed a model to detect diabetic retinopathy from retinal images. The model achieved 96% sensitivity and 94% specificity on a validation set from the same distribution as the training data. After deployment in a rural clinic, the model's sensitivity drops to 80%. The data team analyzes the clinical images from the clinic and finds that the images have lower resolution and different lighting conditions compared to the training dataset. The team has the ability to collect more data from the clinic and retrain the model. What is the BEST course of action?

A retail company uses a time-series model to forecast daily sales for inventory management. The model is a seasonal ARIMA trained on three years of daily data. It performed well during initial validation but after deployment, forecasts became inaccurate during holiday seasons, often underestimating demand by up to 40%. The data science team examined the features and found that the training data did not include any holiday indicators. They also discovered that the model's residuals show strong autocorrelation during holiday weeks. The company needs to improve the forecast for the upcoming holiday season. They have access to historical sales data with holiday dates and are considering several approaches. Which approach will BEST address the issue?

A financial services firm has deployed an AI model for real-time credit scoring. The operations team needs to ensure the model remains reliable and compliant over time. Which TWO actions should the team prioritize? (Choose two.)

The exhibit shows the output of a drift monitoring command for a fraud detection model. The team has an automated pipeline that triggers retraining when the overall average drift score exceeds 0.10. Based on the exhibit, what should the operations team do next?

A hospital's radiology department uses an AI model to detect lung nodules in CT scans. The model was trained on data from a specific brand of scanners and patient demographics common in Europe. Recently, the hospital acquired new scanners from a different manufacturer and started serving a more diverse patient population. Over the past month, the model's false-positive rate has increased by 15% and false-negative rate by 8%. The radiologists are losing confidence and are considering abandoning the AI tool altogether. The IT team has verified that the model inference is running correctly and the hardware is performing as expected. The data science team suspects the problem is related to the change in input data distribution. The hospital's AI operations policy requires that any model update must be validated on at least 500 recent cases before deployment. What is the BEST course of action for the AI operations team?