AI Implementation and Operations

An e-commerce company operates an AI recommendation service. After a marketing campaign, the operations team notices that inference costs have tripled while request volume has only doubled. They need to reduce cost per inference without degrading recommendation quality. Which two actions should the team take? (Choose two.)

A hospital has deployed an AI triage assistant that summarizes patient intake notes and suggests an acuity level for the emergency department. Clinicians report that the assistant sometimes produces confident but unsupported acuity suggestions. The operations team must add safeguards appropriate for a high-stakes clinical deployment. (Choose two.)

Refer to the exhibit. A machine learning pipeline configuration is shown. During a deployment, the model evaluation passes with accuracy 0.86 and precision 0.79. However, the pipeline proceeds to deploy. What is the most likely reason for this behavior?

An insurance company operates an AI claims-triage model that flags suspicious claims for human review. After six months in production, the operations team observes that the model's precision has fallen steadily while recall has stayed roughly constant, and the volume of false-positive flags has grown. The data science team suspects the input data pipeline is the cause rather than the model weights. Which TWO operational checks should the team perform first to diagnose the problem? (Choose two.)

A company uses a large language model (LLM) to generate customer support responses. They notice the model sometimes produces harmful outputs. Which implementation strategy best reduces this risk while maintaining performance?

A company deploys a computer vision model for quality inspection on a manufacturing line. After deployment, the model's accuracy drops from 95% to 80% over two weeks. Which action is most likely to address this issue?

An AI system for fraud detection shows a gradual decline in precision over several weeks, though recall remains stable. Which type of model drift is most likely occurring?

A media company serves personalized article recommendations through a model hosted on a cloud inference service. During a major news event, request volume spikes tenfold and p95 latency rises from 120 ms to over 2 seconds, causing timeouts on the web front end. The model itself is unchanged and the endpoint is healthy. The team wants to keep serving personalized results during spikes without degrading the user experience. Which action should the team take first?

The exhibit shows the output of a drift monitoring command for a fraud detection model. The team has an automated pipeline that triggers retraining when the overall average drift score exceeds 0.10. Based on the exhibit, what should the operations team do next?

A data scientist is deploying a machine learning model to production. The model was trained on an imbalanced dataset. Which technique should be used during deployment to mitigate bias without retraining the model?

A media company serves personalized article recommendations through a model that is retrained weekly. After a major news event, engagement metrics show that recommendations became stale within hours because the model had not yet seen the new topic. The engineering team wants recommendations to reflect breaking topics within minutes without retraining the whole model. Which approach should the team implement?

A financial services firm deploys a credit-scoring model that must produce explanations for adverse action notices. The compliance team requires that each decision be traceable to the exact model version, input features, and the explanation method used at inference time. The data science team currently logs only predictions and timestamps. Which approach best satisfies the traceability requirement?

Based on the exhibit, what is the most likely cause of the pod failure and its solution?

A retail company's ML platform team notices that one of their production models has begun returning predictions with a drastically different distribution than during training. The monitoring dashboard shows the input feature distributions have shifted but no code or model artifacts have changed. The team wants to automatically trigger a retraining pipeline when this condition is detected. Which approach should they implement?

An AI system used for hiring has been found to exhibit racial bias against certain candidates. Which step should the organization take to mitigate this?

An organization is implementing an AI governance framework. Which THREE components are essential for compliance with ethical AI standards?

A global retailer uses an AI model to forecast demand across thousands of stores. After deployment, the model's predictions become less accurate during holiday seasons. The training data included two years of holiday periods. What is the most effective operational strategy to handle this recurring seasonal drift?

A retail company's demand-forecasting model was trained on three years of sales data. After a major competitor closes, regional purchasing patterns shift sharply within two weeks, and forecast error spikes. The operations team wants to detect this kind of abrupt change quickly and trigger a review. Which practice best addresses this requirement?

A large e-commerce company has deployed a real-time product recommendation system using a neural collaborative filtering model. The model was trained on six months of user click and purchase data. For the first three months after deployment, the click-through rate (CTR) improved by 15%. However, starting in the fourth month, CTR began decreasing steadily despite no changes to the system infrastructure or data pipeline. The product manager suspects model decay but the engineering team insists the model is static and should not degrade. The data science lead suggests investigating further. They have access to production logs, A/B testing framework, and historical model versions. What is the BEST course of action to diagnose and address the issue?

A batch inference pipeline fails intermittently with out-of-memory errors when processing large datasets. The pipeline uses pandas DataFrames and feeds a pre-trained model. Which change would most effectively reduce memory consumption?

A media company serves a generative AI assistant to customers through an API. After an update to the system prompt, users begin reporting that the assistant produces responses outside the company's approved tone and occasionally reveals parts of its internal instructions. The operations team must add safeguards that reduce these behaviors in production. (Choose two.)

A logistics company runs an AI route-optimization service that calls a hosted large language model to interpret free-text driver notes and convert them into structured stop instructions. The service works in testing, but in production many requests fail with rate-limit and timeout errors during the morning dispatch window. The team wants the service to survive these failures without losing driver instructions. Which approach should the team implement?

A retail company's demand-forecasting model has been running in production for eight months. Data scientists notice that prediction error has slowly increased, and statistical tests show the distribution of weekly sales figures has shifted relative to the training data, while the model code and pipeline are unchanged. Which phenomenon best describes this situation?

Which THREE factors are most critical to consider when designing a continuous integration/continuous deployment (CI/CD) pipeline for machine learning?

A company wants to roll out a new recommendation model to production. They decide to run an A/B test where 10% of users see the new model and 90% see the old model. After one week, the new model shows a 5% improvement in click-through rate. What is the next best action?

Which THREE are common pitfalls when operationalizing AI models? (Select THREE.)

A retail bank operates a real-time AI service that approves or declines card transactions in under 100 ms. During a marketing campaign, transaction volume triples and the inference service's p99 latency rises to 1.4 seconds, causing checkout timeouts. The model is unchanged and CPU utilization on the inference nodes is only 35%. Which action BEST addresses the latency increase while preserving the sub-100 ms requirement?

A hospital's AI triage assistant occasionally returns confident but incorrect recommendations when it encounters patient records with missing lab values. The clinical team wants the system to avoid acting on unreliable inputs until a human reviews them. Which operational control best addresses this need?

A financial institution deploys an AI credit scoring model. After six months, the model's performance drops significantly. Analysis shows that the relationship between features and labels has changed. Which term describes this phenomenon?

An ML operations team needs to monitor a deployed model's performance. Which TWO metrics are most useful for detecting concept drift in a regression model? (Choose two.)

Which THREE components are essential for implementing a successful MLOps pipeline for a continuously deployed AI system?

Based on the exhibit, which action is most likely to resolve the memory issue?

A financial institution uses a machine learning model to approve personal loans. The model was trained on historical data that includes applicant age, income, credit score, and loan amount. Compliance officers have received customer complaints suggesting the model may be discriminating against applicants over 60 years old. Initial analysis shows that the approval rate for applicants over 60 is 20 percentage points lower than for younger applicants with similar credit profiles. The data science team has been asked to investigate and remediate any bias. They have access to the training data, model coefficients, and can retrain or modify the model. What is the FIRST step the team should take?

A data scientist fine-tunes a large language model for a legal document summarization task. After fine-tuning, the model performs well on test data but produces summaries that include hallucinated legal clauses. Which mitigation strategy is most effective?

During an AI model deployment, the operations team notices that inference requests are taking longer than expected. Which component is most likely causing the bottleneck?

An organization is deploying an AI model on edge devices with limited computational resources. Which model optimization technique is most appropriate?

During model training, the data science team discovers that many input features contain missing values. Which step should be taken to improve data quality?

A hospital's AI triage assistant produces recommendations that clinicians frequently override. An operations review finds the model was trained on data from a different patient population than the one currently served. Which action most directly addresses the root cause of the low acceptance rate?

A company deploys a deep learning model for real-time image classification. After deployment, they notice high inference latency exceeding the 100ms SLA. Which action would most likely reduce latency without significantly impacting accuracy?

A company uses an AI model to predict equipment failures. The model outputs a probability of failure. To minimize false alarms, the operations team wants a high precision. Which deployment strategy should they implement?

An ML engineering team has a retraining pipeline that triggers automatically when model accuracy drops below a threshold. Recently, the model's accuracy has been fluctuating, causing frequent retraining and high compute costs. The team suspects the data distribution is changing slowly. Which approach should the team implement to reduce unnecessary retraining while maintaining model performance?

A machine learning engineer is deploying a model to production. Which TWO practices are essential for ensuring reproducibility of model predictions?

A company is deploying a fraud detection model that must return predictions within 100ms to avoid transaction delays. The team is deciding between batch and real-time inference. Which factor most strongly supports a real-time inference architecture?

Refer to the exhibit. The monitoring dashboard for a deployed churn prediction model shows a drift detected flag. However, the error rate and latency are within acceptable ranges. What is the most appropriate immediate action?

A retail bank runs a batch credit-limit model that scores the entire customer base nightly. The model consumes 40 features, several of which are aggregates computed from transaction history. Downstream systems report that scores for some customers change dramatically between consecutive nights even though nothing about those customers changed. The team needs to make the nightly pipeline reproducible and explainable. Which action should the team take FIRST?

A retailer's recommendation service runs on a managed inference endpoint. During a flash sale, request volume triples and the endpoint's response time exceeds the acceptable threshold. The operations team must reduce latency quickly without retraining the model. Which action should they take first?

A team deploys a machine learning model as a REST API. They want to monitor model drift. Which metric is MOST appropriate for detecting drift in the input data distribution?

Which TWO actions are most appropriate for managing model drift in a production AI system?

A financial services firm has deployed an AI model for real-time credit scoring. The operations team needs to ensure the model remains reliable and compliant over time. Which TWO actions should the team prioritize? (Choose two.)

A healthcare AI startup has developed a model to detect diabetic retinopathy from retinal images. The model achieved 96% sensitivity and 94% specificity on a validation set from the same distribution as the training data. After deployment in a rural clinic, the model's sensitivity drops to 80%. The data team analyzes the clinical images from the clinic and finds that the images have lower resolution and different lighting conditions compared to the training dataset. The team has the ability to collect more data from the clinic and retrain the model. What is the BEST course of action?

An MLOps team uses a CI/CD pipeline to automate model retraining. The pipeline triggers on new labeled data, runs feature engineering, retrains the model, evaluates against a holdout set, and deploys if metrics exceed thresholds. Recently, a retrained model passed validation but caused a 5% accuracy drop in production. Which improvement best prevents this?

An e-commerce company uses a machine learning model to recommend products to users. The model is retrained weekly and deployed to production. For the past three weeks, the model's click-through rate (CTR) has been stable except on Mondays, when it drops by 15%. Analysis reveals that the training data is extracted on Sundays and includes only weekday behavior. On Mondays, user behavior shifts due to weekend browsing patterns not captured in the training data. The team wants to maintain a weekly retraining cadence but fix the Monday performance drop. Which solution best addresses the Monday CTR drop without changing the retraining frequency?

An AI operations team is designing a rollback strategy for a fraud-detection model served behind a feature flag. A new model version shows degraded precision after release. The team wants to restore the previous behavior within minutes without redeploying code or losing the ability to collect data on the new version. Which approach best meets these requirements?

A healthcare AI system that diagnoses medical images must provide explanations for its predictions to comply with regulatory requirements. Which technique should the team implement?

A logistics company wants to detect packages damaged in transit by analyzing photos taken at warehouse checkpoints. The team has only about 200 labeled examples of damaged packages but tens of thousands of photos of undamaged ones. They need a working classifier quickly and cannot collect more damaged-package images in the near term. Which approach should the team use?

An operations team is preparing to deploy a new AI inference service. Security leadership requires that all data in transit between the application and the model endpoint be encrypted and that clients be authenticated before they can submit inference requests. Which combination of controls should the team implement?

A startup has developed a natural language processing model for sentiment analysis. Their CI/CD pipeline includes a step that runs unit tests on the model's output format and a validation step that checks accuracy on a static test dataset. Recently, the pipeline often fails during the validation step, but the failures are inconsistent—sometimes the same model version passes, sometimes fails. The team suspects the test dataset is small and randomly sampled. They need a reliable validation process to deploy models with confidence. Which approach should the team implement?

A model serving endpoint is tested using curl commands. Based on the exhibit, what is the most likely issue?

An ML team monitors a production model using a dashboard that shows daily performance metrics. Over the past month, the model's accuracy has dropped from 92% to 87%, while the data distribution of input features has remained stable according to statistical tests. Which type of model drift is most likely occurring?

Which TWO are best practices for deploying AI models in a containerized production environment? (Select TWO.)

An AI team is preparing a fraud-detection model for production deployment. The model performs well offline, but the team must ensure the deployment is operationally safe and that problems are detected quickly after release. Which TWO practices should be implemented as part of the pre-deployment and post-deployment plan? (Choose two.)

A data scientist is monitoring a deployed image classification model. Which TWO actions are best practices for detecting model drift? (Choose 2.)

A financial services firm runs a credit-scoring AI model in production. The compliance team wants continuous assurance that the deployed model still meets performance and fairness expectations as customer behavior changes. Which TWO operational practices best support this goal? (Choose two.)

A retail company runs a demand-forecasting model in production. Over three weeks, the average order value of incoming transactions has risen by 40 percent because of a promotional campaign, and forecast error has grown steadily. The model was trained on twelve months of historical data with no promotion periods. Which action should the operations team take FIRST to restore forecast reliability?

Which TWO actions should be taken to ensure an AI model complies with GDPR requirements when processing personal data?

A data science team uses Git for version control of model code and DVC for data versioning. They want to implement a model registry to track trained models, their hyperparameters, and performance metrics. Which tool is specifically designed for this purpose and integrates with the existing workflow?

An AI operations team is monitoring a deployed image classification model. They notice a gradual increase in prediction confidence but a drop in accuracy. Which THREE actions should they take to diagnose the issue?

A financial institution needs to integrate an AI-based credit scoring model into an existing mainframe system that processes transactions in COBOL. The model is deployed as a REST API. What is the best strategy to ensure minimal disruption and maintain data integrity?

Which TWO techniques should be considered when optimizing a deep learning model for deployment on edge devices with limited computational resources?

A hospital's radiology department uses an AI model to detect lung nodules in CT scans. The model was trained on data from a specific brand of scanners and patient demographics common in Europe. Recently, the hospital acquired new scanners from a different manufacturer and started serving a more diverse patient population. Over the past month, the model's false-positive rate has increased by 15% and false-negative rate by 8%. The radiologists are losing confidence and are considering abandoning the AI tool altogether. The IT team has verified that the model inference is running correctly and the hardware is performing as expected. The data science team suspects the problem is related to the change in input data distribution. The hospital's AI operations policy requires that any model update must be validated on at least 500 recent cases before deployment. What is the BEST course of action for the AI operations team?

A retail bank deploys an AI model that approves or declines small-business loan applications. Regulators require the bank to explain any adverse decision to the applicant in plain language. The model is a gradient-boosted ensemble over dozens of features, and the bank's data scientists cannot easily describe why a specific applicant was declined. Which approach best satisfies the regulatory requirement?

A small business launched a customer support chatbot powered by a pre-trained language model. The chatbot was fine-tuned on a dataset of past support tickets. For the first week, it performed well, accurately answering 85% of queries. After a routine software update that included a new version of the underlying language model library, the chatbot's accuracy dropped to 60% and it began giving nonsensical responses to some questions. The update did not change any code or configuration specific to the chatbot. The business has a backup of the previous environment. What is the MOST appropriate immediate action?

A logistics company runs an AI route-optimization model on a cloud inference endpoint. The model receives 200 requests per second during business hours and 20 requests per second at night. The operations team wants to reduce cost without violating the 200 ms p95 latency SLA, and they observe that provisioned capacity is sized for peak load. Which approach is MOST appropriate?

A DevOps team is deploying a machine learning model using a CI/CD pipeline. They want to ensure the model is reproducible and traceable. Which TWO practices should they implement?

An AI system is being implemented in a healthcare setting. Which TWO ethical considerations should be prioritized?

A team trained a ResNet-50 model with the configuration shown. The high training accuracy and lower validation accuracy suggest overfitting. Which change to the training configuration is MOST likely to reduce overfitting?

A model serving pod is failing with OOMKilled. What is the most likely cause?

A company deploys an AI model via a REST API that handles sensitive customer data. To secure the endpoint, the security team requires that only authenticated and authorized applications can invoke the API. Which mechanism should be implemented?

A support team deploys a retrieval-augmented generation assistant that answers questions from internal policy documents. Users report that the assistant confidently invents policy details that do not appear in any document. The team wants to reduce these fabricated answers without retraining the language model. Which change is MOST effective?

An AIOps platform monitors server metrics and triggers alerts. The team notices too many false positives. Which adjustment should be made to the anomaly detection model?

A company deployed a machine learning model on a cloud inference service. Users report high latency during peak hours. The model is deployed on a single instance. Which action should the team take to reduce latency without significant architectural changes?

A team deploys a real-time fraud detection model on a streaming platform. The model must produce predictions within 100 milliseconds per event. Initial latency is 150 ms. Which optimization is most likely to meet the latency requirement?

A financial services firm runs a real-time credit-scoring model on an Amazon SageMaker endpoint. The model must not degrade: the team needs automatic detection of distributional drift in the incoming feature data and an alert when drift exceeds a threshold, without retraining the model. Which SageMaker capability should they configure?

Which THREE components are essential in an MLOps pipeline?

A financial services firm runs a credit-scoring model in production on a managed cloud inference endpoint. Over three months, the input distribution of applicant income has shifted substantially because of a regional economic downturn, and the model's predictions have become systematically lower than actual repayment outcomes. The MLOps team needs an operational mechanism that will detect this change automatically and raise an alert before business metrics degrade further. Which approach should the team implement?

A team of data scientists and engineers is working on multiple AI projects. They often struggle to reproduce experiments and manage model versions. Which tool or practice should they adopt?

A media company uses a generative AI assistant to draft customer responses. After an update to the underlying foundation model, agents report that responses sometimes include fabricated policy details. The operations team must detect this regression quickly and prevent fabricated content from reaching customers. Which combination of controls is most appropriate?

A CI/CD pipeline for a computer vision model uses canary deployment. After deploying a new version to 5% of traffic, the pipeline automatically rolls back due to a spike in error rate. The new model's inference time is 20% higher than the previous version. The operations team finds that the error is caused by timeout in the inference service. Which action should be taken to prevent future rollbacks?

An organization deploys an AI model on edge devices for real-time image classification. Which metric is most important to monitor for ensuring the device's operational health?

An operations team runs a real-time fraud-scoring model behind a REST endpoint. Latency is acceptable, but over three weeks the model's predicted positive rate has drifted upward even though the model binary and the feature-extraction code have not changed. The team wants to detect and localize this drift before it degrades business outcomes. Which approach should the team implement?

A company must deploy a new model version with zero downtime. The current model is served via a REST API on a Kubernetes cluster. Which deployment strategy should the team use to gradually shift traffic to the new version while monitoring for errors?

An AI operations team supports a model that scores insurance claims in real time. They need to detect when the live input distribution diverges from the training distribution and alert before claim decisions degrade. Which approach should they implement?

An organization is implementing an AI-powered chatbot for customer service. The chatbot must comply with GDPR and handle data subject access requests (DSARs). Which design approach best ensures compliance?

A data science team uses a CI/CD pipeline for ML models. They need to ensure that each model version is traceable back to the exact training data and hyperparameters. Which practice should be implemented?

A hospital deploys an AI model that summarizes clinical notes for physicians. Before go-live, the AI team must verify that the model does not reproduce patient identifiers in its summaries when they are not clinically necessary. Which activity is the MOST appropriate for this verification?

Which TWO of the following are best practices for monitoring AI models in production?

A team is implementing an ML pipeline using a feature store. Which benefit does a feature store primarily provide in an AI operations context?

A deployed NLP sentiment analysis model experiences a sharp decline in accuracy on customer reviews. The team has verified the input data format and pipeline are correct. Which THREE actions should be taken to diagnose and remediate? (Choose 3.)

A bank plans to deploy a credit-scoring model that will make automated decisions about loan applications. Compliance requires the bank to provide meaningful information about how the system reaches decisions and to give applicants a way to contest outcomes. Which TWO operational practices best support these obligations? (Choose two.)

An operations team runs a computer-vision model that flags manufacturing defects on an assembly line. Auditors require evidence that any single prediction can be reconstructed and explained months later. The team already logs model version, input image hash, and prediction score. Which additional logging practice best satisfies the audit requirement?

A hospital's clinical decision support model was validated at 94 percent accuracy on a held-out set. After go-live, clinicians report that the model's suggestions are frequently irrelevant for elderly patients, even though overall accuracy in the monitoring dashboard has barely moved. Which monitoring practice would have surfaced this problem?

An AI system misclassifies rare but critical events. The team considers using synthetic data. Which consideration is MOST important for ensuring the synthetic data improves performance on real rare events?

An operations team runs a demand-forecasting model on a cloud MLOps platform. The model retrains nightly, and after several weeks the live prediction distribution has drifted away from the distribution captured at training time. The team wants an automated signal that fires before prediction quality visibly degrades. Which practice should they implement?

A retail company runs an AI-powered demand forecasting service in a Kubernetes cluster. The inference pods scale based on CPU utilization, but during flash sales the request queue grows rapidly and p99 latency spikes before new pods become ready. The operations team needs to reduce latency during these spikes without changing the model itself. Which action should the team take?

A bank operates a credit-scoring model in production. Auditors require the team to reproduce the exact score a specific applicant received six months ago, including the model version, the feature values, and the code path used. Which capability must the team have in place to satisfy this requirement?

A company has developed a deep learning model for image classification. The team wants to deploy the model to production with high availability and scalability. Which approach should they use?