Operational Procedures practice questions

An organization is implementing an AI-powered chatbot for customer service. The chatbot must comply with GDPR and handle data subject access requests (DSARs). Which design approach best ensures compliance?

Trap 1: Minimize data collection by not logging any user interactions.

While minimizing data is good, it prevents auditing and debugging; also, DSARs require proving data was not processed if not retained.

Trap 2: Anonymize all user data before logging interactions.

Anonymization may prevent re-identification, but DSARs require access to personal data; anonymized data is not personal data, but the organization must still be able to identify the user's data before anonymization.

Trap 3: Encrypt all chat logs and store them indefinitely for audit…

Indefinite storage violates the storage limitation principle; encryption does not exempt from deletion requests.

A
Minimize data collection by not logging any user interactions.
Why wrong: While minimizing data is good, it prevents auditing and debugging; also, DSARs require proving data was not processed if not retained.
B
Anonymize all user data before logging interactions.
Why wrong: Anonymization may prevent re-identification, but DSARs require access to personal data; anonymized data is not personal data, but the organization must still be able to identify the user's data before anonymization.
C
Implement an audit trail that logs interactions with a unique user identifier, and provide a mechanism to delete logs upon user request.
This ensures compliance with the right to access and erasure under GDPR.
D
Encrypt all chat logs and store them indefinitely for audit purposes.
Why wrong: Indefinite storage violates the storage limitation principle; encryption does not exempt from deletion requests.

Question 2mediummulti select

Which TWO of the following are best practices for monitoring AI models in production?

Trap 1: Monitor model accuracy only at deployment time.

Accuracy should be monitored continuously, not just at deployment.

Trap 2: Regularly retrain without checking performance.

Retraining without performance checks can lead to degraded models.

Trap 3: Freeze the model version once deployed to avoid changes.

Models may need to be updated to handle drift; freezing is not recommended.

A
Set up alerts for prediction latency and error rates.
Operational metrics like latency and errors are critical for production monitoring.
B
Monitor model accuracy only at deployment time.
Why wrong: Accuracy should be monitored continuously, not just at deployment.
C
Regularly retrain without checking performance.
Why wrong: Retraining without performance checks can lead to degraded models.
D
Freeze the model version once deployed to avoid changes.
Why wrong: Models may need to be updated to handle drift; freezing is not recommended.
E
Track input data distribution and compare with training data.
This detects data drift, a key monitoring practice.

Read the full NAT/PAT explanation →

Question 3mediummultiple choice

A team is deploying an AI model that predicts patient readmission risk. The model was trained on data from three hospitals but will be used in a fourth hospital with different patient demographics. What is the most important security risk to assess?

Trap 1: Data poisoning during training

Data poisoning would have occurred before deployment; this scenario focuses on deployment in a new environment.

Trap 2: Model inversion to extract patient data

Model inversion is a privacy risk but not directly related to distribution shift.

Trap 3: Data breach of the inference API

Data breach is a general security concern, not specific to the model's generalization challenge.

A
Data poisoning during training
Why wrong: Data poisoning would have occurred before deployment; this scenario focuses on deployment in a new environment.
B
Adversarial attacks that cause misclassification
The shift in demographics can make the model more vulnerable to adversarial examples that cause incorrect readmission predictions.
C
Model inversion to extract patient data
Why wrong: Model inversion is a privacy risk but not directly related to distribution shift.
D
Data breach of the inference API
Why wrong: Data breach is a general security concern, not specific to the model's generalization challenge.

Question 4easymulti select

Which TWO of the following are essential components of a responsible AI governance framework?

Trap 1: Using ensemble methods to reduce overfitting

This is a data science technique, not governance.

Trap 2: Automated hyperparameter tuning to improve accuracy

This is a model optimization technique, not a governance component.

Trap 3: Deploying models on dedicated hardware to reduce latency

Infrastructure decisions are not core governance.

A
Assignment of a responsible owner for each AI system's outcomes
Accountability is a fundamental governance requirement.
B
Using ensemble methods to reduce overfitting
Why wrong: This is a data science technique, not governance.
C
Clear documentation of model development and decision-making processes
Transparency is a key governance principle.
D
Automated hyperparameter tuning to improve accuracy
Why wrong: This is a model optimization technique, not a governance component.
E
Deploying models on dedicated hardware to reduce latency
Why wrong: Infrastructure decisions are not core governance.

Question 5easymulti select

An organization wants to implement a robust MLOps pipeline. Which THREE components are essential for a complete MLOps framework? (Choose three.)

Trap 1: Automated code review and approval gates

Code review is a practice, not an essential component of the pipeline.

Trap 2: Real-time model monitoring dashboard

Monitoring is operational, not necessarily part of the core MLOps pipeline.

A
Continuous integration and continuous deployment (CI/CD) pipeline
CI/CD automates model building and deployment.
B
Automated testing and validation
Automated tests ensure quality before deployment.
C
Automated code review and approval gates
Why wrong: Code review is a practice, not an essential component of the pipeline.
D
Real-time model monitoring dashboard
Why wrong: Monitoring is operational, not necessarily part of the core MLOps pipeline.
E
Version control for data and model code
Version control is fundamental for reproducibility.

Question 6hardmultiple choice

A global retailer uses an AI model to forecast demand across thousands of stores. After deployment, the model's predictions become less accurate during holiday seasons. The training data included two years of holiday periods. What is the most effective operational strategy to handle this recurring seasonal drift?

Trap 1: Deploy an anomaly detection system to flag holiday prediction…

This is reactive, not proactive.

Trap 2: Use an ensemble of models trained on different time periods

Ensembles can help but not specifically target seasonal retraining.

Trap 3: Increase the volume of training data by including five years of…

More data may not capture the specific seasonal shift adequately.

A
Deploy an anomaly detection system to flag holiday prediction outliers
Why wrong: This is reactive, not proactive.
B
Implement a scheduled retraining cycle just before each holiday period
Proactive retraining with recent holiday data mitigates seasonal drift.
C
Use an ensemble of models trained on different time periods
Why wrong: Ensembles can help but not specifically target seasonal retraining.
D
Increase the volume of training data by including five years of history
Why wrong: More data may not capture the specific seasonal shift adequately.

Question 7easymultiple choice

An organization deploys an AI model on edge devices for real-time image classification. Which metric is most important to monitor for ensuring the device's operational health?

Trap 1: Model calibration error

Calibration is about probability estimates, not hardware health.

Trap 2: Average prediction confidence

Confidence doesn't indicate device resource status.

Trap 3: Model accuracy on local test data

Accuracy is a model metric, not device health.

A
Model calibration error
Why wrong: Calibration is about probability estimates, not hardware health.
B
Inference memory consumption
Memory is a key operational health indicator for edge devices.
C
Average prediction confidence
Why wrong: Confidence doesn't indicate device resource status.
D
Model accuracy on local test data
Why wrong: Accuracy is a model metric, not device health.

Question 8easymulti select

Which THREE are common pitfalls when operationalizing AI models? (Select THREE.)

Trap 1: Using simpler models that are easier to debug

Simpler models are often preferred for operationalization.

Trap 2: Automating the model retraining process

Automation is a best practice, not a pitfall.

A
Training-serving skew due to differences in data preprocessing
Causes model inaccuracies in production.
B
Using simpler models that are easier to debug
Why wrong: Simpler models are often preferred for operationalization.
C
Lack of monitoring for model performance drift
Without monitoring, degradation goes unnoticed.
D
Ignoring infrastructure scalability requirements
Leads to resource contention and failures.
E
Automating the model retraining process
Why wrong: Automation is a best practice, not a pitfall.

Read the full NAT/PAT explanation →

Question 9hardmultiple choice

A large hospital system deploys an AI triage system for emergency rooms. The system uses patient vitals and symptoms to recommend treatment priority. Six months after deployment, complaints arise that the system frequently underestimates the severity of symptoms for patients from certain ethnic backgrounds. A data scientist runs a bias audit and finds that the model's false negative rate is 20% higher for the minority group. The hospital's AI governance board requires immediate corrective action. The data science team has limited resources and cannot retrain the entire model from scratch. They have access to the training data, which is imbalanced. The model is a gradient boosted tree. Which course of action best addresses the bias while minimizing operational impact?

Trap 1: Rebalance the training data using SMOTE and retrain the model

SMOTE retraining requires time and may not fully resolve bias if the model still learns proxy features.

Trap 2: Use adversarial debiasing during training to remove protected…

Adversarial debiasing requires retraining, which contradicts the limited resources constraint.

Trap 3: Replace the model with a simpler logistic regression model to…

Switching models is a major change and may degrade overall performance.

A
Rebalance the training data using SMOTE and retrain the model
Why wrong: SMOTE retraining requires time and may not fully resolve bias if the model still learns proxy features.
B
Use adversarial debiasing during training to remove protected attribute correlations
Why wrong: Adversarial debiasing requires retraining, which contradicts the limited resources constraint.
C
Post-process the model's predictions by adjusting thresholds for the minority group
Threshold adjustment is fast, cheap, and directly minimizes false negative disparity.
D
Replace the model with a simpler logistic regression model to improve interpretability
Why wrong: Switching models is a major change and may degrade overall performance.