Operational Procedures practice questions

A data science team wants to implement a feature store to serve pre-computed features for both training and inference with low latency. Which TWO tools are commonly used for building a feature store?

Trap 1: Kubeflow

Kubeflow is for ML workflows on Kubernetes, not a feature store.

Trap 2: Apache Hive

Hive is a data warehouse infrastructure, not a real-time feature store.

Trap 3: MLflow

MLflow is for experiment tracking and model management, not a feature store.

A
Kubeflow
Why it fails: Kubeflow is for ML workflows on Kubernetes, not a feature store.
B
Apache Hive
Why it fails: Hive is a data warehouse infrastructure, not a real-time feature store.
C
Feast
Feast is an open-source feature store that manages and serves features.
D
Tecton
Tecton is a commercial feature store platform built on top of Spark and Kafka.
E
MLflow
Why it fails: MLflow is for experiment tracking and model management, not a feature store.

Question 2easymultiple choice

An organisation is developing an AI policy. According to the NIST AI RMF, which function involves establishing policies and procedures to ensure the organisation governs AI responsibly?

Trap 1: Manage

Manage allocates resources to the risks identified and treats them after they are measured; it does not establish the policies and procedures themselves. Manage is the correct function once risks have been assessed and require mitigation, monitoring and response.

Trap 2: Measure

Measure quantifies and monitors AI risks using metrics and testing; it does not create the governance policies and procedures. Measure is the correct function when the task is evaluating, benchmarking or tracking identified risks rather than defining organisational governance.

Trap 3: Map

Map catalogues AI context and risks but creates no governance structures; it feeds the Govern function rather than establishing policies and procedures. It is tempting because mapping precedes risk treatment, and would be the right answer if the question asked which function identifies and frames AI risks across the lifecycle.

A
Manage
Why it fails: Manage allocates resources to the risks identified and treats them after they are measured; it does not establish the policies and procedures themselves. Manage is the correct function once risks have been assessed and require mitigation, monitoring and response.
B
Measure
Why it fails: Measure quantifies and monitors AI risks using metrics and testing; it does not create the governance policies and procedures. Measure is the correct function when the task is evaluating, benchmarking or tracking identified risks rather than defining organisational governance.
C
Govern
The Govern function establishes the policies, procedures, roles and accountability structures that ensure AI is developed and used responsibly across the organisation. It satisfies the stem's requirement for setting up governance, whereas Map, Measure and Manage handle risk identification, assessment and treatment.
D
Map
Why it fails: Map catalogues AI context and risks but creates no governance structures; it feeds the Govern function rather than establishing policies and procedures. It is tempting because mapping precedes risk treatment, and would be the right answer if the question asked which function identifies and frames AI risks across the lifecycle.

Question 3hardmultiple choice

A large hospital system deploys an AI triage system for emergency rooms. The system uses patient vitals and symptoms to recommend treatment priority. Six months after deployment, complaints arise that the system frequently underestimates the severity of symptoms for patients from certain ethnic backgrounds. A data scientist runs a bias audit and finds that the model's false negative rate is 20% higher for the minority group. The hospital's AI governance board requires immediate corrective action. The data science team has limited resources and cannot retrain the entire model from scratch. They have access to the training data, which is imbalanced. The model is a gradient boosted tree. Which course of action best addresses the bias while minimizing operational impact?

Trap 1: Rebalance the training data using SMOTE and retrain the model

SMOTE synthesises minority-class examples, but the bias stems from disparate false negative rates across ethnic groups, not class imbalance alone; retraining also exceeds the stated resource limit. It is tempting because imbalance is present, and SMOTE would be correct if underrepresentation of a class were the sole cause.

Trap 2: Use adversarial debiasing during training to remove protected…

Adversarial debiasing requires retraining the gradient boosted tree with a modified objective, which the team cannot do given limited resources. It is tempting because it directly targets protected-attribute correlations, and would be the right choice if full retraining from scratch were feasible and the goal were in-processing fairness.

Trap 3: Replace the model with a simpler logistic regression model to…

Swapping to logistic regression discards the trained gradient boosted tree and does not guarantee reduced false negative disparity for the minority group. It is tempting because interpretability aids auditing, and would suit a scenario where explainability, not measured bias correction, is the governance board's stated requirement.

A
Rebalance the training data using SMOTE and retrain the model
Why it fails: SMOTE synthesises minority-class examples, but the bias stems from disparate false negative rates across ethnic groups, not class imbalance alone; retraining also exceeds the stated resource limit. It is tempting because imbalance is present, and SMOTE would be correct if underrepresentation of a class were the sole cause.
B
Use adversarial debiasing during training to remove protected attribute correlations
Why it fails: Adversarial debiasing requires retraining the gradient boosted tree with a modified objective, which the team cannot do given limited resources. It is tempting because it directly targets protected-attribute correlations, and would be the right choice if full retraining from scratch were feasible and the goal were in-processing fairness.
C
Post-process the model's predictions by adjusting thresholds for the minority group
Threshold adjustment is a post-processing intervention applied at inference, so it corrects the disparate false negative rate without retraining the gradient boosted tree. This satisfies the constraint of limited resources and minimal operational impact, equalising error rates across groups while leaving the existing model intact.
D
Replace the model with a simpler logistic regression model to improve interpretability
Why it fails: Swapping to logistic regression discards the trained gradient boosted tree and does not guarantee reduced false negative disparity for the minority group. It is tempting because interpretability aids auditing, and would suit a scenario where explainability, not measured bias correction, is the governance board's stated requirement.

Question 4hardmultiple choice

A global retailer uses an AI model to forecast demand across thousands of stores. After deployment, the model's predictions become less accurate during holiday seasons. The training data included two years of holiday periods. What is the most effective operational strategy to handle this recurring seasonal drift?

Trap 1: Deploy an anomaly detection system to flag holiday prediction…

Flagging outliers detects drift but does not correct forecasts, so holiday predictions remain wrong. Anomaly detection suits surfacing unexpected events for investigation, whereas recurring seasonal drift needs the model retrained or adjusted to capture the repeating holiday pattern.

Trap 2: Use an ensemble of models trained on different time periods

An ensemble trained on different time periods still learns the same seasonal patterns already present, and averaging does not correct systematic holiday underprediction. Ensembling suits reducing variance across diverse model families, not addressing a known recurring seasonal drift requiring seasonal features or retraining.

Trap 3: Increase the volume of training data by including five years of…

Adding five years of history increases volume but not seasonal relevance; the model already saw two holiday cycles, and more of the same distribution does not fix the drift. Expanding historical data suits improving generalisation when data is scarce, not correcting a recurring seasonal pattern.

A
Deploy an anomaly detection system to flag holiday prediction outliers
Why it fails: Flagging outliers detects drift but does not correct forecasts, so holiday predictions remain wrong. Anomaly detection suits surfacing unexpected events for investigation, whereas recurring seasonal drift needs the model retrained or adjusted to capture the repeating holiday pattern.
B
Implement a scheduled retraining cycle just before each holiday period
Scheduled retraining immediately before each holiday period refreshes the model with the most recent seasonal patterns, directly countering the recurring drift the stem describes. Because the drift is predictable and calendar-bound, a timed cycle restores accuracy before peak demand, unlike reactive monitoring or static thresholds.
C
Use an ensemble of models trained on different time periods
Why it fails: An ensemble trained on different time periods still learns the same seasonal patterns already present, and averaging does not correct systematic holiday underprediction. Ensembling suits reducing variance across diverse model families, not addressing a known recurring seasonal drift requiring seasonal features or retraining.
D
Increase the volume of training data by including five years of history
Why it fails: Adding five years of history increases volume but not seasonal relevance; the model already saw two holiday cycles, and more of the same distribution does not fix the drift. Expanding historical data suits improving generalisation when data is scarce, not correcting a recurring seasonal pattern.

Question 5mediummulti select

A company is implementing an AI ethics board. Which TWO responsibilities should the board typically have?

Trap 1: Writing code for fairness algorithms

An ethics board provides governance, guidance and review of AI use; writing fairness algorithm code is engineering work performed by development teams implementing the board's principles. It is tempting because fairness is a core ethical concern, but the board sets requirements and assesses outcomes rather than authoring implementation code.

Trap 2: Developing marketing strategies for AI products

An ethics board governs responsible AI use through policy, risk review and oversight; developing marketing strategies is a commercial function unrelated to ethical governance. It is tempting because AI products need market positioning, but that belongs to product and marketing teams, not an ethics oversight body.

Trap 3: Conducting daily data privacy audits

An ethics board sets policy, reviews high-risk use cases and provides oversight; conducting daily data privacy audits is operational work owned by privacy, compliance or security teams. It is tempting because privacy is central to AI ethics, but routine auditing is a recurring control activity, not a governance body's remit.

A
Approving or rejecting high-risk AI initiatives
The board holds decision authority over high-risk AI initiatives, approving or rejecting them based on ethical risk assessment. This gatekeeping responsibility matches the stem's expectation that the board governs deployments, ensuring risky projects cannot proceed without ethical sign-off.
B
Writing code for fairness algorithms
Why it fails: An ethics board provides governance, guidance and review of AI use; writing fairness algorithm code is engineering work performed by development teams implementing the board's principles. It is tempting because fairness is a core ethical concern, but the board sets requirements and assesses outcomes rather than authoring implementation code.
C
Reviewing AI projects for ethical compliance and potential biases
Ethics boards examine AI initiatives against ethical standards and detect bias in data or models, satisfying the stem's compliance-review responsibility. This governance oversight ensures projects align with fairness principles before and during deployment, which is the board's core remit.
D
Developing marketing strategies for AI products
Why it fails: An ethics board governs responsible AI use through policy, risk review and oversight; developing marketing strategies is a commercial function unrelated to ethical governance. It is tempting because AI products need market positioning, but that belongs to product and marketing teams, not an ethics oversight body.
E
Conducting daily data privacy audits
Why it fails: An ethics board sets policy, reviews high-risk use cases and provides oversight; conducting daily data privacy audits is operational work owned by privacy, compliance or security teams. It is tempting because privacy is central to AI ethics, but routine auditing is a recurring control activity, not a governance body's remit.

Question 6easymultiple choice

What is the primary function of an AI ethics board within an organization?

Trap 1: Developing algorithms

Algorithm development is typically done by data scientists, not the ethics board.

Trap 2: Managing cloud infrastructure

Cloud infrastructure provisioning sits with platform and operations teams, and an ethics board holds no mandate over compute, storage or network configuration. It is tempting because AI systems do run on cloud infrastructure, so governance discussions touch deployment, but the board's remit is reviewing use cases, bias and societal impact.

Trap 3: Marketing AI products

Promoting AI products is a commercial function owned by product marketing, whereas an ethics board reviews development and deployment for fairness, transparency and compliance. It is tempting because ethical positioning can support market trust, but the board's output is guidance and oversight, not demand generation or sales enablement.

A
Developing algorithms
Why it fails: Algorithm development is typically done by data scientists, not the ethics board.
B
Managing cloud infrastructure
Why it fails: Cloud infrastructure provisioning sits with platform and operations teams, and an ethics board holds no mandate over compute, storage or network configuration. It is tempting because AI systems do run on cloud infrastructure, so governance discussions touch deployment, but the board's remit is reviewing use cases, bias and societal impact.
C
Marketing AI products
Why it fails: Promoting AI products is a commercial function owned by product marketing, whereas an ethics board reviews development and deployment for fairness, transparency and compliance. It is tempting because ethical positioning can support market trust, but the board's output is guidance and oversight, not demand generation or sales enablement.
D
Reviewing AI projects for ethical compliance
An AI ethics board's core remit is governance: examining proposed and live AI projects against ethical principles, flagging risks such as bias or privacy harm, and advising on remediation. It reviews for compliance rather than building models or owning day-to-day operations.

Question 7easymultiple choice

A small marketing team has built an internal AI assistant that answers questions about their product catalog. They want to deploy it quickly with minimal infrastructure management and pay only for what they use. The team has no Kubernetes expertise and wants the provider to handle scaling, patching, and availability. Which deployment option best matches these constraints?

Trap 1: Use a serverless function that loads the model on each invocation…

Serverless functions are pay-per-use and require no server management, but loading a model on every invocation introduces cold-start latency and repeated model loading costs. For an interactive assistant, this can produce unacceptable response times and unpredictable performance. The model may also exceed the memory and execution-time limits of the function runtime. While it minimizes infrastructure, it does not reliably meet the assistant's latency and usability needs.

Trap 2: Provision a Kubernetes cluster, deploy the model behind an…

Kubernetes gives fine-grained control but requires the team to manage nodes, upgrades, networking, and autoscaling policies. The team explicitly lacks Kubernetes expertise and wants minimal infrastructure management. While a managed Kubernetes service reduces some burden, the team would still own cluster operations and patching. This option adds operational overhead that does not match the stated constraints and is not the best fit for a small team.

Trap 3: Run the model on a dedicated virtual machine that the team patches…

A dedicated virtual machine gives the team full control but requires them to patch the operating system, manage the model server, and scale capacity by hand. That conflicts with the goal of minimal infrastructure management and the absence of Kubernetes or deep ops expertise. It also lacks the elastic, pay-as-you-go billing of a managed endpoint, so the team would pay for idle capacity. This option does not best match the constraints.

A
Use a serverless function that loads the model on each invocation and returns a response.
Why it fails: Serverless functions are pay-per-use and require no server management, but loading a model on every invocation introduces cold-start latency and repeated model loading costs. For an interactive assistant, this can produce unacceptable response times and unpredictable performance. The model may also exceed the memory and execution-time limits of the function runtime. While it minimizes infrastructure, it does not reliably meet the assistant's latency and usability needs.
B
Use a managed AI platform endpoint that hosts the model, handles scaling and patching, and bills per request or per token.
A managed AI platform endpoint matches every constraint: the provider handles scaling, patching, and availability, the team avoids Kubernetes operations, and billing is consumption-based. This is the fastest path to production for a small team without infrastructure expertise. It also provides built-in monitoring and integration with the provider's SDKs, reducing the amount of custom operational code the team must write and maintain.
C
Provision a Kubernetes cluster, deploy the model behind an inference server, and configure a horizontal pod autoscaler.
Why it fails: Kubernetes gives fine-grained control but requires the team to manage nodes, upgrades, networking, and autoscaling policies. The team explicitly lacks Kubernetes expertise and wants minimal infrastructure management. While a managed Kubernetes service reduces some burden, the team would still own cluster operations and patching. This option adds operational overhead that does not match the stated constraints and is not the best fit for a small team.
D
Run the model on a dedicated virtual machine that the team patches and scales manually.
Why it fails: A dedicated virtual machine gives the team full control but requires them to patch the operating system, manage the model server, and scale capacity by hand. That conflicts with the goal of minimal infrastructure management and the absence of Kubernetes or deep ops expertise. It also lacks the elastic, pay-as-you-go billing of a managed endpoint, so the team would pay for idle capacity. This option does not best match the constraints.

Question 8hardmultiple choice

A global bank deploys a generative AI assistant that summarizes internal policy documents for loan officers across the European Union and the United States. The compliance team must ensure the system respects regional AI regulations. Which of the following actions is MOST appropriate for aligning the deployment with these requirements?

Trap 1: Restrict the assistant to English-language documents only, because…

Language is not the basis of AI regulatory obligations; risk classification, use context, and data handling are. Limiting language would not satisfy EU AI Act documentation, transparency, or oversight duties, nor US fair lending and consumer protection expectations. This action addresses a translation concern rather than the actual governance requirements of each jurisdiction.

Trap 2: Rely on the foundation model provider's terms of service and…

Provider documentation describes the base model, not the bank's specific deployment, data flows, or use case. Regulatory obligations attach to the deployer as well as the provider, especially for high-risk financial uses. Treating a vendor model card as complete compliance evidence ignores required risk classification, local assessments, and operational controls the bank itself must implement and evidence.

Trap 3: Disable all output logging to minimize the personal data stored,…

Data minimization is one principle among many and does not override accountability, auditability, or record-keeping duties. Removing logs would undermine incident investigation, regulatory reporting, and human oversight evidence. Privacy requirements can be met with retention limits, access controls, and pseudonymization rather than eliminating the audit trail entirely, which would create a different compliance failure.

A
Classify the assistant's risk level per jurisdiction, document the conformity assessment, and maintain human oversight and logging for its outputs.
Generative assistants used in credit-related workflows can fall into higher-risk categories under the EU AI Act and are subject to sector rules such as fair lending in the US. A defensible approach maps each jurisdiction's requirements, performs and documents a conformity or impact assessment, and keeps human review plus audit logs. This addresses transparency, accountability, and oversight obligations simultaneously.
B
Restrict the assistant to English-language documents only, because language localization is the primary regulatory differentiator across regions.
Why it fails: Language is not the basis of AI regulatory obligations; risk classification, use context, and data handling are. Limiting language would not satisfy EU AI Act documentation, transparency, or oversight duties, nor US fair lending and consumer protection expectations. This action addresses a translation concern rather than the actual governance requirements of each jurisdiction.
C
Rely on the foundation model provider's terms of service and published model card as sufficient evidence of regulatory compliance.
Why it fails: Provider documentation describes the base model, not the bank's specific deployment, data flows, or use case. Regulatory obligations attach to the deployer as well as the provider, especially for high-risk financial uses. Treating a vendor model card as complete compliance evidence ignores required risk classification, local assessments, and operational controls the bank itself must implement and evidence.
D
Disable all output logging to minimize the personal data stored, since data minimization is the only regulatory requirement that applies.
Why it fails: Data minimization is one principle among many and does not override accountability, auditability, or record-keeping duties. Removing logs would undermine incident investigation, regulatory reporting, and human oversight evidence. Privacy requirements can be met with retention limits, access controls, and pseudonymization rather than eliminating the audit trail entirely, which would create a different compliance failure.

Question 9easymulti select

Which TWO actions are most appropriate for managing model drift in a production AI system?

Trap 1: Freeze the model to prevent any changes

Freezing the model prevents any weight updates, so it cannot adapt as input distributions shift and drift accumulates unchecked. Freezing suits audit or compliance holds where outputs must remain fixed; drift management requires monitoring and periodic retraining or updating.

Trap 2: Roll back to a previous model version if performance degrades

Rolling back restores earlier behaviour but does not address the underlying distribution shift, so the reverted model drifts again. Rollback suits recovering from a bad deployment or regression; managing drift requires retraining on recent data plus ongoing performance monitoring.

Trap 3: Manually review all model predictions

Manually reviewing every prediction cannot scale to production volumes, so drift detection would lag far behind the data distribution shift. It is tempting because human review suits low-volume, high-stakes auditing or labelling a sample for retraining. Continuous monitoring of input distributions and accuracy metrics is what actually surfaces drift.

A
Freeze the model to prevent any changes
Why it fails: Freezing the model prevents any weight updates, so it cannot adapt as input distributions shift and drift accumulates unchecked. Freezing suits audit or compliance holds where outputs must remain fixed; drift management requires monitoring and periodic retraining or updating.
B
Roll back to a previous model version if performance degrades
Why it fails: Rolling back restores earlier behaviour but does not address the underlying distribution shift, so the reverted model drifts again. Rollback suits recovering from a bad deployment or regression; managing drift requires retraining on recent data plus ongoing performance monitoring.
C
Periodically retrain the model on recent data
Periodic retraining on recent data directly counters data drift by updating learned parameters to reflect the current input distribution, satisfying the requirement to manage drift in production. It addresses gradual distributional shift, which monitoring alone cannot correct, restoring alignment between the model's training assumptions and live data.
D
Manually review all model predictions
Why it fails: Manually reviewing every prediction cannot scale to production volumes, so drift detection would lag far behind the data distribution shift. It is tempting because human review suits low-volume, high-stakes auditing or labelling a sample for retraining. Continuous monitoring of input distributions and accuracy metrics is what actually surfaces drift.
E
Implement automated monitoring to detect drift indicators
Automated monitoring continuously tracks input distributions and performance metrics against the training baseline, detecting data or concept drift before degradation becomes severe. This satisfies the stem's production constraint, where drift emerges gradually and unpredictably, so scheduled manual reviews cannot catch it promptly. Detection triggers retraining or rollback, keeping the deployed model aligned with current data.