AAIR Practice Test 17 — 15 Questions

Question 1

A retail bank's AI risk committee is defining how third-party foundation model providers should be governed across 40 planned use cases. Legal notes that several providers reserve the right to change model versions with limited notice, and business units want to avoid re-approval delays for every update. Which approach BEST balances risk oversight with delivery speed?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Tier use cases by risk and data sensitivity, define pre-approved model versions and change classes per tier, and require impact assessment plus escalation only when a change crosses defined thresholds.. The bank needs governance that is proportionate and fast. Tiering use cases by risk and data sensitivity, pre-approving specific model versions and classes of change, and escalating only when a change crosses defined thresholds gives the committee control over material risk while letting low-impact updates proceed. This answers the vendor's limited-notice practice with pre-defined acceptance criteria, rather than serial re-approval or reliance on contracts alone.

Answer

Delegate all provider oversight to procurement and rely on contractual terms as the primary risk control.

Answer

Standardize on a single foundation model provider to reduce the number of version-change events the committee must review.

Answer

Require full AI risk committee re-approval for every provider model version change before any production use.

Question 2

An AI risk practitioner is designing the human oversight component for a generative AI assistant used by customer service agents. The assistant drafts responses that agents may send after review. Which TWO controls most directly reduce the risk of harmful or non-compliant outputs reaching customers? (Choose two.)

Accepted Answer

Running automated content filters that block outputs containing prohibited topics or personally identifiable information.. Effective oversight for a generative assistant combines a human decision point with an automated safety layer. Requiring logged affirmative approval ensures no draft reaches a customer without human accountability, while content filters catch prohibited or sensitive material deterministically. Together they reduce both the likelihood and the impact of harmful outputs, which is the objective of human oversight in the AI lifecycle.

Answer

Increasing the model's temperature setting to produce more varied and natural-sounding responses.

Answer

Expanding the assistant's knowledge base with additional product documentation to improve answer coverage.

Answer

Reducing the number of agents authorised to use the assistant to a small pilot group.

Question 3

A retail bank's credit-scoring AI was trained on three years of historical loan data. During an ISACA AAIR-aligned risk review, the risk practitioner discovers that the model's training dataset included the final repayment outcomes of loans that were still open at the time of data extraction. Which risk is most directly introduced by this dataset construction?

Accepted Answer

Target leakage, because repayment outcomes encoded future information that would not be available at inference time.. The dataset includes repayment outcomes from loans that were still open when data was extracted, so the model sees information that would not exist at the moment a credit decision is made. That temporal contamination is target leakage, and it inflates validation performance while guaranteeing degradation in production. A lifecycle risk review must detect it during data preparation, before the model is approved for deployment.

Answer

Sampling bias, because the three-year window excludes applicants who never received a loan.

Answer

Concept drift, because credit behaviour changes over the three-year training window.

Answer

Label noise, because repayment outcomes were recorded inconsistently across legacy systems.

Question 4

A global bank has adopted the NIST AI Risk Management Framework (AI RMF) and wants to ensure that its AI risk governance is integrated with its existing enterprise risk management (ERM) processes. The Chief Risk Officer asks the AI governance lead to identify the primary mechanism for achieving this integration. Which action best accomplishes this?

Accepted Answer

Mapping the AI RMF core functions to the bank's existing ERM risk categories and reporting lines.. The correct action is to map AI RMF core functions to the bank's existing ERM risk categories and reporting lines. This integrates AI risk into established governance structures, ensuring consistent oversight and reporting. It leverages existing ERM processes rather than creating parallel systems, which aligns with the AI RMF's emphasis on integration. This approach facilitates holistic risk management and avoids silos.

Answer

Delegating all AI risk decisions to the data science team to ensure technical accuracy.

Answer

Adopting a separate AI risk framework that replaces the bank's existing ERM framework.

Answer

Creating a standalone AI risk register that is maintained separately from the enterprise risk register.

Question 5

An insurance firm's fraud-detection model was approved after validation on a held-out test set. Six months later, investigators find the model performs poorly on claims submitted through a newly launched mobile channel. Which lifecycle risk management activity should have identified this gap before it affected production?

Accepted Answer

Defining and validating the model's intended use and operating domain boundaries during design.. The model was validated against data from existing channels, so its operating domain never included the mobile channel. Documenting intended use and domain boundaries during design would have made mobile claims an explicit out-of-scope condition, prompting revalidation before launch. Without that control, the organisation discovers the gap only through degraded production performance, which is a preventable lifecycle risk.

Answer

Increasing the size of the held-out test set to improve statistical confidence.

Answer

Applying stronger encryption to model inputs and outputs in the production pipeline.

Answer

Retraining the model more frequently on all available claims data.

Question 6

A hospital deploys a sepsis-prediction model. The AI risk manager learns that the vendor retrained the model on new patient data but the hospital's integration still calls the same endpoint, and no notification was issued. Which lifecycle risk control failure does this most directly represent?

Accepted Answer

Absence of model change management and version traceability for a third-party AI component.. When a vendor retrains a model and serves it through the same endpoint without notification, the deploying organisation loses the ability to identify which model version produced a given clinical output. That undermines incident investigation, validation, and regulatory traceability. Effective AI lifecycle risk management requires contractual change notification, version pinning, and revalidation triggers for third-party model updates.

Answer

Inadequate consent management for secondary use of patient data.

Answer

Failure to perform a fairness assessment on protected patient subgroups.

Answer

Insufficient compute capacity to run the updated model in the clinical environment.

Question 7

A global bank's AI risk committee is defining key risk indicators (KRIs) for its portfolio of customer-facing AI systems. The committee wants KRIs that provide early warning of emerging risk rather than lagging confirmation of past failures. Which KRI design best meets this objective?

Accepted Answer

Percentage of AI models whose input data drift exceeds a defined threshold over a rolling 30-day window.. Leading KRIs measure conditions that precede adverse outcomes, giving the committee time to intervene. Input data drift beyond a defined threshold is detectable before model outputs degrade, before customers are harmed, and before regulators take notice. By tracking the proportion of models breaching the drift threshold over a rolling window, the committee gains a forward-looking signal that can trigger retraining, rollback, or heightened oversight.

Answer

Aggregate spend on AI risk remediation activities compared with the annual budget.

Answer

Number of confirmed regulatory findings related to AI systems in the past quarter.

Answer

Total count of AI-related customer complaints resolved by the service desk each month.

Question 8

A healthcare organization is implementing the NIST AI Risk Management Framework (AI RMF) and wants to ensure that its AI governance structure addresses the 'Govern' function's requirement for clear roles and responsibilities. The Chief AI Officer proposes several actions. Which action best satisfies the requirement for accountability and oversight?

Accepted Answer

Establishing an AI Governance Committee with representation from legal, compliance, business units, and data science, and defining a RACI matrix for AI risk decisions.. The correct action is to establish an AI Governance Committee with cross-functional representation and a RACI matrix. This creates clear roles and responsibilities, ensures diverse oversight, and defines accountability for AI risk decisions. It aligns with the AI RMF Govern function's requirement for clear roles and responsibilities, promoting effective challenge and integration with enterprise governance.

Answer

Creating a volunteer-based AI ethics council without formal authority or reporting lines to the board.

Answer

Outsourcing AI risk oversight to an external consulting firm to gain objective expertise.

Answer

Assigning the Chief Data Scientist as the sole owner of all AI risk decisions to ensure technical expertise.

Question 9

A retail company's AI governance committee is drafting its charter. The committee wants to define clear decision rights so that disputes over AI risk acceptance do not stall deployment decisions. Which element is most essential to include in the charter?

Accepted Answer

A RACI matrix that assigns accountability for AI risk decisions by risk tier and defines the escalation path to the board.. A governance charter exists to make authority explicit. A RACI matrix that maps accountability by risk tier, combined with a defined escalation path to the board, ensures that disagreements about AI risk acceptance are resolved by the right decision maker at the right level. This prevents deployment delays caused by ambiguity over who ultimately owns the risk decision.

Answer

The marketing roadmap for AI-powered customer experiences over the next three years.

Answer

A detailed technical specification of every AI model architecture the company plans to use.

Answer

A complete list of every AI vendor contract currently under negotiation.

Question 10

A retail company is developing an AI governance framework based on the NIST AI Risk Management Framework (AI RMF). The Chief Risk Officer wants to ensure that the framework includes a process for continuously monitoring AI risks and updating controls. Which AI RMF function primarily addresses this need?

Accepted Answer

Manage. The Manage function of the AI RMF is designed to continuously monitor AI risks, implement risk responses, and update controls as necessary. It ensures that risk management is an ongoing process, not a one-time activity. This aligns with the need for continuous monitoring and updating of controls to address evolving AI risks.

Answer

Govern

Answer

Measure

Answer

Map

Question 11

A government agency uses an AI system to triage benefit applications. The risk register lists 'automation bias' as a concern. Which monitoring metric would most directly evidence whether this risk is materialising in practice?

Accepted Answer

The rate at which caseworkers accept the AI's triage recommendation without modifying it, broken down by recommendation confidence.. Automation bias is a human-factors risk, so the evidence must come from the human-AI interaction. Tracking how often caseworkers accept the triage recommendation without change, segmented by model confidence, shows whether reviewers defer even when the system is uncertain. A high unmodified acceptance rate on low-confidence outputs indicates the bias is materially affecting decisions and warrants mitigation.

Answer

The number of benefit applications processed per caseworker per day after the AI system was introduced.

Answer

The average latency between the AI producing a recommendation and the caseworker recording a decision.

Answer

The overall accuracy of the AI triage model measured against a gold-standard labelled sample.

Question 12

You are the AI risk lead at a mid-sized insurer. The Chief Risk Officer asks you to establish a formal AI risk taxonomy that will be used consistently across all business units for identifying, assessing, and reporting AI risks. Which action best ensures the taxonomy will be adopted and remain useful over time?

Accepted Answer

Map the taxonomy to the enterprise risk management framework and existing risk register categories, then pilot it with two business units before enterprise rollout.. A durable AI risk taxonomy must integrate with existing ERM structures so risks can be aggregated and compared across the enterprise. Piloting with a subset of business units validates the categories against real AI use cases and generates feedback before broad rollout. This combination of alignment and iterative validation is what drives sustained adoption and keeps the taxonomy relevant as AI use evolves.

Answer

Delegate taxonomy design entirely to the data science team, since they understand AI models best and can define the risks most accurately.

Answer

Draft the taxonomy with only legal and compliance input, then publish it as a mandatory standard to all business units to ensure regulatory alignment.

Answer

Adopt the risk categories directly from a leading industry publication without modification, since external benchmarks carry more credibility than internal design.

Question 13

A fintech company is preparing its annual AI risk report for the board. The Chief Risk Officer wants the report to demonstrate that the AI risk program is operating effectively, not merely listing AI use cases. Which two elements most directly provide evidence of an effective operating AI risk program? (Choose two.)

Accepted Answer

Documented remediation actions and closure rates for issues raised by AI risk assessments and incident reviews.. Evidence of an operating program comes from artifacts that change over time in response to activity. A live AI inventory with tiers, owners, and assessment dates proves continuous identification and accountability, while remediation tracking with closure rates proves that findings translate into corrective action. Together they show the program detects, assigns, and resolves AI risk, which is what the board needs to judge effectiveness.

Answer

The total number of employees who have completed general security awareness training.

Answer

A summary of the AI risk team's organizational chart and reporting lines.

Answer

A list of AI vendor logos and product names used across the organization.

Question 14

A retail insurer is mapping its new generative AI claims assistant against the NIST AI Risk Management Framework. The Chief Risk Officer wants the framework functions to be traceable to existing enterprise risk processes. Which mapping approach best integrates the AI RMF functions with the insurer's ERM framework?

Accepted Answer

Map Govern, Map, Measure, and Manage outcomes to existing ERM risk identification, assessment, treatment, and monitoring activities.. The AI RMF is most effective when its functions are threaded into existing ERM identification, assessment, treatment, and monitoring activities. This gives the insurer a single risk picture, traceable from AI-specific outcomes to enterprise risk registers and board reporting, rather than a parallel compliance exercise.

Answer

Apply the AI RMF only after the generative AI assistant has been deployed to production.

Answer

Treat the AI RMF as a standalone compliance checklist separate from the ERM framework.

Answer

Replace the ERM framework with the AI RMF for all technology-related risks.

Question 15

A hospital network deploys a clinical decision support model that recommends sepsis protocols. During a risk assessment, the AI risk manager identifies that clinicians frequently override the model's recommendations when they conflict with bedside judgment. Which risk treatment strategy is most appropriate for this situation?

Accepted Answer

Mitigate the risk by analyzing override patterns to determine whether they reflect model performance gaps, then recalibrate the model and refine alert thresholds.. Frequent overrides are diagnostic evidence, not an end state. Investigating override patterns can distinguish legitimate clinical judgment from model miscalibration, subgroup bias, or alert fatigue. Recalibrating the model and refining thresholds addresses the root cause directly, improving alignment between recommendations and clinical practice while retaining the safety net of clinician discretion that the hospital already relies on.

Answer

Accept the risk because clinician overrides demonstrate that human oversight is functioning as designed.

Answer

Avoid the risk by disabling the model entirely and reverting to manual sepsis screening protocols.

Answer

Transfer the risk by purchasing additional medical malpractice insurance coverage for AI-assisted decisions.