AAIR AI Risk Program Management Practice Test 5 — 15 Questions

Question 1

You are analyzing the risk of a generative AI implementation. Which metric most effectively measures 'model transparency' for a risk dashboard?

Accepted Answer

Percentage of models deployed with a completed Model Card covering data provenance, limitations, and intended use.. Model transparency is best measured by the percentage of deployed models that have a completed Model Card documenting data provenance, limitations, and intended use. Model Cards (Mitchell et al., 2019) are the industry-standard artifact for transparency, and a coverage metric directly quantifies how transparent the model portfolio is. This gives risk dashboards an actionable, auditable indicator tied to governance requirements.

Answer

The number of unique contributors to the codebase.

Answer

The average latency of model inference.

Answer

The number of parameters in the neural network.

Question 2

When coordinating AI risk across a global organization, why is it critical to include local legal counsel in the AI steering committee?

Accepted Answer

To interpret the varied regulatory landscapes in different operational regions.. Local legal counsel is essential on a global AI steering committee because AI regulations vary significantly by jurisdiction — the EU AI Act, US state laws, China's algorithm rules, Brazil's LGPD, etc. — and local counsel interprets how these apply to the organization's operations in each region. This ensures the committee's risk decisions are legally grounded and regionally compliant rather than assuming a single global standard.

Answer

To audit the training datasets for copyright infringement.

Answer

To handle administrative tasks for the committee.

Answer

To approve the technical architecture of the AI models.

Question 3

Which document is essential for defining the roles and responsibilities within an AI Risk Program?

Accepted Answer

AI Governance RACI Matrix.. The AI Governance RACI Matrix is the document that explicitly defines who is Responsible, Accountable, Consulted, and Informed for AI risk activities across the organization. It is the standard governance artifact for assigning roles and responsibilities, ensuring clear ownership and avoiding gaps or overlaps in the AI Risk Program.

Answer

Data Privacy Impact Assessment (DPIA).

Answer

Model Training Log.

Answer

AI Incident Response Plan.

Question 4

An organization is updating its risk appetite statement for AI. Which specific element should be addressed to manage the 'hallucination' risk of LLMs?

Accepted Answer

Require human-in-the-loop for any AI-generated output used in customer-facing content.. Hallucination risk in LLMs — where the model generates plausible but factually incorrect output — is best managed by requiring human-in-the-loop review for any AI-generated content used in customer-facing communications. This ensures a human validates factual accuracy before the output reaches customers, directly mitigating the reputational, legal, and financial harm of hallucinations. This control aligns with the risk appetite principle of defining acceptable risk thresholds and the corresponding controls.

Answer

Set a performance goal to reduce the model parameter count.

Answer

Increase the frequency of periodic penetration testing.

Answer

Limit the training dataset to under 10GB.

Question 5

When setting up an AI risk monitoring dashboard, what is the most important consideration for ensuring executive buy-in?

Accepted Answer

Map AI performance metrics to business impact indicators like customer churn or operational downtime.. Executive buy-in depends on demonstrating business value, not technical detail. Mapping AI performance metrics to business impact indicators such as customer churn, revenue, or operational downtime translates model behavior into outcomes executives already care about and are measured on. This makes the dashboard a decision-support tool rather than an engineering report.

Answer

Include the number of GPU hours utilized.

Answer

Display the raw training error rates.

Answer

Show a comparison of the organization's AI progress against competitors.

Question 6

You are managing AI supply chain risk. Which control is most critical for third-party AI models?

Accepted Answer

Requiring a documented 'AI Bill of Materials' to understand data lineage and model provenance.. The most critical control for third-party AI models is requiring a documented 'AI Bill of Materials' (AI BOM) to understand data lineage and model provenance. This provides transparency into the components, data sources, and training methods used in the model, enabling effective risk assessment and mitigation. Without this visibility, organizations cannot verify compliance, detect biases, or manage supply chain risks.

Answer

Conducting a manual audit of the vendor's source code.

Answer

Requiring the vendor to submit a monthly performance report.

Answer

Setting a limit on the number of third-party vendors allowed.

Question 7

In the context of AI risk management, what does the 'Human-in-the-Loop' (HITL) concept primarily aim to achieve?

Accepted Answer

To maintain accountability and oversight for AI-driven decisions.. Human-in-the-Loop (HITL) primarily ensures that a human reviews, approves, or can override AI-driven decisions, preserving accountability and meaningful oversight. It addresses the governance need that a responsible person remains answerable for outcomes, especially in high-risk or regulated contexts. This maps directly to accountability principles in the NIST AI RMF and OECD AI Principles.

Answer

To reduce the cost of model development.

Answer

To increase the speed of model inference.

Answer

To automate the labeling of training data.

Question 8

Which document is the most appropriate starting point for an AI Risk assessment for a new machine learning project?

Accepted Answer

An AI Risk Assessment Protocol/Template.. An AI Risk Assessment Protocol/Template provides a standardized framework for identifying, evaluating, and mitigating risks specific to AI systems, such as bias, privacy, and model drift. It ensures the assessment is systematic, repeatable, and aligned with governance requirements. Starting with a template guarantees that all critical risk dimensions are considered from the outset, rather than ad-hoc or incomplete analysis.

Answer

The marketing slide deck for the AI feature.

Answer

The technical API documentation.

Answer

The project budget spreadsheet.

Question 9

When addressing 'Explainability' as an AI risk, which approach is most effective for a non-technical stakeholder?

Accepted Answer

Providing counterfactual explanations like 'If your income had been $5k higher, the loan would have been approved'.. Counterfactual explanations are the most effective for non-technical stakeholders because they describe what would need to change in the input for a different outcome, using plain language and concrete scenarios. For example, 'If your income had been $5k higher, the loan would have been approved' is immediately understandable without any mathematical background. This approach aligns with GDPR Article 22 and the right to meaningful information about automated decisions.

Answer

Presenting the feature weights of a neural network.

Answer

Explaining the gradient descent optimization process.

Answer

Showing the mathematical formula used for loss reduction.

Question 10

Which phase of the AI lifecycle is most critical for initial risk mitigation?

Accepted Answer

Design and Problem Formulation.. The Design and Problem Formulation phase is most critical for initial risk mitigation because this is where the problem is scoped, objectives are defined, and potential risks are identified before any model is built. Decisions made here — such as choosing the use case, defining success metrics, identifying affected stakeholders, and assessing ethical/legal constraints — determine the risk profile of the entire project. Once a model is trained and deployed, many risks are already baked in and far more costly to remediate.

Answer

Deployment.

Answer

Decommissioning.

Answer

Model Monitoring.

Question 11

Which of the following is a key objective of an AI Risk Management Program?

Accepted Answer

To enable the ethical and safe adoption of AI technologies.. The key objective of an AI Risk Management Program is to enable the ethical and safe adoption of AI technologies by identifying, assessing, mitigating, and monitoring AI-specific risks such as bias, opacity, privacy violations, and safety failures. It supports responsible innovation rather than blocking AI use. This aligns with frameworks like NIST AI RMF and ISO/IEC 42001.

Answer

To eliminate all AI risks.

Answer

To maximize the budget spent on AI tools.

Answer

To replace human employees with AI.

Question 12

You are auditing an AI project. Which finding suggests a failure in AI risk management governance?

Accepted Answer

No individual is designated as the accountable owner for the model's production outcomes.. The absence of a designated accountable owner for the model's production outcomes is a clear governance failure. AI risk management frameworks (NIST AI RMF, ISO/IEC 42001) require clear accountability structures so that someone is responsible for model performance, incident response, and compliance. Without an accountable owner, risks go unmanaged, issues go unreported, and remediation stalls.

Answer

The model was trained on historical data.

Answer

The team uses a cloud-based GPU provider.

Answer

The model has a 2% prediction error rate.

Question 13

When establishing KPIs for an AI program, what is the risk of focusing solely on 'Model Accuracy'?

Accepted Answer

Other critical risks such as bias, security, and fairness may be overlooked.. Focusing solely on model accuracy creates a narrow optimization target that ignores other critical AI risks such as bias, fairness, security, privacy, and robustness. Accuracy alone does not guarantee that a model is ethical, compliant, or safe in production, so other risk dimensions can be neglected.

Answer

The model will be too computationally expensive.

Answer

The data scientists will take too long to develop the model.

Answer

The model will not be deployable in production.

Question 14

An organization discovers that their AI model exhibits bias toward a specific demographic. What is the most important step in the Incident Response process?

Accepted Answer

Suspend the model's production use until the issue is investigated and remediated.. When an AI model is found to exhibit harmful bias, the immediate priority in incident response is containment — suspending production use prevents ongoing discriminatory decisions, legal exposure, and harm to affected individuals while the root cause is investigated. This mirrors the 'containment before eradication' principle in standard IR frameworks (NIST SP 800-61). Retraining or deleting data without first stopping the model leaves the biased system live and compounding harm.

Answer

Retrain the model immediately.

Answer

Delete all training data related to the demographic.

Answer

Issue a public press release.

Question 15

When documenting an AI Risk Program, what should be included in the 'AI Asset Register'?

Accepted Answer

The model purpose, data lineage, stakeholders, and risk classification.. An AI Asset Register is an inventory of AI systems and models that captures governance-relevant metadata: the model's purpose, the data sources and lineage used to train it, the stakeholders accountable for it, and its risk classification. This information enables risk management, regulatory reporting, and lifecycle oversight across the AI portfolio.

Answer

The raw hardware serial numbers of the servers.

Answer

The exact lines of code in the model.

Answer

The home addresses of all developers.