ISACA Advanced in AI Risk (AAIR) (AAIR) (AAIR) — Questions 1–75

The model should be assessed for retraining and the production environment should be audited.

Identifying and documenting the context and intended use is the foundation of mapping AI risks to an enterprise framework.

Risk is measured by performance stability (drift), accuracy, and operational health.

The 1st line (Model Owners) and 2nd line (Risk/Compliance) are critical for effective AI risk management.

Mapping specific risk indicators to model performance monitoring allows metrics to evolve as the model's environment changes.

The Risk Assessment report documents the analysis, and the Mitigation Plan shows how identified risks will be addressed.

The first step is identifying the AI system and defining its intended purpose (Context).

Board oversight of AI requires clear policies, defined accountability, and reporting mechanisms that link AI performance to the enterprise's risk and compliance goals.

Risk appetite must be specific to the impact and the tolerance levels for failure.

Shadow deployments allow real-world data validation without exposing end-users to potential model failures.

The Third Line provides independent, objective assurance on the effectiveness of the risk management and governance processes.

Risk culture maturity is demonstrated by transparently reporting not just incidents, but the effectiveness of the risk management process itself, including the 'Risk Appetite' vs. 'Actual Risk' gap.

The RACI matrix clearly defines who is Responsible, Accountable, Consulted, and Informed for AI risk activities.

Post-implementation reviews assess whether the AI model's performance and risk levels remain consistent with the original design and risk appetite.

AI risks should not be siloed; they should be categorized alongside existing operational and reputational risks to ensure they are visible to the Board.

A complete report covers the risk identification, the impact analysis, and the control strategy.

Explainability allows stakeholders to audit decisions and ensure they comply with regulatory requirements (like GDPR).

Lineage allows for traceability, enabling auditability and root cause analysis in case of a model failure.

An asset register must track the model's purpose, data sources, and business owner to ensure oversight.

HITL ensures that human judgment and ethics remain integrated into high-risk AI decision processes to prevent unmonitored failures.

Model cards provide transparency and documentation about the intended use, limitations, and performance characteristics.

The Board/Executive Leadership defines the appetite, while Legal/Compliance ensures that the definition meets legal requirements.

Without human checks, bad models can propagate quickly, causing cascading failures and security gaps.

The Chief Legal Officer or Legal counsel is accountable for IP rights and contractual obligations related to AI output.

A cross-functional approach involving data science, legal, compliance, and IT security ensures risk is addressed at every SDLC phase.

Immediate containment is necessary to prevent further harm while the root cause is investigated.

The 'Manage' function involves implementing controls. Essential components include clear policies on prioritization, documented operational procedures, and a process for ongoing monitoring and improvement.

Data quality must be validated prior to training to ensure model reliability.

Feature distribution, target distribution, and model confidence scores all provide signals of drift.

Continuous monitoring and version control are fundamental for governing the model's entire lifespan.

COSO ERM requires integrating risk into strategy and performance. Governance structures and risk appetite alignment are key.

Feedback loops occur when model outputs influence the future training data, which must be detected by comparing output distributions over time.

AI systems impacting physical safety or fundamental rights require the highest level of governance oversight.

A policy needs to define roles, risk classification, and the lifecycle process.

Shadow AI refers to AI tools or models used within an organization without the knowledge or approval of the IT/Risk governance teams.

Data quality and security (access control) are the pillars of data governance for AI.

Model lineage, inference logs, and the original model card are critical for understanding *what* happened.

The manager should facilitate a risk assessment to quantify the concern and identify potential technical controls to mitigate the privacy risk.

Model drift and retraining frequency serve as leading indicators for the degradation of AI reliability over time.

A/B testing (or canary deployments) allows for traffic splitting to validate performance before full rollout.

The second line acts as the risk management and compliance oversight function, separate from the business owners.

Data lineage, model validation results, and incident response logs are critical audit points for enterprise risk accountability.

Legal/Compliance and Data Privacy are essential for addressing the regulatory and ethical risks inherent in AI deployments.

Bias can enter the model through the training data, the algorithm's objective function, or through post-processing of results.

The Chief Risk Officer (CRO) is responsible for the enterprise risk framework. Their involvement is essential to ensure AI risks are measured consistently with other business risks.

Online learning models are constantly updating, making them highly susceptible to 'poisoning' attacks.

Counterfactual explanations (e.g., 'What would have to change for this decision to be different?') are highly intuitive and effective for non-technical stakeholders.

Risk appetite must consider the organization's business strategy, regulatory environment, and technical capability.

Concept drift occurs when the relationship between input variables and target variables changes, requiring adjustments to data drift detection parameters.

Right-to-audit clauses ensure that the organization can verify the vendor's AI risk controls, which is essential for third-party risk management.

When ground truth is delayed, practitioners must rely on feature drift detection rather than accuracy metrics.

The potential for reputational or legal harm (consequence) is the most critical component when assessing the impact of AI-generated content.

The rate of false negatives in a credit approval model is a direct measure of AI-specific operational risk.

The model owner is accountable for the entire lifecycle and risk profile of a specific AI model.

Management needs to see the current risk levels and compliance status in a summarized format.

The inventory provides a central repository for tracking all AI assets, which is the baseline for risk management.

An AI chatbot producing hallucinated information is a clear AI risk involving inaccuracy and potential harm.

Intended use and performance boundaries are essential for governance.

Standardizing a taxonomy ensures that all business units report risks using the same language and impact levels.

Documentation completeness (Model Cards) provides a standardized measure of transparency, enabling auditors to assess accountability.

Real-time performance metrics, a process for human oversight, and a log of model decisions are foundational for continuous monitoring.

Risk appetite for AI depends on the domain, data sensitivity, and potential for harm.

The Risk Appetite Statement is the formal document approved by the board or senior management that defines the level of risk the organization is willing to accept in pursuit of objectives, including AI-specific tolerances.

Criticality of the decision, the nature of the data involved, and the potential impact on individuals are the standard pillars of AI risk assessment.

Oversight requires representatives from business, technology, and risk management.

Cross-functional engagement ensures that legal, ethics, and technical teams validate the model against disparate impact standards.

Coordination requires clear communication, defined roles, and a shared vocabulary.

Data leakage is a significant risk when sending proprietary or sensitive data to external AI service providers.

Data lineage allows you to trace data back to its source, which is critical for identifying and mitigating bias or training data contamination.

Integrating privacy controls into the development lifecycle (DevOps/MLOps) is a key requirement of modern AI governance frameworks.

This is a Data Quality/Data Representation risk, where the model's understanding of data is limited by the training corpus.

Human fatigue and cognitive bias can lead to poor oversight, potentially creating a false sense of security.

Drift metrics indicate whether the model's accuracy is degrading, which is a proxy for reliability.

The 'Measure' function uses quantitative and qualitative assessments to test whether the model is meeting its goals and risk requirements.

An inventory ensures that no 'shadow AI' exists, allowing for comprehensive risk assessment and oversight.