AAIR Flashcards — Free ISACA Advanced in AI Risk (AAIR) (AAIR) Study Cards

Demographic parity ratio in training data samples

Demographic parity ratio in training data samples is a leading indicator because it measures the representation of protected groups before the model is deployed. A skewed ratio (e.g., far from 1.0) signals that the model may learn biased patterns, allowing proactive mitigation. Unlike lagging metrics such as complaints, it provides early warning during data collection and preprocessing.

Identifying and documenting the context and intended use

Identifying and documenting the context and intended use is the foundation of mapping AI risks to an enterprise framework.

During the data ingestion and preprocessing stage.

Data quality assessment must happen during data ingestion and preprocessing, because that is when data is collected, cleaned, validated, and transformed before it reaches model training. Catching quality issues here prevents garbage-in-garbage-out propagation and is far cheaper than remediating after deployment. This early-stage control minimizes long-term risk across the entire AI lifecycle.

The test methodology, criteria for success, and the specific mitigation steps taken.

For an AI bias test audit trail, the defensible record must show how the test was conducted, what threshold defined pass/fail, and what was done when bias was detected — this is what regulators and auditors need to reconstruct the decision. Documenting methodology, success criteria, and mitigation steps demonstrates due diligence and reproducibility, which is the core purpose of an audit trail. Operational trivia like hours, salaries, or tool names does not establish that the model was actually tested for fairness.

To ensure auditability, reproducibility, and the ability to roll back to a known stable state.

Model versioning exists to provide auditability (who changed what and when), reproducibility (recreate any past model and its results), and rollback capability (revert to a known stable version when a new release degrades). These properties are foundational for incident response, regulatory review, and safe continuous delivery of AI systems.

To provide high-level visibility into AI risk posture and trends

An AI Risk Dashboard aggregates key risk indicators, model performance metrics, and compliance status into a concise executive view. Its primary purpose is to give leadership high-level visibility into the organization's AI risk posture and emerging trends so they can make informed strategic decisions. It does not replace governance bodies or expose raw technical logs.

Ensuring that there is clear ownership and responsibility for AI system performance and safety.

The Accountability principle in the NIST AI RMF (part of the Govern function) is about ensuring clear ownership and responsibility for AI system performance, safety, and outcomes throughout the lifecycle. It answers 'who is responsible?' so that risks are managed, issues are escalated, and redress is possible. It is not about explainability, bug-freeness, or speed.

The definition of roles and responsibilities

A key component of AI Risk Governance is the definition of roles and responsibilities, which establishes accountability for risk management across the organization. This includes specifying who owns AI risks, who approves deployments, and who monitors ongoing performance. Without clear roles, governance breaks down and risks go unmanaged.

To confirm the vendor's model satisfies the internal 'Model Card' and transparency requirements for risk assessment.

The primary governance objective when evaluating a vendor's AI model transparency is to ensure the vendor provides sufficient documentation — such as model cards, datasheets, and transparency reports — to assess risks related to bias, explainability, and compliance. Confirming that the vendor's model satisfies internal Model Card and transparency requirements enables the organization to perform due diligence and integrate the model into its risk management framework.

AI models deployed in production are missing from the centralized enterprise risk register.

A centralized enterprise risk register is the authoritative inventory of organizational risks; if production AI models are absent from it, the governance structure has failed to integrate AI risk into ERM — meaning those models are unmanaged, unreviewed, and invisible to executive and Board oversight. This is a structural governance failure because it breaks the linkage between AI risk identification and enterprise-level risk aggregation, violating the principle that all material risks must flow into the ERM.

Require Data Science to translate technical model output into plain language documentation

Explainability requirements exist so that non-technical stakeholders — Legal, compliance, regulators, and affected individuals — can understand how a model reaches decisions. The Data Science team owns the technical model and is best positioned to translate its behavior into plain-language documentation that Legal can review and rely on. This preserves the division of labor: Data Science explains, Legal interprets and applies.

Design and Problem Formulation.

The Design and Problem Formulation phase is most critical for initial risk mitigation because this is where the problem is scoped, objectives are defined, and potential risks are identified before any model is built. Decisions made here — such as choosing the use case, defining success metrics, identifying affected stakeholders, and assessing ethical/legal constraints — determine the risk profile of the entire project. Once a model is trained and deployed, many risks are already baked in and far more costly to remediate.

Apply synthetic data generation techniques to balance the training distribution.

When a training dataset underrepresents a demographic, the most effective proactive mitigation at the design phase is to correct the data distribution itself — synthetic data generation (e.g., SMOTE, CTGAN, or targeted augmentation) can increase representation of the minority group so the model learns its patterns rather than defaulting to the majority. Addressing the root cause in the data is preferable to downstream patches because it improves model behavior for the affected group across all future inferences.

Creating an enterprise AI inventory

A sustainable AI risk program requires knowing what AI exists across the enterprise — an AI inventory is the foundational artifact that every other activity (policy, monitoring, audit, vendor review) depends on. Without it, risk assessments are incomplete and controls cannot be scoped. This is why inventory creation is consistently the first and most cited foundational requirement.

Identify and classify all AI applications by criticality and risk level.

Integrating AI risk into ERM begins with discovery and classification — you cannot govern, monitor, or write policy for AI applications you haven't identified. Identifying and classifying all AI applications by criticality and risk level establishes the risk register baseline that every subsequent control (monitoring, policy, ownership) depends on. This mirrors the standard ERM sequence: identify, assess, treat, monitor.

Comprehensive audit trails of model lineage, versions, and validation results.

External regulators require demonstrable traceability: who built the model, what data it used, which version is in production, and what validation it passed. Comprehensive audit trails of model lineage, versions, and validation results (B) provide exactly this evidence and are the cornerstone of MRM compliance. The other options are cosmetic or irrelevant to regulatory requirements.

They do not capture social and compliance risk dimensions

Model performance metrics (accuracy, precision, recall, latency) measure technical behavior but say nothing about whether the model complies with regulations, treats protected groups fairly, respects privacy, or aligns with organizational values. Risk management must span social, ethical, legal, and compliance dimensions that performance metrics cannot capture. Relying on performance alone leaves material risk unmanaged.

Operational risk of deploying a poorly performing model to production.

Shadow deployments allow real-world data validation without exposing end-users to potential model failures.

To enable the ethical and safe adoption of AI technologies.

The key objective of an AI Risk Management Program is to enable the ethical and safe adoption of AI technologies by identifying, assessing, mitigating, and monitoring AI-specific risks such as bias, opacity, privacy violations, and safety failures. It supports responsible innovation rather than blocking AI use. This aligns with frameworks like NIST AI RMF and ISO/IEC 42001.