AAIA Practice Test 3 — 30 Questions

Question 1

You are assessing a company's 'AI Governance Committee'. Which action best demonstrates effective board-level oversight?

Accepted Answer

Approving the annual AI strategy and corresponding risk mitigation plans. Board-level oversight in an AI Governance Committee centers on strategic direction and risk appetite, not operational execution. Approving the annual AI strategy and its corresponding risk mitigation plans is a fiduciary, governance-level action that sets direction, allocates accountability, and ensures risk is managed at the highest level. This mirrors how boards approve corporate strategy and enterprise risk registers rather than getting involved in tactical decisions.

Answer

Managing the day-to-day procurement of data labeling services

Answer

Reviewing individual code commits for AI model training

Answer

Selecting the machine learning framework used by data scientists

Question 2

What is the risk of 'Feature Skew' in an AI pipeline?

Accepted Answer

Incorrect model predictions. Feature skew (also called training-serving skew) occurs when the feature values used at training time differ from those computed at inference time — for example, different preprocessing, aggregation windows, or data sources. Because the model learned relationships based on the training distribution, mismatched inputs at serving time cause it to produce incorrect predictions. This is a data/feature consistency problem, not an infrastructure failure.

Answer

Database corruption

Answer

Increased hardware costs

Answer

System crash

Question 3

What is the most effective way to address 'Adversarial Input' during inference?

Accepted Answer

Adversarial training and input filtering. Implementing input validation/sanitization and adversarial training (training on adversarial examples) improves robustness.

Answer

Reducing training time

Answer

Increasing model depth

Answer

Ignoring the input

Question 4

For a production LLM deployment, which operational metric is most critical to monitor to identify 'hallucination' or grounding failure in real-time?

Accepted Answer

Context faithfulness score. Retrieval metrics, specifically faithfulness or answer relevance, indicate if the model is hallucinating outside of retrieved context.

Answer

Throughput (TPS)

Answer

Token generation latency

Answer

Error code distribution

Answer

Memory footprint

Question 5

You are auditing a model's 'Data Pre-processing' step. Which practice is a major concern regarding data privacy?

Accepted Answer

Storing training data that can be re-identified without proper access controls. Storing training data that can be re-identified without proper access controls is the core privacy risk: even 'anonymized' datasets can often be re-linked to individuals via quasi-identifiers, and without RBAC, encryption, or tokenization, that data becomes a direct PII exposure. Auditors flag this because it violates data-minimization and purpose-limitation principles.

Answer

Documenting the transformations applied to the data

Answer

Compressing the data to save storage costs

Answer

Using open-source data transformation libraries

Question 6

Which THREE factors should an auditor consider when evaluating the suitability of an AI model for production?

Accepted Answer

Inference latency and throughput. Option A (Inference latency and throughput) is correct because production suitability depends on whether the model can serve requests within required response-time SLAs and handle the expected request volume, making latency and throughput key operational metrics. Option C (Fairness and bias metrics) is correct because an auditor must verify that the model does not produce discriminatory or skewed outcomes across protected groups, which is a core governance and compliance concern. Option D (Model accuracy metrics) is correct because accuracy, precision, recall, F1, or similar performance measures determine whether the model reliably performs its intended task in production. Options B (Developer age demographics) and E (Operating system color schemes) are irrelevant to model suitability, as neither affects the model's performance, fairness, or operational viability.

Answer

Developer age demographics

Answer

Operating system color schemes

Question 7

Which TWO of the following are core components of an AI incident response strategy?

Accepted Answer

Defined communication protocols for internal and external stakeholders. Option B is correct because a core part of any AI incident response strategy is having defined communication protocols that specify how internal teams (e.g., security, legal, engineering) and external stakeholders (e.g., customers, regulators) are notified during an AI-related incident, ensuring timely and coordinated disclosure. Option D is correct because procedures for rolling back or disabling the model are essential containment and remediation mechanisms—if an AI model behaves harmfully or is compromised, responders must be able to revert to a prior version or shut it down to stop the impact. In contrast, option A (marketing campaigns for AI product launches) relates to go-to-market activities, not incident response. Option C (software license management for all developers) is a compliance and asset-management concern, not an incident response component. Option E (automated email server maintenance) is routine IT operations and unrelated to handling AI incidents.

Answer

Marketing campaigns for AI product launches

Answer

Software license management for all developers

Answer

Automated email server maintenance

Question 8

Which THREE of the following are core components of a robust AI Governance and Risk Assessment Framework?

Accepted Answer

Standardized model risk assessment templates. A robust AI Governance and Risk Assessment Framework requires standardized model risk assessment templates (B) so that every AI/ML model is evaluated consistently against the same criteria, such as fairness, explainability, robustness, and data quality, enabling comparable and auditable risk scoring across the model portfolio. It also requires defined AI risk appetite and threshold levels (D), because governance depends on explicit, approved boundaries that determine which risks are acceptable and trigger escalation, mitigation, or rejection decisions. Finally, it requires clearly assigned roles and responsibilities for model ownership and accountability (E), since frameworks such as NIST AI RMF and ISO/IEC 42001 rely on named accountable owners (e.g., model owners, validators, and oversight committees) to enforce controls and remediation. Network bandwidth optimization plans (A) are an infrastructure/performance concern, not a governance or risk-assessment component, and automated model training hyperparameter tuning (C) is an MLOps/model-development activity that improves model performance but does not establish governance, risk appetite, or accountability structures.

Answer

Network bandwidth optimization plans

Answer

Automated model training hyperparameter tuning

Question 9

Which role is primarily responsible for ensuring that the AI governance framework is followed in daily operations?

Accepted Answer

The Model Manager or Product Owner. The Model Manager or Product Owner is accountable for the day-to-day adherence to the AI governance framework, ensuring that models are developed, deployed, and monitored according to policy. This role bridges governance requirements and operational execution, making it the primary owner of daily compliance. They coordinate with data scientists, engineers, and risk teams to enforce controls.

Answer

The company's external auditor

Answer

The office manager

Answer

The chief financial officer

Question 10

During an AI audit, what is a 'PII Scrubber' tool used for?

Accepted Answer

Anonymizing sensitive data. A PII Scrubber is used to detect and anonymize personally identifiable information such as names, addresses, SSNs, and emails so that sensitive data can be used in AI training or analysis without exposing individuals. It supports privacy compliance by removing or masking identifiers before data flows into models or logs. This is a core privacy-preserving control in AI audits.

Answer

Cleaning up logs

Answer

Compressing images

Answer

Improving accuracy

Question 11

In Azure Machine Learning, which component should an auditor verify to ensure model registry access is restricted to authorized CI/CD service principals?

Accepted Answer

Workspace Access Control (IAM). Access to the Model Registry is governed by Azure RBAC and specifically Managed Identities/Service Principals assigned to the workspace.

Answer

Environment YAML files

Answer

Compute Instance logs

Answer

Data labeling projects

Question 12

You find that the 'Inference API' for a model allows 'Prompt Injection' attacks. Which governance failure is most likely?

Accepted Answer

Inadequate input validation controls and adversarial testing. Prompt injection is a failure of input validation and security design within the model governance framework.

Answer

Lack of a social media presence for the company

Answer

Using a low-cost cloud provider

Answer

Failure to update the company logo

Question 13

If an AI pipeline fails, which component provides the best 'log' to diagnose the failure?

Accepted Answer

Orchestrator task execution logs. The pipeline orchestrator's task logs indicate specifically which step failed and why.

Answer

Model documentation

Answer

Company mission statement

Answer

Developer's email

Question 14

You are auditing a model's 'Robustness' against adversarial attacks. Which test is most appropriate?

Accepted Answer

Performing adversarial perturbation testing. Adversarial testing involves introducing perturbations to inputs to see if the model's output changes unexpectedly.

Answer

Checking if the server is plugged in

Answer

Checking the model's training accuracy

Answer

Asking the team to describe their favorite movies

Question 15

What is 'Model Versioning' used for in an AI project?

Accepted Answer

Ensuring reproducibility and rollback. It allows tracking changes, rolling back to previous versions, and ensuring reproducibility.

Answer

Reducing model size

Answer

Improving training time

Answer

Increasing security

Question 16

You are auditing a model using IBM Watson OpenScale. Which feature should be configured to detect 'Disparate Impact' to ensure regulatory compliance?

Accepted Answer

Fairness monitor. The Fairness monitor in IBM Watson OpenScale is specifically designed to detect and measure bias in machine learning models, including metrics like Disparate Impact. It evaluates whether a model's predictions disproportionately favor or disadvantage certain groups based on protected attributes (e.g., gender, race). This directly supports regulatory compliance by identifying discriminatory outcomes, making it the correct choice for detecting Disparate Impact.

Answer

Quality monitor

Answer

Drift detection monitor

Answer

Explainability monitor

Answer

Payload logging

Question 17

When auditing an MLflow experiment, which object allows the auditor to verify the exact parameters and code version used during training?

Accepted Answer

MLflow Run. An MLflow Run is the fundamental unit that captures all details of a single execution of a machine learning experiment, including parameters, metrics, artifacts, and the source code version (via Git commit or other tags). Therefore, the Run object allows an auditor to verify the exact parameters and code version used during training. The Model Registry manages model versions and stages but does not store the original training parameters or code version directly.

Answer

MLflow Model Registry

Answer

MLflow Experiment

Answer

MLflow Tracking Server

Question 18

Which THREE factors must be evaluated when assessing the 'Fairness' of an AI model?

Accepted Answer

Disparities in model error rates across protected groups. Fairness assessment requires examining disparities in model error rates across protected groups (B), because unequal false-positive or false-negative rates (e.g., via equalized odds or error-rate parity) directly reveal discriminatory performance. Historical bias present in the training datasets (C) must be evaluated since biased or unrepresentative data propagates and amplifies unfair outcomes in the model. Demographic parity metrics for the model's predictions (D) are also essential, as they quantify whether favorable outcomes are distributed equally across groups (e.g., via disparate impact ratio). The programming language used to build the model (A) is irrelevant to fairness, as fairness depends on data, features, and evaluation metrics rather than implementation language. The number of CPU cores allocated for training (E) affects computational performance only, not the fairness of the model's predictions.

Answer

The programming language used to build the model

Answer

The number of CPU cores allocated for training

Question 19

What is the consequence of 'Pipeline Staleness' in an AI system?

Accepted Answer

Model degradation due to outdated data. If the pipeline is not refreshed, it trains on outdated data, leading to model degradation.

Answer

Unauthorized access

Answer

Increased hardware costs

Answer

System crash

Question 20

You are auditing a 'Model Monitoring' dashboard. Which metric is most indicative of a potential degradation in the model's reliability over time?

Accepted Answer

Statistical drift between the production input data and training baseline. Data drift and concept drift are primary indicators that a model's performance may be degrading in a production environment.

Answer

The physical location of the cloud server hosting the model

Answer

The number of lines of code in the model deployment script

Answer

The number of users accessing the model API