AAIA Practice Test 2 — 10 Questions

Question 1

Which TWO techniques should an auditor employ to detect bias in a model where the training data has imbalanced demographic representation?

Accepted Answer

Implementation of fairness-aware loss functions. Fairness-aware loss functions (B) are correct because they explicitly incorporate group-fairness constraints or penalties (e.g., equalized odds, demographic parity regularization) into the model's objective during training, directly counteracting bias caused by imbalanced demographic representation. Stratified cross-validation (E) is correct because it preserves the proportional class and demographic distribution in each fold, ensuring that performance and bias metrics are evaluated reliably across underrepresented groups rather than being masked by skewed splits. The remaining options do not belong: updating the corporate logo (A) and changing the font size in the UI (C) are cosmetic and have no effect on model training or bias, while increasing the number of project meetings (D) is a process activity that does not alter data imbalance or model fairness.

Answer

Updating the corporate logo

Answer

Changing the font size in the UI

Answer

Increasing the number of project meetings

Question 2

What is the primary function of an 'AI Audit Log'?

Accepted Answer

To track and document all interactions and changes made to the AI system. An audit log records all actions performed by or on the AI system, ensuring accountability and traceability.

Answer

To automatically order pizza for the team

Answer

To store the personal contact information of the IT staff

Answer

To provide entertainment for employees during breaks

Question 3

Which TWO of the following are essential for ensuring 'Model Observability' in an enterprise production environment?

Accepted Answer

System latency monitoring. Observability requires both performance metrics and input monitoring (drift).

Answer

Manual code refactoring

Answer

Database backup frequency

Answer

Developer salary audits

Question 4

What is the purpose of 'Model Validation' in the audit lifecycle?

Accepted Answer

To verify model performance on test data. Model validation is the process of evaluating a trained model's performance on a separate test dataset to ensure it generalizes well and meets the intended objectives. It verifies metrics such as accuracy, precision, recall, or fairness before deployment, catching overfitting or data leakage issues.

Answer

To speed up data ingestion

Answer

To lower API costs

Answer

To increase training time

Question 5

When auditing data pipelines, what is the function of a 'Data Quality' assertion?

Accepted Answer

To validate data integrity before training. Assertions (e.g., in Great Expectations) check if data meets predefined schemas and constraints before ingestion.

Answer

To rename data columns

Answer

To increase prediction speed

Answer

To compress input files

Question 6

You are assessing 'Model Security'. Which attack vector specifically exploits the AI training phase?

Accepted Answer

Data poisoning. Data poisoning involves injecting malicious data into the training set to influence the model's behavior.

Answer

Man-in-the-middle attack

Answer

Denial-of-service attack

Answer

Phishing attack

Question 7

Which document is the primary source for establishing the scope of an AI audit?

Accepted Answer

The internal AI Policy and the organization's Risk Management Framework. The internal AI Policy defines the rules, boundaries, and requirements for AI use, while the Risk Management Framework establishes how risks are identified, assessed, and mitigated. Together they define what an AI audit must cover, including scope, criteria, and risk areas. Auditors use these documents to determine which systems, processes, and controls fall within the audit boundary.

Answer

A list of all employees who have taken an AI training course

Answer

The company's social media posting guidelines

Answer

The employee handbook for the engineering department

Question 8

When drafting an AI policy, you need to define the 'Human-in-the-Loop' (HITL) requirement. Which of the following represents a best-practice control for HITL?

Accepted Answer

Manual review of high-risk model outputs prior to execution. Human-in-the-Loop (HITL) is a control pattern where a human reviews and approves model outputs before consequential actions are taken. The best-practice control is targeted manual review of high-risk outputs prior to execution, which balances risk mitigation with operational efficiency by focusing human attention where the impact is greatest. This aligns with risk-based AI governance frameworks that scale oversight to the severity of the decision.

Answer

Logging all model interactions for post-mortem analysis

Answer

Setting all model confidence thresholds to 100%

Answer

Automated approval for all decisions

Question 9

What does a 'Backtesting' report verify in an AI Audit?

Accepted Answer

Historical model performance. Backtesting uses historical data to verify how the model *would* have performed, ensuring reliability.

Answer

Future accuracy

Answer

Cloud service costs

Answer

Developer salary

Question 10

In a CI/CD pipeline for AI, where should the 'Model Validation' gate be located?

Accepted Answer

Between training and registration. Validation must occur after training but before model registration to prevent bad models from reaching production.

Answer

During feature engineering

Answer

Before source code check-in

Answer

After deployment