AAIA Practice Test 6 — 10 Questions

Question 1

Which TWO metrics provide the best insights into 'Model Quality' during an audit?

Accepted Answer

Recall score. Precision and Recall are fundamental metrics for assessing the quality of classification models.

Answer

Disk space usage

Answer

Number of server restarts

Answer

Team's average coffee intake

Question 2

Which TWO of the following are examples of 'Data Leakage' that an auditor should look for in a pipeline?

Accepted Answer

Training on test data. Target leakage and future data leakage are primary types of data leakage in AI pipelines.

Answer

Using cloud-based storage

Answer

Documenting model parameters

Answer

Using open-source libraries

Question 3

When auditing an AI system for regulatory compliance, you find that data lineage is broken between the feature store and the training pipeline. Which control is most likely deficient?

Accepted Answer

Metadata logging and transformation documentation. Feature stores must maintain end-to-end traceability to ensure that data provenance is clear and auditable.

Answer

Access control lists for the model production environment

Answer

Encryption of data at rest in the data warehouse

Answer

Load balancing configuration for the inference API

Question 4

Which TWO of the following are primary components of an AI model risk management inventory?

Accepted Answer

Training data provenance and lineage information. An inventory must track metadata and lifecycle status for accountability.

Answer

Employee performance review scores

Answer

List of office furniture used by the team

Answer

Marketing budget for the model launch

Question 5

Which environment should be used to perform 'Model Validation' before moving to production?

Accepted Answer

Staging Environment. Staging or UAT environments mirror production and are used for testing.

Answer

Training Cluster

Answer

Development Environment

Answer

Public Internet

Answer

Local Desktop

Question 6

Which metric would best detect 'Label Drift' in a production model?

Accepted Answer

Target distribution shift (PSI). Label drift occurs when the output (target) distribution changes significantly from training.

Answer

Model latency

Answer

CPU utilization

Answer

Number of users

Question 7

What is the most effective way to address 'Adversarial Input' during inference?

Accepted Answer

Adversarial training and input filtering. Implementing input validation/sanitization and adversarial training (training on adversarial examples) improves robustness.

Answer

Reducing training time

Answer

Increasing model depth

Answer

Ignoring the input

Question 8

Which THREE items should be tracked in a 'Model Registry' for audit readiness?

Accepted Answer

Training dataset lineage. Registration must link code, data, and model artifacts.

Answer

Cloud provider logo

Answer

Employee birthday

Question 9

During an AI risk assessment, you notice that the 'Model Risk Management' (MRM) framework does not mandate independent validation for models categorized as 'High Impact'. What is the most significant consequence?

Accepted Answer

Increased probability of undetected systemic model bias or failure. Independent validation is a core pillar of model risk governance, especially for high-impact systems.

Answer

Inability to detect model drift in production environments

Answer

Non-compliance with internal cybersecurity patching cycles

Answer

Operational silos between the AI team and the IT department

Question 10

What is the goal of a 'Human-in-the-loop' (HITL) control in an AI system?

Accepted Answer

To provide human oversight for automated decisions to mitigate risk and error. HITL ensures that a human makes final decisions for high-stakes AI outputs, reducing the risk of automation error.

Answer

To reduce the time spent on manual data entry

Answer

To replace all human employees with automated systems

Answer

To increase the speed of model training