AAIA Practice Test 3 — 25 Questions

Question 1

Which TWO metrics provide the best insights into 'Model Quality' during an audit?

Accepted Answer

Recall score. Precision and Recall are fundamental metrics for assessing the quality of classification models.

Answer

Disk space usage

Answer

Number of server restarts

Answer

Team's average coffee intake

Question 2

Which TWO of the following are examples of 'Data Leakage' that an auditor should look for in a pipeline?

Accepted Answer

Training on test data. Target leakage and future data leakage are primary types of data leakage in AI pipelines.

Answer

Using cloud-based storage

Answer

Documenting model parameters

Answer

Using open-source libraries

Question 3

When auditing an AI system for regulatory compliance, you find that data lineage is broken between the feature store and the training pipeline. Which control is most likely deficient?

Accepted Answer

Metadata logging and transformation documentation. Feature stores must maintain end-to-end traceability to ensure that data provenance is clear and auditable.

Answer

Access control lists for the model production environment

Answer

Encryption of data at rest in the data warehouse

Answer

Load balancing configuration for the inference API

Question 4

Which TWO of the following are primary components of an AI model risk management inventory?

Accepted Answer

Training data provenance and lineage information. An inventory must track metadata and lifecycle status for accountability.

Answer

Employee performance review scores

Answer

List of office furniture used by the team

Answer

Marketing budget for the model launch

Question 5

Which environment should be used to perform 'Model Validation' before moving to production?

Accepted Answer

Staging Environment. Staging or UAT environments mirror production and are used for testing.

Answer

Training Cluster

Answer

Development Environment

Answer

Public Internet

Answer

Local Desktop

Question 6

Which metric would best detect 'Label Drift' in a production model?

Accepted Answer

Target distribution shift (PSI). Label drift occurs when the output (target) distribution changes significantly from training.

Answer

Model latency

Answer

CPU utilization

Answer

Number of users

Question 7

What is the most effective way to address 'Adversarial Input' during inference?

Accepted Answer

Adversarial training and input filtering. Implementing input validation/sanitization and adversarial training (training on adversarial examples) improves robustness.

Answer

Reducing training time

Answer

Increasing model depth

Answer

Ignoring the input

Question 8

Which THREE items should be tracked in a 'Model Registry' for audit readiness?

Accepted Answer

Training dataset lineage. Registration must link code, data, and model artifacts.

Answer

Cloud provider logo

Answer

Employee birthday

Question 9

During an AI risk assessment, you notice that the 'Model Risk Management' (MRM) framework does not mandate independent validation for models categorized as 'High Impact'. What is the most significant consequence?

Accepted Answer

Increased probability of undetected systemic model bias or failure. Independent validation is a core pillar of model risk governance, especially for high-impact systems.

Answer

Inability to detect model drift in production environments

Answer

Non-compliance with internal cybersecurity patching cycles

Answer

Operational silos between the AI team and the IT department

Question 10

What is the goal of a 'Human-in-the-loop' (HITL) control in an AI system?

Accepted Answer

To provide human oversight for automated decisions to mitigate risk and error. HITL ensures that a human makes final decisions for high-stakes AI outputs, reducing the risk of automation error.

Answer

To reduce the time spent on manual data entry

Answer

To replace all human employees with automated systems

Answer

To increase the speed of model training

Question 11

You are assessing 'Model Security'. Which attack vector specifically exploits the AI training phase?

Accepted Answer

Data poisoning. Data poisoning involves injecting malicious data into the training set to influence the model's behavior.

Answer

Man-in-the-middle attack

Answer

Denial-of-service attack

Answer

Phishing attack

Question 12

During an audit of AI procurement, you find that the 'Vendor AI Assessment' questionnaire is missing. What is the correct next step?

Accepted Answer

Immediately halt the use of the vendor until a formal risk assessment is completed. An auditor must ensure that third-party AI risks are evaluated using a formal, documented process.

Answer

None of the above

Answer

Update the office building safety guidelines

Answer

Ask the vendor for their marketing presentation

Answer

Ignore the missing document if the vendor is well-known

Question 13

Which THREE components are critical for a robust 'AI Governance' framework?

Accepted Answer

Established AI ethics and compliance policies. Effective governance requires policy, accountability, and continuous monitoring.

Answer

Increasing the size of the marketing team

Answer

Choosing the most expensive cloud platform

Question 14

When auditing an 'AI Model Lifecycle', which phase occurs immediately after the 'Model Training' phase?

Accepted Answer

Model Validation. After training, the model must be validated before it is deployed to production.

Answer

Writing the initial requirements

Answer

Marketing the model to customers

Answer

Decommissioning

Question 15

You are reviewing a model risk management (MRM) framework for an AI system using NVIDIA NeMo Guardrails. What is the primary purpose of this tool in a risk mitigation strategy?

Accepted Answer

To enforce safety boundaries on LLM inputs and outputs. NeMo Guardrails allows developers to define programmatic boundaries for LLM behavior, effectively serving as an automated control for AI governance.

Answer

To automate model retraining

Answer

To improve model training speed

Answer

To encrypt training datasets

Question 16

What is the primary function of a 'Canary Deployment' in AI model release?

Accepted Answer

Gradual rollout to test stability. Releasing the new model to a small subset of traffic to monitor performance before full rollout.

Answer

To update documentation

Answer

To delete old models

Answer

To increase training performance

Answer

To encrypt model weights

Question 17

In an AI Audit, what does 'Model Fairness' assessment typically involve?

Accepted Answer

Disparity analysis across segments. Checking for bias across protected groups (e.g., race, gender) in model performance metrics.

Answer

Counting model parameters

Answer

Checking model speed

Answer

Verifying code documentation

Question 18

Which TWO are standard practices for securing AI model endpoints?

Accepted Answer

Mutual TLS (mTLS). Security requires network and authentication controls.

Answer

Unprotected endpoints

Answer

Storing logs on public S3

Answer

Public internet access

Question 19

What is the primary function of an 'AI Ethics Board'?

Accepted Answer

Reviewing AI projects for alignment with corporate values and societal norms. The ethics board ensures that AI projects align with corporate values and societal standards.

Answer

Managing the HR payroll systems

Answer

Fixing bugs in the server infrastructure

Answer

Writing the code for all production models

Question 20

Which AI risk is best mitigated by conducting a 'bias audit' on the training dataset?

Accepted Answer

Unfairness and discrimination in model outputs. Bias audits analyze datasets for underrepresented groups or historical prejudices that would lead to discriminatory outputs.

Answer

Hardware overheating

Answer

System latency

Answer

Server cost overruns