AAIA Timed Practice Test 3 — 10 Questions

Question 1

When documenting audit findings for an AI system, which TWO of the following are critical to include to ensure the audit can be replicated?

Accepted Answer

Specific Git commit SHA for the model code. Version control information and the exact dataset version are essential for reproducing the audit results.

Answer

The office location of the data scientists

Answer

The brand of the coffee machine used

Answer

The developer's personal email address

Question 2

You are auditing a deployment on Kubernetes using Kubeflow. Which component should the auditor examine to ensure that the pipeline steps are reproducible?

Accepted Answer

Kubeflow Pipelines. Kubeflow Pipelines (KFP) ensure reproducibility through containerized steps and pipeline definitions stored in YAML.

Answer

Kubelet

Answer

Prometheus

Answer

Istio

Question 3

An auditor is evaluating an AI system for 'Model Inversion' risk. What is this?

Accepted Answer

Reconstructing training data from outputs. Model Inversion is an attack where an adversary reconstructs training data from model outputs.

Answer

Reducing inference latency

Answer

Increasing training bias

Answer

Overfitting the data

Question 4

What is the primary role of a 'Human-in-the-Loop' (HITL) audit requirement?

Accepted Answer

To review critical AI decisions. HITL ensures that sensitive or critical decisions made by AI are reviewed by human experts.

Answer

To automate model retraining

Answer

To generate marketing reports

Answer

To improve GPU performance

Question 5

When auditing model explainability, why is it risky to rely solely on 'Global Feature Importance'?

Accepted Answer

It fails to explain individual decisions. Global importance explains the model overall, but individual predictions (local) may be driven by different factors.

Answer

It lacks local prediction accuracy

Answer

It uses too much memory

Answer

It is only for regression models

Question 6

An auditor is evaluating the 'Safety Filter' of an LLM. Which approach is most suitable for detecting 'jailbreak' vulnerabilities?

Accepted Answer

Adversarial red teaming. Red teaming with adversarial prompts (jailbreaks) is the standard method for testing safety filter resilience.

Answer

Running unit tests for code

Answer

Reviewing server logs

Answer

Checking network bandwidth

Question 7

Which THREE factors should an auditor consider when evaluating the suitability of an AI model for production?

Accepted Answer

Inference latency and throughput. Accuracy, fairness metrics, and latency/throughput are standard KPIs for production readiness.

Answer

Developer age demographics

Answer

Operating system color schemes

Question 8

When documenting findings, which THREE elements should be included for each finding?

Accepted Answer

The root cause. The condition, the root cause, and the recommendation are essential components of an audit finding.

Answer

The font size used in the report

Answer

The names of all employees who disagreed

Question 9

Which TWO of the following are examples of AI governance 'Inputs'?

Accepted Answer

Internal AI Policy and guidelines. Inputs include regulatory requirements and internal policy documents.

Answer

A photograph of the company founder

Answer

The company's social media password

Answer

The office printer manual

Question 10

An organization is using 'Federated Learning' to maintain data privacy while training models. What is the primary audit risk associated with this architecture?

Accepted Answer

Model inversion attacks targeting the central aggregator. In Federated Learning, local model updates (gradients) can potentially reveal information about the underlying training data, necessitating 'Differential Privacy' as a control.

Answer

Increased training latency

Answer

Lack of access to training hardware

Answer

Centralized data breaches