AAIA Practice Exam 2 — 20 Questions

Question 1

When documenting audit findings for an AI system, which TWO of the following are critical to include to ensure the audit can be replicated?

Accepted Answer

Specific Git commit SHA for the model code. Version control information and the exact dataset version are essential for reproducing the audit results.

Answer

The office location of the data scientists

Answer

The brand of the coffee machine used

Answer

The developer's personal email address

Question 2

You are auditing a deployment on Kubernetes using Kubeflow. Which component should the auditor examine to ensure that the pipeline steps are reproducible?

Accepted Answer

Kubeflow Pipelines. Kubeflow Pipelines (KFP) ensure reproducibility through containerized steps and pipeline definitions stored in YAML.

Answer

Kubelet

Answer

Prometheus

Answer

Istio

Question 3

An auditor is evaluating an AI system for 'Model Inversion' risk. What is this?

Accepted Answer

Reconstructing training data from outputs. Model Inversion is an attack where an adversary reconstructs training data from model outputs.

Answer

Reducing inference latency

Answer

Increasing training bias

Answer

Overfitting the data

Question 4

What is the primary role of a 'Human-in-the-Loop' (HITL) audit requirement?

Accepted Answer

To review critical AI decisions. HITL ensures that sensitive or critical decisions made by AI are reviewed by human experts.

Answer

To automate model retraining

Answer

To generate marketing reports

Answer

To improve GPU performance

Question 5

When auditing model explainability, why is it risky to rely solely on 'Global Feature Importance'?

Accepted Answer

It fails to explain individual decisions. Global importance explains the model overall, but individual predictions (local) may be driven by different factors.

Answer

It lacks local prediction accuracy

Answer

It uses too much memory

Answer

It is only for regression models

Question 6

An auditor is evaluating the 'Safety Filter' of an LLM. Which approach is most suitable for detecting 'jailbreak' vulnerabilities?

Accepted Answer

Adversarial red teaming. Red teaming with adversarial prompts (jailbreaks) is the standard method for testing safety filter resilience.

Answer

Running unit tests for code

Answer

Reviewing server logs

Answer

Checking network bandwidth

Question 7

Which THREE factors should an auditor consider when evaluating the suitability of an AI model for production?

Accepted Answer

Inference latency and throughput. Accuracy, fairness metrics, and latency/throughput are standard KPIs for production readiness.

Answer

Developer age demographics

Answer

Operating system color schemes

Question 8

When documenting findings, which THREE elements should be included for each finding?

Accepted Answer

The root cause. The condition, the root cause, and the recommendation are essential components of an audit finding.

Answer

The font size used in the report

Answer

The names of all employees who disagreed

Question 9

Which TWO of the following are examples of AI governance 'Inputs'?

Accepted Answer

Internal AI Policy and guidelines. Inputs include regulatory requirements and internal policy documents.

Answer

A photograph of the company founder

Answer

The company's social media password

Answer

The office printer manual

Question 10

An organization is using 'Federated Learning' to maintain data privacy while training models. What is the primary audit risk associated with this architecture?

Accepted Answer

Model inversion attacks targeting the central aggregator. In Federated Learning, local model updates (gradients) can potentially reveal information about the underlying training data, necessitating 'Differential Privacy' as a control.

Answer

Increased training latency

Answer

Lack of access to training hardware

Answer

Centralized data breaches

Question 11

Which THREE of the following are potential risks associated with AI model 'Drift' that an auditor must account for?

Accepted Answer

Bias against specific demographic groups that emerged post-deployment. Drift impacts operational accuracy, fairness, and compliance, making it a critical governance risk.

Answer

Increased electricity consumption of servers

Answer

Inability to access the source code of the model

Question 12

In an audit of differential privacy implementations, what is the 'epsilon' parameter used for?

Accepted Answer

Privacy budget control. Epsilon is the privacy budget; lower values provide stronger privacy guarantees but potentially lower utility.

Answer

Model training speed

Answer

Learning rate scheduling

Answer

Database query limit

Question 13

Which technique is recommended for auditing 'data leakage' in a feature engineering pipeline?

Accepted Answer

Temporal cross-validation. Cross-validation using temporal splits (rather than random splits) prevents training on future information.

Answer

Feature ablation

Answer

Normalization

Answer

Random sampling

Question 14

When conducting an audit, what is the 'Model Registry' entry for a model version supposed to contain?

Accepted Answer

The model artifact, environment, and metadata. It should contain the model binary, the environment definition (requirements.txt), and the experiment metadata (the 'Run' ID).

Answer

Only the model binary

Answer

Only the training dataset

Answer

Only the developer's credentials

Question 15

Which TWO methods are commonly used by auditors to verify the 'Provenance' of training data?

Accepted Answer

Reviewing data lineage logs in the metadata store. Hashing files and maintaining lineage logs are standard technical methods to verify data provenance.

Answer

Watching the training video

Answer

Interviewing the janitorial staff

Answer

Checking if the model is open source

Question 16

You are auditing a 'Model Monitoring' dashboard. Which metric is most indicative of a potential degradation in the model's reliability over time?

Accepted Answer

Statistical drift between the production input data and training baseline. Data drift and concept drift are primary indicators that a model's performance may be degrading in a production environment.

Answer

The physical location of the cloud server hosting the model

Answer

The number of lines of code in the model deployment script

Answer

The number of users accessing the model API

Question 17

You find that the 'Inference API' for a model allows 'Prompt Injection' attacks. Which governance failure is most likely?

Accepted Answer

Inadequate input validation controls and adversarial testing. Prompt injection is a failure of input validation and security design within the model governance framework.

Answer

Lack of a social media presence for the company

Answer

Using a low-cost cloud provider

Answer

Failure to update the company logo

Question 18

What is the primary function of an 'AI Audit Log'?

Accepted Answer

To track and document all interactions and changes made to the AI system. An audit log records all actions performed by or on the AI system, ensuring accountability and traceability.

Answer

To automatically order pizza for the team

Answer

To store the personal contact information of the IT staff

Answer

To provide entertainment for employees during breaks

Question 19

When auditing data pipelines, what is the function of a 'Data Quality' assertion?

Accepted Answer

To validate data integrity before training. Assertions (e.g., in Great Expectations) check if data meets predefined schemas and constraints before ingestion.

Answer

To rename data columns

Answer

To increase prediction speed

Answer

To compress input files

Question 20

When using 'LIME' (Local Interpretable Model-agnostic Explanations) for auditing, what is the auditor looking for?

Accepted Answer

Local decision transparency. LIME creates a local surrogate model to explain individual predictions, helping the auditor understand why a specific decision was made.

Answer

Data distribution drift

Answer

Overall model performance

Answer

Adversarial robustness