AAIA Practice Test 2 — 25 Questions

Question 1

An auditor is evaluating the 'Safety Filter' of an LLM. Which approach is most suitable for detecting 'jailbreak' vulnerabilities?

Accepted Answer

Adversarial red teaming. Red teaming with adversarial prompts (jailbreaks) is the standard method for testing safety filter resilience.

Answer

Running unit tests for code

Answer

Reviewing server logs

Answer

Checking network bandwidth

Question 2

Which THREE factors should an auditor consider when evaluating the suitability of an AI model for production?

Accepted Answer

Inference latency and throughput. Accuracy, fairness metrics, and latency/throughput are standard KPIs for production readiness.

Answer

Developer age demographics

Answer

Operating system color schemes

Question 3

When documenting findings, which THREE elements should be included for each finding?

Accepted Answer

The root cause. The condition, the root cause, and the recommendation are essential components of an audit finding.

Answer

The font size used in the report

Answer

The names of all employees who disagreed

Question 4

Which TWO of the following are examples of AI governance 'Inputs'?

Accepted Answer

Internal AI Policy and guidelines. Inputs include regulatory requirements and internal policy documents.

Answer

A photograph of the company founder

Answer

The company's social media password

Answer

The office printer manual

Question 5

An organization is using 'Federated Learning' to maintain data privacy while training models. What is the primary audit risk associated with this architecture?

Accepted Answer

Model inversion attacks targeting the central aggregator. In Federated Learning, local model updates (gradients) can potentially reveal information about the underlying training data, necessitating 'Differential Privacy' as a control.

Answer

Increased training latency

Answer

Lack of access to training hardware

Answer

Centralized data breaches

Question 6

Which THREE of the following are potential risks associated with AI model 'Drift' that an auditor must account for?

Accepted Answer

Bias against specific demographic groups that emerged post-deployment. Drift impacts operational accuracy, fairness, and compliance, making it a critical governance risk.

Answer

Increased electricity consumption of servers

Answer

Inability to access the source code of the model

Question 7

In an audit of differential privacy implementations, what is the 'epsilon' parameter used for?

Accepted Answer

Privacy budget control. Epsilon is the privacy budget; lower values provide stronger privacy guarantees but potentially lower utility.

Answer

Model training speed

Answer

Learning rate scheduling

Answer

Database query limit

Question 8

Which technique is recommended for auditing 'data leakage' in a feature engineering pipeline?

Accepted Answer

Temporal cross-validation. Cross-validation using temporal splits (rather than random splits) prevents training on future information.

Answer

Feature ablation

Answer

Normalization

Answer

Random sampling

Question 9

When conducting an audit, what is the 'Model Registry' entry for a model version supposed to contain?

Accepted Answer

The model artifact, environment, and metadata. It should contain the model binary, the environment definition (requirements.txt), and the experiment metadata (the 'Run' ID).

Answer

Only the model binary

Answer

Only the training dataset

Answer

Only the developer's credentials

Question 10

Which TWO methods are commonly used by auditors to verify the 'Provenance' of training data?

Accepted Answer

Reviewing data lineage logs in the metadata store. Hashing files and maintaining lineage logs are standard technical methods to verify data provenance.

Answer

Watching the training video

Answer

Interviewing the janitorial staff

Answer

Checking if the model is open source

Question 11

You are auditing a 'Model Monitoring' dashboard. Which metric is most indicative of a potential degradation in the model's reliability over time?

Accepted Answer

Statistical drift between the production input data and training baseline. Data drift and concept drift are primary indicators that a model's performance may be degrading in a production environment.

Answer

The physical location of the cloud server hosting the model

Answer

The number of lines of code in the model deployment script

Answer

The number of users accessing the model API

Question 12

You find that the 'Inference API' for a model allows 'Prompt Injection' attacks. Which governance failure is most likely?

Accepted Answer

Inadequate input validation controls and adversarial testing. Prompt injection is a failure of input validation and security design within the model governance framework.

Answer

Lack of a social media presence for the company

Answer

Using a low-cost cloud provider

Answer

Failure to update the company logo

Question 13

What is the primary function of an 'AI Audit Log'?

Accepted Answer

To track and document all interactions and changes made to the AI system. An audit log records all actions performed by or on the AI system, ensuring accountability and traceability.

Answer

To automatically order pizza for the team

Answer

To store the personal contact information of the IT staff

Answer

To provide entertainment for employees during breaks

Question 14

When auditing data pipelines, what is the function of a 'Data Quality' assertion?

Accepted Answer

To validate data integrity before training. Assertions (e.g., in Great Expectations) check if data meets predefined schemas and constraints before ingestion.

Answer

To rename data columns

Answer

To increase prediction speed

Answer

To compress input files

Question 15

When using 'LIME' (Local Interpretable Model-agnostic Explanations) for auditing, what is the auditor looking for?

Accepted Answer

Local decision transparency. LIME creates a local surrogate model to explain individual predictions, helping the auditor understand why a specific decision was made.

Answer

Data distribution drift

Answer

Overall model performance

Answer

Adversarial robustness

Question 16

Which component of an AI policy is most important for establishing organizational accountability?

Accepted Answer

Defining the roles and responsibilities of the AI Governance Committee and model owners. Accountability requires clearly defined roles and responsibilities for AI system oversight.

Answer

Describing the history of artificial intelligence research

Answer

Listing the hardware specifications of the training clusters

Answer

Providing the contact information for the company's PR department

Question 17

When auditing model versioning, what is the recommended practice for maintaining evidence?

Accepted Answer

Use immutable versioning. Immutable model repositories and versioning control (e.g., Git LFS, MLflow) ensure the exact state can be audited.

Answer

Delete old versions

Answer

Use mutable file names

Answer

Store models in cache

Question 18

Which TWO monitoring tools are typically used to detect 'Data Drift' in production pipelines?

Accepted Answer

AWS SageMaker Model Monitor. AWS SageMaker Model Monitor and Google Cloud Vertex AI Model Monitoring are industry standard tools for drift detection.

Answer

Web Browser

Answer

Microsoft Excel

Answer

Adobe Photoshop

Question 19

A board of directors asks you to define the 'AI Risk Appetite'. Which approach best satisfies the governance requirement?

Accepted Answer

Defining quantitative thresholds for acceptable model performance and ethical impact. Risk appetite must align with the organization's business objectives and ethical standards.

Answer

Creating a repository of all AI ethical guidelines published by NGOs

Answer

Providing the board with a list of all AI vendors used in the organization

Answer

Restricting all AI development to on-premises environments

Question 20

When scoping an AI audit engagement, which TWO of the following documents should the auditor request to understand the AI model's governance structure?

Accepted Answer

Technical system architecture documentation. Model Cards and technical documentation provide the necessary governance and implementation context for an auditor.

Answer

Public social media posts

Answer

Company marketing brochures

Answer

Developer salary structures