AAIA AI Auditing Tools And Techniques Practice Test 2 — 15 Questions

Question 1

An auditor is using 'CleverHans' for model auditing. What kind of vulnerability is this library designed to detect?

Accepted Answer

Adversarial vulnerability. CleverHans is a library for benchmarking the robustness of neural networks against adversarial examples.

Answer

Memory leaks in model deployment

Answer

Infrastructure misconfiguration

Answer

PII leaks in training data

Question 2

Which metric is commonly used to audit classification models?

Accepted Answer

F1-score. The F1-score balances precision and recall, providing a single metric for classification performance.

Answer

Logarithmic growth

Answer

Mean Squared Error

Answer

CPU utilization

Question 3

Which of the following is a common 'drift' symptom an auditor should look for in production models?

Accepted Answer

Decreasing model accuracy. When data distribution changes (Concept Drift), the model's accuracy typically degrades over time.

Answer

Increasing server costs

Answer

Increased team size

Answer

Slow Git repository commits

Question 4

An auditor is evaluating an AI system for 'Model Inversion' risk. What is this?

Accepted Answer

Reconstructing training data from outputs. Model Inversion is an attack where an adversary reconstructs training data from model outputs.

Answer

Reducing inference latency

Answer

Increasing training bias

Answer

Overfitting the data

Question 5

What is the primary role of a 'Human-in-the-Loop' (HITL) audit requirement?

Accepted Answer

To review critical AI decisions. HITL ensures that sensitive or critical decisions made by AI are reviewed by human experts.

Answer

To automate model retraining

Answer

To generate marketing reports

Answer

To improve GPU performance

Question 6

In an audit of differential privacy implementations, what is the 'epsilon' parameter used for?

Accepted Answer

Privacy budget control. Epsilon is the privacy budget; lower values provide stronger privacy guarantees but potentially lower utility.

Answer

Model training speed

Answer

Learning rate scheduling

Answer

Database query limit

Question 7

When auditing data pipelines, what is the function of a 'Data Quality' assertion?

Accepted Answer

To validate data integrity before training. Assertions (e.g., in Great Expectations) check if data meets predefined schemas and constraints before ingestion.

Answer

To rename data columns

Answer

To increase prediction speed

Answer

To compress input files

Question 8

When auditing model versioning, what is the recommended practice for maintaining evidence?

Accepted Answer

Use immutable versioning. Immutable model repositories and versioning control (e.g., Git LFS, MLflow) ensure the exact state can be audited.

Answer

Delete old versions

Answer

Use mutable file names

Answer

Store models in cache

Question 9

Which tool provides visual confirmation of data lineage in an end-to-end AI project?

Accepted Answer

Apache Airflow. Apache Airflow provides DAG (Directed Acyclic Graph) visualization, which auditors use to map data flow and lineage.

Answer

Docker

Answer

Jenkins

Answer

Kubernetes

Question 10

What is the purpose of 'Model Validation' in the audit lifecycle?

Accepted Answer

To verify model performance on test data. Validation confirms that the model performs as expected against independent, unseen data.

Answer

To speed up data ingestion

Answer

To lower API costs

Answer

To increase training time

Question 11

What is a 'Model Repository' in an AI audit context?

Accepted Answer

A versioned registry of models. A central, versioned store (like MLflow or Azure Model Registry) for managing trained models.

Answer

A backup hard drive

Answer

A list of developers

Answer

A documentation folder

Question 12

When auditing model explainability, why is it risky to rely solely on 'Global Feature Importance'?

Accepted Answer

It fails to explain individual decisions. Global importance explains the model overall, but individual predictions (local) may be driven by different factors.

Answer

It lacks local prediction accuracy

Answer

It uses too much memory

Answer

It is only for regression models

Question 13

Which technique is recommended for auditing 'data leakage' in a feature engineering pipeline?

Accepted Answer

Temporal cross-validation. Cross-validation using temporal splits (rather than random splits) prevents training on future information.

Answer

Feature ablation

Answer

Normalization

Answer

Random sampling

Question 14

When using 'LIME' (Local Interpretable Model-agnostic Explanations) for auditing, what is the auditor looking for?

Accepted Answer

Local decision transparency. LIME creates a local surrogate model to explain individual predictions, helping the auditor understand why a specific decision was made.

Answer

Data distribution drift

Answer

Overall model performance

Answer

Adversarial robustness

Question 15

When auditing the 'training dataset' for a classification model, what should the auditor confirm regarding class balance?

Accepted Answer

That classes are equally represented or addressed. Class imbalance can cause the model to be biased toward the majority class; the auditor should check for techniques like re-sampling or cost-sensitive learning.

Answer

That all data is in CSV format

Answer

That the dataset size is at least 1TB

Answer

That training time is minimized