AAIA AI Auditing Tools And Techniques Practice Test 3 — 15 Questions

Question 1

Which of these is a typical 'audit finding' in an AI governance review?

Accepted Answer

Insufficient model documentation. Lack of version control or undocumented model changes are classic audit findings in AI systems.

Answer

Too many GPU cores

Answer

Cloud-based deployment

Answer

Use of Python language

Question 2

What is the primary benefit of using 'Containerization' for AI model auditability?

Accepted Answer

Environment reproducibility. Containers (e.g., Docker) encapsulate code, libraries, and dependencies, ensuring that the model runs the same way in audit as it did in training.

Answer

Reduced data storage costs

Answer

Increased hardware speed

Answer

Simplified UI design

Question 3

When auditing fairness metrics, what does the 'Demographic Parity' metric measure?

Accepted Answer

Equal positive outcome rates. Demographic Parity requires that the probability of a positive outcome be equal across different demographic groups.

Answer

Equal training data representation

Answer

Equal error rates

Answer

Equal accuracy rates

Question 4

An auditor is evaluating the 'Safety Filter' of an LLM. Which approach is most suitable for detecting 'jailbreak' vulnerabilities?

Accepted Answer

Adversarial red teaming. Red teaming with adversarial prompts (jailbreaks) is the standard method for testing safety filter resilience.

Answer

Running unit tests for code

Answer

Reviewing server logs

Answer

Checking network bandwidth

Question 5

During an AI audit, what is a 'PII Scrubber' tool used for?

Accepted Answer

Anonymizing sensitive data. PII scrubbers remove or anonymize Personally Identifiable Information from training sets to ensure privacy compliance.

Answer

Cleaning up logs

Answer

Compressing images

Answer

Improving accuracy

Question 6

When conducting an audit, what is the 'Model Registry' entry for a model version supposed to contain?

Accepted Answer

The model artifact, environment, and metadata. It should contain the model binary, the environment definition (requirements.txt), and the experiment metadata (the 'Run' ID).

Answer

Only the model binary

Answer

Only the training dataset

Answer

Only the developer's credentials

Question 7

When auditing an AI system's 'Explainability', which issue is highlighted by 'Explanation Faithfulness'?

Accepted Answer

Whether the explanation accurately represents the model's logic. Faithfulness measures whether the explanation accurately reflects the model's internal decision process rather than just being a plausible simplification.

Answer

Whether the model is fast enough

Answer

Whether the user trusts the model

Answer

Whether the explanation is easy to read

Question 8

When scoping an AI audit engagement, which TWO of the following documents should the auditor request to understand the AI model's governance structure?

Accepted Answer

Technical system architecture documentation. Model Cards and technical documentation provide the necessary governance and implementation context for an auditor.

Answer

Public social media posts

Answer

Company marketing brochures

Answer

Developer salary structures

Question 9

Which THREE factors should an auditor consider when evaluating the suitability of an AI model for production?

Accepted Answer

Inference latency and throughput. Accuracy, fairness metrics, and latency/throughput are standard KPIs for production readiness.

Answer

Developer age demographics

Answer

Operating system color schemes

Question 10

When documenting audit findings for an AI system, which TWO of the following are critical to include to ensure the audit can be replicated?

Accepted Answer

Specific Git commit SHA for the model code. Version control information and the exact dataset version are essential for reproducing the audit results.

Answer

The office location of the data scientists

Answer

The brand of the coffee machine used

Answer

The developer's personal email address

Question 11

Which TWO techniques should an auditor employ to detect bias in a model where the training data has imbalanced demographic representation?

Accepted Answer

Implementation of fairness-aware loss functions. Stratified sampling and fairness-aware loss functions are common techniques to address and audit bias.

Answer

Updating the corporate logo

Answer

Changing the font size in the UI

Answer

Increasing the number of project meetings

Question 12

When conducting an audit, which THREE of the following represent potential 'Model Risk' areas that require documentation?

Accepted Answer

Susceptibility to adversarial attacks. Adversarial vulnerability, data drift, and bias are three primary risks requiring documentation in an AI audit report.

Answer

The color of the server racks

Answer

The speed of the local internet connection

Question 13

Which TWO methods are commonly used by auditors to verify the 'Provenance' of training data?

Accepted Answer

Reviewing data lineage logs in the metadata store. Hashing files and maintaining lineage logs are standard technical methods to verify data provenance.

Answer

Watching the training video

Answer

Interviewing the janitorial staff

Answer

Checking if the model is open source

Question 14

Which TWO metrics provide the best insights into 'Model Quality' during an audit?

Accepted Answer

Recall score. Precision and Recall are fundamental metrics for assessing the quality of classification models.

Answer

Disk space usage

Answer

Number of server restarts

Answer

Team's average coffee intake

Question 15

An auditor is evaluating the data pipeline security. Which THREE controls should be verified?

Accepted Answer

Full audit log of data provenance. Access control, encryption at rest, and data provenance are standard security controls for AI data pipelines.

Answer

The number of lines of code in the pipeline

Answer

Office access badge logs