AAIA Practice Test 14 — 15 Questions

Question 1

When scoping an audit for a model using SHAP values, what is the primary objective of the auditor?

Accepted Answer

Explainability of model predictions. SHAP (SHapley Additive exPlanations) is used to quantify the contribution of each feature to a specific prediction, aiding interpretability.

Answer

Detecting data drift

Answer

Security vulnerability scanning

Answer

Infrastructure scaling

Question 2

When auditing an AI pipeline, where should 'Environment Variables' be stored to ensure security?

Accepted Answer

Secrets Manager. Secrets/variables should be stored in a Secrets Manager, not in the code or environment files.

Answer

Git repository

Answer

Hardcoded in script

Answer

Publicly accessible folder

Question 3

Why must you conduct a 'Bias Audit' on a model that has been fine-tuned on new data?

Accepted Answer

Fine-tuning can introduce new biases. Fine-tuning on new data can inadvertently introduce new biases or exacerbate existing ones.

Answer

It speeds up inference

Answer

It fixes hardware bugs

Answer

It reduces model size

Question 4

When auditing an MLflow experiment, which object allows the auditor to verify the exact parameters and code version used during training?

Accepted Answer

MLflow Run. The MLflow Run object stores the code version (git commit), parameters, and metrics for a training session.

Answer

MLflow Model Registry

Answer

MLflow Experiment

Answer

MLflow Tracking Server

Question 5

When auditing fairness using the 'Aequitas' toolkit, what is the first step an auditor should take?

Accepted Answer

Select bias metrics. The first step in Aequitas is to define the bias metrics (e.g., False Positive Rate Parity) relevant to the context.

Answer

Encrypt the dataset

Answer

Train a new model

Answer

Delete sensitive attributes

Question 6

You are auditing a model using IBM Watson OpenScale. Which feature should be configured to detect 'Disparate Impact' to ensure regulatory compliance?

Accepted Answer

Fairness monitor. Watson OpenScale features a 'Fairness' monitor that specifically tracks Disparate Impact ratios.

Answer

Quality monitor

Answer

Drift detection monitor

Answer

Explainability monitor

Answer

Payload logging

Question 7

An auditor is using 'CleverHans' for model auditing. What kind of vulnerability is this library designed to detect?

Accepted Answer

Adversarial vulnerability. CleverHans is a library for benchmarking the robustness of neural networks against adversarial examples.

Answer

Memory leaks in model deployment

Answer

Infrastructure misconfiguration

Answer

PII leaks in training data

Question 8

Which THREE of the following are core components of a robust AI Governance and Risk Assessment Framework?

Accepted Answer

Standardized model risk assessment templates. These three options represent the strategic, operational, and accountability pillars of a comprehensive AI governance program.

Answer

Network bandwidth optimization plans

Answer

Automated model training hyperparameter tuning

Question 9

Which tool provides visual confirmation of data lineage in an end-to-end AI project?

Accepted Answer

Apache Airflow. Apache Airflow provides DAG (Directed Acyclic Graph) visualization, which auditors use to map data flow and lineage.

Answer

Docker

Answer

Jenkins

Answer

Kubernetes

Question 10

You are auditing an organization's AI deployment in Microsoft Azure. The Chief Risk Officer requires a central dashboard to monitor AI model performance drift and data quality metrics across all deployed services. Which tool should you confirm is configured to provide these insights?

Accepted Answer

Azure Machine Learning Model Monitor. Azure Machine Learning Model Monitor provides the specific functionality to track performance drift, data quality, and model health, which is essential for AI governance compliance.

Answer

Microsoft Purview AI Compliance Manager

Answer

Azure Policy for AI

Answer

Azure AI Content Safety

Question 11

Which artifact should an auditor request to verify 'Model Governance' during the model life cycle?

Accepted Answer

Independent model validation reports. Model validation reports provide evidence that the model was checked against performance and safety standards.

Answer

The personal notes of the lead developer

Answer

A photograph of the development team

Answer

A list of office keys issued to employees

Question 12

Which document is the primary source for establishing the scope of an AI audit?

Accepted Answer

The internal AI Policy and the organization's Risk Management Framework. The AI Policy and the AI Risk Management Framework (RMF) define the boundaries and expectations for the audit.

Answer

A list of all employees who have taken an AI training course

Answer

The company's social media posting guidelines

Answer

The employee handbook for the engineering department

Question 13

Which role is primarily responsible for ensuring that the AI governance framework is followed in daily operations?

Accepted Answer

The Model Manager or Product Owner. The AI Product Owner or Model Manager is responsible for the operational compliance of their specific AI systems.

Answer

The company's external auditor

Answer

The office manager

Answer

The chief financial officer

Question 14

Which THREE technical artifacts should the auditor collect to verify the 'Model Training Process'?

Accepted Answer

Environment requirements (e.g., requirements.txt). Training logs, environment requirements (dependencies), and the training script are essential to reconstruct the training process.

Answer

The office address of the IT department

Answer

Employee performance reviews

Question 15

During an audit of Google Cloud Vertex AI, you discover that sensitive PII is being logged during model inference. You need to implement a control that enforces privacy at the model serving layer. Which configuration should you verify?

Accepted Answer

Vertex AI Model privacy and data masking controls. Vertex AI supports data masking and privacy controls during inference to ensure PII is not stored in logs, which is a key component of AI governance.

Answer

Vertex AI Model Monitoring for data drift

Answer

Vertex AI Feature Store access logs

Answer

Cloud IAM role binding for Vertex AI