AAIA Practice Test 17 — 15 Questions

Question 1

What is the primary objective of 'Board-level AI Governance'?

Accepted Answer

To ensure AI aligns with the organization's business strategy and risk appetite. The board ensures that AI aligns with the firm's strategic risk appetite and legal obligations.

Answer

To decide which programming language is used

Answer

To fix bugs in the model code

Answer

To organize the annual team lunch

Question 2

You are auditing a firm's AI policy and find that the 'Model Inventory' in the enterprise risk management platform lacks versioning metadata. Which specific control gap should be prioritized to align with NIST AI RMF?

Accepted Answer

Lack of immutable audit logs for model architecture changes. Traceability and version control are essential to establish accountability and reproducibility in AI risk management.

Answer

Absence of automated model deployment pipelines

Answer

Failure to define the RPO for the primary database

Answer

Insufficient budget for cloud storage of historical data

Question 3

When auditing model fairness in a production environment using IBM Watson OpenScale, which metric would you specifically validate to ensure the model is not violating protected class regulations?

Accepted Answer

Disparate Impact. Disparate Impact is the standard statistical metric used in OpenScale to identify bias against protected groups during inference.

Answer

Precision-Recall AUC

Answer

Model Latency

Answer

Root Mean Square Error

Question 4

An auditor is asked to review the AI model inventory. What information must be present in the inventory for every model to satisfy basic regulatory requirements?

Accepted Answer

Model owner, business purpose, and status. An inventory must track model ownership, purpose, and lifecycle stage to ensure accountability and oversight.

Answer

The complete list of all training algorithms used

Answer

The total number of GPUs used during training

Answer

All hyperparameter values

Question 5

Which THREE factors must be evaluated when assessing the 'Fairness' of an AI model?

Accepted Answer

Disparities in model error rates across protected groups. Fairness assessment requires looking at demographic parity, historical bias, and error rate disparities.

Answer

The programming language used to build the model

Answer

The number of CPU cores allocated for training

Question 6

What is the risk of 'Feature Skew' in an AI pipeline?

Accepted Answer

Incorrect model predictions. If features used in training differ from those used in serving, the model produces incorrect results.

Answer

Database corruption

Answer

Increased hardware costs

Answer

System crash

Question 7

In Google Vertex AI Pipelines, which mechanism prevents 'training-serving skew' when deploying a custom container model?

Accepted Answer

Model Monitoring skew detection. Vertex AI Model Monitoring provides built-in skew detection by comparing training data statistics against serving traffic statistics.

Answer

Custom prediction routines

Answer

Artifact metadata locks

Answer

Pipeline concurrency limits

Question 8

Which TWO of the following are considered 'Model Metadata' that should be tracked in a registry?

Accepted Answer

Hyperparameters. Registry must store enough info to recreate the model state.

Answer

Employee vacation schedule

Answer

Cafeteria menu

Answer

Office Wi-Fi password

Question 9

What is the main role of the 'AI Risk Assessment' document in a project lifecycle?

Accepted Answer

To identify and evaluate potential risks before the model is deployed. The risk assessment identifies potential hazards and guides the implementation of controls before the project proceeds.

Answer

To provide a history of the company's annual profits

Answer

To record the names of all HR staff

Answer

To list the office parking spaces

Question 10

When auditing Kubeflow pipelines, which configuration file is used to define the resource requests for individual pipeline steps?

Accepted Answer

resource_spec in the SDK. The 'pipeline.yaml' or the step-specific resource configuration in the SDK defines CPU/Memory limits.

Answer

Namespace policy

Answer

Kubeconfig file

Answer

Service mesh config

Question 11

An organization using AWS SageMaker is experiencing model drift in production. You need to implement automated monitoring to trigger alerts when the distribution of input data deviates from the baseline. Which tool should be configured?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor uses baseline statistics to detect data quality issues in production endpoints.

Answer

AWS CloudWatch Alarms

Answer

SageMaker Clarify

Answer

SageMaker Pipelines

Question 12

When implementing drift detection in AWS SageMaker Model Monitor, which metric should be monitored to detect feature attribution shift for structured data?

Accepted Answer

Feature Attribution Drift (SHAP). Feature attribution drift monitors how the model's reliance on specific input features changes over time.

Answer

Cosine Similarity

Answer

Mean Square Error

Answer

Precision-Recall AUC

Question 13

As part of an AI audit, you are checking the 'AI Transparency' requirement. Which documentation artifact is essential to provide to external regulators to explain the model's design, intended use, and limitations?

Accepted Answer

Model Card. The Model Card is a standardized, transparent document that summarizes a model's characteristics, intended use, and limitations, as recommended by AI ethics standards.

Answer

Unit testing suite

Answer

API Swagger documentation

Answer

Infrastructure-as-Code (IaC) templates

Question 14

When auditing an AI model, what does 'Concept Drift' signify to an operational team?

Accepted Answer

The model's predictive accuracy is likely degrading. Concept drift occurs when the statistical relationship between input and target variable changes, rendering the model obsolete.

Answer

The training data was leaked

Answer

The model is overfitted

Answer

The model is running too fast

Question 15

When using DVC (Data Version Control) in an AI project, what is the best practice to ensure the 'data version' is immutable for audit purposes?

Accepted Answer

Commit .dvc files to git. Committing the '.dvc' file (which contains a hash of the data) to Git ensures that the specific data version is locked to the code version.

Answer

Use absolute file paths

Answer

Disable data caching

Answer

Store data on local drive