AAIA Practice Test 16 — 15 Questions

Question 1

An organization uses AWS SageMaker. As an auditor, you need to verify that model lineage and versioning are being captured to satisfy the AI risk assessment requirement for reproducibility. Which feature ensures that every model training job is recorded with its data sources?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

SageMaker Lineage Tracking. SageMaker Lineage Tracking automatically captures the relationship between data, models, and training jobs to ensure full auditability.

Answer

AWS CloudTrail

Answer

SageMaker Clarify

Answer

SageMaker Model Registry

Question 2

An organization using AWS SageMaker is experiencing model drift in production. You need to implement automated monitoring to trigger alerts when the distribution of input data deviates from the baseline. Which tool should be configured?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor is the purpose-built service for detecting data drift, model quality drift, bias drift, and feature attribution drift in deployed endpoints. It compares incoming inference data against a baseline statistics and constraints file, and can emit CloudWatch metrics that trigger alarms. This directly addresses the requirement for automated monitoring of input distribution deviation.

Answer

AWS CloudWatch Alarms

Answer

SageMaker Clarify

Answer

SageMaker Pipelines

Question 3

An auditor finds that a model's 'input feature importance' has changed significantly after a retrain. What is the most appropriate action?

Accepted Answer

Investigate source data and feature logic. Investigate if the change is due to a shift in data or a feature engineering error.

Answer

Ignore the change

Answer

Delete the model

Answer

Upgrade the hardware

Question 4

An auditor is evaluating the 'Model Monitoring' dashboard. Which TWO items should the dashboard display to alert the team of potential issues?

Accepted Answer

Real-time model accuracy metrics. Option B (Real-time model accuracy metrics) is correct because a Model Monitoring dashboard must continuously track how well the deployed model performs against ground truth or validation data, so degradation in accuracy (e.g., precision, recall, F1, or RMSE) can trigger timely alerts before business impact occurs. Option D (Data drift alerts) is correct because monitoring must detect distributional shifts between production input data and the training baseline—using statistical measures such as PSI, KL divergence, or KS tests—since drift is a leading indicator that model predictions will become unreliable. The remaining options do not belong: A (developer's vacation schedule) is an HR/administrative concern, C (public stock prices) is unrelated external market data, and E (office building temperature) is a facilities metric, none of which signal model performance or data integrity issues.

Answer

The model developer's vacation schedule

Answer

Public stock prices of the firm

Answer

Office building temperature

Question 5

Which TWO of the following are primary components of an AI model risk management inventory?

Accepted Answer

Training data provenance and lineage information. Option B is correct because training data provenance and lineage information is a core element of an AI model risk management inventory, as it documents where data originated, how it was transformed, and its chain of custody, which is essential for assessing bias, compliance, and reproducibility risks. Option D is correct because the model purpose and business application description defines what the model does, the decisions it informs, and its operational context, which is necessary to scope and prioritize risk assessments. Options A, C, and E are incorrect because employee performance review scores, office furniture lists, and marketing budgets are unrelated to the technical, ethical, and regulatory risk characteristics of AI models and therefore do not belong in a model risk management inventory.

Answer

Employee performance review scores

Answer

List of office furniture used by the team

Answer

Marketing budget for the model launch

Question 6

Which security control is most effective against 'Prompt Injection' attacks in a deployed LLM?

Accepted Answer

Prompt Guardrails. Prompt Guardrails are input/output filtering layers that detect and block malicious or policy-violating prompts, including injection attempts like 'ignore previous instructions' or role-override payloads. They work by classifying, pattern-matching, or using a secondary LLM to evaluate prompts before they reach the target model, and by sanitizing outputs before they reach the user. This is the most direct and effective control because it operates at the exact attack surface — the prompt itself.

Answer

Enabling GPU acceleration

Answer

Increasing model parameters

Answer

Regular re-training

Question 7

When documenting audit findings for an AI system, which TWO of the following are critical to include to ensure the audit can be replicated?

Accepted Answer

Specific Git commit SHA for the model code. Option A is correct because recording the specific Git commit SHA pins the exact revision of the model code that was audited, so anyone can check out that same commit and reproduce the training or inference behavior without ambiguity from later changes. Option C is correct because capturing the version ID or hash of the training dataset uniquely identifies the exact data snapshot used, ensuring the same inputs can be retrieved and the results independently verified. Together, the code commit SHA and dataset version/hash form the minimal provenance pair needed for a replicable audit. Options B, D, and E are not relevant to reproducibility: office location, coffee machine brand, and a developer's personal email address have no bearing on the model's code, data, or computational results.

Answer

The office location of the data scientists

Answer

The brand of the coffee machine used

Answer

The developer's personal email address

Question 8

When auditing an AI pipeline, where should 'Environment Variables' be stored to ensure security?

Accepted Answer

Secrets Manager. Environment variables frequently hold credentials, API keys, and connection strings, so they must be stored in a dedicated secrets management service such as Secrets Manager. This provides encryption at rest, access control, auditing, and rotation, which are the security properties an audit expects. Storing them anywhere else exposes them to leakage.

Answer

Git repository

Answer

Hardcoded in script

Answer

Publicly accessible folder

Question 9

Which technique is best for verifying that a model has not been subject to 'training data poisoning'?

Accepted Answer

Review of data provenance. Verifying data provenance — tracing where training data came from, who handled it, and whether it was tampered with — is the most direct way to detect training data poisoning. Poisoning attacks insert malicious or corrupted samples into training data, so reviewing the origin, custody, and integrity of that data is the primary control. Accuracy or latency testing cannot reliably reveal deliberate poisoning because poisoned models often perform normally on standard benchmarks.

Answer

Latency testing

Answer

Model Accuracy testing

Answer

Hyperparameter tuning

Question 10

Which THREE items must be included in a 'Model Card' to comply with regulatory model risk management standards?

Accepted Answer

Intended use cases. Model cards summarize intended use, limitations, and performance characteristics.

Answer

Internal employee passwords

Answer

Cloud service provider revenue

Question 11

When using DVC (Data Version Control) in an AI project, what is the best practice to ensure the 'data version' is immutable for audit purposes?

Accepted Answer

Commit .dvc files to git. DVC tracks data artifacts by storing lightweight .dvc pointer files (containing MD5 hashes and metadata) in Git while the actual data lives in remote storage (S3, GCS, Azure Blob, etc.). Committing the .dvc files to Git creates an immutable, versioned record of exactly which data hash was used for each experiment or model, providing the cryptographic provenance auditors require. Because the hash is content-addressed, any modification to the underlying data produces a different hash, breaking the link and exposing tampering.

Answer

Use absolute file paths

Answer

Disable data caching

Answer

Store data on local drive

Question 12

In an AI Audit, what does 'Model Fairness' assessment typically involve?

Accepted Answer

Disparity analysis across segments. Checking for bias across protected groups (e.g., race, gender) in model performance metrics.

Answer

Counting model parameters

Answer

Checking model speed

Answer

Verifying code documentation

Question 13

When documenting findings, which THREE elements should be included for each finding?

Accepted Answer

The root cause. Option C is correct because every documented finding must state the condition — the specific factual situation the auditor observed (what was found), which is the core of the finding. Option A is correct because the root cause explains why the condition exists, identifying the underlying reason so the finding is actionable rather than merely descriptive. Option B is correct because a recommendation for improvement provides the corrective or mitigating action needed to address the condition and its root cause, completing the finding's value. Options D and E are incorrect: font size is a formatting detail irrelevant to the substance of a finding, and listing employees who disagreed is not a required element and could introduce unnecessary or inappropriate personal information.

Answer

The font size used in the report

Answer

The names of all employees who disagreed

Question 14

What is the most effective way to audit 'Data Lineage' in a complex AI ecosystem?

Accepted Answer

Automated Metadata Tracking. Using metadata tracking tools that record the transformations from source to final model input.

Answer

Asking developers

Answer

Manual spreadsheet tracking

Answer

Checking project folder names

Question 15

You are performing an audit on an ML project utilizing Kubeflow. The team is using Katib for hyperparameter tuning. Which configuration step is critical to ensure that individual trials do not starve the production inference service of resources?

Accepted Answer

Implementing Kubernetes ResourceQuotas. Kubernetes ResourceQuotas let administrators cap aggregate CPU, memory, and object counts within a namespace, ensuring that Katib experiment trials running in that namespace cannot consume unbounded cluster resources and starve the production inference service. By scoping quotas to the tuning namespace, the platform team guarantees a resource floor for production workloads. This is the standard Kubernetes-native mechanism for multi-tenant resource isolation.

Answer

Increasing node count on the master node

Answer

Disabling auto-scaling on the cluster

Answer

Setting the Katib algorithm to random search