AAIA Practice Test 3 — 10 Questions

Question 1

Which technique is most appropriate for mitigating 'Concept Drift' in a production model?

Accepted Answer

Implementing an automated re-training pipeline based on performance threshold triggers. Continuous re-training or fine-tuning based on recent data is the standard mitigation for concept drift.

Answer

Increasing the number of neurons in the hidden layers

Answer

Moving the model to a larger GPU cluster

Answer

Restricting access to the training dataset

Question 2

When deploying an AI model on a public cloud, what is the best practice for 'Secret Management' of API keys?

Accepted Answer

Using a dedicated Secret Vault service. Secrets must be stored in a dedicated vault (e.g., AWS Secrets Manager, Azure Key Vault) and injected at runtime.

Answer

Printing to logs

Answer

Hardcoding in the script

Answer

Storing in a Git repo

Question 3

What does a 'Model Approval' workflow ensure?

Accepted Answer

Human oversight of deployment. That a human reviewer has vetted the model against compliance and quality standards before production deployment.

Answer

Faster training

Answer

Reduced cloud costs

Answer

Automatic deletion

Question 4

An auditor is asked to review the AI model inventory. What information must be present in the inventory for every model to satisfy basic regulatory requirements?

Accepted Answer

Model owner, business purpose, and status. For basic regulatory compliance, an AI model inventory must capture accountability (model owner), the reason the model exists (business purpose), and its current lifecycle stage (status). These three elements are consistently required by frameworks like the EU AI Act, NIST AI RMF, and ISO/IEC 42001 to ensure traceability, risk management, and governance. Without them, an auditor cannot determine who is responsible, why the model is deployed, or whether it should still be in use.

Answer

The complete list of all training algorithms used

Answer

The total number of GPUs used during training

Answer

All hyperparameter values

Question 5

An auditor is evaluating the 'Safety Filter' of an LLM. Which approach is most suitable for detecting 'jailbreak' vulnerabilities?

Accepted Answer

Adversarial red teaming. Red teaming with adversarial prompts (jailbreaks) is the standard method for testing safety filter resilience.

Answer

Running unit tests for code

Answer

Reviewing server logs

Answer

Checking network bandwidth

Question 6

Which TWO items should be included in an automated 'Deployment Log'?

Accepted Answer

Deployed model version/ID. Option A (Deployed model version/ID) is correct because a deployment log must record exactly which artifact was released so that any running model can be traced back to a specific version for rollback, auditing, and reproducibility. Option E (Deployment timestamp) is correct because recording when the deployment occurred establishes a chronological audit trail, enabling correlation with incidents, monitoring metrics, and change-management records. The remaining options do not belong: office temperature (B) is an environmental reading irrelevant to release tracking, team member home address (C) is personal data with no deployment significance and would raise privacy concerns, and favorite programming language (D) is a personal preference that has no bearing on what was deployed or when.

Answer

Office temperature

Answer

Team member home address

Answer

Favorite programming language

Question 7

Which TWO are common types of 'AI Model Drift'?

Accepted Answer

Concept drift. Concept drift (A) is a correct answer because it describes the situation where the statistical relationship between input features and the target variable changes over time, so a model's learned mapping becomes stale and its predictions degrade. Data drift (B) is also correct because it refers to a change in the distribution of the input features (also called covariate shift) relative to the training data, which is one of the most common forms of model drift monitored in production ML systems. The remaining options are not recognized categories of AI model drift: temperature drift (C) is not an ML drift type (temperature is a sampling parameter in LLMs, not a drift class), while CPU drift (D) and memory drift (E) are hardware/infrastructure phenomena unrelated to changes in data or concept distributions.

Answer

Temperature drift

Answer

CPU drift

Answer

Memory drift

Question 8

When scoping an audit, which TWO stakeholders should the auditor interview?

Accepted Answer

Model Owners. Model Owners (A) are correct because they hold accountability for the model's purpose, risk classification, and governance decisions, making them essential for defining the audit's scope and objectives. Data Scientists (D) are correct because they possess the technical detail on data sources, feature engineering, training pipelines, and model behavior needed to bound the audit's technical scope. The lobby receptionist (B), external marketing agency (C), and office cleaning staff (E) are not relevant stakeholders for scoping an audit, as they have no ownership, development, or operational responsibility for the model or its data.

Answer

The lobby receptionist

Answer

The company's external marketing agency

Answer

Office cleaning staff

Question 9

What is the purpose of 'Model Versioning' in an AI audit context?

Accepted Answer

To allow for reproducibility and auditability of past model versions. Versioning ensures that the specific state of a model used for a specific decision can be retrieved and audited.

Answer

To automatically delete old models to save space

Answer

To hide the model code from competitors

Answer

To reduce the number of files on the server

Question 10

An auditor notices high 'latency' in an online model deployment. Which component should be inspected first?

Accepted Answer

Inference service resource limits. The inference container resource allocation or the request preprocessing overhead is the common bottleneck.

Answer

Developer laptop specs

Answer

Model documentation

Answer

Data labeling tool settings

Answer

Git repository history