AAIA Practice Exam 2 — 20 Questions

Question 1

Which technique is most appropriate for mitigating 'Concept Drift' in a production model?

Accepted Answer

Implementing an automated re-training pipeline based on performance threshold triggers. Continuous re-training or fine-tuning based on recent data is the standard mitigation for concept drift.

Answer

Increasing the number of neurons in the hidden layers

Answer

Moving the model to a larger GPU cluster

Answer

Restricting access to the training dataset

Question 2

When deploying an AI model on a public cloud, what is the best practice for 'Secret Management' of API keys?

Accepted Answer

Using a dedicated Secret Vault service. Secrets must be stored in a dedicated vault (e.g., AWS Secrets Manager, Azure Key Vault) and injected at runtime.

Answer

Printing to logs

Answer

Hardcoding in the script

Answer

Storing in a Git repo

Question 3

What does a 'Model Approval' workflow ensure?

Accepted Answer

Human oversight of deployment. That a human reviewer has vetted the model against compliance and quality standards before production deployment.

Answer

Faster training

Answer

Reduced cloud costs

Answer

Automatic deletion

Question 4

An auditor is asked to review the AI model inventory. What information must be present in the inventory for every model to satisfy basic regulatory requirements?

Accepted Answer

Model owner, business purpose, and status. For basic regulatory compliance, an AI model inventory must capture accountability (model owner), the reason the model exists (business purpose), and its current lifecycle stage (status). These three elements are consistently required by frameworks like the EU AI Act, NIST AI RMF, and ISO/IEC 42001 to ensure traceability, risk management, and governance. Without them, an auditor cannot determine who is responsible, why the model is deployed, or whether it should still be in use.

Answer

The complete list of all training algorithms used

Answer

The total number of GPUs used during training

Answer

All hyperparameter values

Question 5

An auditor is evaluating the 'Safety Filter' of an LLM. Which approach is most suitable for detecting 'jailbreak' vulnerabilities?

Accepted Answer

Adversarial red teaming. Red teaming with adversarial prompts (jailbreaks) is the standard method for testing safety filter resilience.

Answer

Running unit tests for code

Answer

Reviewing server logs

Answer

Checking network bandwidth

Question 6

Which TWO items should be included in an automated 'Deployment Log'?

Accepted Answer

Deployed model version/ID. Option A (Deployed model version/ID) is correct because a deployment log must record exactly which artifact was released so that any running model can be traced back to a specific version for rollback, auditing, and reproducibility. Option E (Deployment timestamp) is correct because recording when the deployment occurred establishes a chronological audit trail, enabling correlation with incidents, monitoring metrics, and change-management records. The remaining options do not belong: office temperature (B) is an environmental reading irrelevant to release tracking, team member home address (C) is personal data with no deployment significance and would raise privacy concerns, and favorite programming language (D) is a personal preference that has no bearing on what was deployed or when.

Answer

Office temperature

Answer

Team member home address

Answer

Favorite programming language

Question 7

Which TWO are common types of 'AI Model Drift'?

Accepted Answer

Concept drift. Concept drift (A) is a correct answer because it describes the situation where the statistical relationship between input features and the target variable changes over time, so a model's learned mapping becomes stale and its predictions degrade. Data drift (B) is also correct because it refers to a change in the distribution of the input features (also called covariate shift) relative to the training data, which is one of the most common forms of model drift monitored in production ML systems. The remaining options are not recognized categories of AI model drift: temperature drift (C) is not an ML drift type (temperature is a sampling parameter in LLMs, not a drift class), while CPU drift (D) and memory drift (E) are hardware/infrastructure phenomena unrelated to changes in data or concept distributions.

Answer

Temperature drift

Answer

CPU drift

Answer

Memory drift

Question 8

When scoping an audit, which TWO stakeholders should the auditor interview?

Accepted Answer

Model Owners. Model Owners (A) are correct because they hold accountability for the model's purpose, risk classification, and governance decisions, making them essential for defining the audit's scope and objectives. Data Scientists (D) are correct because they possess the technical detail on data sources, feature engineering, training pipelines, and model behavior needed to bound the audit's technical scope. The lobby receptionist (B), external marketing agency (C), and office cleaning staff (E) are not relevant stakeholders for scoping an audit, as they have no ownership, development, or operational responsibility for the model or its data.

Answer

The lobby receptionist

Answer

The company's external marketing agency

Answer

Office cleaning staff

Question 9

What is the purpose of 'Model Versioning' in an AI audit context?

Accepted Answer

To allow for reproducibility and auditability of past model versions. Versioning ensures that the specific state of a model used for a specific decision can be retrieved and audited.

Answer

To automatically delete old models to save space

Answer

To hide the model code from competitors

Answer

To reduce the number of files on the server

Question 10

An auditor notices high 'latency' in an online model deployment. Which component should be inspected first?

Accepted Answer

Inference service resource limits. The inference container resource allocation or the request preprocessing overhead is the common bottleneck.

Answer

Developer laptop specs

Answer

Model documentation

Answer

Data labeling tool settings

Answer

Git repository history

Question 11

When scoping an audit for a model using SHAP values, what is the primary objective of the auditor?

Accepted Answer

Explainability of model predictions. SHAP (SHapley Additive exPlanations) values are a game-theoretic method for attributing a model's prediction to its input features, so an audit scoped around SHAP is fundamentally about explainability of model predictions. The auditor's primary objective is to verify that the model's decisions can be understood and justified per-feature. This supports regulatory, fairness, and trust requirements rather than operational concerns.

Answer

Detecting data drift

Answer

Security vulnerability scanning

Answer

Infrastructure scaling

Question 12

Which THREE factors influence 'AI Pipeline' latency?

Accepted Answer

Model parameter size. Option B (Model parameter size) is correct because larger models require more computation per inference, increasing processing time and thus pipeline latency. Option C (Network transfer speed) is correct because AI pipelines often move data between storage, preprocessing, and inference services, and slower bandwidth or higher latency links directly add to end-to-end delay. Option D (Data preprocessing complexity) is correct because heavy operations such as tokenization, resizing, normalization, or feature engineering consume CPU/GPU cycles before inference, extending total pipeline latency. Options A (Company CEO name) and E (Office building height) are irrelevant metadata that have no technical effect on compute, transfer, or processing time in an AI pipeline.

Answer

Company CEO name

Answer

Office building height

Question 13

In a financial services firm, you are auditing the 'Explainability' of a credit scoring model. The model uses complex non-linear features. Which technique should you verify is being used to provide local explanations for individual credit decisions?

Accepted Answer

SHAP values. SHAP (SHapley Additive exPlanations) values are grounded in cooperative game theory and assign each feature a contribution to a single prediction, producing faithful local explanations even for complex non-linear models. For a credit-scoring audit, SHAP lets you explain why one specific applicant was approved or denied by decomposing that individual prediction into per-feature additive contributions. Permutation importance, pruning, and normalization do not provide per-instance local explanations.

Answer

Feature Importance Permutation

Answer

Model pruning

Answer

Data normalization

Question 14

An auditor is evaluating an AI system for 'Model Inversion' risk. What is this?

Accepted Answer

Reconstructing training data from outputs. Model Inversion is an attack where an adversary reconstructs training data from model outputs.

Answer

Reducing inference latency

Answer

Increasing training bias

Answer

Overfitting the data

Question 15

As part of an AI audit, you are checking the 'AI Transparency' requirement. Which documentation artifact is essential to provide to external regulators to explain the model's design, intended use, and limitations?

Accepted Answer

Model Card. A Model Card is the documentation artifact specifically designed to disclose a model's design, intended use, limitations, performance characteristics, and ethical considerations to external stakeholders. It provides regulators with a standardized, transparent summary of the model's purpose and constraints. This directly satisfies AI transparency requirements in audit and regulatory contexts.

Answer

Unit testing suite

Answer

API Swagger documentation

Answer

Infrastructure-as-Code (IaC) templates

Question 16

Which of the following is a symptom of 'Overfitting' during the training phase?

Accepted Answer

High training accuracy, low validation accuracy. Overfitting happens when the model performs well on training data but poorly on unseen test data.

Answer

Low training accuracy, low validation accuracy

Answer

Model crashes during training

Answer

Model is too small

Question 17

Which THREE of the following are potential risks associated with AI model 'Drift' that an auditor must account for?

Accepted Answer

Bias against specific demographic groups that emerged post-deployment. Drift impacts operational accuracy, fairness, and compliance, making it a critical governance risk.

Answer

Increased electricity consumption of servers

Answer

Inability to access the source code of the model

Question 18

Which environment should be used to perform 'Model Validation' before moving to production?

Accepted Answer

Staging Environment. Staging or UAT environments mirror production and are used for testing.

Answer

Training Cluster

Answer

Development Environment

Answer

Public Internet

Answer

Local Desktop

Question 19

Which THREE tasks are required for a 'Model Decommissioning' workflow?

Accepted Answer

Terminate serving infrastructure. Decommissioning requires archival, notification, and resource cleanup.

Answer

Ask developer for coffee

Answer

Delete all company records

Question 20

If a model's performance drops, and the input data distribution has changed, what operational process must be triggered?

Accepted Answer

Model Retraining. Data distribution change triggers model retraining using updated data samples.

Answer

Security audit

Answer

Model Deletion

Answer

Hardware Upgrade

Answer

Code Refactoring