AAIA Practice Test 14 — 15 Questions

Question 1

Which TWO of the following are examples of 'Model Risk Management' (MRM) controls that must be validated during an audit?

Accepted Answer

Post-deployment performance and drift monitoring. Option B is correct because post-deployment performance and drift monitoring is a core MRM control: it continuously tracks live model behavior against baselines (e.g., PSI, KS statistic, or accuracy degradation thresholds) to detect data drift, concept drift, and performance decay, and auditors must verify these monitoring mechanisms are in place and effective. Option E is correct because independent model validation post-development is a foundational MRM control requiring a party separate from the model developers to assess conceptual soundness, data quality, assumptions, and outcomes; audit validation confirms this independence and rigor exist. Option A is not an MRM control but a CI/CD engineering practice, option C concerns developer tooling standards rather than model risk, and option D is a data/training change that may affect a model but is not itself a validation control.

Answer

Automated deployment pipeline triggers

Answer

Standardized IDE choice for developers

Answer

Increasing training dataset size by 50%

Question 2

Which AI risk is best mitigated by conducting a 'bias audit' on the training dataset?

Accepted Answer

Unfairness and discrimination in model outputs. Bias audits analyze datasets for underrepresented groups or historical prejudices that would lead to discriminatory outputs.

Answer

Hardware overheating

Answer

System latency

Answer

Server cost overruns

Question 3

A company uses MLflow for model registry and tracking. You need to transition a model from the 'Staging' environment to 'Production' only after it passes an automated integration test. How is this state transition managed in MLflow?

Accepted Answer

Using the 'transition_model_version_stage' API. Transitioning model versions in the MLflow Model Registry is done via the 'transition_model_version_stage' API or UI.

Answer

Updating the 'version' tag in the git repo

Answer

Deleting the model in Staging and re-uploading to Production

Answer

Manually copying files in the S3 bucket

Question 4

What is the primary objective of a 'Model Inventory' in an enterprise AI audit?

Accepted Answer

Centralized visibility of all models. To have a centralized, authoritative list of all models in production for compliance and risk tracking.

Answer

To log user feedback

Answer

To optimize GPU usage

Answer

To store model weights

Question 5

You are assessing the risk of 'Data Poisoning' in a retail AI model. Which control should you implement during the data ingestion pipeline to mitigate this risk?

Accepted Answer

Input data validation and sanitization. Data validation and sanitization during the ingestion pipeline ensure that malicious or corrupted data samples are blocked before entering the training set.

Answer

Model version control

Answer

Increasing the GPU compute capacity

Answer

Enabling Multi-Factor Authentication (MFA) on the model registry

Question 6

When scoping an AI audit engagement, which TWO of the following documents should the auditor request to understand the AI model's governance structure?

Accepted Answer

Technical system architecture documentation. Option D, technical system architecture documentation, is correct because it describes how the AI model is built, deployed, and integrated with surrounding systems, revealing the governance-relevant controls, data flows, and ownership boundaries an auditor needs to assess. Option E, the Model Card, is correct because it is the standard artifact documenting a model's intended use, training data, performance metrics, limitations, and ethical considerations, which directly evidences the model's governance structure. The unmarked options do not belong: public social media posts (A) and company marketing brochures (B) are promotional or informal and not authoritative governance records, while developer salary structures (C) are HR compensation data unrelated to AI model governance.

Answer

Public social media posts

Answer

Company marketing brochures

Answer

Developer salary structures

Question 7

Which THREE criteria are necessary for an AI system to be deemed 'Trustworthy' under most governance frameworks?

Accepted Answer

Reliability and consistent performance over time. Option A is correct because trustworthy AI frameworks (e.g., EU HLEG, NIST AI RMF, OECD) require reliability and consistent performance over time, meaning the system behaves as intended and maintains accuracy across varying conditions and lifecycles. Option B is correct because transparency of the decision-making process is a core principle, enabling stakeholders to understand how outputs are produced, supporting explainability, auditability, and accountability. Option D is correct because robustness and security against adversarial attacks ensure the system resists manipulation, data poisoning, and evasion, which is essential for safety and trustworthiness. Option C is not a governance criterion; revenue generation is a business objective, not a trustworthiness requirement. Option E is not a criterion either, as marketing buzzwords have no bearing on ethical, technical, or regulatory trustworthiness standards.

Answer

The ability to generate high revenue for the firm

Answer

The use of the latest marketing buzzwords

Question 8

When selecting testing techniques for an AI model, which THREE are considered 'Model-Agnostic'?

Accepted Answer

SHAP (SHapley Additive exPlanations). SHAP (A) is model-agnostic because it computes Shapley values from cooperative game theory using only the model's input-output behavior, so it can explain any classifier or regressor regardless of its internal architecture. Partial Dependence Plots (C) are also model-agnostic: they estimate the marginal effect of features on predictions by averaging model outputs over the dataset, requiring only a predict function rather than access to model internals. LIME (E) is explicitly model-agnostic, as its name states, since it explains individual predictions by fitting a simple interpretable surrogate (e.g., linear model) to perturbed samples around the instance, treating the original model as a black box. In contrast, neural network weight pruning (B) is a model-specific optimization technique that requires access to a neural network's weights and architecture, and linear regression coefficients (D) are intrinsic parameters only available for linear models, so neither generalizes across arbitrary model types.

Answer

Neural network weight pruning

Answer

Linear regression coefficients

Question 9

What is the role of a 'Deployment Pipeline' in MLOps?

Accepted Answer

Automating deployment steps. To automate the packaging, testing, and deployment of a model.

Answer

Storing logs

Answer

Writing code

Answer

Labeling data

Question 10

An auditor is using 'CleverHans' for model auditing. What kind of vulnerability is this library designed to detect?

Accepted Answer

Adversarial vulnerability. CleverHans is an open-source Python library developed by Google that implements reference implementations of adversarial attacks (e.g., FGSM, PGD, Carlini-Wagner) and defenses against machine learning models. It is specifically designed to help researchers and auditors probe models for adversarial vulnerabilities — inputs crafted to cause misclassification. Therefore, the vulnerability class it detects is adversarial vulnerability.

Answer

Memory leaks in model deployment

Answer

Infrastructure misconfiguration

Answer

PII leaks in training data

Question 11

Which component of an AI policy is most important for establishing organizational accountability?

Accepted Answer

Defining the roles and responsibilities of the AI Governance Committee and model owners. Accountability requires clearly defined roles and responsibilities for AI system oversight.

Answer

Describing the history of artificial intelligence research

Answer

Listing the hardware specifications of the training clusters

Answer

Providing the contact information for the company's PR department

Question 12

What is the function of a 'Circuit Breaker' in a microservice-based AI architecture?

Accepted Answer

Preventing failure propagation. To stop requests to a failing AI service to prevent system-wide collapse.

Answer

Increasing latency

Answer

Training the model

Answer

Managing databases

Question 13

A board of directors asks you to define the 'AI Risk Appetite'. Which approach best satisfies the governance requirement?

Accepted Answer

Defining quantitative thresholds for acceptable model performance and ethical impact. Risk appetite must align with the organization's business objectives and ethical standards.

Answer

Creating a repository of all AI ethical guidelines published by NGOs

Answer

Providing the board with a list of all AI vendors used in the organization

Answer

Restricting all AI development to on-premises environments

Question 14

You are managing an Azure Machine Learning workspace and need to ensure that model training jobs are reproducible across environments. Which feature should be configured to capture the exact environment state, including Python packages and system dependencies?

Accepted Answer

Environment objects. Azure ML Environment objects are the mechanism designed to capture and version the exact runtime state of a training job — including the Python interpreter version, pip/conda packages, environment variables, and Docker base image. When an Environment is registered and referenced by a job, Azure ML rebuilds the identical containerized context on any compute target, which is what makes training reproducible across environments. Datasets, registries, and compute targets do not encapsulate package or system dependencies.

Answer

Azure ML Datasets

Answer

Model Registries

Answer

Compute Targets

Question 15

When designing an AI governance framework, which THREE roles should have oversight responsibility?

Accepted Answer

Legal Counsel. Legal Counsel (A) is correct because AI governance must address regulatory compliance, liability, intellectual property, and contractual obligations, which require legal expertise to interpret laws such as the EU AI Act or GDPR. ML Engineer (D) is correct because technical oversight of model development, training data, bias testing, and performance monitoring requires deep machine-learning knowledge to ensure systems behave safely and as intended. Data Privacy Officer (E) is correct because AI systems typically process personal data, and the DPO is responsible for ensuring compliance with data protection regulations, conducting DPIAs, and safeguarding data subject rights. Marketing Intern (B) and Janitorial Staff (C) are not oversight roles in an AI governance framework, as they lack the domain authority, regulatory accountability, and technical or legal expertise required for governance decisions.

Answer

Marketing Intern

Answer

Janitorial Staff