AAIA Practice Test 5 — 25 Questions

Question 1

When auditing an Azure Machine Learning pipeline, which functionality within 'Responsible AI dashboard' is required to perform counterfactual analysis on model predictions?

Accepted Answer

Counterfactuals component. The 'Counterfactuals' component in the Azure Responsible AI dashboard allows auditors to see how input changes impact predictions.

Answer

Causal Inference component

Answer

Model Overview component

Answer

Error Analysis component

Question 2

Which metric would best detect 'Label Drift' in a production model?

Accepted Answer

Target distribution shift (PSI). Label drift occurs when the output (target) distribution changes significantly from training.

Answer

Model latency

Answer

CPU utilization

Answer

Number of users

Question 3

Which artifact should an auditor request to verify 'Model Governance' during the model life cycle?

Accepted Answer

Independent model validation reports. Independent model validation reports are artifacts that provide objective evidence of a model's performance, limitations, and risks, as assessed by a party separate from the development team. They are a key governance control to ensure models are fit for purpose and comply with internal policies and regulations.

Answer

The personal notes of the lead developer

Answer

A photograph of the development team

Answer

A list of office keys issued to employees

Question 4

What is the purpose of 'Containerization' (e.g., Docker) in MLOps?

Accepted Answer

Ensuring environment consistency. Ensures the environment (dependencies, OS) is consistent across development, testing, and production.

Answer

Encrypting the model

Answer

Reducing data size

Answer

Increasing model accuracy

Question 5

You are reviewing the 'AI Incident Response Plan'. Which element is essential for compliance?

Accepted Answer

Defined escalation paths for ethical or performance incidents. An AI Incident Response Plan must specify clear escalation paths for ethical or performance incidents to ensure that issues such as bias, safety violations, or model failures are promptly reported and addressed by the appropriate stakeholders. This is essential for compliance with regulations like the EU AI Act, which mandates incident reporting and accountability. Without defined escalation paths, organizations cannot demonstrate timely and effective response, risking legal and reputational damage.

Answer

The history of the company's founding

Answer

A log of all office supply purchases

Answer

A list of all cafeteria menus in the building

Question 6

Which THREE technical artifacts should the auditor collect to verify the 'Model Training Process'?

Accepted Answer

Environment requirements (e.g., requirements.txt). The auditor should collect the environment requirements file (A), such as requirements.txt, because it documents the exact library versions, dependencies, and runtime configuration used to reproduce the model training process. Training execution logs (C) are essential because they provide a timestamped record of the actual training runs, including metrics, errors, hyperparameters, and resource usage that prove the process occurred as described. The training script/code (E) is also required because it contains the model architecture, data preprocessing steps, loss function, optimizer settings, and training loop that define how the model was built. Options B and D do not belong: an office address is irrelevant to technical verification of model training, and employee performance reviews are HR records that do not evidence the training process itself.

Answer

The office address of the IT department

Answer

Employee performance reviews

Question 7

An organization experiences 'Data Leakage' in a machine learning project. What is the most likely cause during the training process?

Accepted Answer

Using target data in training features. Using test/validation data during feature selection or training leads to leakage.

Answer

Using standard Python libraries

Answer

Using secure cloud storage

Answer

Using too many GPUs

Question 8

What is the key difference between 'AI Ethics' and 'AI Compliance'?

Accepted Answer

Ethics focuses on societal values, while compliance focuses on legal and regulatory requirements. Ethics concerns what *should* be done based on values, while compliance concerns what *must* be done based on rules.

Answer

There is no difference between the two

Answer

Ethics is only for AI, while compliance is for all departments

Answer

Compliance is only for technical roles, while ethics is for executives

Question 9

Which TWO of the following are typical 'Model Risk Management' (MRM) pillars?

Accepted Answer

Independent model validation. MRM is structured around independent validation and continuous monitoring.

Answer

Designing office holiday parties

Answer

Negotiating salaries for the data scientists

Answer

Public relations management

Question 10

What is the primary function of an 'AI Pipeline Orchestrator' like Kubeflow or Airflow?

Accepted Answer

Automating task dependencies. Orchestration automates the sequence of tasks: data ingestion, training, and deployment.

Answer

Encrypting network traffic

Answer

Writing code for the model

Answer

Optimizing SQL queries

Question 11

What is the primary purpose of a 'Champion-Challenger' deployment pattern in MLOps?

Accepted Answer

To safely test a new model vs production. Champion-Challenger deployment runs the current production model (champion) alongside one or more candidate models (challengers), routing a small percentage of live traffic to the challengers and comparing business/ML metrics. This enables safe, data-driven promotion decisions without risking the full user base.

Answer

To manage database indexes

Answer

To compress model weights

Answer

To increase training speed

Question 12

In the context of 'AI Pipelines', what is the role of a 'Model Registry'?

Accepted Answer

Version control for model artifacts. The Model Registry provides a central repository to store, version, and manage the lifecycle of trained models.

Answer

Storing training logs

Answer

Monitoring compute costs

Answer

Optimizing SQL queries

Question 13

Which document is essential for an AI audit to identify the 'intended use' of an AI system?

Accepted Answer

Model Card. The Model Card explicitly documents a model's intended use, out-of-scope uses, limitations, and performance characteristics — making it the primary artifact auditors consult to determine whether a model is being used as designed. It is the canonical source for 'intended use' statements in AI governance. Without it, auditors cannot assess fitness-for-purpose or misuse risk.

Answer

Server Hardware Specs

Answer

Financial Statement

Answer

Employee Training Log

Question 14

Why must you conduct a 'Bias Audit' on a model that has been fine-tuned on new data?

Accepted Answer

Fine-tuning can introduce new biases. Fine-tuning updates a model's weights on new data, and if that data reflects historical or sampling bias, the model will absorb and potentially amplify it. A bias audit after fine-tuning verifies that the update did not introduce or worsen disparate outcomes across protected groups. This is a governance and fairness requirement, not a performance optimization.

Answer

It speeds up inference

Answer

It fixes hardware bugs

Answer

It reduces model size

Question 15

In an audit of differential privacy implementations, what is the 'epsilon' parameter used for?

Accepted Answer

Privacy budget control. Epsilon is the privacy budget; lower values provide stronger privacy guarantees but potentially lower utility.

Answer

Model training speed

Answer

Learning rate scheduling

Answer

Database query limit

Question 16

Which AWS SageMaker feature allows auditors to verify that a model was trained on encrypted data?

Accepted Answer

KMS Key Usage in CloudTrail. SageMaker Training Jobs support 'Data Encryption' via KMS keys, which is logged in CloudTrail.

Answer

Instance type selection

Answer

Model registry labels

Answer

Notebook instance tags

Question 17

When reviewing an AI system log, what is the 'inference request' ID used for?

Accepted Answer

Audit trail and traceability. The inference ID links the specific request to its model output, facilitating trace-ability during audits.

Answer

Data encryption

Answer

Hyperparameter storage

Answer

Model compression

Question 18

Which TWO metrics provide the best insights into 'Model Quality' during an audit?

Accepted Answer

Recall score. Precision and Recall are fundamental metrics for assessing the quality of classification models.

Answer

Disk space usage

Answer

Number of server restarts

Answer

Team's average coffee intake

Question 19

What is the goal of a 'Human-in-the-loop' (HITL) control in an AI system?

Accepted Answer

To provide human oversight for automated decisions to mitigate risk and error. HITL ensures that a human makes final decisions for high-stakes AI outputs, reducing the risk of automation error.

Answer

To reduce the time spent on manual data entry

Answer

To replace all human employees with automated systems

Answer

To increase the speed of model training

Question 20

Which THREE criteria are essential when selecting an AI deployment strategy?

Accepted Answer

Service Level Agreements (SLAs). Option C (Service Level Agreements) is essential because SLAs define the required availability, latency, throughput, and uptime guarantees the AI deployment must meet, directly shaping whether a cloud, on-premises, or hybrid strategy is viable. Option D (Risk tolerance) is essential because it determines acceptable exposure to data privacy, regulatory compliance, model accuracy, and vendor lock-in risks, which dictates how much control and isolation the deployment environment must provide. Option E (Compute resource availability) is essential because AI workloads—especially training and inference—demand specific GPU/TPU capacity, memory, and scaling characteristics that constrain which deployment strategy can actually run the model. Options A (UI color) and B (number of coffee machines) are purely cosmetic or unrelated workplace factors and have no bearing on the technical, operational, or compliance requirements of an AI deployment strategy.

Answer

Color of the UI

Answer

Number of coffee machines