AAIA Practice Test 17 — 15 Questions

Question 1

You are auditing a deployment on Kubernetes using Kubeflow. Which component should the auditor examine to ensure that the pipeline steps are reproducible?

Accepted Answer

Kubeflow Pipelines. Kubeflow Pipelines (KFP) ensure reproducibility through containerized steps and pipeline definitions stored in YAML.

Answer

Kubelet

Answer

Prometheus

Answer

Istio

Question 2

Which TWO of the following are examples of AI governance 'Inputs'?

Accepted Answer

Internal AI Policy and guidelines. Option B is correct because an Internal AI Policy and guidelines is a governance input: it supplies the organization's own rules, principles, and constraints that shape how AI systems are designed, deployed, and monitored. Option C is correct because external regulatory frameworks such as the EU AI Act are governance inputs: they impose legal and compliance requirements that the organization's AI governance program must incorporate. Option A is not a governance input; a photograph of the company founder has no bearing on AI policies, controls, or compliance obligations. Option D is not a governance input; a social media password is a credential and a security concern, not a source of AI governance requirements. Option E is not a governance input; an office printer manual documents hardware operation and is unrelated to AI governance.

Answer

A photograph of the company founder

Answer

The company's social media password

Answer

The office printer manual

Question 3

An auditor is evaluating an AI system's robustness against adversarial attacks. Which technique involves perturbing input data to see if the model output changes significantly?

Accepted Answer

Adversarial testing. Adversarial testing is the practice of deliberately perturbing inputs (adding noise, changing pixels, altering tokens) to probe whether a model's output changes in ways that reveal fragility or exploitable decision boundaries. It is the standard technique auditors use to assess robustness against adversarial attacks because it directly simulates the attacker's behavior. Pruning, bias assessment, and augmentation serve different purposes and do not test adversarial robustness.

Answer

Model pruning

Answer

Bias assessment

Answer

Data augmentation

Question 4

Which THREE of the following are essential components of an MLOps maturity model for ensuring operational stability?

Accepted Answer

Centralized model registry and artifact management. Option B is correct because a centralized model registry and artifact management system provides versioning, lineage, and governance for models and their artifacts, which is essential for reproducible deployments and operational stability in MLOps. Option C is correct because automated retraining pipelines enable consistent, repeatable model updates in response to data drift or performance degradation, reducing manual error and keeping models reliable in production. Option E is correct because real-time monitoring and alerting for model performance detects drift, degradation, and anomalies promptly, allowing teams to respond before business impact occurs. Option A is not appropriate because storing raw data directly in the model repository conflates data and model artifacts, harming governance, reproducibility, and storage management. Option D is not essential because manual model validation processes do not scale and undermine the automation and consistency that MLOps maturity models require for operational stability.

Answer

Storing raw data directly in the model repository

Answer

Manual model validation processes

Question 5

Which tool provides visual confirmation of data lineage in an end-to-end AI project?

Accepted Answer

Apache Airflow. Apache Airflow is a workflow orchestration platform that provides a web-based UI showing DAG (Directed Acyclic Graph) runs, task dependencies, and data flow across pipeline stages. This visual representation allows stakeholders to trace data lineage from ingestion through transformation to model training and deployment, confirming end-to-end data provenance.

Answer

Docker

Answer

Jenkins

Answer

Kubernetes

Question 6

In an AI Audit, what does an 'Input Feature Schema' define?

Accepted Answer

Expected data format and constraints. It defines the expected data types, shapes, and constraints for input features, ensuring integration stability.

Answer

The training duration

Answer

The database name

Answer

The model architecture

Question 7

Which TWO of the following are considered 'Model Metadata' that should be tracked in a registry?

Accepted Answer

Hyperparameters. Model metadata in a registry captures the configuration and data lineage needed to reproduce, audit, and govern a model. Option A (Hyperparameters) is correct because values such as learning rate, batch size, number of layers, and regularization strength define how the model was configured and are essential for reproducibility and comparison across runs. Option B (Training dataset version) is correct because recording the exact dataset version (e.g., a DVC hash, S3 object version, or snapshot ID) establishes data lineage, enabling audits, rollback, and detection of training-serving skew. The remaining options do not belong: an employee vacation schedule (C) is HR data unrelated to model artifacts, a cafeteria menu (D) is facilities information, and an office Wi-Fi password (E) is a security credential, none of which describe model configuration, data, or lineage.

Answer

Employee vacation schedule

Answer

Cafeteria menu

Answer

Office Wi-Fi password

Question 8

Which of these is a valid reason to 'Roll Back' a model deployment?

Accepted Answer

Detected production performance issues. If the model in production starts showing unexpected errors or performance drops, rolling back is the immediate safety action.

Answer

To save storage space

Answer

To change the model name

Answer

Developer preference

Question 9

You are configuring a CI/CD pipeline for ML using GitHub Actions to deploy to a Kubernetes cluster. To ensure sensitive credentials for your container registry are not exposed in logs, what is the best practice?

Accepted Answer

Store credentials in GitHub Secrets. Using GitHub Secrets is the standard way to inject sensitive data into CI/CD pipelines securely.

Answer

Use environment variables in the workflow file

Answer

Hardcode credentials in the Dockerfile

Answer

Commit a .env file to the repository

Question 10

Which of the following is a key component of 'MLOps Maturity'?

Accepted Answer

Automated pipeline triggers. High maturity involves full CI/CD, automated testing, and active model monitoring.

Answer

Manual data entry

Answer

Single engineer silo

Answer

No documentation

Question 11

What is the purpose of 'Performance Monitoring' in AI?

Accepted Answer

Ensuring SLA compliance. To detect if the system is meeting its latency, throughput, and accuracy targets.

Answer

Encrypting data

Answer

Changing model weights

Answer

Renaming models

Question 12

What is the role of an 'A/B Test' in AI production?

Accepted Answer

Comparing two models in production. A/B testing allows comparing the performance of a new model against an existing one with real user traffic.

Answer

Encrypting data

Answer

Reducing training time

Answer

Deleting unused models

Question 13

An AI audit team is evaluating a bank's production ML system that processes loan applications. The system uses an automated pipeline that retrains the model every month. The audit team needs to verify that the retraining process does not introduce unintended bias. Which operational control should the audit team examine first?

Accepted Answer

The model validation gate that includes fairness metrics before deployment. The most direct operational control for verifying that retraining does not introduce bias is a validation gate that enforces fairness metrics before deployment. Such a gate acts as a preventive control, ensuring that any model failing bias thresholds is blocked from production. Auditors should examine the gate's design, the fairness metrics selected, and the approval process to confirm it operates effectively.

Answer

The data drift detection thresholds configured for the pipeline

Answer

The logging of model predictions for post-hoc analysis

Answer

The version control system used for model artifacts

Question 14

An AI auditor is reviewing a healthcare organization's ML system that predicts patient readmission risk. The system uses a feature store to serve features to both training and inference pipelines. The auditor notices that the feature store does not enforce point-in-time correctness for training data. What is the most significant operational risk this poses?

Accepted Answer

Data leakage leading to overly optimistic performance estimates. The absence of point-in-time correctness in a feature store allows future information to leak into training features, creating data leakage. This causes the model to learn from information unavailable at prediction time, inflating validation metrics and leading to poor generalization. In healthcare, such leakage could result in a model that fails to identify high-risk patients in real time, compromising patient safety and regulatory compliance.

Answer

Inability to reproduce model training runs

Answer

Increased latency in the inference pipeline

Answer

Training-serving skew due to inconsistent feature transformations

Question 15

An AI auditor is reviewing the operational documentation for a computer vision model used in quality control on a manufacturing line. The documentation includes a section on 'model serving infrastructure.' Which element is most important for the auditor to verify regarding the model's ability to handle peak production loads?

Accepted Answer

The autoscaling configuration and resource limits of the inference service. The autoscaling configuration and resource limits of the inference service are most important for handling peak production loads. These settings determine whether the system can dynamically allocate resources to meet demand without performance degradation. Auditors should verify that scaling policies are tested, monitored, and capable of maintaining throughput during peak periods, ensuring operational reliability.

Answer

The number of layers in the neural network architecture

Answer

The model's inference latency on a single test image

Answer

The model's training dataset size and diversity