AAIA Practice Test 8 — 10 Questions

Question 1

In Azure Machine Learning, which component should an auditor verify to ensure model registry access is restricted to authorized CI/CD service principals?

Accepted Answer

Workspace Access Control (IAM). Access to the Model Registry is governed by Azure RBAC and specifically Managed Identities/Service Principals assigned to the workspace.

Answer

Environment YAML files

Answer

Compute Instance logs

Answer

Data labeling projects

Question 2

You find that the 'Inference API' for a model allows 'Prompt Injection' attacks. Which governance failure is most likely?

Accepted Answer

Inadequate input validation controls and adversarial testing. Prompt injection is a failure of input validation and security design within the model governance framework.

Answer

Lack of a social media presence for the company

Answer

Using a low-cost cloud provider

Answer

Failure to update the company logo

Question 3

If an AI pipeline fails, which component provides the best 'log' to diagnose the failure?

Accepted Answer

Orchestrator task execution logs. The pipeline orchestrator's task logs indicate specifically which step failed and why.

Answer

Model documentation

Answer

Company mission statement

Answer

Developer's email

Question 4

You are auditing a model's 'Robustness' against adversarial attacks. Which test is most appropriate?

Accepted Answer

Performing adversarial perturbation testing. Adversarial testing involves introducing perturbations to inputs to see if the model's output changes unexpectedly.

Answer

Checking if the server is plugged in

Answer

Checking the model's training accuracy

Answer

Asking the team to describe their favorite movies

Question 5

What is 'Model Versioning' used for in an AI project?

Accepted Answer

Ensuring reproducibility and rollback. It allows tracking changes, rolling back to previous versions, and ensuring reproducibility.

Answer

Reducing model size

Answer

Improving training time

Answer

Increasing security

Question 6

You are auditing a model using IBM Watson OpenScale. Which feature should be configured to detect 'Disparate Impact' to ensure regulatory compliance?

Accepted Answer

Fairness monitor. The Fairness monitor in IBM Watson OpenScale is specifically designed to detect and measure bias in machine learning models, including metrics like Disparate Impact. It evaluates whether a model's predictions disproportionately favor or disadvantage certain groups based on protected attributes (e.g., gender, race). This directly supports regulatory compliance by identifying discriminatory outcomes, making it the correct choice for detecting Disparate Impact.

Answer

Quality monitor

Answer

Drift detection monitor

Answer

Explainability monitor

Answer

Payload logging

Question 7

When auditing an MLflow experiment, which object allows the auditor to verify the exact parameters and code version used during training?

Accepted Answer

MLflow Run. An MLflow Run is the fundamental unit that captures all details of a single execution of a machine learning experiment, including parameters, metrics, artifacts, and the source code version (via Git commit or other tags). Therefore, the Run object allows an auditor to verify the exact parameters and code version used during training. The Model Registry manages model versions and stages but does not store the original training parameters or code version directly.

Answer

MLflow Model Registry

Answer

MLflow Experiment

Answer

MLflow Tracking Server

Question 8

Which THREE factors must be evaluated when assessing the 'Fairness' of an AI model?

Accepted Answer

Disparities in model error rates across protected groups. Fairness assessment requires examining disparities in model error rates across protected groups (B), because unequal false-positive or false-negative rates (e.g., via equalized odds or error-rate parity) directly reveal discriminatory performance. Historical bias present in the training datasets (C) must be evaluated since biased or unrepresentative data propagates and amplifies unfair outcomes in the model. Demographic parity metrics for the model's predictions (D) are also essential, as they quantify whether favorable outcomes are distributed equally across groups (e.g., via disparate impact ratio). The programming language used to build the model (A) is irrelevant to fairness, as fairness depends on data, features, and evaluation metrics rather than implementation language. The number of CPU cores allocated for training (E) affects computational performance only, not the fairness of the model's predictions.

Answer

The programming language used to build the model

Answer

The number of CPU cores allocated for training

Question 9

What is the consequence of 'Pipeline Staleness' in an AI system?

Accepted Answer

Model degradation due to outdated data. If the pipeline is not refreshed, it trains on outdated data, leading to model degradation.

Answer

Unauthorized access

Answer

Increased hardware costs

Answer

System crash

Question 10

You are auditing a 'Model Monitoring' dashboard. Which metric is most indicative of a potential degradation in the model's reliability over time?

Accepted Answer

Statistical drift between the production input data and training baseline. Data drift and concept drift are primary indicators that a model's performance may be degrading in a production environment.

Answer

The physical location of the cloud server hosting the model

Answer

The number of lines of code in the model deployment script

Answer

The number of users accessing the model API