AAIA Practice Test 4 — 25 Questions

Question 1

An organization is scaling its AI governance program. You need to automate the identification of 'Shadow AI' in the corporate network. Which tool would be the most effective to gain visibility into unauthorized AI model API usage?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Microsoft Defender for Cloud Apps. Cloud Access Security Brokers (CASBs) like Microsoft Defender for Cloud Apps provide visibility into unsanctioned SaaS and API usage, including AI platforms.

Answer

Azure AI Content Safety

Answer

GitHub Copilot Business logs

Answer

Azure Policy

Question 2

A firm uses a 'Federated Learning' approach. Which governance benefit does this provide?

Accepted Answer

Enhanced data privacy by keeping raw data locally on devices. Federated learning allows training on decentralized data, keeping raw data on local devices, which enhances privacy.

Answer

Reduced usage of electricity

Answer

Faster model training times

Answer

Improved model interpretability

Question 3

A company's AI governance policy requires 'Model explainability'. Which tool or technique best satisfies this requirement for complex deep learning models?

Accepted Answer

Using SHAP (SHapley Additive exPlanations) to interpret model features. SHAP (SHapley Additive exPlanations) or LIME are industry-standard techniques for explaining deep learning decisions.

Answer

Asking the model's developer to write a summary of their feelings about the model

Answer

Printing the model's architecture as a JPEG file

Answer

Reducing the number of parameters in the model to one

Question 4

During an audit of an AI system, you notice that 'Model validation' is performed by the same team that developed the model. Why is this a concern?

Accepted Answer

Lack of independence compromises the integrity of the validation process. Self-validation lacks the independence required to identify errors, bias, or lack of robustness effectively.

Answer

The team might be too friendly with each other

Answer

The validation team might work too slowly

Answer

It is not a concern

Answer

The model will be too accurate

Question 5

What is the primary function of an 'AI Pipeline Orchestrator' like Kubeflow or Airflow?

Accepted Answer

Automating task dependencies. Orchestration automates the sequence of tasks: data ingestion, training, and deployment.

Answer

Encrypting network traffic

Answer

Writing code for the model

Answer

Optimizing SQL queries

Question 6

When designing an AI governance framework, which THREE roles should have oversight responsibility?

Accepted Answer

Legal Counsel. Governance involves cross-functional stakeholders.

Answer

Marketing Intern

Answer

Janitorial Staff

Question 7

If an AI pipeline fails, which component provides the best 'log' to diagnose the failure?

Accepted Answer

Orchestrator task execution logs. The pipeline orchestrator's task logs indicate specifically which step failed and why.

Answer

Model documentation

Answer

Company mission statement

Answer

Developer's email

Question 8

Which action is required when a model is found to be 'Non-compliant' with the internal AI Governance policy?

Accepted Answer

Immediate remediation or removal from the production environment. Non-compliant models must be removed from production or restricted until remediated to prevent risk.

Answer

Writing a blog post about the failure

Answer

Ignoring the finding if the model is profitable

Answer

Giving the model a new name

Question 9

You are reviewing the AI Incident Response Plan. Which scenario constitutes a 'High' severity AI incident that requires immediate board notification under most enterprise risk frameworks?

Accepted Answer

Systemic bias detected in a core automated lending model. An incident where a model makes a biased decision impacting a protected class carries significant regulatory and reputational risk, requiring board visibility.

Answer

A 200ms latency increase during inference

Answer

A minor change in the training dataset size

Answer

A 2% drop in model prediction accuracy

Question 10

Which THREE of the following should be included in an AI Policy document to ensure compliance and ethical alignment?

Accepted Answer

Requirements for bias assessment and mitigation. These items cover accountability, ethical guidelines, and legal compliance, which are central to AI policy.

Answer

Instructions for renaming local variables

Answer

Specific hardware purchase limits

Question 11

You are auditing a firm using LLMs (Large Language Models). Which risk is most specific to generative AI compared to traditional predictive models?

Accepted Answer

Hallucinations and generation of misinformation. Hallucination is a unique risk of generative models where they output plausible but factually incorrect information.

Answer

Database login credentials being shared

Answer

Server power consumption

Answer

Missing documentation for the training code

Question 12

You are auditing a 'Model Lifecycle' and find that the decommissioning process is not defined. What is the impact?

Accepted Answer

Increased risk of using outdated or insecure models in production. Without a decommissioning process, deprecated or risky models may remain in production, posing security and performance risks.

Answer

The office wifi will stop working

Answer

The company's stock price will drop immediately

Answer

The data scientists will leave the company

Question 13

What is the main benefit of 'Automated Retraining'?

Accepted Answer

Keeps models updated with new data. It ensures the model adapts to new data patterns without manual intervention.

Answer

Eliminates the need for humans

Answer

Reduces electricity costs

Answer

Speeds up inference

Question 14

During an audit of an LLM pipeline, you discover that 'System Prompt' versioning is missing. What is the operational risk?

Accepted Answer

Unpredictable model behavior changes. Without system prompt versioning, changes to the instructions can cause unpredictable behavior changes that cannot be reverted.

Answer

Higher cloud bills

Answer

Increased latency

Answer

Database corruption

Question 15

Which TWO are common types of 'AI Model Drift'?

Accepted Answer

Concept drift. Drift is split between inputs (data) and outputs (concept).

Answer

Temperature drift

Answer

CPU drift

Answer

Memory drift

Question 16

Which TWO are common 'AI Operational' challenges?

Accepted Answer

Data quality management. Managing data quality and model maintenance are core challenges.

Answer

Selecting a font

Answer

Writing poetry

Answer

Buying new office chairs

Question 17

Which THREE risks should be explicitly managed in an 'AI Governance' program?

Accepted Answer

Operational performance degradation. Governance must manage bias, security, and performance risks.

Answer

The choice of font for the final report

Answer

The color of the server room lights

Question 18

In Azure Machine Learning, which component should an auditor verify to ensure model registry access is restricted to authorized CI/CD service principals?

Accepted Answer

Workspace Access Control (IAM). Access to the Model Registry is governed by Azure RBAC and specifically Managed Identities/Service Principals assigned to the workspace.

Answer

Environment YAML files

Answer

Compute Instance logs

Answer

Data labeling projects

Question 19

In the context of 'AI Pipelines', what is the role of a 'Model Registry'?

Accepted Answer

Version control for model artifacts. The Model Registry provides a central repository to store, version, and manage the lifecycle of trained models.

Answer

Storing training logs

Answer

Monitoring compute costs

Answer

Optimizing SQL queries

Question 20

An organization uses AWS SageMaker. As an auditor, you need to verify that model lineage and versioning are being captured to satisfy the AI risk assessment requirement for reproducibility. Which feature ensures that every model training job is recorded with its data sources?

Accepted Answer

SageMaker Lineage Tracking. SageMaker Lineage Tracking automatically captures the relationship between data, models, and training jobs to ensure full auditability.

Answer

AWS CloudTrail

Answer

SageMaker Clarify

Answer

SageMaker Model Registry