Courseiva

ISACA Advanced in AI Audit (AAIA) (AAIA) (AAIA) — Questions 151209

209 questions total · 3pages · All types, answers revealed

Page 2

Page 3 of 3

151
MCQmedium

An organization is scaling its AI governance program. You need to automate the identification of 'Shadow AI' in the corporate network. Which tool would be the most effective to gain visibility into unauthorized AI model API usage?

A.Azure AI Content Safety
B.Microsoft Defender for Cloud Apps
C.GitHub Copilot Business logs
D.Azure Policy
AnswerB

CASB functionality allows for the discovery and control of unauthorized AI services (Shadow AI).

Why this answer

Cloud Access Security Brokers (CASBs) like Microsoft Defender for Cloud Apps provide visibility into unsanctioned SaaS and API usage, including AI platforms.

152
MCQmedium

During an audit of an AI system, you notice that 'Model validation' is performed by the same team that developed the model. Why is this a concern?

A.The team might be too friendly with each other
B.Lack of independence compromises the integrity of the validation process
C.The validation team might work too slowly
D.It is not a concern
E.The model will be too accurate
AnswerB

Conflict of interest prevents the team from objectively reviewing their own work.

Why this answer

Self-validation lacks the independence required to identify errors, bias, or lack of robustness effectively.

153
MCQhard

Which action is required when a model is found to be 'Non-compliant' with the internal AI Governance policy?

A.Writing a blog post about the failure
B.Ignoring the finding if the model is profitable
C.Immediate remediation or removal from the production environment
D.Giving the model a new name
AnswerC

Risk management dictates that non-compliant systems must be controlled immediately.

Why this answer

Non-compliant models must be removed from production or restricted until remediated to prevent risk.

154
MCQeasy

Which role is primarily responsible for ensuring that the AI governance framework is followed in daily operations?

A.The Model Manager or Product Owner
B.The company's external auditor
C.The office manager
D.The chief financial officer
AnswerA

They own the system and are responsible for its lifecycle and compliance.

Why this answer

The AI Product Owner or Model Manager is responsible for the operational compliance of their specific AI systems.

155
MCQeasy

What is 'Model Versioning' used for in an AI project?

A.Reducing model size
B.Improving training time
C.Increasing security
D.Ensuring reproducibility and rollback
AnswerD

Core MLOps capability.

Why this answer

It allows tracking changes, rolling back to previous versions, and ensuring reproducibility.

156
MCQmedium

A firm uses a third-party AI service. Which control is most critical to ensure compliance with the firm's own AI governance policy?

A.Including AI-specific compliance requirements in the vendor service-level agreement (SLA)
B.Conducting weekly penetration tests on the vendor's office premises
C.Requesting the vendor's employee turnover rates for the last fiscal year
D.Requiring the vendor to submit copies of all their marketing brochures
AnswerA

Contractual obligations are the primary mechanism for enforcing internal standards on external partners.

Why this answer

Supply chain risk management requires mapping vendor controls to internal governance requirements.

157
Multi-Selectmedium

Which TWO are standard practices for securing AI model endpoints?

Select 2 answers
A.Mutual TLS (mTLS)
B.Unprotected endpoints
C.Storing logs on public S3
D.Public internet access
E.API Rate Limiting
AnswersA, E

Authenticates both client and server.

Why this answer

Security requires network and authentication controls.

158
MCQhard

In a financial services firm, you are auditing the 'Explainability' of a credit scoring model. The model uses complex non-linear features. Which technique should you verify is being used to provide local explanations for individual credit decisions?

A.Feature Importance Permutation
B.SHAP values
C.Model pruning
D.Data normalization
AnswerB

SHAP provides specific, mathematically sound local explanations for individual model outputs.

Why this answer

SHAP (SHapley Additive exPlanations) is a widely accepted, model-agnostic technique for providing local explanations of individual predictions.

159
Multi-Selectmedium

Which TWO are common 'AI Operational' challenges?

Select 2 answers
A.Selecting a font
B.Data quality management
C.Writing poetry
D.Buying new office chairs
E.Model maintenance
AnswersB, E

Ensures model health.

Why this answer

Managing data quality and model maintenance are core challenges.

160
MCQeasy

What is the main benefit of 'Automated Retraining'?

A.Eliminates the need for humans
B.Reduces electricity costs
C.Keeps models updated with new data
D.Speeds up inference
AnswerC

Reduces drift impact.

Why this answer

It ensures the model adapts to new data patterns without manual intervention.

161
MCQeasy

What is the main role of the 'AI Risk Assessment' document in a project lifecycle?

A.To provide a history of the company's annual profits
B.To record the names of all HR staff
C.To identify and evaluate potential risks before the model is deployed
D.To list the office parking spaces
AnswerC

Proactive identification of risk allows for mitigation before harmful deployment.

Why this answer

The risk assessment identifies potential hazards and guides the implementation of controls before the project proceeds.

162
MCQhard

Why must you conduct a 'Bias Audit' on a model that has been fine-tuned on new data?

A.It speeds up inference
B.It fixes hardware bugs
C.Fine-tuning can introduce new biases
D.It reduces model size
AnswerC

Data distribution shifts can correlate with sensitive attributes.

Why this answer

Fine-tuning on new data can inadvertently introduce new biases or exacerbate existing ones.

163
MCQhard

In Google Vertex AI Pipelines, which mechanism prevents 'training-serving skew' when deploying a custom container model?

A.Custom prediction routines
B.Artifact metadata locks
C.Model Monitoring skew detection
D.Pipeline concurrency limits
AnswerC

Automates comparison of feature distributions.

Why this answer

Vertex AI Model Monitoring provides built-in skew detection by comparing training data statistics against serving traffic statistics.

164
MCQmedium

A firm uses a 'Federated Learning' approach. Which governance benefit does this provide?

A.Reduced usage of electricity
B.Faster model training times
C.Improved model interpretability
D.Enhanced data privacy by keeping raw data locally on devices
AnswerD

Because raw data is not centralized, the privacy risk is significantly reduced.

Why this answer

Federated learning allows training on decentralized data, keeping raw data on local devices, which enhances privacy.

165
MCQhard

What is the most effective way to address 'Adversarial Input' during inference?

A.Reducing training time
B.Adversarial training and input filtering
C.Increasing model depth
D.Ignoring the input
AnswerB

Hardens the model against inputs.

Why this answer

Implementing input validation/sanitization and adversarial training (training on adversarial examples) improves robustness.

166
Multi-Selecthard

When selecting testing techniques for an AI model, which THREE are considered 'Model-Agnostic'?

Select 3 answers
A.SHAP (SHapley Additive exPlanations)
B.Neural network weight pruning
C.Partial Dependence Plots (PDPs)
D.Linear regression coefficients
E.LIME (Local Interpretable Model-agnostic Explanations)
AnswersA, C, E

A model-agnostic method for feature contribution.

Why this answer

LIME, SHAP, and Partial Dependence Plots (PDPs) are model-agnostic methods that work with any ML model.

167
Multi-Selectmedium

Which TWO techniques should an auditor employ to detect bias in a model where the training data has imbalanced demographic representation?

Select 2 answers
A.Updating the corporate logo
B.Implementation of fairness-aware loss functions
C.Changing the font size in the UI
D.Increasing the number of project meetings
E.Stratified cross-validation
AnswersB, E

Addresses bias directly during training/audit adjustments.

Why this answer

Stratified sampling and fairness-aware loss functions are common techniques to address and audit bias.

168
MCQmedium

In SageMaker, what does 'Model Monitor' specifically detect?

A.Database uptime
B.Model size
C.Employee attendance
D.Data and model drift
AnswerD

Core feature of SageMaker Model Monitor.

Why this answer

It monitors for data drift and model quality degradation in production.

169
MCQmedium

When auditing data pipelines, what is the function of a 'Data Quality' assertion?

A.To rename data columns
B.To validate data integrity before training
C.To increase prediction speed
D.To compress input files
AnswerB

Assertions act as guardrails for data quality.

Why this answer

Assertions (e.g., in Great Expectations) check if data meets predefined schemas and constraints before ingestion.

170
MCQeasy

What is the purpose of 'Model Validation' in the audit lifecycle?

A.To speed up data ingestion
B.To lower API costs
C.To increase training time
D.To verify model performance on test data
AnswerD

Validation ensures the model generalizes well to new data.

Why this answer

Validation confirms that the model performs as expected against independent, unseen data.

171
MCQeasy

Which document is the primary source for establishing the scope of an AI audit?

A.A list of all employees who have taken an AI training course
B.The internal AI Policy and the organization's Risk Management Framework
C.The company's social media posting guidelines
D.The employee handbook for the engineering department
AnswerB

These documents define what is permitted, what is tracked, and how risks are managed.

Why this answer

The AI Policy and the AI Risk Management Framework (RMF) define the boundaries and expectations for the audit.

172
Multi-Selecthard

Which THREE criteria are essential when selecting an AI deployment strategy?

Select 3 answers
A.Color of the UI
B.Number of coffee machines
C.Service Level Agreements (SLAs)
D.Risk tolerance
E.Compute resource availability
AnswersC, D, E

Defines uptime requirements.

Why this answer

Strategy must balance risk, speed, and resource availability.

173
Multi-Selectmedium

Which THREE of the following should be included in an AI Policy document to ensure compliance and ethical alignment?

Select 3 answers
A.Requirements for bias assessment and mitigation
B.Transparency and disclosure requirements for automated decisions
C.Instructions for renaming local variables
D.Specific hardware purchase limits
E.Requirements for data privacy and security protections
AnswersA, B, E

Essential for ethical AI alignment.

Why this answer

These items cover accountability, ethical guidelines, and legal compliance, which are central to AI policy.

174
MCQmedium

A data scientist proposes using a 'black-box' model for a high-stakes loan approval process. Which action does your AI audit framework require?

A.Mandating the use of XAI (Explainable AI) tools to interpret model decisions
B.Requiring the data scientist to publish the code on an internal repository
C.Allowing the model if it achieves 99% accuracy on the test set
D.Asking the marketing team to verify the model's fairness
AnswerA

For high-stakes decisions, auditability and transparency require that decisions be explainable.

Why this answer

Explainability is a prerequisite for models that have significant impact on individuals, as per most AI governance frameworks.

175
MCQmedium

You are reviewing a model risk management (MRM) framework for an AI system using NVIDIA NeMo Guardrails. What is the primary purpose of this tool in a risk mitigation strategy?

A.To automate model retraining
B.To improve model training speed
C.To enforce safety boundaries on LLM inputs and outputs
D.To encrypt training datasets
AnswerC

NeMo Guardrails provides a programmatic way to constrain model behavior to compliant, safe interactions.

Why this answer

NeMo Guardrails allows developers to define programmatic boundaries for LLM behavior, effectively serving as an automated control for AI governance.

176
MCQmedium

You are reviewing the 'AI Incident Response Plan'. Which element is essential for compliance?

A.The history of the company's founding
B.Defined escalation paths for ethical or performance incidents
C.A log of all office supply purchases
D.A list of all cafeteria menus in the building
AnswerB

An AI incident requires a specific response team to handle potential ethical or safety impacts.

Why this answer

Incident response must include specific triggers for escalation and communication regarding AI failures.

177
MCQhard

When auditing an AI system for regulatory compliance, you find that data lineage is broken between the feature store and the training pipeline. Which control is most likely deficient?

A.Access control lists for the model production environment
B.Encryption of data at rest in the data warehouse
C.Load balancing configuration for the inference API
D.Metadata logging and transformation documentation
AnswerD

Metadata is the key to maintaining lineage; without it, the link between features and training data is obscured.

Why this answer

Feature stores must maintain end-to-end traceability to ensure that data provenance is clear and auditable.

178
MCQmedium

What does a 'Backtesting' report verify in an AI Audit?

A.Future accuracy
B.Cloud service costs
C.Developer salary
D.Historical model performance
AnswerD

Validates reliability.

Why this answer

Backtesting uses historical data to verify how the model *would* have performed, ensuring reliability.

179
MCQmedium

You are auditing a firm's AI policy and find that the 'Model Inventory' in the enterprise risk management platform lacks versioning metadata. Which specific control gap should be prioritized to align with NIST AI RMF?

A.Absence of automated model deployment pipelines
B.Failure to define the RPO for the primary database
C.Lack of immutable audit logs for model architecture changes
D.Insufficient budget for cloud storage of historical data
AnswerC

Without versioning metadata, the lineage of model iterations is lost, violating traceability requirements.

Why this answer

Traceability and version control are essential to establish accountability and reproducibility in AI risk management.

180
MCQhard

Which technique is most appropriate for mitigating 'Concept Drift' in a production model?

A.Increasing the number of neurons in the hidden layers
B.Moving the model to a larger GPU cluster
C.Implementing an automated re-training pipeline based on performance threshold triggers
D.Restricting access to the training dataset
AnswerC

Automated re-training using recent, representative data corrects the drift in the model's concept.

Why this answer

Continuous re-training or fine-tuning based on recent data is the standard mitigation for concept drift.

181
Multi-Selecthard

Which TWO of the following are legitimate 'AI Governance' concerns regarding third-party AI service providers (SaaS)?

Select 2 answers
A.The programming language used by the vendor
B.Data residency and compliance with local regulations
C.Third-party audit rights and transparency of the vendor's model development
D.Vendor's model update and retraining frequency
E.The physical location of the vendor's corporate office
AnswersB, C

Ensuring the provider processes data in compliant regions is a major risk.

Why this answer

Auditors must ensure that third-party AI tools comply with internal data handling standards and that the vendor's own governance is adequate.

182
MCQmedium

What is the role of a 'Deployment Pipeline' in MLOps?

A.Automating deployment steps
B.Storing logs
C.Writing code
D.Labeling data
AnswerA

Reduces human error.

Why this answer

To automate the packaging, testing, and deployment of a model.

183
MCQeasy

An auditor is asked to review the AI model inventory. What information must be present in the inventory for every model to satisfy basic regulatory requirements?

A.Model owner, business purpose, and status
B.The complete list of all training algorithms used
C.The total number of GPUs used during training
D.All hyperparameter values
AnswerA

These are the fundamental requirements for maintaining an accurate and auditable AI inventory.

Why this answer

An inventory must track model ownership, purpose, and lifecycle stage to ensure accountability and oversight.

184
Multi-Selectmedium

Which TWO items should be included in an automated 'Deployment Log'?

Select 2 answers
A.Deployed model version/ID
B.Office temperature
C.Team member home address
D.Favorite programming language
E.Deployment timestamp
AnswersA, E

Provenance requirement.

Why this answer

Logs must provide the 'who, when, and what' of the deployment.

185
Multi-Selecthard

Which THREE technical artifacts should the auditor collect to verify the 'Model Training Process'?

Select 3 answers
A.Environment requirements (e.g., requirements.txt)
B.The office address of the IT department
C.Training execution logs
D.Employee performance reviews
E.Training script/code
AnswersA, C, E

Ensures reproducible environments.

Why this answer

Training logs, environment requirements (dependencies), and the training script are essential to reconstruct the training process.

186
Multi-Selectmedium

Which TWO are common types of 'AI Model Drift'?

Select 2 answers
A.Concept drift
B.Data drift
C.Temperature drift
D.CPU drift
E.Memory drift
AnswersA, B

Relationship shift.

Why this answer

Drift is split between inputs (data) and outputs (concept).

187
Multi-Selectmedium

When scoping an audit, which TWO stakeholders should the auditor interview?

Select 2 answers
A.Model Owners
B.The lobby receptionist
C.The company's external marketing agency
D.Data Scientists
E.Office cleaning staff
AnswersA, D

They understand the business objectives and risk ownership.

Why this answer

Data Scientists and Model Owners have the necessary technical and business context for the AI system.

188
MCQhard

You find that the 'Inference API' for a model allows 'Prompt Injection' attacks. Which governance failure is most likely?

A.Inadequate input validation controls and adversarial testing
B.Lack of a social media presence for the company
C.Using a low-cost cloud provider
D.Failure to update the company logo
AnswerA

Testing for and preventing adversarial inputs is a core requirement of secure AI development.

Why this answer

Prompt injection is a failure of input validation and security design within the model governance framework.

189
Multi-Selecthard

Which THREE criteria are necessary for an AI system to be deemed 'Trustworthy' under most governance frameworks?

Select 3 answers
A.Reliability and consistent performance over time
B.Transparency of the decision-making process
C.The ability to generate high revenue for the firm
D.Robustness and security against adversarial attacks
E.The use of the latest marketing buzzwords
AnswersA, B, D

Consistent performance is a requirement for trust.

Why this answer

Trustworthiness is derived from transparency, reliability, and security.

190
MCQmedium

An organization uses MLflow for model tracking. To ensure auditability of model lineage, what must be captured in the 'mlflow.log_artifact' call?

A.Environment variable list
B.User IP address
C.Compute cluster name
D.Git commit hash and data URI
AnswerD

This links code state to the model artifact.

Why this answer

Capturing the specific code version (git hash) and data snapshot ID ensures reproducibility.

191
MCQeasy

What is the purpose of 'Model Versioning' in an AI audit context?

A.To automatically delete old models to save space
B.To allow for reproducibility and auditability of past model versions
C.To hide the model code from competitors
D.To reduce the number of files on the server
AnswerB

If a decision is challenged, you must be able to reproduce it using the exact model version used at that time.

Why this answer

Versioning ensures that the specific state of a model used for a specific decision can be retrieved and audited.

192
MCQmedium

An auditor notices high 'latency' in an online model deployment. Which component should be inspected first?

A.Developer laptop specs
B.Model documentation
C.Inference service resource limits
D.Data labeling tool settings
E.Git repository history
AnswerC

CPU/RAM bottlenecks cause latency.

Why this answer

The inference container resource allocation or the request preprocessing overhead is the common bottleneck.

193
MCQhard

An auditor is evaluating an AI system for 'Model Inversion' risk. What is this?

A.Reducing inference latency
B.Increasing training bias
C.Overfitting the data
D.Reconstructing training data from outputs
AnswerD

Model inversion aims to leak sensitive training data.

Why this answer

Model Inversion is an attack where an adversary reconstructs training data from model outputs.

194
MCQmedium

What is the function of a 'Feature Store' in an MLOps pipeline?

A.Storing raw video files
B.Compressing model weights
C.Centralized feature serving and storage
D.Monitoring compute costs
AnswerC

Ensures consistency between training and serving.

Why this answer

A Feature Store provides a centralized repository for standardized feature definitions and storage for both training and inference.

195
Multi-Selectmedium

Which TWO of the following are essential for ensuring 'Model Observability' in an enterprise production environment?

Select 2 answers
A.System latency monitoring
B.Manual code refactoring
C.Database backup frequency
D.Developer salary audits
E.Prediction drift monitoring
AnswersA, E

Essential for SLA compliance.

Why this answer

Observability requires both performance metrics and input monitoring (drift).

196
MCQeasy

What is the primary objective of 'Board-level AI Governance'?

A.To ensure AI aligns with the organization's business strategy and risk appetite
B.To decide which programming language is used
C.To fix bugs in the model code
D.To organize the annual team lunch
AnswerA

The board's duty is strategic oversight and protecting shareholder value.

Why this answer

The board ensures that AI aligns with the firm's strategic risk appetite and legal obligations.

197
MCQeasy

Which of the following is a symptom of 'Overfitting' during the training phase?

A.Low training accuracy, low validation accuracy
B.Model crashes during training
C.Model is too small
D.High training accuracy, low validation accuracy
AnswerD

Classic sign of overfitting.

Why this answer

Overfitting happens when the model performs well on training data but poorly on unseen test data.

198
MCQmedium

What is the primary function of an 'AI Model Gateway'?

A.Storing model weights
B.Centralized request management and security
C.Training the model
D.Labeling data
AnswerB

Standard API gateway function.

Why this answer

A gateway acts as a security and management layer for AI requests, providing throttling, logging, and authentication.

199
MCQmedium

When auditing fairness metrics, what does the 'Demographic Parity' metric measure?

A.Equal positive outcome rates
B.Equal training data representation
C.Equal error rates
D.Equal accuracy rates
AnswerA

Demographic parity specifically looks at the rate of positive predictions across groups.

Why this answer

Demographic Parity requires that the probability of a positive outcome be equal across different demographic groups.

200
MCQeasy

What is the purpose of an 'AI Audit trail'?

A.Reducing cloud latency
B.Providing transparency and accountability
C.Increasing model performance
D.Changing model parameters
AnswerB

Key compliance requirement.

Why this answer

To provide a transparent, verifiable record of all actions taken on an AI system.

201
MCQmedium

You are assessing a company's 'AI Governance Committee'. Which action best demonstrates effective board-level oversight?

A.Managing the day-to-day procurement of data labeling services
B.Reviewing individual code commits for AI model training
C.Selecting the machine learning framework used by data scientists
D.Approving the annual AI strategy and corresponding risk mitigation plans
AnswerD

The board's role is to ensure that AI risk is integrated into the overall enterprise risk strategy.

Why this answer

Board oversight must focus on strategic alignment and the establishment of high-level accountability frameworks.

202
MCQmedium

In Azure Machine Learning, which component should an auditor verify to ensure model registry access is restricted to authorized CI/CD service principals?

A.Workspace Access Control (IAM)
B.Environment YAML files
C.Compute Instance logs
D.Data labeling projects
AnswerA

RBAC controls access to the workspace registry.

Why this answer

Access to the Model Registry is governed by Azure RBAC and specifically Managed Identities/Service Principals assigned to the workspace.

203
MCQhard

If an AI pipeline fails, which component provides the best 'log' to diagnose the failure?

A.Orchestrator task execution logs
B.Model documentation
C.Company mission statement
D.Developer's email
AnswerA

Shows the point of failure.

Why this answer

The pipeline orchestrator's task logs indicate specifically which step failed and why.

204
MCQmedium

Which artifact should an auditor request to verify 'Model Governance' during the model life cycle?

A.The personal notes of the lead developer
B.A photograph of the development team
C.Independent model validation reports
D.A list of office keys issued to employees
AnswerC

These reports prove that the model underwent required governance checks before production deployment.

Why this answer

Model validation reports provide evidence that the model was checked against performance and safety standards.

205
MCQmedium

You are reviewing the AI Incident Response Plan. Which scenario constitutes a 'High' severity AI incident that requires immediate board notification under most enterprise risk frameworks?

A.A 200ms latency increase during inference
B.Systemic bias detected in a core automated lending model
C.A minor change in the training dataset size
D.A 2% drop in model prediction accuracy
AnswerB

Systemic bias in high-impact models poses critical regulatory and compliance risk.

Why this answer

An incident where a model makes a biased decision impacting a protected class carries significant regulatory and reputational risk, requiring board visibility.

206
MCQeasy

What is the goal of an AI 'Rollout' plan?

A.Reducing disk space
B.Deleting logs
C.Controlled deployment to users
D.Increasing training time
AnswerC

Risk management.

Why this answer

To manage the deployment of a model to users in a controlled way.

207
Multi-Selectmedium

Which TWO of the following are examples of 'Data Leakage' that an auditor should look for in a pipeline?

Select 2 answers
A.Using cloud-based storage
B.Training on test data
C.Documenting model parameters
D.Using open-source libraries
E.Including target-related features that are not available at inference time
AnswersB, E

Training on the test set is a major leakage violation.

Why this answer

Target leakage and future data leakage are primary types of data leakage in AI pipelines.

208
MCQhard

If a model's performance drops, and the input data distribution has changed, what operational process must be triggered?

A.Security audit
B.Model Deletion
C.Hardware Upgrade
D.Code Refactoring
E.Model Retraining
AnswerE

Adapts model to new data patterns.

Why this answer

Data distribution change triggers model retraining using updated data samples.

209
MCQhard

When implementing drift detection in AWS SageMaker Model Monitor, which metric should be monitored to detect feature attribution shift for structured data?

A.Cosine Similarity
B.Mean Square Error
C.Feature Attribution Drift (SHAP)
D.Precision-Recall AUC
AnswerC

This directly measures shift in SHAP values.

Why this answer

Feature attribution drift monitors how the model's reliance on specific input features changes over time.

Page 2

Page 3 of 3

All pages