ISACA · Free Practice Questions · Last reviewed May 2026
18 real exam-style questions organised by domain, each with the correct answer highlighted and a plain-English explanation of why it's right — and why the others are wrong.
What is the primary purpose of a 'Champion-Challenger' deployment pattern in MLOps?
To manage database indexes
To compress model weights
To safely test a new model vs production
Champion-Challenger routes a portion of live traffic to the challenger model while the champion continues serving most requests, comparing performance on real data. This satisfies the stem's safety constraint: the new model is validated against production traffic without exposing all users to unproven behaviour, enabling evidence-based promotion or rollback.
To increase training speed
For a production LLM deployment, which operational metric is most critical to monitor to identify 'hallucination' or grounding failure in real-time?
Throughput (TPS)
Context faithfulness score
A context faithfulness score measures whether generated output is grounded in the retrieved source documents, directly detecting hallucination in real time. Latency, token throughput, or cost metrics cannot reveal grounding failure, so this metric uniquely satisfies the stem's requirement to identify hallucination during production serving.
Token generation latency
Error code distribution
Memory footprint
When implementing drift detection in AWS SageMaker Model Monitor, which metric should be monitored to detect feature attribution shift for structured data?
Cosine Similarity
Mean Square Error
Feature Attribution Drift (SHAP)
Feature attribution drift applies SHAP values to compare each feature's contribution between baseline and current data, revealing changes in how features influence predictions. Monitoring it satisfies the requirement to detect feature attribution shift for structured datasets in SageMaker Model Monitor.
Precision-Recall AUC
Which AWS SageMaker feature allows auditors to verify that a model was trained on encrypted data?
Instance type selection
Model registry labels
Notebook instance tags
KMS Key Usage in CloudTrail
CloudTrail records every KMS API call, including which key encrypted the training data and who used it. Auditors can therefore trace key usage events to verify that SageMaker training jobs consumed encrypted data, providing the evidential trail the scenario demands.
In Azure Machine Learning, which component should an auditor verify to ensure model registry access is restricted to authorized CI/CD service principals?
Workspace Access Control (IAM)
Workspace Access Control (IAM) holds the role assignments governing who can read or write to the model registry. Auditors verify these assignments to confirm only authorised CI/CD service principals hold registry permissions, satisfying the restriction requirement.
Environment YAML files
Compute Instance logs
Data labeling projects
What is the primary function of an 'AI Pipeline Orchestrator' like Kubeflow or Airflow?
Automating task dependencies
Orchestrators such as Kubeflow and Airflow model pipelines as directed acyclic graphs, scheduling each step to run only after its upstream dependencies succeed, plus retries and monitoring, rather than performing model training or inference themselves.
Encrypting network traffic
Writing code for the model
Optimizing SQL queries
Want more AI Operations practice?
Practice this domainAn auditor is evaluating bias in a model deployed on AWS SageMaker. Which tool should be used to generate a report on pre-training and post-training bias metrics?
SageMaker Model Monitor
SageMaker Clarify
SageMaker Clarify generates pre-training and post-training bias metrics, directly satisfying the auditor's requirement for both measurement stages. It computes metrics such as class imbalance and disparate impact before training, then compares predicted labels against ground truth afterwards, producing a consolidated bias report within SageMaker.
AWS Glue DataBrew
Amazon Inspector
When auditing an Azure Machine Learning pipeline, which functionality within 'Responsible AI dashboard' is required to perform counterfactual analysis on model predictions?
Causal Inference component
Model Overview component
Counterfactuals component
Counterfactuals generates perturbed feature values and shows the smallest change needed to flip a prediction, which is exactly the counterfactual analysis the audit requires. Other Responsible AI components (error analysis, fairness, interpretability) do not produce these what-if contrasts.
Error Analysis component
You are auditing a model using IBM Watson OpenScale. Which feature should be configured to detect 'Disparate Impact' to ensure regulatory compliance?
Fairness monitor
The Fairness monitor in IBM Watson OpenScale computes disparate impact by comparing favourable outcome rates across protected and reference groups, flagging bias when the ratio breaches the four-fifths threshold. This directly satisfies the regulatory compliance requirement to detect disparate impact, unlike drift or quality monitors, which track accuracy degradation rather than group-level fairness.
Quality monitor
Drift detection monitor
Explainability monitor
Payload logging
To verify the provenance of a model in Google Cloud Vertex AI, which service should the auditor inspect to review the lineage graph of the artifacts?
Vertex AI ML Metadata
Vertex AI ML Metadata stores the lineage graph tracking datasets, training runs, models and artefacts, letting an auditor trace a model's provenance through each pipeline step. Inspecting it reveals the exact artefacts and executions that produced the model.
Vertex AI Feature Store
Cloud Logging
Vertex AI Model Registry
An auditor is evaluating an AI system's robustness against adversarial attacks. Which technique involves perturbing input data to see if the model output changes significantly?
Model pruning
Bias assessment
Data augmentation
Adversarial testing
Adversarial testing deliberately perturbs inputs, often via gradient-based or perturbation methods, then measures whether model output changes materially. That directly satisfies the robustness evaluation described, unlike standard accuracy or drift monitoring, which never probe deliberate input manipulation.
Which technique is best for verifying that a model has not been subject to 'training data poisoning'?
Review of data provenance
Reviewing data provenance traces each training record's origin, collection method and transformation history, exposing tampering or unauthorised injection before the model learns from it. This directly satisfies the stem's requirement to verify integrity of the training set, since poisoning occurs at data ingestion rather than inference, making downstream output testing insufficient.
Latency testing
Model Accuracy testing
Hyperparameter tuning
Want more AI Auditing Tools And Techniques practice?
Practice this domainWhen auditing an AI system for regulatory compliance, you find that data lineage is broken between the feature store and the training pipeline. Which control is most likely deficient?
Access control lists for the model production environment
Encryption of data at rest in the data warehouse
Load balancing configuration for the inference API
Metadata logging and transformation documentation
Broken lineage between feature store and training pipeline means the provenance of features cannot be traced, so metadata logging and transformation documentation are deficient. Without recorded transformations and versions, reproducibility and regulatory auditability of the model fail.
A firm uses a third-party AI service. Which control is most critical to ensure compliance with the firm's own AI governance policy?
Including AI-specific compliance requirements in the vendor service-level agreement (SLA)
Contractual enforcement is the only mechanism that binds an external provider to the firm's governance requirements, since the firm cannot apply its internal controls directly to third-party infrastructure. The SLA translates policy obligations into auditable, enforceable commitments, satisfying the compliance constraint the scenario demands.
Conducting weekly penetration tests on the vendor's office premises
Requesting the vendor's employee turnover rates for the last fiscal year
Requiring the vendor to submit copies of all their marketing brochures
You are auditing a 'Model Monitoring' dashboard. Which metric is most indicative of a potential degradation in the model's reliability over time?
Statistical drift between the production input data and training baseline
Statistical drift between production inputs and the training baseline signals that the data distribution the model sees has shifted away from what it learned. This covariate shift typically precedes accuracy loss, making it the earliest reliable indicator of degrading reliability on the dashboard.
The physical location of the cloud server hosting the model
The number of lines of code in the model deployment script
The number of users accessing the model API
You are assessing a company's 'AI Governance Committee'. Which action best demonstrates effective board-level oversight?
Managing the day-to-day procurement of data labeling services
Reviewing individual code commits for AI model training
Selecting the machine learning framework used by data scientists
Approving the annual AI strategy and corresponding risk mitigation plans
Board-level oversight means governing direction and risk appetite, not operational detail. Approving the annual AI strategy alongside its risk mitigation plans exercises that authority, ensuring AI initiatives align with organisational objectives and acceptable risk tolerances.
Which component of an AI policy is most important for establishing organizational accountability?
Describing the history of artificial intelligence research
Listing the hardware specifications of the training clusters
Providing the contact information for the company's PR department
Defining the roles and responsibilities of the AI Governance Committee and model owners
Assigning explicit roles and responsibilities to the AI Governance Committee and model owners creates named accountability, ensuring every AI system has an owner answerable for its risks and decisions. This directly establishes the organisational accountability the stem asks for, unlike broader policy statements.
During an AI risk assessment, you notice that the 'Model Risk Management' (MRM) framework does not mandate independent validation for models categorized as 'High Impact'. What is the most significant consequence?
Inability to detect model drift in production environments
Non-compliance with internal cybersecurity patching cycles
Operational silos between the AI team and the IT department
Increased probability of undetected systemic model bias or failure
Without independent validation, high-impact models escape scrutiny by parties outside the development team, so bias, drift and failure modes remain undetected until production impact occurs. This directly satisfies the consequence of removing that control from the MRM framework.
Want more AI Governance And Risk practice?
Practice this domainThe vendor does not publish an exact question count for the AAIA. Always confirm current details on the vendor's official exam page.
Scenario-based questions covering exam objectives with detailed answer explanations.
The exam covers 3 domains: AI Operations, AI Auditing Tools And Techniques, AI Governance And Risk. Questions are weighted by domain — higher-weight domains appear more on your actual exam.
No. These are original exam-style practice questions written against the official ISACA AAIA exam objectives. They are not copied from the real exam. Courseiva focuses on genuine understanding, not memorisation of braindumps.
Courseiva tracks your accuracy per domain and routes you toward weak areas automatically. Free, no account required.