AI Auditing Tools And Techniques

An auditor is evaluating bias in a model deployed on AWS SageMaker. Which tool should be used to generate a report on pre-training and post-training bias metrics?

When auditing an Azure Machine Learning pipeline, which functionality within 'Responsible AI dashboard' is required to perform counterfactual analysis on model predictions?

You are auditing a model using IBM Watson OpenScale. Which feature should be configured to detect 'Disparate Impact' to ensure regulatory compliance?

To verify the provenance of a model in Google Cloud Vertex AI, which service should the auditor inspect to review the lineage graph of the artifacts?

An auditor is evaluating an AI system's robustness against adversarial attacks. Which technique involves perturbing input data to see if the model output changes significantly?

Which technique is best for verifying that a model has not been subject to 'training data poisoning'?

When auditing an MLflow experiment, which object allows the auditor to verify the exact parameters and code version used during training?

When scoping an audit for a model using SHAP values, what is the primary objective of the auditor?

When reviewing an AI system log, what is the 'inference request' ID used for?

When auditing model deployment, what is the primary purpose of 'Shadow Mode' testing?

You are auditing a deployment on Kubernetes using Kubeflow. Which component should the auditor examine to ensure that the pipeline steps are reproducible?

Which document is essential for an AI audit to identify the 'intended use' of an AI system?

During an audit of an LLM-based application, which technique is most effective for testing the robustness against 'Prompt Injection'?

When auditing fairness using the 'Aequitas' toolkit, what is the first step an auditor should take?

Which aspect of 'Data Drift' does the 'Population Stability Index' (PSI) measure?

An auditor is using 'CleverHans' for model auditing. What kind of vulnerability is this library designed to detect?

Which metric is commonly used to audit classification models?

Which of the following is a common 'drift' symptom an auditor should look for in production models?

An auditor is evaluating an AI system for 'Model Inversion' risk. What is this?

What is the primary role of a 'Human-in-the-Loop' (HITL) audit requirement?

In an audit of differential privacy implementations, what is the 'epsilon' parameter used for?

When auditing data pipelines, what is the function of a 'Data Quality' assertion?

When auditing model versioning, what is the recommended practice for maintaining evidence?

Which tool provides visual confirmation of data lineage in an end-to-end AI project?

What is the purpose of 'Model Validation' in the audit lifecycle?

What is a 'Model Repository' in an AI audit context?

When auditing model explainability, why is it risky to rely solely on 'Global Feature Importance'?

Which technique is recommended for auditing 'data leakage' in a feature engineering pipeline?

When using 'LIME' (Local Interpretable Model-agnostic Explanations) for auditing, what is the auditor looking for?

When auditing the 'training dataset' for a classification model, what should the auditor confirm regarding class balance?

Which of these is a typical 'audit finding' in an AI governance review?

What is the primary benefit of using 'Containerization' for AI model auditability?

When auditing fairness metrics, what does the 'Demographic Parity' metric measure?

An auditor is evaluating the 'Safety Filter' of an LLM. Which approach is most suitable for detecting 'jailbreak' vulnerabilities?

During an AI audit, what is a 'PII Scrubber' tool used for?

When conducting an audit, what is the 'Model Registry' entry for a model version supposed to contain?

When auditing an AI system's 'Explainability', which issue is highlighted by 'Explanation Faithfulness'?

When scoping an AI audit engagement, which TWO of the following documents should the auditor request to understand the AI model's governance structure?

Which THREE factors should an auditor consider when evaluating the suitability of an AI model for production?

When documenting audit findings for an AI system, which TWO of the following are critical to include to ensure the audit can be replicated?

Which TWO techniques should an auditor employ to detect bias in a model where the training data has imbalanced demographic representation?

When conducting an audit, which THREE of the following represent potential 'Model Risk' areas that require documentation?

Which TWO methods are commonly used by auditors to verify the 'Provenance' of training data?

Which TWO metrics provide the best insights into 'Model Quality' during an audit?

An auditor is evaluating the data pipeline security. Which THREE controls should be verified?

When auditing an AI pipeline, which THREE items should be part of the 'Evidence Collection' plan?

When scoping an audit, which TWO stakeholders should the auditor interview?

When documenting findings, which THREE elements should be included for each finding?

When selecting testing techniques for an AI model, which THREE are considered 'Model-Agnostic'?

An auditor is evaluating the 'Model Monitoring' dashboard. Which TWO items should the dashboard display to alert the team of potential issues?

Which TWO of the following are examples of 'Data Leakage' that an auditor should look for in a pipeline?

Which THREE actions are appropriate when you find a significant 'Bias' finding in an AI audit?

Which THREE technical artifacts should the auditor collect to verify the 'Model Training Process'?

Which TWO monitoring tools are typically used to detect 'Data Drift' in production pipelines?

An audit team must verify that a credit-scoring model's predictions can be explained to a regulator on a case-by-case basis. The data science team used a gradient-boosted tree ensemble. Which technique should the auditor recommend to produce locally faithful explanations of individual decisions?

An AI audit team is validating a fraud-detection model deployed through a REST API. The model card claims 94% recall, but the auditor wants independent evidence that the live endpoint behaves as documented. The team has read access to the model registry and the API logs, but cannot access the training pipeline. Which technique should the auditor use to validate the deployed model's behavior?

An auditor needs to confirm that a deployed model in production is bit-for-bit identical to the artifact that passed validation. Which artifact should the auditor use to verify integrity?

An auditor is reviewing an AI-based loan underwriting system and wants to verify that the model's predictions are stable when small, non-semantic changes are made to input features. The auditor plans to submit the same applicant profile with trivial formatting variations. Which auditing technique does this describe?

An auditor is examining a computer vision model used for medical image triage. The vendor provides only a compiled binary and no architecture documentation. The auditor wants to estimate how much the model's output changes when individual pixels are altered slightly, in order to assess decision boundary sharpness. Which technique is most appropriate?

An auditor is testing whether a recommendation model leaks information about individual users in its training set. The team exposes only top-N ranked outputs through the API. Which technique is most appropriate to estimate membership inference risk under this constraint?

An auditor is assessing an AI recruitment screening tool that ranks candidates. The auditor needs to gather evidence that the model's decisions can be explained to rejected applicants and to a regulator. Which TWO techniques should the auditor apply to produce local, case-level explanations for individual candidate rankings? (Choose two.)

During an audit of a natural language processing system, the auditor needs to determine whether a fine-tuned model reproduces verbatim passages from its training corpus. Which technique should the auditor apply to test for training-data extraction?

An auditor is evaluating a generative AI customer service assistant that produces free-text responses. The auditor wants to systematically test whether the assistant can be induced to reveal its system prompt or produce prohibited content. Which technique should the auditor use?