AI Auditing Tools And Techniques

An auditor is evaluating the data pipeline security. Which THREE controls should be verified?

An audit team must verify that a credit-scoring model's predictions can be explained to a regulator on a case-by-case basis. The data science team used a gradient-boosted tree ensemble. Which technique should the auditor recommend to produce locally faithful explanations of individual decisions?

When auditing an Azure Machine Learning pipeline, which functionality within 'Responsible AI dashboard' is required to perform counterfactual analysis on model predictions?

What is a 'Model Repository' in an AI audit context?

In an audit of differential privacy implementations, what is the 'epsilon' parameter used for?

When reviewing an AI system log, what is the 'inference request' ID used for?

An auditor needs to confirm that a deployed model in production is bit-for-bit identical to the artifact that passed validation. Which artifact should the auditor use to verify integrity?

When using 'LIME' (Local Interpretable Model-agnostic Explanations) for auditing, what is the auditor looking for?

Which TWO monitoring tools are typically used to detect 'Data Drift' in production pipelines?

Which of these is a typical 'audit finding' in an AI governance review?

Which THREE actions are appropriate when you find a significant 'Bias' finding in an AI audit?

When conducting an audit, which THREE of the following represent potential 'Model Risk' areas that require documentation?

Which of the following is a common 'drift' symptom an auditor should look for in production models?

When auditing model explainability, why is it risky to rely solely on 'Global Feature Importance'?

Which TWO methods are commonly used by auditors to verify the 'Provenance' of training data?

When auditing an AI pipeline, which THREE items should be part of the 'Evidence Collection' plan?

To verify the provenance of a model in Google Cloud Vertex AI, which service should the auditor inspect to review the lineage graph of the artifacts?

An auditor is evaluating bias in a model deployed on AWS SageMaker. Which tool should be used to generate a report on pre-training and post-training bias metrics?

Which document is essential for an AI audit to identify the 'intended use' of an AI system?

An auditor is reviewing an AI-based loan underwriting system and wants to verify that the model's predictions are stable when small, non-semantic changes are made to input features. The auditor plans to submit the same applicant profile with trivial formatting variations. Which auditing technique does this describe?

Which aspect of 'Data Drift' does the 'Population Stability Index' (PSI) measure?

Which technique is recommended for auditing 'data leakage' in a feature engineering pipeline?

Which THREE factors should an auditor consider when evaluating the suitability of an AI model for production?

An auditor is evaluating the 'Model Monitoring' dashboard. Which TWO items should the dashboard display to alert the team of potential issues?

When documenting audit findings for an AI system, which TWO of the following are critical to include to ensure the audit can be replicated?

An auditor is examining a computer vision model used for medical image triage. The vendor provides only a compiled binary and no architecture documentation. The auditor wants to estimate how much the model's output changes when individual pixels are altered slightly, in order to assess decision boundary sharpness. Which technique is most appropriate?

When auditing the 'training dataset' for a classification model, what should the auditor confirm regarding class balance?

Which technique is best for verifying that a model has not been subject to 'training data poisoning'?

When conducting an audit, what is the 'Model Registry' entry for a model version supposed to contain?

Which TWO metrics provide the best insights into 'Model Quality' during an audit?

When auditing model deployment, what is the primary purpose of 'Shadow Mode' testing?

An auditor is using 'CleverHans' for model auditing. What kind of vulnerability is this library designed to detect?

When scoping an audit for a model using SHAP values, what is the primary objective of the auditor?

During an AI audit, what is a 'PII Scrubber' tool used for?

During an audit of an LLM-based application, which technique is most effective for testing the robustness against 'Prompt Injection'?

An auditor is evaluating the 'Safety Filter' of an LLM. Which approach is most suitable for detecting 'jailbreak' vulnerabilities?

When auditing an AI system's 'Explainability', which issue is highlighted by 'Explanation Faithfulness'?

When documenting findings, which THREE elements should be included for each finding?

When scoping an AI audit engagement, which TWO of the following documents should the auditor request to understand the AI model's governance structure?

You are auditing a deployment on Kubernetes using Kubeflow. Which component should the auditor examine to ensure that the pipeline steps are reproducible?

An auditor is assessing an AI recruitment screening tool that ranks candidates. The auditor needs to gather evidence that the model's decisions can be explained to rejected applicants and to a regulator. Which TWO techniques should the auditor apply to produce local, case-level explanations for individual candidate rankings? (Choose two.)

An auditor is evaluating an AI system's robustness against adversarial attacks. Which technique involves perturbing input data to see if the model output changes significantly?

What is the primary benefit of using 'Containerization' for AI model auditability?

Which metric is commonly used to audit classification models?

Which tool provides visual confirmation of data lineage in an end-to-end AI project?

What is the primary role of a 'Human-in-the-Loop' (HITL) audit requirement?

When auditing model versioning, what is the recommended practice for maintaining evidence?

You are auditing a model using IBM Watson OpenScale. Which feature should be configured to detect 'Disparate Impact' to ensure regulatory compliance?

When auditing an MLflow experiment, which object allows the auditor to verify the exact parameters and code version used during training?

An auditor is testing whether a recommendation model leaks information about individual users in its training set. The team exposes only top-N ranked outputs through the API. Which technique is most appropriate to estimate membership inference risk under this constraint?

An AI audit team is validating a fraud-detection model deployed through a REST API. The model card claims 94% recall, but the auditor wants independent evidence that the live endpoint behaves as documented. The team has read access to the model registry and the API logs, but cannot access the training pipeline. Which technique should the auditor use to validate the deployed model's behavior?

When auditing fairness using the 'Aequitas' toolkit, what is the first step an auditor should take?

During an audit of a natural language processing system, the auditor needs to determine whether a fine-tuned model reproduces verbatim passages from its training corpus. Which technique should the auditor apply to test for training-data extraction?

An auditor is evaluating a generative AI customer service assistant that produces free-text responses. The auditor wants to systematically test whether the assistant can be induced to reveal its system prompt or produce prohibited content. Which technique should the auditor use?

When selecting testing techniques for an AI model, which THREE are considered 'Model-Agnostic'?

Which TWO techniques should an auditor employ to detect bias in a model where the training data has imbalanced demographic representation?

When auditing data pipelines, what is the function of a 'Data Quality' assertion?

What is the purpose of 'Model Validation' in the audit lifecycle?

Which THREE technical artifacts should the auditor collect to verify the 'Model Training Process'?

When scoping an audit, which TWO stakeholders should the auditor interview?

An auditor is evaluating an AI system for 'Model Inversion' risk. What is this?

When auditing fairness metrics, what does the 'Demographic Parity' metric measure?

Which TWO of the following are examples of 'Data Leakage' that an auditor should look for in a pipeline?