AIF-C01 Guidelines for Responsible AI Practice Test 3 — 15 Questions

Question 1

A data scientist wants to detect potential bias in a binary classification model before deployment. Which AWS service can analyze the model's predictions across different demographic groups?

Accepted Answer

Amazon SageMaker Clarify. Amazon SageMaker Clarify is the correct service because it is specifically designed to detect bias in machine learning models by analyzing predictions across demographic groups. It provides pre-training and post-training bias metrics, such as disparate impact and difference in positive proportions, enabling data scientists to evaluate fairness before deployment.

Answer

Amazon SageMaker Ground Truth

Answer

Amazon CloudWatch Logs Insights

Answer

Amazon SageMaker Model Monitor

Question 2

A team is deploying a regression model for loan approval. To ensure transparency for regulators, they need to explain individual predictions. Which interpretability method can provide local explanations by approximating the model with a simpler surrogate?

Accepted Answer

LIME. LIME (Local Interpretable Model-agnostic Explanations) is the correct choice because it generates local explanations by fitting a simpler, interpretable surrogate model (e.g., linear regression or decision tree) around a single prediction. This allows the team to explain why a specific loan application was approved or rejected, meeting regulatory transparency requirements without needing access to the original model's internals.

Answer

SHAP values

Answer

Partial dependence plots

Answer

Permutation feature importance

Question 3

A healthcare company must train a model on sensitive patient data while complying with privacy regulations. They want to add noise to the training process to prevent re-identification. Which technique should they implement?

Accepted Answer

Differential privacy. Differential privacy is the correct technique because it adds calibrated noise to the training process (e.g., via gradient clipping and noise injection in stochastic gradient descent) to ensure that the model's outputs do not reveal whether any individual's data was included in the training set. This provides a formal mathematical guarantee (ε-differential privacy) that limits the risk of re-identification, which is essential for complying with privacy regulations like HIPAA or GDPR when training on sensitive patient data.

Answer

k-anonymity

Answer

Federated learning

Answer

Homomorphic encryption

Question 4

After deploying a model, a company notices that the distribution of the input features has shifted compared to the training data. Which feature of Amazon SageMaker Model Monitor can alert them to this change?

Accepted Answer

Data quality monitoring. Amazon SageMaker Model Monitor's data quality monitoring feature is specifically designed to detect changes in the distribution of input features compared to the training data. It uses statistical tests (e.g., Kolmogorov-Smirnov, Chi-squared) to compare baseline and live data distributions, alerting when drift is detected. This directly addresses the scenario of input feature distribution shift.

Answer

Model quality monitoring

Answer

Bias drift monitoring

Answer

Feature importance drift

Question 5

A company uses Amazon SageMaker Ground Truth to label a dataset for a binary classifier. To reduce labeling bias, which workforce configuration is most appropriate?

Accepted Answer

Private workforce of domain experts. A private workforce of domain experts ensures that labeling is performed by individuals with deep knowledge of the data domain, which directly reduces labeling bias. Domain experts are less likely to misinterpret ambiguous data points and can apply consistent, informed judgment, thereby minimizing systematic errors that could skew the binary classifier's training data.

Answer

Automatic labeling with Active Learning

Answer

Public workforce with no qualification

Answer

Vendor managed workforce

Question 6

A machine learning team is building a credit risk model and discovers that the training data has a significant imbalance in loan approval rates between two demographic groups. They decide to reweight the training samples using a preprocessing technique. Which SageMaker Clarify feature can help compute the appropriate sample weights to achieve demographic parity?

Accepted Answer

Clarify preprocessing (reweighting). SageMaker Clarify's preprocessing (reweighting) feature directly computes sample weights to adjust for imbalances in training data, enabling demographic parity by assigning higher weights to underrepresented groups. This is a pre-training bias mitigation technique that modifies the dataset before model training, aligning with the team's goal of reweighting samples to address loan approval rate disparities.

Answer

Clarify post-training bias metrics

Answer

Model Monitor bias drift

Answer

Clarify explainability (SHAP)

Question 7

Which of the following is a key principle of responsible AI according to AWS?

Accepted Answer

Transparency. Transparency is a core principle of responsible AI according to AWS, meaning that customers should be able to understand how an AI system makes decisions, including its inputs, outputs, and limitations. AWS emphasizes this through features like model cards in Amazon SageMaker, which document model performance, intended uses, and biases. This principle ensures that AI systems are explainable and auditable, building trust with users.

Answer

Complexity

Answer

Speed

Answer

Profitability

Question 8

Which TWO actions should a data scientist take to evaluate fairness of a binary classification model using Amazon SageMaker Clarify? (Choose two.)

Accepted Answer

Use post-training bias metrics like Difference in Positive Proportions. Option A is correct because SageMaker Clarify's post-training bias metrics, such as Difference in Positive Proportions (DPP), compare predicted outcomes across facets (e.g., gender or age groups) to quantify disparate impact in the model's actual predictions, which is essential for evaluating fairness of a binary classifier. Option D is correct because pre-training bias metrics like Class Imbalance (CI) measure skew in the underlying training data's label distribution across facets before any model is trained, helping detect whether the data itself could lead to biased outcomes. Together, these two metric types cover both data-level and model-level fairness assessment as Clarify is designed to do. Option B is not a Clarify fairness evaluation action but a data preprocessing technique, and balancing data does not by itself measure bias. Option C, SHAP values, explains feature importance/attribution for interpretability, not fairness metrics. Option E, data quality monitoring on unlabeled data, addresses data drift/quality issues and does not evaluate model fairness.

Answer

Ensure the training dataset is balanced by resampling

Answer

Generate SHAP values for feature importance

Answer

Run a data quality monitoring job on unlabeled data

Question 9

Which THREE considerations are important when implementing responsible AI for a production NLP system? (Choose three.)

Accepted Answer

Continuously monitor model outputs for bias and drift. Option B is correct because responsible AI in production requires ongoing monitoring of model outputs to detect bias and data/concept drift, since model behavior can degrade or become unfair as real-world data changes over time. Option D is correct because model cards are a standard transparency artifact that document a model's intended use, performance metrics across relevant subgroups, and known limitations, enabling informed and accountable use by stakeholders. Option E is correct because embedding bias detection as an automated gate in the CI/CD pipeline ensures every model update is evaluated for fairness regressions before deployment, making responsible AI a repeatable part of the ML lifecycle rather than an afterthought. Option A does not belong because FDA approval applies to regulated medical devices and clinical AI, not to general production NLP systems. Option C does not belong because encrypting training code at rest is a generic security control and is not a responsible-AI consideration such as fairness, transparency, or accountability.

Answer

Obtain FDA approval for the model

Answer

Apply encryption at rest for all training code

Question 10

Which TWO techniques provide interpretability for machine learning models at a local (per-prediction) level? (Choose two.)

Accepted Answer

SHAP values. SHAP (SHapley Additive exPlanations) values (option A) are correct because they decompose an individual prediction into additive per-feature contributions based on Shapley values from cooperative game theory, giving a local, per-instance explanation of how each feature pushed the model output. LIME (Local Interpretable Model-agnostic Explanations) (option D) is also correct because it fits a simple interpretable surrogate model (e.g., sparse linear model) around a single prediction by perturbing the instance's neighborhood, thereby explaining that specific prediction locally. Partial dependence plots (option B) are not local — they show the average marginal effect of a feature across the entire dataset, a global technique. A confusion matrix (option C) is a global performance summary of classification outcomes (TP/FP/TN/FN counts), not a per-prediction explanation. Permutation feature importance (option E) is a global method that measures the drop in overall model performance when a feature's values are shuffled across the dataset, so it does not explain individual predictions.

Answer

Partial dependence plots

Answer

Confusion matrix

Answer

Permutation feature importance

Question 11

Refer to the exhibit. A data scientist runs an Amazon SageMaker Clarify bias analysis on a binary classifier. The pre-training ClassImbalance is 1.5 and the post-training DPPL is 0.15. What should the data scientist conclude?

Accepted Answer

The data has a mild class imbalance, but the model shows a noticeable bias in predictions.. The pre-training ClassImbalance metric of 1.5 indicates a mild class imbalance (values close to 1.0 indicate balance, while values significantly above 1.0 indicate imbalance). The post-training DPPL (Difference in Positive Proportions in Labels) metric of 0.15 exceeds the commonly accepted fairness threshold of 0.10, indicating a noticeable bias in the model's predictions. Therefore, the data has a mild imbalance, but the model exhibits a bias that warrants further investigation.

Answer

The data is highly imbalanced and the model is unbiased.

Answer

The pre-training metric indicates a fairness issue, but the post-training metric is acceptable.

Answer

The data is perfectly balanced and the model is fair.

Question 12

Refer to the exhibit. A developer is reviewing CloudWatch Logs for a deployed model and notices the same input appears multiple times with slightly different probabilities. What responsible AI concern does this pattern suggest?

Accepted Answer

The model is not robust; it produces inconsistent predictions for the same input.. The pattern of the same input producing slightly different probabilities indicates that the model's predictions are not deterministic for identical inputs. This violates the principle of robustness in responsible AI, which requires that a model should produce consistent outputs for the same input under the same conditions. Inconsistent predictions for identical inputs undermine trust and reliability, making the model non-robust.

Answer

The model is overfitting to the training data.

Answer

The model is exhibiting bias against a demographic group.

Answer

The input data is drifting from the training distribution.

Question 13

A company uses Amazon Rekognition for facial analysis. They want to ensure the model doesn't exhibit bias based on skin tone. What should they do?

Accepted Answer

Ensure the training dataset includes diverse skin tones. Bias in facial analysis models, such as those used by Amazon Rekognition, often stems from imbalanced training data. By ensuring the training dataset includes diverse skin tones, the model learns to recognize features across all demographic groups, reducing performance disparities and promoting fairness. This directly addresses the root cause of bias in machine learning models.

Answer

Apply data augmentation to increase dataset size

Answer

Use a larger neural network

Answer

Use a pre-trained model from AWS Marketplace

Question 14

A healthcare startup uses Amazon SageMaker to train a model predicting patient readmission. They need to ensure the model's predictions do not discriminate based on protected attributes like age or race. Which SageMaker feature allows them to monitor and mitigate bias during training?

Accepted Answer

SageMaker Clarify. SageMaker Clarify is the correct choice because it is specifically designed to detect and mitigate bias in machine learning models. It provides built-in capabilities to analyze training data and model predictions for bias against protected attributes such as age or race, and can generate bias reports and suggest mitigation strategies during training.

Answer

SageMaker Model Monitor

Answer

SageMaker Autopilot

Answer

SageMaker Debugger

Question 15

A media company uses Amazon Transcribe for automatic speech recognition. They discover the model has higher error rates for non-native English speakers. Which Responsible AI principle are they failing to uphold?

Accepted Answer

Fairness. The model's higher error rates for non-native English speakers indicate a bias in the training data or model design that leads to disparate performance across demographic groups. This directly violates the Fairness principle of Responsible AI, which requires that AI systems treat all groups equitably and do not amplify existing societal biases. Amazon Transcribe's underlying acoustic and language models may have been trained predominantly on native English speech, causing systematic underperformance for non-native accents.

Answer

Explainability

Answer

Robustness

Answer

Privacy