AIF-C01 Guidelines for Responsible AI Practice Test 4 — 15 Questions

Question 1

A company develops a chatbot using Amazon Lex. To ensure transparency, what should the chatbot do when it cannot answer a question?

Accepted Answer

Clearly state that it cannot answer and offer alternatives. Responsible AI guidelines, including those from AWS for Amazon Lex, require that when a chatbot cannot answer a question, it should clearly state its inability and offer alternatives (e.g., rephrasing the query or providing related topics). This maintains transparency and user trust, aligning with the 'Explainability' principle under the Guidelines for Responsible AI.

Answer

Remain silent and wait for the next input

Answer

Provide a random answer from a predefined list

Answer

Automatically escalate all unanswered questions to a human

Question 2

Which THREE considerations are essential for ensuring responsible AI in a model that predicts employee performance? (Choose 3)

Accepted Answer

Incorporate human review before final decisions. Responsible AI frameworks, such as those outlined by AWS, mandate human-in-the-loop (HITL) oversight for high-stakes decisions like employee performance predictions. This ensures that automated outputs are reviewed for context, fairness, and potential errors before affecting employment outcomes, aligning with the AIF-C01 domain's emphasis on human accountability.

Answer

Minimize the number of features to reduce cost

Answer

Publish the model's predictions publicly for transparency

Question 3

Which TWO practices help ensure transparency in AI systems? (Choose 2)

Accepted Answer

Use model-agnostic explainability tools like SHAP. Model-agnostic explainability tools like SHAP (SHapley Additive exPlanations) provide post-hoc explanations for any machine learning model by computing feature contributions based on cooperative game theory. This allows stakeholders to understand how each input feature influences a prediction, directly supporting transparency without requiring access to the model's internal structure.

Answer

Combine multiple models to obscure decision logic

Answer

Remove all features except the most predictive ones

Answer

Use black-box models to protect proprietary algorithms

Question 4

An AI team uses the IAM policy shown in the exhibit to control endpoint creation. Why does this policy support responsible AI?

Accepted Answer

It ensures data capture is enabled for model monitoring. The IAM policy includes a condition that enforces the `DataCaptureConfig.EnableCapture` parameter to be set to `true` when creating a SageMaker endpoint. This ensures that model monitoring data is automatically collected, which is a key practice for responsible AI as it allows continuous monitoring of model performance, bias detection, and drift analysis. Without data capture, teams cannot audit or validate model behavior in production, undermining accountability and transparency.

Answer

It requires human approval before deploying any model

Answer

It prevents the use of GPU instances to reduce cost

Answer

It restricts endpoints to only use models built in SageMaker

Question 5

A data scientist runs the SageMaker Clarify job shown in the exhibit for a credit risk model. After reviewing the results, they find a high bias metric for the gender facet. Which action is most consistent with responsible AI?

Accepted Answer

Investigate the root cause and retrain with balanced data. Responsible AI requires understanding and mitigating bias at its source, not just masking it. Investigating the root cause (e.g., data collection bias, labeling bias, or proxy features) and retraining with balanced data directly addresses the high bias metric detected by SageMaker Clarify, aligning with AWS's principle of fairness. Simply removing the gender attribute may not eliminate bias if other features act as proxies, and increasing the threshold does not fix the underlying model bias.

Answer

Proceed with deployment because the model is already in production

Answer

Remove the gender attribute from the training data and retrain

Answer

Increase the acceptance threshold for the model

Question 6

A financial services company is deploying a generative AI chatbot to assist customers with account inquiries. The company wants to ensure the chatbot does not generate biased or harmful responses. Which combination of AWS services and practices should the company implement to monitor and mitigate these risks?

Accepted Answer

Use Amazon SageMaker Clarify to detect bias in model outputs and implement a human-in-the-loop workflow with Amazon A2I to review flagged responses.. Amazon SageMaker Clarify is specifically designed to detect bias in machine learning models and their outputs, while Amazon Augmented AI (A2I) enables a human-in-the-loop workflow to review flagged responses. This combination directly addresses the requirement to monitor and mitigate biased or harmful responses from a generative AI chatbot, ensuring responsible AI practices.

Answer

Configure the chatbot to use a pre-trained model from SageMaker JumpStart and disable logging to avoid storing sensitive customer data.

Answer

Use Amazon Rekognition to analyze chat logs for biased language and automatically block responses with a confidence score above 90%.

Answer

Deploy Amazon Lex with built-in sentiment analysis to detect negative customer emotions and automatically escalate to a human agent.

Question 7

A healthcare organization is developing a clinical decision support system using Amazon Bedrock with a large language model (LLM) to analyze patient symptoms and suggest potential diagnoses. The system must comply with HIPAA and internal responsible AI guidelines. During testing, the model occasionally generates diagnoses that are inconsistent with established medical guidelines and shows a tendency to recommend more aggressive treatments for patients from certain demographic groups. The team has already implemented data encryption, access controls, and basic content filtering. They need to further reduce biased and unsafe outputs without delaying the deployment timeline. What should the team do next?

Accepted Answer

Fine-tune the model using a curated dataset of anonymized patient records that is balanced across demographic groups and aligned with clinical guidelines.. Fine-tuning the model with a balanced, curated dataset directly addresses both the bias and clinical accuracy issues at the model level, which is the most effective approach for reducing biased and unsafe outputs without delaying deployment. This method adjusts the model's internal weights to align with established medical guidelines and demographic fairness, rather than relying on post-processing filters or logging that do not fix the root cause. Since the team has already implemented basic content filtering, fine-tuning provides a targeted, efficient solution that can be completed within a reasonable timeline.

Answer

Increase the logging of all model inputs and outputs to Amazon CloudWatch and set up alarms for any mentions of protected attributes.

Answer

Replace the current LLM with a different pre-trained model that has been benchmarked for lower bias on medical datasets.

Answer

Apply stronger content filtering rules using Amazon Comprehend Medical to block any diagnosis that contains demographic-related terms.

Question 8

A data scientist wants to detect potential bias in a binary classification model before deployment. Which AWS service can analyze the model's predictions across different demographic groups?

Accepted Answer

Amazon SageMaker Clarify. Amazon SageMaker Clarify is the correct service because it is specifically designed to detect bias in machine learning models by analyzing predictions across demographic groups. It provides pre-training and post-training bias metrics, such as disparate impact and difference in positive proportions, enabling data scientists to evaluate fairness before deployment.

Answer

Amazon SageMaker Ground Truth

Answer

Amazon CloudWatch Logs Insights

Answer

Amazon SageMaker Model Monitor

Question 9

A team is deploying a regression model for loan approval. To ensure transparency for regulators, they need to explain individual predictions. Which interpretability method can provide local explanations by approximating the model with a simpler surrogate?

Accepted Answer

LIME. LIME (Local Interpretable Model-agnostic Explanations) is the correct choice because it generates local explanations by fitting a simpler, interpretable surrogate model (e.g., linear regression or decision tree) around a single prediction. This allows the team to explain why a specific loan application was approved or rejected, meeting regulatory transparency requirements without needing access to the original model's internals.

Answer

SHAP values

Answer

Partial dependence plots

Answer

Permutation feature importance

Question 10

After deploying a model, a company notices that the distribution of the input features has shifted compared to the training data. Which feature of Amazon SageMaker Model Monitor can alert them to this change?

Accepted Answer

Data quality monitoring. Amazon SageMaker Model Monitor's data quality monitoring feature is specifically designed to detect changes in the distribution of input features compared to the training data. It uses statistical tests (e.g., Kolmogorov-Smirnov, Chi-squared) to compare baseline and live data distributions, alerting when drift is detected. This directly addresses the scenario of input feature distribution shift.

Answer

Model quality monitoring

Answer

Bias drift monitoring

Answer

Feature importance drift

Question 11

A company uses Amazon SageMaker Ground Truth to label a dataset for a binary classifier. To reduce labeling bias, which workforce configuration is most appropriate?

Accepted Answer

Private workforce of domain experts. A private workforce of domain experts ensures that labeling is performed by individuals with deep knowledge of the data domain, which directly reduces labeling bias. Domain experts are less likely to misinterpret ambiguous data points and can apply consistent, informed judgment, thereby minimizing systematic errors that could skew the binary classifier's training data.

Answer

Automatic labeling with Active Learning

Answer

Public workforce with no qualification

Answer

Vendor managed workforce

Question 12

A machine learning team is building a credit risk model and discovers that the training data has a significant imbalance in loan approval rates between two demographic groups. They decide to reweight the training samples using a preprocessing technique. Which SageMaker Clarify feature can help compute the appropriate sample weights to achieve demographic parity?

Accepted Answer

Clarify preprocessing (reweighting). SageMaker Clarify's preprocessing (reweighting) feature directly computes sample weights to adjust for imbalances in training data, enabling demographic parity by assigning higher weights to underrepresented groups. This is a pre-training bias mitigation technique that modifies the dataset before model training, aligning with the team's goal of reweighting samples to address loan approval rate disparities.

Answer

Clarify post-training bias metrics

Answer

Model Monitor bias drift

Answer

Clarify explainability (SHAP)

Question 13

A security team is concerned about adversarial attacks on their image classification model deployed on Amazon SageMaker. They want to test robustness against carefully crafted inputs that cause misclassification. What approach should they use?

Accepted Answer

Generating adversarial examples using SageMaker Clarify. SageMaker Clarify includes built-in capabilities for generating adversarial examples to evaluate model robustness against inputs designed to cause misclassification. This directly addresses the security team's concern by simulating adversarial attacks, allowing them to measure and improve the model's resilience before deployment.

Answer

Data augmentation on the training set

Answer

A/B testing between two similar models

Answer

SageMaker Model Monitor with adversarial drift

Question 14

Which THREE considerations are important when implementing responsible AI for a production NLP system? (Choose three.)

Accepted Answer

Continuously monitor model outputs for bias and drift. Continuous monitoring of model outputs for bias and drift is essential for maintaining responsible AI in production NLP systems. As language patterns and data distributions evolve over time, a model that was initially fair and accurate can develop harmful biases or performance degradation, so automated monitoring ensures ongoing alignment with ethical standards and regulatory requirements.

Answer

Obtain FDA approval for the model

Answer

Apply encryption at rest for all training code

Question 15

Refer to the exhibit. A data scientist runs an Amazon SageMaker Clarify bias analysis on a binary classifier. The pre-training ClassImbalance is 1.5 and the post-training DPPL is 0.15. What should the data scientist conclude?

Accepted Answer

The data has a mild class imbalance, but the model shows a noticeable bias in predictions.. The pre-training ClassImbalance metric of 1.5 indicates a mild class imbalance (values close to 1.0 indicate balance, while values significantly above 1.0 indicate imbalance). The post-training DPPL (Difference in Positive Proportions in Labels) metric of 0.15 exceeds the commonly accepted fairness threshold of 0.10, indicating a noticeable bias in the model's predictions. Therefore, the data has a mild imbalance, but the model exhibits a bias that warrants further investigation.

Answer

The data is highly imbalanced and the model is unbiased.

Answer

The pre-training metric indicates a fairness issue, but the post-training metric is acceptable.

Answer

The data is perfectly balanced and the model is fair.