Guidelines for Responsible AI practice questions

Use only black-box models to protect intellectual property

Why wrong: Black-box models reduce transparency.

Increase model complexity to improve accuracy

Why wrong: Increasing complexity may reduce explainability.

Minimize the amount of training data collected

Why wrong: Data minimization does not guarantee fairness.

Increase the prediction threshold for the affected group

Why wrong: Adjusting thresholds can mask bias.

Retrain the model with more data from the affected group

Why wrong: Without analysis, retraining may not fix bias.

Immediately retire the model to prevent harm

Why wrong: Premature retirement may disrupt operations.

Use demographic features but with minimal monitoring

Why wrong: Minimal monitoring can allow bias to persist.

Use a complex black-box model and rely on post-hoc explanations

Why wrong: Post-hoc explanations may be unreliable.

Optimize the model solely for accuracy on historical data

Why wrong: Accuracy alone does not guarantee fairness.

Rely solely on user feedback to improve

Why wrong: User feedback alone may not prevent harm.

Disable the recommendation system entirely

Why wrong: Disabling the system is too extreme.

Increase the volume of training data

Why wrong: More data does not guarantee safety.

Restrict model use to only standard English

Why wrong: Restricting use excludes users.

Remove slang from input before inference

Why wrong: Removing slang may lose meaning.

Adjust the confidence threshold only for those groups

Why wrong: Group-specific thresholds may be unfair.

Send all flagged transactions to customers for confirmation

Why wrong: Customer fatigue may occur.

Focus only on precision to minimize false positives

Why wrong: Ignoring recall can miss fraud.

Increase the detection threshold to reduce false positives

Why wrong: Higher threshold reduces recall more.

Remain silent and wait for the next input

Why wrong: Silence is not transparent.

Provide a random answer from a predefined list

Why wrong: Random answers can mislead.

Automatically escalate all unanswered questions to a human

Why wrong: Escalation may not always be necessary.

Collect as much data as possible without quality checks

Why wrong: Data quality is more important than quantity.

Ensure the development team is homogeneous to avoid conflicts

Why wrong: Homogeneous teams increase bias risk.

Use the most complex model available for maximum accuracy

Why wrong: Complexity can reduce explainability.

Minimize the number of features to reduce cost

Why wrong: Cost minimization is not a responsible AI principle.

Publish the model's predictions publicly for transparency

Why wrong: Public disclosure may violate privacy.

Combine multiple models to obscure decision logic

Why wrong: Obscuring logic is opposite of transparency.

Remove all features except the most predictive ones

Why wrong: Removing features may reduce explainability.

Use black-box models to protect proprietary algorithms

Why wrong: Black-box models reduce transparency.

It requires human approval before deploying any model

Why wrong: The policy does not mention human approval.

It prevents the use of GPU instances to reduce cost

Why wrong: The policy does not restrict instance types.

It restricts endpoints to only use models built in SageMaker

Why wrong: The policy does not restrict model origin.

Proceed with deployment because the model is already in production

Why wrong: Deploying a biased model is irresponsible.

Remove the gender attribute from the training data and retrain

Why wrong: Removing attributes may not remove proxy bias.

Increase the acceptance threshold for the model

Why wrong: Threshold adjustment does not fix bias.

Configure the chatbot to use a pre-trained model from SageMaker JumpStart and disable logging to avoid storing sensitive customer data.

Why wrong: Disabling logging removes the ability to audit and monitor for harmful outputs.

Use Amazon Rekognition to analyze chat logs for biased language and automatically block responses with a confidence score above 90%.

Why wrong: Amazon Rekognition is designed for image and video analysis, not text-based bias detection.

Deploy Amazon Lex with built-in sentiment analysis to detect negative customer emotions and automatically escalate to a human agent.

Why wrong: Amazon Lex does not have built-in bias detection for generative AI responses.

Increase the logging of all model inputs and outputs to Amazon CloudWatch and set up alarms for any mentions of protected attributes.

Why wrong: Logging helps with monitoring but does not prevent biased outputs from being generated.

Replace the current LLM with a different pre-trained model that has been benchmarked for lower bias on medical datasets.

Why wrong: Switching models without fine-tuning may still result in biased or unsafe outputs, and benchmarking does not guarantee suitability.

Apply stronger content filtering rules using Amazon Comprehend Medical to block any diagnosis that contains demographic-related terms.

Why wrong: Filtering based on demographic terms does not address the underlying model bias and may block legitimate clinical content.

Amazon SageMaker Ground Truth

Why wrong: Ground Truth is for data labeling, not bias detection in predictions.

Amazon CloudWatch Logs Insights

Why wrong: CloudWatch Logs is for monitoring logs, not analyzing model bias.

Amazon SageMaker Model Monitor

Why wrong: Model Monitor tracks model quality over time, not pre-deployment bias analysis.

SHAP values

Why wrong: SHAP is a local method but based on cooperative game theory, not surrogate models.

Partial dependence plots

Why wrong: Partial dependence is a global interpretability method.

Permutation feature importance

Why wrong: Permutation importance is a global method that measures feature importance by shuffling.

k-anonymity

Why wrong: k-anonymity generalizes data to make individuals indistinguishable, but does not add noise during training.

Federated learning

Why wrong: Federated learning trains across decentralized data but requires differential privacy for formal guarantees.

Homomorphic encryption

Why wrong: Homomorphic encryption allows computation on encrypted data, not noise injection for privacy during training.

Model quality monitoring

Why wrong: Model quality monitors prediction accuracy against ground truth, not feature drift.

Bias drift monitoring

Why wrong: Bias drift monitors fairness metrics, not input feature distributions.

Feature importance drift

Why wrong: SageMaker Model Monitor does not have a built-in feature importance drift monitor.

Automatic labeling with Active Learning

Why wrong: Active Learning reduces manual labeling but does not address labeler bias.

Public workforce with no qualification

Why wrong: Unqualified public workers may introduce inconsistent or biased labels.

Vendor managed workforce

Why wrong: AWS Ground Truth does not have a built-in vendor workforce; vendors are third-party.

Clarify post-training bias metrics

Why wrong: Post-training metrics measure bias after training, not reweighting before.

Model Monitor bias drift

Why wrong: Model Monitor detects drift post-deployment, not preprocessing.

Clarify explainability (SHAP)

Why wrong: SHAP explains predictions, not reweights data.