AI Safety Security And Governance practice questions

Write a Lambda function to strip logs before writing to S3

Why it fails: This adds unnecessary complexity and latency compared to native service features.

Use IAM policies to prevent logs from being written

Why it fails: This violates the enterprise requirement to have logs for compliance.

Use AWS CloudTrail to log API calls and enable redaction in CloudWatch

Why it fails: CloudTrail logs metadata, not the prompt/response content itself.

Amazon CloudWatch Metrics

Why it fails: CloudWatch tracks invocation count and latency, but not cost per se.

AWS Trusted Advisor

Why it fails: Trusted Advisor provides optimization recommendations, not custom usage reports.

Amazon QuickSight

Why it fails: QuickSight is a BI tool; while it can visualize cost data, Cost Explorer is the primary service for this.

Use a custom system prompt to tell the model not to use URLs

Why it fails: System prompts are suggestions and cannot be relied upon for strict compliance/safety requirements.

Disable model streaming

Why it fails: Streaming is a performance feature; it does not filter content.

Use IAM to block internet access for the application

Why it fails: The model generates text internally; blocking the application's internet access does not stop the model from generating URL strings.

Create an IAM role for each model

Why it fails: This is unmanageable and does not prevent users from choosing other models via the API.

Use Amazon Cognito

Why it fails: Cognito handles authentication, not API-level governance of specific foundation models.

Hardcode the model ID in the application code

Why it fails: This relies on developer discipline and is not a security governance mechanism.

Publicly accessible S3 buckets

Why it fails: Public buckets are a security violation.

Temporary local storage on the training instance

Why it fails: Training instances are ephemeral; data should be stored in durable, compliant storage.

DynamoDB tables

Why it fails: Fine-tuning typically requires file-based input (S3), not NoSQL table formats.

Use Lambda to truncate text

Why it fails: This is inefficient; the token cost is already incurred by the model before truncation occurs.

Disable streaming

Why it fails: Streaming impacts the user experience, not the total token cost.

Use Guardrails to block long responses

Why it fails: Guardrails filter content, they don't natively enforce token counts.

Disable filters for non-English languages

Why it fails: This is a major security risk.

Translate all inputs to English first

Why it fails: Adding an extra translation step adds latency and potential for translation errors.

Use a regex-based filter only

Why it fails: Regex is not sufficient for multilingual semantic content filtering.

Use an AWS Lambda function after retrieval to call the OpenSearch _search API and remove documents whose department does not match the user.

Why it fails: Post-retrieval filtering in Lambda means unauthorized documents have already been fetched and leave the vector store, which weakens the security boundary and adds latency and custom code. It also contradicts the requirement that filtering occur inside the vector search without application changes.

Enable OpenSearch Serverless security analytics and define a field-level security policy that restricts the department field.

Why it fails: Field-level security in OpenSearch Serverless controls which fields a role can read or write; it does not filter individual documents by their content during a k-nearest-neighbor vector query. Applying it here would hide the department field rather than exclude unauthorized documents, so the model could still receive restricted content.

Create a separate OpenSearch Serverless collection for each department and route queries based on the caller's IAM role.

Why it fails: Separate collections per department do isolate data, but they require duplicating ingestion pipelines, indexes, and IAM wiring, and they force routing logic into the application. This is far more operational effort than the requirement allows, and it does not perform the filtering inside a single vector search as specified.

Use Amazon Bedrock in the designated Region and rely on the default data handling policy, which guarantees that data is never used for model training.

Why it fails: While Amazon Bedrock does not use your data to train the base models by default, relying solely on the default policy without explicit configuration does not provide the auditable proof that the agency requires. The agency needs to document and enforce region restrictions and data usage settings, not just assume defaults. This option lacks the necessary verification and control.

Configure Amazon Bedrock to use a VPC endpoint in the designated Region and enable model invocation logging, then provide the logs to auditors as evidence of data residency and non-training.

Why it fails: A VPC endpoint ensures private connectivity and can help enforce regional data flow, but model invocation logging captures request and response data, not proof that the data was excluded from training. Logging alone does not demonstrate that the model provider did not use the data for training. This option conflates network isolation with data usage attestation, which auditors require separately.

Deploy the application on Amazon SageMaker instead of Amazon Bedrock, because SageMaker guarantees that no data is ever used for training.

Why it fails: Amazon SageMaker is a machine learning platform where you control training data and can enforce isolation, but it does not inherently guarantee that data is never used for training unless you configure and manage that yourself. Switching services does not automatically provide the required compliance attestation, and it may introduce additional operational burden. This option does not directly address the agency's need for auditable evidence.

Encrypt the model weights with a CMK

Why it fails: CMK encryption protects model weights at rest against unauthorised disk or snapshot access; it cannot detect or block malicious instructions embedded in a prompt. It is tempting because encryption is a security control, and it would be correct for meeting data-at-rest protection requirements for model artefacts in Amazon S3 or Amazon EBS.

Enable Amazon Inspector on the endpoint

Why it fails: Amazon Inspector scans compute workloads and container images for software vulnerabilities and unintended network exposure; it does not inspect prompt text or model inputs at inference time. It is tempting because it is a security service, and it would be the right choice for continuously assessing EC2 instances or images for CVEs.

Disable the API Gateway execute-api method

Why it fails: Disabling the execute-api method removes the invocation path entirely, breaking the application rather than filtering adversarial prompts; prompt injection arrives through legitimate requests. It is tempting because it blocks access, and it would be correct when an API endpoint must be taken out of service or restricted during an incident.

Create a custom fine-tuned model

Why it fails: Fine-tuning adjusts a model's behaviour for a domain or style; it does not reliably block hate speech or violence and requires training data and cost. It is tempting because fine-tuning customises outputs, and would be correct when adapting tone or task performance rather than enforcing content safety.

Use AWS Config rules to monitor the model

Why it fails: AWS Config rules evaluate resource configuration compliance; they do not inspect prompts or responses, so they cannot block hate speech or violence. They are tempting because Config provides governance guardrails, and would be correct for detecting non-compliant infrastructure settings rather than filtering model output.

Add a system prompt to the application code

Why it fails: A system prompt only guides the model's tone and instructions; it cannot enforce content filtering, so hate speech or violence can still be generated. It is tempting because system prompts are the standard way to shape model behaviour, and would be correct for steering style, persona or task scope rather than safety enforcement.

Encrypt the documents with different KMS keys

Why it fails: KMS encryption protects data at rest but does not filter retrieval results; Bedrock Knowledge Bases enforce per-user access through metadata filtering tied to identity, not key permissions. KMS keys are tempting because they segment data cryptographically, and would suit encrypting distinct document sets, yet retrieval would still return any indexed chunk the query matches.

Create separate Knowledge Bases for every user

Why it fails: Per-user Knowledge Bases duplicate the entire indexed corpus and cannot scale, and Bedrock offers no per-user access control at retrieval. It is tempting as an isolation approach, and would suit a handful of tenants needing fully separate data stores rather than shared documents with per-user permissions.

Apply IAM policies to the S3 bucket

Why it fails: IAM policies on the S3 bucket govern direct S3 access, not documents returned by a Bedrock Knowledge Base retrieval, so unauthorised content still reaches the model. It is tempting because S3 policies are the standard access mechanism, and would be correct for direct S3 reads.

VPC Endpoint policies

Why it fails: VPC endpoint policies control which resources can reach a service through an endpoint, not which AWS regions the application may process data in. They cannot enforce regional residency. Endpoint policies would be correct when restricting access to a specific service endpoint.

IAM policies for all users

Why it fails: IAM policies govern which identities can call which actions on which resources; they do not restrict the AWS regions where a Generative AI application processes data. Region residency needs region-scoped enforcement. IAM policies would be correct when controlling user permissions.

CloudFront geographic restrictions

Why it fails: CloudFront geographic restrictions block viewer access by country at edge locations; they do not constrain which AWS regions process data. Region-level residency needs region-scoped controls. Geo restriction would be correct when the requirement is blocking users from specific countries.

Enable AWS Organizations SCPs only

Why it fails: SCPs set the maximum permissions boundary for accounts in an organisation but grant nothing themselves, and they cannot target individual IAM users. SCPs suit centrally capping what member accounts may do; permitting specific users to invoke particular Bedrock models needs identity-based IAM policy statements.

Use Amazon Cognito user pools

Why it fails: Cognito user pools authenticate end users of applications and issue OIDC tokens; they do not authorise IAM principals to invoke Bedrock model actions. Cognito is the right choice for adding sign-up and sign-in to a customer-facing app, not for restricting IAM users' access to foundation models.

Use Resource-Based Policies on the foundation model

Why it fails: Bedrock foundation models are not resources that accept resource-based policies; access is governed through identity-based IAM policies and, where required, guardrails. Resource-based policies fit services such as S3 buckets or KMS keys, where a resource itself can name permitted principals.

AWS KMS with envelope encryption

Why it fails: KMS envelope encryption protects data confidentiality at rest and in transit, but the model still receives plaintext once decrypted, so PII reaches it unmasked. It is tempting as the standard AWS encryption control, and would be correct for securing stored objects or volumes, not prompt redaction.

Amazon Macie

Why it fails: Macie discovers and classifies sensitive data in S3, it cannot intercept or redact a live Bedrock inference request. It is tempting because it is AWS's PII detection service, and would be correct for auditing stored data at rest, not for masking prompts in transit.

AWS WAF with custom regex rules

Why it fails: WAF inspects HTTP requests at the edge against regex rules, but Bedrock's InvokeModel API payload is not parsed for PII masking, and regex alone cannot reliably identify personal data. It is tempting for filtering malicious input, and would be correct for blocking SQL injection or known bad actors.

Create separate Amazon Q applications

Why it fails: Separate Amazon Q applications duplicate the index and identity configuration, and each user still needs per-application subscriptions. Data source access is governed by Access Control Lists and principal mappings within one application. Multiple applications suit isolating entirely separate business units or tenants.

Manually delete documents for specific users

Why it fails: Deleting documents removes content for every user and breaks the index's consistency with the source repository, which is re-crawled. Per-user restriction requires ACL-aware ingestion so the same document is visible only to authorised principals. Manual deletion suits permanently removing obsolete or sensitive content.

Apply IAM policies to the user's login

Why it fails: IAM policies control access to AWS APIs and the Amazon Q service itself, not which documents a user retrieves during chat. Document-level restriction relies on ACLs crawled from the data source and mapped to user principals. IAM suits controlling who may create or configure applications.

Query Amazon SageMaker Model Monitor

Why it fails: Model Monitor detects data and model quality drift in captured traffic; it does not emit a per-request record of the model version served. Model Monitor is correct when you need alerts on feature distribution shift or accuracy degradation.

Enable CloudTrail logs

Why it fails: CloudTrail records AWS API activity, not the model artefact version invoked per inference; that metadata must be captured by the inference layer itself. It is tempting because CloudTrail is the default AWS audit trail, and would be correct for auditing who called control-plane APIs such as creating or modifying endpoints.

Use AWS Config to track model changes

Why it fails: AWS Config records resource configuration changes, not per-request inference metadata such as the foundation model version invoked. It cannot capture the model ID or version tied to each individual request. AWS Config suits compliance auditing of resource state changes over time, for example detecting when an endpoint's configuration drifts from a baseline.

AWS WAF

Why it fails: AWS WAF filters HTTP traffic against web exploits; it neither establishes private network paths nor encrypts Bedrock API calls in transit. It is tempting because WAF sits in front of public endpoints, but it is the right choice for blocking SQL injection or bot traffic, not for private connectivity.

AWS CloudFront

Why it fails: CloudFront caches and delivers content over public edge locations, so traffic still traverses the internet rather than a private path to Bedrock. It is tempting because it terminates TLS at the edge, but it suits accelerating global static or dynamic content delivery, not private endpoint connectivity.

AWS Certificate Manager

Why it fails: AWS Certificate Manager issues and renews TLS certificates but does not create private network connectivity or encrypt traffic itself. It is tempting because the scenario mentions encryption in transit, yet ACM is correct when you need managed public or private certificates for load balancers and CloudFront distributions.

Amazon Inspector

Why it fails: Amazon Inspector scans workloads for software vulnerabilities and unintended network exposure; it does not measure application latency or Bedrock call performance. It is tempting because Inspector assesses resource security posture, which suits vulnerability management rather than the runtime performance monitoring this scenario requires.

AWS CloudTrail

Why it fails: CloudTrail records API activity for auditing and governance, not runtime latency or invocation performance metrics. It is tempting because it captures Bedrock API calls, but it logs who called what and when, not duration or error rates. Amazon CloudWatch provides the performance and latency metrics this scenario requires.

AWS Config

Why it fails: AWS Config records resource configuration changes and evaluates compliance, not runtime latency or invocation performance of Bedrock calls. It is tempting because Config provides visibility into AWS resource state, which suits auditing and drift detection rather than application-level performance monitoring.

Use fine-tuning to force the model to watermark

Why it fails: Fine-tuning adjusts model weights on example data; it cannot guarantee a deterministic watermark on every generated document and offers no verifiable extraction mechanism. It is tempting because fine-tuning shapes output style and format, but watermarking requires deterministic post-processing or a watermark-aware inference pipeline, not weight adjustment.

Enable watermarking in the Bedrock console settings

Why it fails: Bedrock exposes no console watermarking toggle; generated text is returned as plain output, so no traceability mark is applied. It is tempting because console settings do govern guardrails, logging and model access, but watermarking requires post-processing the response, for example via a Lambda function or application layer.

Use AWS KMS to sign the generated text

Why it fails: KMS signs data cryptographically for integrity and provenance verification, but a signature is not embedded in the document text and does not survive as a visible or statistical watermark. It is tempting because KMS provides key custody and audit trails, yet traceability here needs an in-band mark, not detached signature metadata.