AIP-C01 Exam Questions and Answers

The model's support for citation and source attribution.

Citation and source attribution let the model tie each generated claim to retrieved passages, which is essential for verifying grounding accuracy. Without attribution, hallucinations cannot be distinguished from grounded statements, so this capability directly satisfies the high-grounding-accuracy requirement in the RAG scenario.

The context window size of the foundation model.

A larger context window allows more retrieved passages to be supplied to the model, improving the chance that relevant evidence is present during generation. Insufficient context forces truncation of retrieved chunks, which degrades grounding accuracy, so window size directly constrains the RAG scenario's requirement.

Using a reranking model to score retrieved documents

A reranking model rescores the initial candidate set from vector search, applying a cross-encoder that evaluates query-document pairs jointly rather than comparing independent embeddings. This directly improves semantic relevance by promoting contextually accurate passages, satisfying the stem's requirement for higher-quality retrieval in the RAG pipeline.

Applying semantic chunking based on document structure

Semantic chunking preserves coherent units such as sections and headings, so each embedding represents one complete idea rather than a fragment split mid-topic. This directly satisfies the stem's semantic-relevance constraint: retrieval matches meaning, not arbitrary token windows, lifting recall precision for the RAG pipeline.

Implementing hybrid search combining keyword and vector search

Hybrid search fuses BM25 keyword matching with dense vector similarity, so exact terms such as product codes or acronyms that embeddings blur are still retrieved. This directly raises semantic search relevance by covering both lexical and meaning-based recall gaps.

Bedrock Model Evaluation jobs.

Bedrock Model Evaluation jobs run automatic and human evaluations against candidate foundation models using your own dataset, producing metrics on accuracy, robustness and latency. This lets the developer compare reasoning capability against latency for the chatbot.

Deploy the Bedrock Knowledge Base using a PrivateLink interface endpoint for Bedrock and a VPC endpoint for OpenSearch Serverless.

PrivateLink interface endpoints for Bedrock and a VPC endpoint for OpenSearch Serverless keep traffic on the AWS private network, so requests never traverse the public internet. This satisfies the constraint that all data remains within the VPC.

Use Amazon Bedrock Knowledge Base metadata filtering during the retrieval phase to apply access controls.

Metadata filtering applies attribute-based predicates during retrieval, so each user's query only returns chunks whose metadata matches their identity claims. This enforces per-user access control at query time without duplicating knowledge bases or pre-filtering the entire corpus.

Amazon Bedrock Guardrails

Bedrock Guardrails applies configurable PII detection and masking policies directly to prompts and completions before the foundation model processes them, satisfying the requirement that sensitive data be filtered prior to inference rather than relying on post-processing or application-side scrubbing.

The application is exceeding the tokens-per-minute (TPM) limit for the specific model in that AWS Region.

Exceeding the model-specific tokens-per-minute quota in that Region triggers throttling independently of account-level limits, so 429s appear despite unused account capacity. Bedrock enforces per-model, per-Region TPM and RPM quotas separately, meaning a single high-volume Lambda workload can saturate the model's throughput ceiling while aggregate account usage remains low.

Implement a RAGAS-based evaluation pipeline to measure faithfulness and answer relevance.

RAGAS quantifies retrieval-augmented generation quality by scoring faithfulness (whether claims are entailed by retrieved context) and answer relevance, directly measuring ground-truth accuracy against the internal knowledge base. This satisfies the stem's requirement to quantify hallucination reduction in the RAG pipeline.

Answer Relevance

Answer Relevance scores how directly the generated response addresses the user's original question, independent of factual grounding. In a RAG test suite it catches responses that are faithful to retrieved context yet drift off-topic, satisfying the need to evaluate end-to-end answer quality.

Context Precision

Context Precision measures the proportion of retrieved chunks that are genuinely relevant to the question, penalising noise in the ranked results. This isolates the retrieval stage of the RAG pipeline, letting the test suite verify that the retriever surfaces useful passages before generation occurs.

Faithfulness

Faithfulness measures whether every claim in the generated answer is actually supported by the retrieved context, directly exposing hallucinations. For a RAG test suite this is essential, since it isolates whether the generation stage stayed grounded in the documents retrieved, independently of retrieval quality.

Check CloudWatch Logs for the associated Action Group Lambda functions.

Action Group invocations run in Lambda, so CloudWatch Logs capture the request payloads, errors and timeouts that surface as agent failures. Reviewing these logs isolates whether the fault lies in the Lambda function rather than the agent's orchestration or prompt configuration.

Review the Agent's 'Trace' output to identify the failure point.

Trace output records each orchestration step, prompt, and action-group invocation, so it pinpoints exactly where the Agent fails rather than guessing. This directly isolates the issue, satisfying the stem's requirement to identify the failure point before attempting any fix.

Validating the reasoning path of the agent.

Validating the reasoning path verifies the agent's chain-of-thought and orchestration logic, not just the final response. This satisfies the test plan requirement to assess how the agent reaches conclusions, exposing flawed planning or prompt interpretation before production deployment.

Testing the accuracy of tool invocations (Action Groups).

Action Groups expose tools the agent invokes, so testing invocation accuracy confirms the agent selects the correct tool with correctly formed parameters. This satisfies the test plan's requirement to validate tool-use behaviour rather than only natural-language responses.

Ensuring Guardrails are applied to both input and output.

Guardrails must be verified on both the user input and the agent's generated output, since either direction can carry harmful or non-compliant content. Testing both satisfies the plan's coverage of safety controls across the full request-response cycle.

Implement an exponential backoff and jitter retry strategy using the AWS SDK for the Bedrock InvokeModel API call.

Bedrock throttling is transient and retrying with exponential backoff and jitter is the AWS-recommended approach. The SDK's built-in retry mechanism can be customized, and adding jitter prevents synchronized retries from multiple Lambda instances. This directly addresses the throttling without changing quotas.

Use a prompt-based filter system that detects malicious intent via a secondary validation model

A secondary validation model inspects prompts for malicious intent before inference, catching injection attempts that static keyword filters miss because attackers rephrase payloads. This layered detection satisfies the prompt-injection protection requirement for the SageMaker JumpStart endpoint more effectively than input sanitisation alone.

Implement Amazon Bedrock Guardrails with content filters

Bedrock Guardrails content filters evaluate prompts and responses against configurable hate, violence, sexual and insult categories, blocking matching content. This satisfies the requirement centrally at the model invocation layer, without custom prompt engineering or separate moderation infrastructure per application.

Use the filters attribute in the Retrieve API call based on document metadata

Filtering on document metadata during retrieval enforces per-user authorisation at query time, satisfying the requirement that users only see documents they are permitted to access. Metadata tags such as department or group are matched against the caller's identity, so unauthorised chunks never enter the context passed to the model.

Service Control Policies (SCPs)

SCPs are AWS Organizations guardrails that deny actions or regions across member accounts, evaluated before IAM policies. Attaching a region-restriction SCP to the relevant OUs enforces data residency at the account boundary, blocking Generative AI workloads from invoking services outside approved regions.

Attach an IAM policy with bedrock:InvokeModel allowing access to specific model ARNs

Attaching an IAM policy scoped to specific model ARNs satisfies the least-privilege constraint, because Bedrock authorises each InvokeModel call against the resource ARN in the request. Only principals granted bedrock:InvokeModel on those ARNs succeed; other IAM users receive AccessDenied, restricting access to the named foundation models.

Amazon Bedrock Guardrails

Amazon Bedrock Guardrails apply sensitive-information filters that detect and mask PII in prompts before inference, satisfying the requirement that data be redacted prior to reaching the foundation model. Configured via `sensitiveInformationPolicyConfig`, they intercept the request at the service boundary, so the model never receives raw personal data.

toolConfig

toolConfig is the correct parameter to define tools.

Configure Subnets and Security Groups in the Model creation

Specifying subnets and security groups during model creation places the inference container's network interfaces inside the VPC, giving it private connectivity and the correct security group rules. Without this configuration, the endpoint cannot reach private VPC resources.

Action Group

An action group defines the API operations the agent may invoke, including the OpenAPI schema describing parameters and the Lambda function that fulfils each call. It is the required binding that lets Bedrock Agents interact with the external API.

OpenAPI Schema

Bedrock Agents require an OpenAPI schema to define the external API's operations, parameters and request/response structures, which the agent uses to construct valid action-group invocations. Without this machine-readable contract, the agent cannot map user intent to the correct API call, so it satisfies the requirement to interact with an external API.

Lambda Function

A Lambda function implements the business logic that Bedrock Agents invokes to call the external API, handling request formatting, authentication and response parsing. Without it, the action group has no executable backend, so the agent cannot complete the API interaction.

Amazon OpenSearch Serverless

Amazon OpenSearch Serverless is the vector store natively integrated with Bedrock Knowledge Bases, providing managed vector indexing and similarity search without separate provisioning. This satisfies the managed RAG requirement by removing manual database administration from the retrieval pipeline.

Base64 encoded string in the JSON input field

SageMaker JumpStart vision models expect image bytes serialised as a Base64 encoded string within the JSON input field, not raw binary or a URL. Supplying the image in that encoding resolves the schema mismatch at InvokeEndpoint.

Increase the max_tokens parameter

max_tokens caps the number of tokens generated in the response; raising it allows longer output before the model stops. Truncation occurs because generation halts at that limit, so increasing the value directly satisfies the requirement to capture the full response.

SageMaker Serverless Inference

Serverless Inference scales to zero when no requests arrive, so you pay only for compute during actual invocations. This directly addresses the unpredictable traffic and significant idle periods, unlike real-time endpoints that bill for continuously provisioned instances regardless of load.

AWS Budgets

AWS Budgets lets you define a monthly cost budget and configure alerts when actual or forecasted spend crosses your threshold. This directly satisfies the requirement to be notified before the SageMaker inference endpoint exceeds its specified monthly budget.

Configuring Application Auto Scaling with scheduled actions

Application Auto Scaling scheduled actions adjust the endpoint's desired instance count against a cron or rate schedule, scaling in during off-peak hours and out afterwards. This directly satisfies the off-peak cost-reduction constraint without manual intervention or redeploying the endpoint.

Amazon CloudWatch

Amazon CloudWatch is the primary telemetry service receiving Bedrock metrics such as ModelInvocationLatency, letting you graph, alarm and analyse them. It satisfies the monitoring requirement without additional agents, since Bedrock publishes these metrics natively to CloudWatch.

Implementing dynamic batching in your inference handler

Dynamic batching groups concurrent inference requests into a single forward pass, raising GPU utilisation and cutting per-request latency. It directly addresses the low memory utilisation and high latency described, since the GPU is otherwise underfed by one-at-a-time requests. This suits real-time endpoints with concurrent traffic.

SageMaker Model Monitor

SageMaker Model Monitor continuously captures endpoint inference data and compares it against a baseline, detecting data drift, model quality degradation and bias shifts. This directly satisfies the requirement to observe model performance drifts over time, unlike one-off evaluation tools that provide only point-in-time metrics.