AIF-C01 Fundamentals of Generative AI — Timed 20 Questions

Question 1

A data science team is evaluating foundation models for a code generation task. They need a model that is fine-tuned for code and can be deployed on Amazon SageMaker. Which THREE criteria are important to consider when selecting a model?

20 minute time limit — choose an answer to begin.

Accepted Answer

Licensing and usage terms. When selecting a foundation model for deployment on Amazon SageMaker, particularly for code generation, three primary criteria are licensing and usage terms, cost per token for inference, and context window length. Licensing ensures legal compliance for commercial use; cost per token directly impacts operational expenses; context window length determines the maximum input size, which is critical for generating large code snippets. While model architecture and training algorithms are relevant to model performance, they are not the top three criteria for initial selection, as they are often evaluated after the practical constraints of licensing, cost, and context window are met.

Answer

The training algorithm used

Answer

Model size and architecture

Question 2

A company wants to generate product descriptions from a few keywords without managing infrastructure. Which AWS service provides a serverless API for accessing foundation models?

Accepted Answer

Amazon Bedrock. Amazon Bedrock is a fully managed, serverless service that provides a single API to access and invoke foundation models (FMs) from leading AI providers like AI21 Labs, Anthropic, Cohere, Meta, and Stability AI. It eliminates the need to manage underlying infrastructure, making it the correct choice for generating product descriptions from keywords without provisioning servers.

Answer

Amazon Lex

Answer

Amazon SageMaker

Answer

Amazon Comprehend

Question 3

A data scientist is evaluating foundation models for a text summarization task and wants to use a standard metric. Which metric is commonly used to assess the quality of generated summaries?

Accepted Answer

ROUGE. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is the standard metric for text summarization. It measures the overlap of n-grams, word sequences, or word pairs between the generated summary and reference summaries, focusing on recall. This makes it particularly suitable for evaluating how well the generated summary captures the key content of the reference.

Answer

F1 score

Answer

BLEU

Answer

Accuracy

Question 4

A machine learning engineer notices that a generative AI model occasionally produces biased outputs. Which AWS feature can automatically filter harmful content before it reaches users?

Accepted Answer

Amazon Bedrock Guardrails. Amazon Bedrock Guardrails is specifically designed to implement safeguards for generative AI applications, including the ability to filter harmful, biased, or inappropriate content before it reaches users. It allows you to define denied topics, content filters, and sensitive information filters that are applied at inference time, directly addressing the need to automatically filter biased outputs from a generative AI model.

Answer

Amazon CloudWatch alarms

Answer

Amazon SageMaker Clarify

Answer

AWS Identity and Access Management (IAM) policies

Question 5

A team is using Amazon Bedrock with a Claude model and wants to ensure responses adhere to a specific output format such as JSON. Which technique should be applied?

Accepted Answer

Include a system prompt with explicit formatting instructions. Amazon Bedrock with Claude models supports system prompts that can include explicit formatting instructions, such as 'Respond in valid JSON format.' This technique directly controls the model's output structure without requiring external tools or retraining, making it the simplest and most effective method for enforcing a specific output format like JSON.

Answer

Use a retrieval-augmented generation (RAG) approach

Answer

Attach a guardrail with a JSON schema

Answer

Customize the model with a JSON training dataset

Question 6

A startup wants to integrate a generative AI chatbot into their mobile app with minimal latency. Which AWS service is purpose-built for deploying foundation models with low latency and high throughput?

Accepted Answer

Amazon Bedrock. Amazon Bedrock is a fully managed service that provides access to foundation models (FMs) from leading AI providers via a serverless API, purpose-built for deploying generative AI applications with low latency and high throughput. It handles the underlying infrastructure, model hosting, and scaling automatically, making it ideal for integrating a generative AI chatbot into a mobile app with minimal latency.

Answer

AWS Lambda

Answer

Amazon SageMaker

Answer

Amazon Transcribe

Question 7

A company wants to personalize its generative AI model for its specific domain without sharing data with third-party model providers. Which method should they use?

Accepted Answer

Fine-tuning the foundation model on their proprietary data. Fine-tuning the foundation model on proprietary data allows the company to adapt the model's weights to its specific domain without sharing data with third parties. This method trains the model on private datasets, enabling it to learn domain-specific patterns and terminology while keeping data in-house, which is critical for data privacy and compliance.

Answer

Prompt engineering with domain-specific examples

Answer

Retrieval-augmented generation (RAG) with a domain-specific knowledge base

Answer

Model distillation using a larger foundation model

Question 8

A developer is using Amazon Bedrock's Converse API to build a multi-turn conversation. They notice the model forgets earlier context after a few exchanges. What is the most likely cause?

Accepted Answer

The developer is not sending the previous messages in each request. The Converse API is stateless by design, meaning it does not retain conversation history between API calls. To maintain context across multiple turns, the developer must explicitly include the entire message history (previous user and assistant messages) in each new request. If the developer omits these previous messages, the model has no memory of earlier exchanges and will appear to forget context.

Answer

The API has a rate limit that truncates history

Answer

The model's context window is too small for the conversation

Answer

The model's maximum output length is set too low

Question 9

Which TWO actions are best practices for reducing hallucinations in generative AI models? (Choose 2)

Accepted Answer

Use retrieval-augmented generation (RAG). Retrieval-Augmented Generation (RAG) is a best practice for reducing hallucinations because it grounds the model's output in factual, retrieved context from an external knowledge base. By providing relevant documents or data chunks at inference time, RAG constrains the model to generate responses based on verified information rather than relying solely on its parametric memory, which can be incomplete or outdated.

Answer

Increase the model size

Answer

Fine-tune the model on proprietary data

Answer

Use a smaller model to limit complexity

Question 10

Which THREE are key capabilities of Amazon Bedrock? (Choose 3)

Accepted Answer

Model customization through fine-tuning. Amazon Bedrock allows you to customize foundation models (FMs) using your own data through fine-tuning, enabling the model to adapt to domain-specific tasks without building a model from scratch. This is a key capability for enterprises that need models tailored to their unique requirements, such as legal document analysis or medical terminology understanding.

Answer

Automatic model selection based on use case

Answer

Built-in vector database for knowledge bases

Question 11

Which TWO factors are most important when selecting a foundation model for a sentiment analysis task? (Choose 2)

Accepted Answer

Composition of the model's training data (domain, language). The composition of the model's training data directly determines its ability to understand domain-specific language, slang, and cultural nuances. For sentiment analysis, a model trained on social media text will outperform one trained on academic papers when analyzing tweets. Similarly, language coverage is critical: a model trained only on English will fail on French or code-switched text, leading to inaccurate sentiment predictions.

Answer

API pricing per invocation

Answer

Inference latency

Answer

Color scheme of the model's documentation

Question 12

A developer runs this AWS CLI command to invoke a model in us-west-2 but receives an error: 'An error occurred (ModelNotFoundException) when calling the InvokeModel operation: Model not found'. What is the most likely cause?

Accepted Answer

The model is not available in the us-west-2 region. The error 'ModelNotFoundException' specifically indicates that the model ID or ARN is not recognized in the specified region. AWS Bedrock models are region-specific; not all foundation models are available in every AWS region. The developer invoked the model in us-west-2, but the requested model may only be available in regions like us-east-1 or us-west-1, causing the service to return a ModelNotFoundException rather than a permissions or formatting error.

Answer

The request body is not properly formatted

Answer

The --region parameter is missing from the command

Answer

The user's IAM role lacks permissions to invoke the model

Question 13

A user has this IAM policy and attempts to invoke the model in the us-west-2 region. They receive an AccessDenied error. What is the reason?

Accepted Answer

The policy does not allow the us-west-2 region. The policy explicitly denies access to the 'us-west-2' region via the 'Deny' effect with a condition on 'aws:RequestedRegion'. Since the user is attempting to invoke the model in us-west-2, the request matches the deny condition, resulting in an AccessDenied error. The policy does not grant any explicit allow for that region, so the default implicit deny combined with the explicit deny blocks the action.

Answer

The resource ARN is malformed

Answer

The action 'bedrock:InvokeModel' is incorrect

Answer

The model ID is case-sensitive

Question 14

A developer deployed this guardrail to block sensitive topics and sexual content. However, the model still generates responses about a specific sensitive topic that is not in the TopicPolicy. What should the developer do to prevent this?

Accepted Answer

Add the specific topic to the TopicPolicy list. The guardrail's TopicPolicy is designed to block specific topics by listing them. Since the model is generating responses about a sensitive topic not currently in the policy, adding that topic to the list directly extends the policy's coverage to prevent those responses. This is the precise mechanism for blocking new unwanted topics without altering other guardrail components.

Answer

Add a SensitiveInformationPolicy to filter PII

Answer

Increase the InputStrength of the content filter to MAX

Answer

Change the TopicPolicy Type from DENY to ALLOW

Question 15

A data scientist is deploying a fine-tuned Mistral model on Amazon Bedrock. After deployment, inference latency is too high for real-time applications. Which configuration change can reduce latency without significantly impacting output quality?

Accepted Answer

Reduce max tokens from 1024 to 256. Reducing max tokens from 1024 to 256 directly decreases the number of tokens the model generates per request, which lowers the computational load and memory usage during inference. Since the model only processes up to the specified token limit, this reduces latency proportionally while preserving output quality for short-form responses, as the model's core reasoning and generation capabilities remain unchanged.

Answer

Switch to a larger model variant

Answer

Decrease the top-p to 0.5

Answer

Increase the temperature to 0.9

Question 16

A bank is using Amazon Bedrock to summarize customer support transcripts. The summaries often contain factual inaccuracies (hallucinations). Which approach is most effective for reducing hallucinations?

Accepted Answer

Implement RAG by grounding summarization on retrieved transcripts. Retrieval-Augmented Generation (RAG) grounds the model's output on actual retrieved chunks of the customer support transcripts, providing factual context that reduces the likelihood of hallucination. By retrieving relevant transcript segments and feeding them as context to the LLM, the model generates summaries based on verified source material rather than relying solely on its parametric knowledge, which is the primary cause of factual inaccuracies.

Answer

Decrease the top-p to 0.1

Answer

Increase the model's temperature to make outputs more diverse

Answer

Fine-tune a smaller model on a large dataset of transcripts

Question 17

A media company uses a foundation model on Amazon Bedrock to generate article summaries. The model occasionally omits important details. Which prompt engineering technique is most likely to improve completeness?

Accepted Answer

Include a list of required key points in the prompt. Explicitly listing required key points in the prompt guides the foundation model to cover all specified elements, directly addressing the omission issue. This technique, often called 'constrained generation' or 'structured prompting,' forces the model to attend to each required detail, improving completeness without altering model parameters.

Answer

Use a lower temperature setting

Answer

Increase the max tokens limit

Answer

Add 'Be concise' to the prompt

Question 18

A company fine-tunes a foundation model using SageMaker to create a domain-specific chatbot. After deployment on Bedrock, the model shows high confidence in incorrect answers. What is the most likely cause and its solution?

Accepted Answer

The model is overfitting; apply regularization techniques during fine-tuning. When a fine-tuned model exhibits high confidence in incorrect answers, it is a classic symptom of overfitting, where the model has memorized the training data too closely and fails to generalize to new inputs. Applying regularization techniques such as dropout, weight decay, or early stopping during fine-tuning reduces overfitting by penalizing overly complex decision boundaries, forcing the model to learn more robust patterns. This is the most direct cause and solution given the scenario of a domain-specific chatbot deployed on Bedrock.

Answer

The model was not pre-trained on enough data; use a larger base model

Answer

The training data was imbalanced; collect more diverse data

Answer

The inference temperature is too low; increase it

Question 19

What is a foundation model?

Accepted Answer

A model trained on diverse data that can be adapted to many tasks. A foundation model is a large-scale AI model trained on vast, diverse datasets (e.g., text, images, code) using self-supervised learning, enabling it to be adapted to a wide range of downstream tasks through fine-tuning or few-shot learning. Option C correctly captures this core property of broad adaptability, which distinguishes foundation models from task-specific models. For example, GPT-4 and Claude are foundation models that can handle translation, summarization, and coding without being retrained from scratch.

Answer

A model that only works with tabular data

Answer

A model that requires no additional tuning for new tasks

Answer

A model that is specifically trained for one task, like image classification

Question 20

Which TWO strategies can help reduce inference costs when using Amazon Bedrock? (Select TWO.)

Accepted Answer

Use provisioned throughput for high-volume, predictable workloads. Provisioned throughput allows you to reserve capacity for a specific foundation model, ensuring predictable performance and cost savings for high-volume workloads compared to on-demand pricing. By committing to a certain throughput level, you avoid per-token charges that can accumulate with variable usage, reducing overall inference costs.

Answer

Use a higher temperature setting to generate fewer tokens

Answer

Increase the max tokens to allow longer responses

Answer

Cache frequently used responses in Amazon ElastiCache