AIF-C01 Practice Test 3 — 25 Questions

Question 1

A data science team is evaluating the output quality of a text generation model. They want to use both automated metrics and human judgment. Which THREE approaches should they include in their evaluation strategy? (Select THREE.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Conduct human evaluation with a rating rubric. Option A is correct because human evaluation with a rating rubric directly captures subjective quality dimensions such as fluency, coherence, relevance, and factual consistency that automated metrics cannot fully assess, which is essential when the team explicitly wants human judgment. Option D is correct because ROUGE (recall-oriented n-gram/LCS overlap) and BLEU (precision-oriented n-gram overlap with brevity penalty) are standard automated reference-based metrics for comparing generated text against reference texts, matching the team's goal of using automated metrics. Option E is correct because running the model on a task-specific benchmark dataset provides standardized, objective, task-relevant performance measurement (e.g., accuracy, F1, exact match) that complements both human ratings and text-overlap metrics. Option B does not belong because inference latency under load measures operational performance and scalability, not output quality of the generated text. Option C does not belong because perplexity is typically computed on a language model's token likelihoods over a corpus and is not a reliable measure of the quality of generated text, especially for open-ended generation.

Answer

Measure inference latency under load

Answer

Compute perplexity on the generated text

Question 2

A company is using Amazon Bedrock to generate responses for customer support. They want to ensure that the model does not expose personally identifiable information (PII) in its outputs. Which AWS feature can be configured to automatically redact PII from model responses?

Accepted Answer

Amazon Bedrock Guardrails. Amazon Bedrock Guardrails is the correct choice because it provides configurable policies that can automatically detect and redact personally identifiable information (PII) from model inputs and outputs. This feature is specifically designed for Amazon Bedrock to enforce content safety and compliance requirements, including PII redaction, without requiring custom code or external services.

Answer

Amazon Macie

Answer

Amazon SageMaker Model Monitor

Answer

AWS CloudTrail

Question 3

Which TWO statements about the bias-variance tradeoff are correct? (Choose TWO.)

Accepted Answer

High bias typically leads to underfitting. Option C is correct because high bias means the model makes overly simplistic assumptions about the data, so it fails to capture the underlying pattern and systematically underfits both training and test data. Option E is correct because high variance means the model is overly sensitive to the particular training samples, fitting noise as well as signal, which produces excellent training performance but poor generalization—the hallmark of overfitting. Option A is wrong because high bias leads to underfitting, not overfitting; overfitting is associated with high variance. Option B is wrong because high variance models are highly sensitive to changes in training data, not insensitive. Option D is wrong because increasing model complexity typically increases variance (and decreases bias), not decreases variance.

Answer

High bias typically leads to overfitting

Answer

High variance models are insensitive to changes in training data

Answer

Increasing model complexity typically decreases variance

Question 4

A research organization is using Amazon SageMaker Studio to collaborate on building machine learning models. The security policy requires that all data and code remain within a VPC and cannot be accessed from the public internet. Additionally, the organization wants to enforce that only approved base images are used for the Studio environment. How should the organization configure SageMaker Studio to meet these requirements?

Accepted Answer

Configure the Studio domain in VPC Only mode and use a SageMaker Studio lifecycle configuration to restrict the list of available base images to those in a private Amazon ECR repository.. Configuring the SageMaker Studio domain in VPC Only mode ensures that all data and code remain within the VPC and cannot be accessed from the public internet. Additionally, using a lifecycle configuration script to restrict available base images to those in a private Amazon ECR repository enforces the policy that only approved base images are used, as the lifecycle configuration can modify the Jupyter Server settings to limit the image registry.

Answer

Use an AWS Transit Gateway to connect the VPC, and enforce HTTPS for all traffic.

Answer

Configure the Studio domain to disable direct internet access, and let users choose any base image from the SageMaker public registry.

Answer

Use AWS CloudFormation to create a Studio domain in a VPC and rely on individuals to use approved images.

Question 5

A company uses Amazon SageMaker to build and deploy models. They want to enforce compliance that all model endpoints are encrypted in transit and use least privilege access. Which THREE steps should they take? (Choose THREE.)

Accepted Answer

Configure the SageMaker endpoint to use a custom SSL certificate via AWS Certificate Manager. Configuring a SageMaker endpoint to use a custom SSL certificate from AWS Certificate Manager (ACM) ensures that all data transmitted between clients and the endpoint is encrypted in transit using TLS. This enforces the compliance requirement for encryption in transit by replacing the default SageMaker certificate with a customer-managed certificate, which can be validated and rotated as needed.

Answer

Enable AWS CloudTrail to log all endpoint invocations

Answer

Disable root access on the SageMaker notebook instances

Question 6

A company wants to use Amazon Bedrock to translate customer emails from English to Spanish. The emails contain occasional personal names and addresses. Which Guardrail configuration should be applied to protect customer privacy?

Accepted Answer

PII detection. PII detection in Bedrock Guardrails can identify and redact personally identifiable information such as names and addresses. Content filtering, topic denial, and grounding check do not specifically target PII.

Answer

Grounding check

Answer

Topic denial

Answer

Content filtering

Question 7

A financial services company uses Amazon SageMaker to train models with sensitive customer data. They must ensure that no data leaves a specific AWS Region due to data residency regulations. The training data is in S3. Which architecture meets this requirement while minimizing data transfer?

Accepted Answer

Use S3 VPC endpoints and place SageMaker training job in a private subnet with no internet access. Using a VPC with S3 VPC endpoints ensures data stays within the AWS network and does not traverse the internet. Data remains in the same Region because S3 endpoints are Regional.

Answer

Place SageMaker training job in a private subnet with a NAT gateway and route traffic through the internet

Answer

Configure S3 Transfer Acceleration and use a public SageMaker training job

Answer

Use AWS Glue to copy data to an EBS volume attached to the training instance

Question 8

A company wants to use a third-party foundation model from Amazon Bedrock but is concerned about data privacy because the model provider might store prompts and responses. How should they address this concern?

Accepted Answer

Review the third-party model provider's data handling policy and choose a model that does not retain data. Each third-party model provider in Bedrock has its own data handling policies. Customers should review those policies and can choose models that do not store data or use Bedrock features like Guardrails to redact sensitive data. However, the direct action is to review the provider's policy and select a model that meets privacy requirements.

Answer

Enable Amazon Bedrock model invocation logging to capture all interactions

Answer

Use Amazon Macie to scan prompts before they are sent

Answer

Use AWS KMS to encrypt the prompts and responses before sending to the model

Question 9

A machine learning team wants to detect bias in a deployed model's predictions on new data. They use Amazon SageMaker. Which service should they use to generate bias reports after deployment?

Accepted Answer

Amazon SageMaker Clarify. Amazon SageMaker Clarify provides bias detection and explainability for ML models, both during training and after deployment. SageMaker Model Monitor detects data drift but not bias. SageMaker Debugger is for training debugging. SageMaker Role Manager is for managing IAM roles.

Answer

Amazon SageMaker Debugger

Answer

Amazon SageMaker Model Monitor

Answer

Amazon SageMaker Role Manager

Question 10

Which AWS service is used as a vector store in Amazon Bedrock Knowledge Bases for storing and retrieving embeddings?

Accepted Answer

Amazon OpenSearch Serverless. Amazon Bedrock Knowledge Bases support several vector stores, including Amazon OpenSearch Serverless, Aurora pgvector, Pinecone, and MongoDB Atlas. OpenSearch Serverless is the AWS-native managed solution frequently used for this purpose.

Answer

Amazon DynamoDB

Answer

Amazon Redshift

Answer

Amazon S3

Question 11

Which THREE considerations are essential for ensuring responsible AI in a model that predicts employee performance? (Choose 3)

Accepted Answer

Incorporate human review before final decisions. Option C is correct because responsible AI in an HR context requires human-in-the-loop oversight: performance predictions should inform, not automate, decisions about employees, so a human reviewer can catch errors and provide context before any action is taken. Option D is correct because employee performance data is personal and often sensitive, so the model must comply with privacy and consent requirements (e.g., GDPR lawful basis, purpose limitation, and data minimization) to be ethically and legally sound. Option E is correct because bias testing across demographic groups (e.g., comparing error rates, selection rates, and disparate impact metrics by gender, age, or ethnicity) is essential to detect and mitigate discriminatory outcomes in performance predictions. Option A does not belong because minimizing features to cut cost is an efficiency concern, not a responsible-AI safeguard, and can even harm fairness if it removes relevant variables. Option B does not belong because publishing individual employees' predicted performance publicly would violate privacy and confidentiality rather than constitute meaningful transparency, which is better served by model documentation and explainability to authorized stakeholders.

Answer

Minimize the number of features to reduce cost

Answer

Publish the model's predictions publicly for transparency

Question 12

Refer to the exhibit. A developer runs the CLI command to summarize text using Claude v2 in Bedrock. The output is shorter than expected. Which change should the developer make to allow a longer response?

Accepted Answer

Increase 'max_tokens_to_sample' to 1000. The 'max_tokens_to_sample' parameter in the Bedrock InvokeModel API directly controls the maximum number of tokens the model can generate in its response. By default, this value is often set low (e.g., 256 tokens), which truncates the output. Increasing it to 1000 allows Claude v2 to produce a longer summary up to that token limit.

Answer

Change the prompt to include 'Write a long summary'

Answer

Set 'stop_reason' to 'none'

Answer

Use a different region like us-west-2

Question 13

An e-commerce company uses an Amazon Lex chatbot to handle customer inquiries. They want to implement human oversight for sensitive interactions, such as when the chatbot cannot provide a confident response. Which AWS service should they integrate?

Accepted Answer

Amazon Augmented AI (A2I). Amazon Augmented AI (A2I) is the correct service because it provides built-in human review workflows for ML predictions, allowing you to route low-confidence responses from Amazon Lex to human reviewers for oversight. This directly addresses the requirement for human oversight on sensitive interactions where the chatbot cannot provide a confident response.

Answer

Amazon Rekognition

Answer

Amazon Comprehend

Answer

Amazon SageMaker Ground Truth

Question 14

A large enterprise has multiple teams deploying ML models on AWS. To ensure governance and accountability, they need to enforce that all models pass a fairness review before production deployment. Which SageMaker feature should they use to implement this approval workflow?

Accepted Answer

SageMaker Model Registry. SageMaker Model Registry is the correct choice because it provides a centralized catalog for managing ML models, including versioning, approval status, and metadata. It supports approval workflows by allowing you to define model groups, set approval statuses (e.g., PendingApproval, Approved, Rejected), and integrate with CI/CD pipelines to enforce that only approved models are deployed to production.

Answer

SageMaker Studio

Answer

SageMaker Experiments

Answer

SageMaker Model Monitor

Question 15

Which AWS service can be used to extract text and data from scanned documents such as invoices and receipts?

Accepted Answer

Amazon Textract. Amazon Textract is a machine learning service specifically designed to extract text, handwriting, and structured data (like tables and forms) from scanned documents. It goes beyond simple optical character recognition (OCR) by using ML models to understand the layout and relationships within documents, making it ideal for processing invoices and receipts.

Answer

Amazon Transcribe

Answer

Amazon Comprehend

Answer

Amazon Rekognition

Question 16

A media company uses Amazon Transcribe for automatic speech recognition. They discover the model has higher error rates for non-native English speakers. Which Responsible AI principle are they failing to uphold?

Accepted Answer

Fairness. The model's higher error rates for non-native English speakers indicate a bias in the training data or model design that leads to disparate performance across demographic groups. This directly violates the Fairness principle of Responsible AI, which requires that AI systems treat all groups equitably and do not amplify existing societal biases. Amazon Transcribe's underlying acoustic and language models may have been trained predominantly on native English speech, causing systematic underperformance for non-native accents.

Answer

Explainability

Answer

Robustness

Answer

Privacy

Question 17

A team is using Amazon Bedrock to generate images from text prompts. The generated images often contain artifacts and do not match the prompt description. Which combination of steps should the team take to improve image quality?

Accepted Answer

Refine the prompt with more descriptive language and adjust the CFG scale and inference steps.. Refining the prompt with more descriptive language helps the model better interpret the user's intent, while adjusting the CFG (Classifier-Free Guidance) scale controls how strictly the model adheres to the prompt, and increasing inference steps allows the diffusion process to produce higher-quality, artifact-free images. These are standard hyperparameters in diffusion-based image generation models on Amazon Bedrock, directly addressing both artifacts and prompt mismatch.

Answer

Fine-tune the model using SageMaker Ground Truth and increase the training epochs.

Answer

Increase the max token count and use a larger model variant.

Answer

Use a different foundation model and increase the image resolution.

Question 18

A company uses Amazon Bedrock to generate product descriptions. They notice that the output sometimes contains incorrect information. What should they do to improve accuracy?

Accepted Answer

Implement Retrieval-Augmented Generation (RAG).. Retrieval-Augmented Generation (RAG) enhances the accuracy of foundation model outputs by grounding the generation in authoritative, up-to-date external knowledge sources. Instead of relying solely on the model's parametric memory, RAG retrieves relevant documents or data from a vector database (e.g., Amazon OpenSearch Serverless) and injects them into the prompt context, reducing hallucinations and incorrect information in product descriptions.

Answer

Increase the temperature parameter.

Answer

Use a larger foundation model.

Answer

Use AWS WAF to filter outputs.

Question 19

Which of the following correctly describes the purpose of pre-training in the context of large language models?

Accepted Answer

To learn general language representations from a large, unlabeled corpus. Pre-training in large language models involves learning general language representations from a large, unlabeled corpus using self-supervised objectives like masked language modeling or next-token prediction. This phase builds a foundational understanding of syntax, semantics, and world knowledge without task-specific labels, which can later be fine-tuned for downstream tasks.

Answer

To evaluate the model's performance on a benchmark dataset

Answer

To reduce the model's size for deployment on edge devices

Answer

To adapt the model to a specific downstream task using labeled data

Question 20

A company uses Amazon SageMaker Pipelines for MLOps. The security team requires that all pipeline steps use only approved Docker images from a private Amazon ECR repository, and that all pipeline artifacts are encrypted with a customer managed KMS key. Which THREE steps must the company configure to meet these requirements? (Choose three.)

Accepted Answer

Specify a KMS key in the pipeline definition for encrypting output artifacts.. Amazon SageMaker Pipelines allows you to specify a KMS key in the pipeline definition to encrypt output artifacts at rest. This ensures that all artifacts generated by pipeline steps are encrypted using a customer managed KMS key, meeting the security team's encryption requirement.

Answer

Set an ECR lifecycle policy to delete untagged images older than 30 days.

Answer

Enable AWS Config rules to check for public ECR repositories.