AIF-C01 Practice Test 6 — 10 Questions

Question 1

A data science team is evaluating the output quality of a text generation model. They want to use both automated metrics and human judgment. Which THREE approaches should they include in their evaluation strategy? (Select THREE.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Conduct human evaluation with a rating rubric. Option A is correct because human evaluation with a rating rubric directly captures subjective quality dimensions such as fluency, coherence, relevance, and factual consistency that automated metrics cannot fully assess, which is essential when the team explicitly wants human judgment. Option D is correct because ROUGE (recall-oriented n-gram/LCS overlap) and BLEU (precision-oriented n-gram overlap with brevity penalty) are standard automated reference-based metrics for comparing generated text against reference texts, matching the team's goal of using automated metrics. Option E is correct because running the model on a task-specific benchmark dataset provides standardized, objective, task-relevant performance measurement (e.g., accuracy, F1, exact match) that complements both human ratings and text-overlap metrics. Option B does not belong because inference latency under load measures operational performance and scalability, not output quality of the generated text. Option C does not belong because perplexity is typically computed on a language model's token likelihoods over a corpus and is not a reliable measure of the quality of generated text, especially for open-ended generation.

Answer

Measure inference latency under load

Answer

Compute perplexity on the generated text

Question 2

A company is using Amazon Bedrock to generate responses for customer support. They want to ensure that the model does not expose personally identifiable information (PII) in its outputs. Which AWS feature can be configured to automatically redact PII from model responses?

Accepted Answer

Amazon Bedrock Guardrails. Amazon Bedrock Guardrails is the correct choice because it provides configurable policies that can automatically detect and redact personally identifiable information (PII) from model inputs and outputs. This feature is specifically designed for Amazon Bedrock to enforce content safety and compliance requirements, including PII redaction, without requiring custom code or external services.

Answer

Amazon Macie

Answer

Amazon SageMaker Model Monitor

Answer

AWS CloudTrail

Question 3

Which TWO statements about the bias-variance tradeoff are correct? (Choose TWO.)

Accepted Answer

High bias typically leads to underfitting. Option C is correct because high bias means the model makes overly simplistic assumptions about the data, so it fails to capture the underlying pattern and systematically underfits both training and test data. Option E is correct because high variance means the model is overly sensitive to the particular training samples, fitting noise as well as signal, which produces excellent training performance but poor generalization—the hallmark of overfitting. Option A is wrong because high bias leads to underfitting, not overfitting; overfitting is associated with high variance. Option B is wrong because high variance models are highly sensitive to changes in training data, not insensitive. Option D is wrong because increasing model complexity typically increases variance (and decreases bias), not decreases variance.

Answer

High bias typically leads to overfitting

Answer

High variance models are insensitive to changes in training data

Answer

Increasing model complexity typically decreases variance

Question 4

A research organization is using Amazon SageMaker Studio to collaborate on building machine learning models. The security policy requires that all data and code remain within a VPC and cannot be accessed from the public internet. Additionally, the organization wants to enforce that only approved base images are used for the Studio environment. How should the organization configure SageMaker Studio to meet these requirements?

Accepted Answer

Configure the Studio domain in VPC Only mode and use a SageMaker Studio lifecycle configuration to restrict the list of available base images to those in a private Amazon ECR repository.. Configuring the SageMaker Studio domain in VPC Only mode ensures that all data and code remain within the VPC and cannot be accessed from the public internet. Additionally, using a lifecycle configuration script to restrict available base images to those in a private Amazon ECR repository enforces the policy that only approved base images are used, as the lifecycle configuration can modify the Jupyter Server settings to limit the image registry.

Answer

Use an AWS Transit Gateway to connect the VPC, and enforce HTTPS for all traffic.

Answer

Configure the Studio domain to disable direct internet access, and let users choose any base image from the SageMaker public registry.

Answer

Use AWS CloudFormation to create a Studio domain in a VPC and rely on individuals to use approved images.

Question 5

A company uses Amazon SageMaker to build and deploy models. They want to enforce compliance that all model endpoints are encrypted in transit and use least privilege access. Which THREE steps should they take? (Choose THREE.)

Accepted Answer

Configure the SageMaker endpoint to use a custom SSL certificate via AWS Certificate Manager. Configuring a SageMaker endpoint to use a custom SSL certificate from AWS Certificate Manager (ACM) ensures that all data transmitted between clients and the endpoint is encrypted in transit using TLS. This enforces the compliance requirement for encryption in transit by replacing the default SageMaker certificate with a customer-managed certificate, which can be validated and rotated as needed.

Answer

Enable AWS CloudTrail to log all endpoint invocations

Answer

Disable root access on the SageMaker notebook instances

Question 6

A company wants to use Amazon Bedrock to translate customer emails from English to Spanish. The emails contain occasional personal names and addresses. Which Guardrail configuration should be applied to protect customer privacy?

Accepted Answer

PII detection. PII detection in Bedrock Guardrails can identify and redact personally identifiable information such as names and addresses. Content filtering, topic denial, and grounding check do not specifically target PII.

Answer

Grounding check

Answer

Topic denial

Answer

Content filtering

Question 7

A financial services company uses Amazon SageMaker to train models with sensitive customer data. They must ensure that no data leaves a specific AWS Region due to data residency regulations. The training data is in S3. Which architecture meets this requirement while minimizing data transfer?

Accepted Answer

Use S3 VPC endpoints and place SageMaker training job in a private subnet with no internet access. Using a VPC with S3 VPC endpoints ensures data stays within the AWS network and does not traverse the internet. Data remains in the same Region because S3 endpoints are Regional.

Answer

Place SageMaker training job in a private subnet with a NAT gateway and route traffic through the internet

Answer

Configure S3 Transfer Acceleration and use a public SageMaker training job

Answer

Use AWS Glue to copy data to an EBS volume attached to the training instance

Question 8

A company wants to use a third-party foundation model from Amazon Bedrock but is concerned about data privacy because the model provider might store prompts and responses. How should they address this concern?

Accepted Answer

Review the third-party model provider's data handling policy and choose a model that does not retain data. Each third-party model provider in Bedrock has its own data handling policies. Customers should review those policies and can choose models that do not store data or use Bedrock features like Guardrails to redact sensitive data. However, the direct action is to review the provider's policy and select a model that meets privacy requirements.

Answer

Enable Amazon Bedrock model invocation logging to capture all interactions

Answer

Use Amazon Macie to scan prompts before they are sent

Answer

Use AWS KMS to encrypt the prompts and responses before sending to the model

Question 9

A machine learning team wants to detect bias in a deployed model's predictions on new data. They use Amazon SageMaker. Which service should they use to generate bias reports after deployment?

Accepted Answer

Amazon SageMaker Clarify. Amazon SageMaker Clarify provides bias detection and explainability for ML models, both during training and after deployment. SageMaker Model Monitor detects data drift but not bias. SageMaker Debugger is for training debugging. SageMaker Role Manager is for managing IAM roles.

Answer

Amazon SageMaker Debugger

Answer

Amazon SageMaker Model Monitor

Answer

Amazon SageMaker Role Manager

Question 10

Which AWS service is used as a vector store in Amazon Bedrock Knowledge Bases for storing and retrieving embeddings?

Accepted Answer

Amazon OpenSearch Serverless. Amazon Bedrock Knowledge Bases support several vector stores, including Amazon OpenSearch Serverless, Aurora pgvector, Pinecone, and MongoDB Atlas. OpenSearch Serverless is the AWS-native managed solution frequently used for this purpose.

Answer

Amazon DynamoDB

Answer

Amazon Redshift

Answer

Amazon S3