AIP-C01 Operational Efficiency And Optimization — Timed 15 Questions

Question 1

Your team is using Amazon SageMaker JumpStart to deploy a pre-trained model. To minimize the cost of storage for the model artifacts, which S3 storage class should you configure for the bucket?

Accepted Answer

S3 Standard-Infrequent Access. S3 Standard is generally used, but for long-term storage of model artifacts that are rarely accessed, S3 Standard-Infrequent Access is the correct cost-optimization choice.

Answer

S3 Standard

Answer

S3 Glacier Deep Archive

Answer

S3 Intelligent-Tiering

Question 2

To track the resource usage of a SageMaker training job, you should look at which CloudWatch metric?

Accepted Answer

GPUUtilization. GPUUtilization is the CloudWatch metric that reports the percentage of GPU compute capacity actively used by a SageMaker training job, making it the direct indicator of how heavily the training instance's accelerators are being consumed. It is emitted per-instance under the /aws/sagemaker/TrainingJobs namespace, so it answers the question of resource usage for GPU-backed training. The other listed metrics belong to real-time inference endpoints, not training jobs.

Answer

RequestCount

Answer

ModelLatency

Answer

Invocations

Question 3

You are optimizing a SageMaker inference endpoint for an LLM that requires high memory bandwidth. Which instance family should you choose to balance performance and cost?

Accepted Answer

inf2 instances. For an LLM requiring high memory bandwidth while balancing performance and cost, inf2 instances are the best choice. Inf2 instances are powered by AWS Inferentia2 chips, purpose-built for high-throughput, low-latency inference with large memory bandwidth and cost efficiency. They are optimized for generative AI inference, making them ideal for LLMs.

Answer

g5 instances

Answer

m5 instances

Answer

p4d instances

Question 4

Your Generative AI application uses a vector database on Amazon OpenSearch Service. You notice high CPU usage during ingestion. How can you optimize performance?

Accepted Answer

Use bulk requests with an optimized refresh interval. Increasing the number of dedicated primary shards or using a more performant instance type for the ingestion nodes are standard optimization patterns.

Answer

Reduce the index replication factor

Answer

Disable vector search

Answer

Increase the number of replica shards

Question 5

Which THREE techniques can improve the performance (latency) of an inference endpoint?

Accepted Answer

Model quantization. Model optimization, hardware-specific instance types, and batching are standard techniques for improving inference speed.

Answer

Moving model artifacts to S3 Standard

Answer

Increasing the number of replicas

Question 6

A team operates a Retrieval Augmented Generation assistant on Amazon Bedrock Knowledge Bases backed by Amazon OpenSearch Serverless. Latency has crept up and per-query cost is high. Which TWO changes most directly reduce retrieval cost and latency without degrading answer faithfulness? (Choose two.)

Accepted Answer

Cache frequent query embeddings and their retrieved passages in Amazon ElastiCache for Redis.. Retrieval cost and latency in RAG come from two places: the vector search itself and the number of tokens stuffed into the generation prompt. Lowering top-k shrinks both, and caching repeated query embeddings plus their retrieved passages eliminates redundant work entirely for common questions. Together they preserve grounding because the same relevant passages are still supplied, just fewer of them and served faster on repeats.

Answer

Increase the embedding model dimension to improve semantic fidelity of stored vectors.

Answer

Enable hybrid search on the knowledge base to combine keyword and vector retrieval.

Answer

Switch the vector store to a larger OpenSearch Serverless OCU floor to absorb spikes.

Question 7

You are deploying a generative AI application that uses Amazon Bedrock. The application's workload is predictable during business hours but drops to near zero at night. You want to minimize cost while maintaining the ability to serve requests during peak hours without throttling. Which combination of actions should you take?

Accepted Answer

Request a quota increase for on-demand throughput to cover peak hours, and use on-demand for all hours.. For a workload with predictable peaks and near-zero off-peak usage, requesting a higher on-demand quota ensures peak capacity without throttling, while on-demand pricing avoids charges during idle periods. This combination minimizes cost and maintains performance. Provisioned throughput would be more expensive because it is billed continuously, even when not used.

Answer

Purchase provisioned throughput for 24/7 capacity and use it for both peak and off-peak hours.

Answer

Use provisioned throughput only during peak hours by scheduling it, and on-demand during off-peak.

Answer

Use on-demand throughput during peak hours and rely on the default quotas, and do nothing during off-peak hours.

Question 8

You are optimizing a generative AI application that uses Amazon Bedrock. The application generates summaries of customer support tickets. You notice that many tickets are similar and the model often produces redundant summaries. You want to reduce the number of model invocations and improve operational efficiency. (Choose two.)

Accepted Answer

Implement a semantic caching layer that stores and retrieves summaries for similar ticket content.. Semantic caching avoids redundant model calls by reusing summaries for similar tickets, directly reducing invocations. Prompt caching reduces the cost and latency of the static instructions within each call, improving per-invocation efficiency. Together, these two optimizations address the goal of reducing redundant work and improving operational efficiency without changing the model or degrading output quality.

Answer

Switch to a smaller model to reduce the cost per invocation.

Answer

Increase the temperature to generate more diverse summaries and avoid redundancy.

Answer

Batch multiple tickets into a single Amazon Bedrock invocation to generate multiple summaries at once.

Question 9

You are deploying a foundation model on Amazon SageMaker. To optimize inference costs for an application with unpredictable traffic patterns and significant idle periods, which deployment option should you select?

Accepted Answer

SageMaker Serverless Inference. Serverless Inference is the most cost-effective solution for unpredictable traffic as it eliminates the cost of idle instances by automatically scaling resources based on request volume.

Answer

Batch Transform

Answer

Real-time inference with provisioned Auto Scaling instances

Answer

Asynchronous Inference

Question 10

You need to ensure that your SageMaker inference endpoint does not exceed a specific monthly budget. Which AWS feature allows you to set up notifications when spending thresholds are crossed?

Accepted Answer

AWS Budgets. AWS Budgets allows you to set custom cost and usage budgets and receive email or SNS alerts when thresholds are approached or exceeded.

Answer

SageMaker Cost Explorer

Answer

CloudWatch Alarms on billing metrics

Answer

AWS Cost and Usage Reports

Question 11

You have a fleet of SageMaker endpoints. You want to implement an automatic cost reduction strategy during off-peak hours for specific endpoints. Which approach is most efficient?

Accepted Answer

Configuring Application Auto Scaling with scheduled actions. Application Auto Scaling for SageMaker endpoints supports scheduled scaling actions that adjust the desired instance count based on a cron or rate expression. This allows you to automatically scale in (reduce instances) during off-peak hours and scale out during peak hours, directly reducing cost without manual intervention. It is the most efficient because it is automated, precise, and integrates natively with SageMaker endpoints.

Answer

Using spot instances for all endpoints

Answer

Deactivating the endpoint manually

Answer

Updating the endpoint configuration

Question 12

An application using Amazon Bedrock is experiencing high latency. You need to monitor the 'ModelInvocationLatency' metric to identify performance bottlenecks. Which AWS service provides the primary integration for this telemetry?

Accepted Answer

Amazon CloudWatch. Amazon CloudWatch is the primary monitoring service for AWS services including Amazon Bedrock. Bedrock automatically publishes metrics such as ModelInvocationLatency, InvocationCount, and InvocationThrottles to CloudWatch under the AWS/Bedrock namespace. You can create alarms, dashboards, and custom metrics from this telemetry to identify performance bottlenecks.

Answer

AWS CloudTrail

Answer

AWS X-Ray

Answer

Amazon OpenSearch Service

Question 13

You are tuning a Large Language Model on SageMaker and notice that GPU memory utilization is consistently low despite high latency. Which action is most appropriate to optimize performance?

Accepted Answer

Implementing dynamic batching in your inference handler. Dynamic batching groups multiple incoming inference requests into a single forward pass on the GPU, which raises GPU utilization and throughput while reducing per-request latency overhead. Low GPU utilization with high latency is the classic signature of under-batched inference where the accelerator sits idle between small requests. Implementing dynamic batching in the inference handler directly addresses this by keeping the GPU busy with concurrent work.

Answer

Increasing the number of instances

Answer

Switching to a larger instance type

Answer

Moving the model to CPU instances

Question 14

Which SageMaker feature allows you to observe model performance drifts over time?

Accepted Answer

SageMaker Model Monitor. SageMaker Model Monitor automatically detects data drift, concept drift, and model quality issues.

Answer

SageMaker Pipelines

Answer

SageMaker Data Wrangler

Answer

SageMaker Clarify

Question 15

You are deploying a model using SageMaker Multi-Model Endpoints (MME). What is the primary benefit of this strategy for operational efficiency?

Accepted Answer

Reducing the number of instances required. MME allows you to host multiple models on a single set of instances, reducing the total cost by sharing resources.

Answer

Increasing model accuracy

Answer

Lowering training time

Answer

Enabling faster model deployment