Operational Efficiency And Optimization practice questions

S3 Standard

Why it fails: Standard is the baseline, but IA is cheaper for rarely accessed artifacts.

S3 Glacier Deep Archive

Why it fails: Glacier is for long-term archival, which may be too slow for active development.

S3 Intelligent-Tiering

Why it fails: While good, IA is the specific answer for known access patterns.

RequestCount

Why it fails: This is for API metrics, not internal training resource usage.

ModelLatency

Why it fails: This is for inference endpoints, not training jobs.

Invocations

Why it fails: Invocations are specific to endpoint health.

g5 instances

Why it fails: G5s are good, but inf2 instances are more cost-optimized for inference.

m5 instances

Why it fails: General-purpose instances lack the high memory bandwidth needed for LLMs.

p4d instances

Why it fails: P4d instances are for training, not cost-optimized for inference.

Reduce the index replication factor

Why it fails: This affects availability, not ingestion performance.

Disable vector search

Why it fails: This defeats the purpose of the application.

Increase the number of replica shards

Why it fails: Replica shards add overhead to index writes.

Moving model artifacts to S3 Standard

Why it fails: Storage location does not impact latency once deployed.

Increasing the number of replicas

Why it fails: Replicas improve throughput, not individual request latency.

Increase the embedding model dimension to improve semantic fidelity of stored vectors.

Why it fails: Higher-dimensional embeddings enlarge the index, raise memory pressure in OpenSearch Serverless, and slow approximate nearest neighbor search. They also increase storage and OCU consumption. The scenario is about reducing cost and latency, not improving recall, so this change moves every relevant metric in the wrong direction and requires a full re-index of the corpus.

Enable hybrid search on the knowledge base to combine keyword and vector retrieval.

Why it fails: Hybrid search can improve recall by blending lexical and semantic signals, but it adds a second retrieval pass and score fusion, which increases latency and compute rather than reducing them. It is a quality lever, not an efficiency lever, and the scenario already assumes acceptable faithfulness. Enabling it here would work against both the latency and cost goals the team is trying to hit.

Switch the vector store to a larger OpenSearch Serverless OCU floor to absorb spikes.

Why it fails: Raising the minimum OCU allocation increases baseline spend and does not reduce per-query latency for steady-state traffic. It only helps with burst absorption, which is not the described symptom. Because cost is explicitly a concern, paying for idle capacity contradicts the goal and leaves the retrieval pipeline's inefficiencies untouched.

Purchase provisioned throughput for 24/7 capacity and use it for both peak and off-peak hours.

Why it fails: Provisioned throughput is billed continuously, so using it 24/7 would incur cost even when the workload is near zero at night. This is wasteful for a workload with predictable idle periods. While it guarantees capacity, it does not minimize cost. The scenario asks to minimize cost while handling peaks, so this is not optimal.

Use provisioned throughput only during peak hours by scheduling it, and on-demand during off-peak.

Why it fails: Amazon Bedrock provisioned throughput is purchased in fixed terms (e.g., one month or longer) and cannot be scheduled on and off hourly. This option assumes a scheduling flexibility that does not exist. Therefore, it is not a valid or cost-effective approach for the described workload.

Use on-demand throughput during peak hours and rely on the default quotas, and do nothing during off-peak hours.

Why it fails: Relying solely on on-demand throughput with default quotas may lead to throttling during peak hours if the workload exceeds the quota. Doing nothing during off-peak hours is fine because on-demand has no idle cost, but the lack of quota planning during peak could cause failures. This option does not proactively manage throughput for predictable peaks, so it is not the best combination.

Switch to a smaller model to reduce the cost per invocation.

Why it fails: Switching to a smaller model may reduce per-invocation cost, but it does not reduce the number of invocations. It could also degrade summary quality. The scenario asks to reduce the number of model invocations, so this option does not meet the primary requirement. It might be a separate optimization but is not one of the two best actions here.

Increase the temperature to generate more diverse summaries and avoid redundancy.

Why it fails: Increasing temperature increases randomness, which can lead to less accurate and less consistent summaries. It does not reduce the number of model invocations; it only changes the output variability. For summarization, higher temperature is generally undesirable because it can introduce hallucinations. This option does not address the goal of reducing invocations.

Batch multiple tickets into a single Amazon Bedrock invocation to generate multiple summaries at once.

Why it fails: Batching can reduce the number of API calls, but it does not reduce the total number of model invocations in the sense of processing each ticket; it combines them into one call, which increases prompt size and complexity. For similar tickets, semantic caching is more effective because it avoids processing altogether. Batching may also hit context limits and is not the best fit for this scenario.

Batch Transform

Why it fails: Batch Transform processes a finite dataset as an offline job, so it cannot serve unpredictable live traffic during idle periods. It is tempting because Batch Transform is the correct, cost-effective choice for large offline inference jobs where no real-time endpoint is required.

Real-time inference with provisioned Auto Scaling instances

Why it fails: Provisioned instances bill continuously whether or not requests arrive, so idle periods still incur full instance-hour charges — the opposite of the cost goal. Auto Scaling reacts too slowly for spiky, unpredictable traffic. It suits steady, predictable workloads needing consistent low-latency responses.

Asynchronous Inference

Why it fails: Asynchronous Inference queues requests and returns results via Amazon S3, targeting large payloads and long processing times rather than idle-period cost. Queued requests still occupy provisioned instances, so idle periods bill identically. It would be correct for near-real-time workloads with payloads up to 1 GB.

SageMaker Cost Explorer

Why it fails: SageMaker Cost Explorer does not exist as a distinct feature offering threshold notifications; cost visibility comes from AWS Cost Explorer, which reports spend but sends no alerts. It is tempting because it sounds like the billing console, yet budget breach notifications require AWS Budgets with alert thresholds.

CloudWatch Alarms on billing metrics

Why it fails: CloudWatch billing alarms track the estimated AWS charges metric, which is account-wide and lags by hours; they cannot scope to a single SageMaker endpoint's monthly spend. They are tempting for generic cost alerting, but AWS Budgets supports service- and tag-filtered thresholds with notifications.

AWS Cost and Usage Reports

Why it fails: Cost and Usage Reports deliver granular billing data to S3 for analysis; they publish on a schedule and trigger no notifications when a threshold is crossed. It is tempting because it gives the most detailed spend data, but it is intended for cost analysis and forecasting pipelines, not proactive budget alerting.

Using spot instances for all endpoints

Why it fails: Spot instances suit training jobs and managed spot training, not real-time inference endpoints, which cannot tolerate interruption; SageMaker endpoints do not run on spot capacity. It tempts because spot pricing cuts compute cost dramatically, which is correct for interruptible training workloads.

Deactivating the endpoint manually

Why it fails: Manual deactivation requires human intervention, so it cannot deliver the automatic off-peak cost reduction the scenario demands. It is tempting because deleting or stopping endpoints genuinely eliminates compute charges, and would suit one-off teardown of unused endpoints rather than recurring scheduled scaling.

Updating the endpoint configuration

Why it fails: Updating endpoint configuration changes instance type or count but does not schedule automatic off-peak scaling; that requires Application Auto Scaling with scheduled policies. It tempts because manual right-sizing reduces cost, and would be correct for a one-off resize rather than recurring off-peak reduction.

AWS CloudTrail

Why it fails: CloudTrail records API activity for auditing — who invoked Bedrock and when — not performance telemetry such as ModelInvocationLatency. CloudTrail would be correct when investigating which principal made a call, or when reconstructing changes to Bedrock configuration for compliance purposes.

AWS X-Ray

Why it fails: X-Ray traces request paths across distributed services, revealing where latency accrues, but Bedrock publishes ModelInvocationLatency as a CloudWatch metric, and X-Ray does not surface that metric directly. X-Ray would be correct for pinpointing which downstream call within a traced request chain introduced delay.

Amazon OpenSearch Service

Why it fails: OpenSearch Service ingests and visualises logs and traces, but it does not natively receive Bedrock's ModelInvocationLatency metric stream; that telemetry flows to Amazon CloudWatch, which the stem's monitoring requirement targets. OpenSearch would be correct for searching application log data or building custom dashboards from exported logs.

Increasing the number of instances

Why it fails: Adding instances scales horizontal throughput, but each replica still stalls on the same per-GPU bottleneck, so single-job latency is unchanged. Low utilisation with high latency signals data-loading starvation or undersized batches. More instances suit distributed training that already saturates each GPU.

Switching to a larger instance type

Why it fails: A larger instance adds GPU memory and compute that the job is not using, so latency stays unchanged while cost rises. Low utilisation with high latency indicates the GPU is starved by input pipelines or small batches. Larger instances suit genuine out-of-memory errors or compute-bound training.

Moving the model to CPU instances

Why it fails: CPU instances remove GPU acceleration entirely, so training throughput collapses rather than improving. Low GPU utilisation with high latency points to data-loading or batch-size bottlenecks, addressed by tuning input pipelines or batch size. CPU instances suit small inference workloads where GPU cost is unjustified.

SageMaker Pipelines

Why it fails: Pipelines orchestrate training, processing and deployment steps as a workflow. They fit CI/CD automation of model building; runtime drift detection requires Model Monitor scheduling baseline comparisons against captured endpoint data, which Pipelines does not perform.

SageMaker Data Wrangler

Why it fails: Data Wrangler transforms and visualises tabular data for feature engineering before training. It fits preparing datasets, not production monitoring; drift observation requires Model Monitor comparing live endpoint data against a baseline, which Data Wrangler never ingests.

SageMaker Clarify

Why it fails: Clarify detects bias and explains feature attributions, plus bias drift, but not model performance decay against ground truth. It fits pre-deployment fairness auditing and explainability reporting; performance drift needs Model Monitor with baseline and endpoint data capture.

Increasing model accuracy

Why it fails: MME hosts many models behind one endpoint sharing compute, which affects hosting cost and resource use, not predictive quality. Accuracy depends on training data, features and algorithms. It is tempting because consolidating models feels like an improvement, but accuracy is unchanged; MME would be chosen to serve numerous models cost-effectively.

Lowering training time

Why it fails: MME changes inference hosting by loading models dynamically from Amazon S3 onto shared instances; training occurs before deployment and is unaffected. It is tempting because endpoint consolidation reduces operational overhead, but training duration depends on data volume, algorithm and compute. MME would be correct when the goal is reducing hosting cost across many models.

Enabling faster model deployment

Why it fails: MME reduces hosting cost by loading models on demand onto shared instances; it does not accelerate the deployment pipeline itself, since models must still be trained and uploaded to Amazon S3. It is tempting because fewer endpoints feel quicker to manage, but deployment speed is unchanged; MME suits serving many models economically.

Increasing the number of input tokens

Why it fails: Input tokens don't optimize retrieval performance.

Deleting the data source

Why it fails: This would break the application.

Disabling encryption

Why it fails: Security should not be disabled for performance.

Amazon S3

Why it fails: SageMaker container logs are written to Amazon CloudWatch Logs, not S3; the endpoint streams stdout/stderr there for real-time debugging. S3 is tempting because it stores model artefacts, training data and batch transform output, and is the correct choice when you need durable object storage rather than live log inspection.

Amazon DynamoDB

Why it fails: DynamoDB is a key-value NoSQL database for application data, and it holds no container stdout or stderr. It is tempting because it is a common AWS data store, and would be correct for persisting inference results, session state, or feature lookups in an application.

AWS CloudTrail

Why it fails: CloudTrail records AWS API activity and management events, not the stdout/stderr output written by an inference container. It is tempting because it is the default audit and logging service, and would be correct for tracing who called CreateEndpoint or InvokeEndpoint.

Increasing the instance count

Why it fails: Instance count doesn't directly solve cold starts for serverless.

Changing the container image

Why it fails: Container size affects cold starts, but Provisioned Concurrency is the direct feature.

Using a smaller model

Why it fails: Model size is not the only factor in cold starts.

Disabling versioning on the S3 bucket

Why it fails: Disabling S3 versioning removes prior object versions but leaves current pipeline artifacts accumulating, so storage costs persist. Versioning is tempting because it multiplies stored copies, yet lifecycle policies transitioning or expiring artifacts are what actually reduce ongoing storage spend.

Using smaller instance types

Why it fails: Smaller instance types reduce compute charges for training jobs, not the S3 storage consumed by pipeline artifacts. Instance sizing is tempting because it lowers overall run cost, but the question targets artifact storage, which instance selection does not affect.

Deleting logs every day

Why it fails: Deleting logs addresses CloudWatch log retention, not the S3 artifacts SageMaker Pipelines writes. Log deletion is tempting because logs are a visible recurring cost, but pipeline artifacts sit in S3 and require lifecycle or retention configuration instead.

CloudWatch Dashboards

Why it fails: CloudWatch Dashboards visualise operational metrics such as invocation counts and latency, not cost allocation across SageMaker projects. Cost Explorer, filtered by cost-allocation tags, is the service that centralises and breaks down spend. CloudWatch is tempting because it aggregates per-resource telemetry, but it carries no billing data.

SageMaker Studio Dashboard

Why it fails: The SageMaker Studio Dashboard shows resource usage and instance metrics within Studio, not consolidated billing across multiple projects. It is tempting because it is the native SageMaker monitoring surface, and it would be correct if the requirement were tracking notebook instance utilisation rather than centralised cost visualisation.

AWS Trusted Advisor

Why it fails: Trusted Advisor inspects account-wide best practise checks and cannot aggregate or visualise SageMaker project spend. It is tempting because it surfaces cost-optimisation recommendations, and it would be correct if the requirement were identifying idle resources or service quota warnings rather than centralised cost visualisation.