Hard Difficulty Questions

Use SageMaker Batch Transform with multiple instances

Why wrong: Batch Transform is designed for offline, asynchronous inference on large datasets, not for real-time serving with sub-10 ms latency and 1000 requests per second; it lacks a persistent endpoint and cannot meet the latency or throughput requirements of this scenario. It is tempting because it can process high volumes cost-effectively for batch jobs, such as generating predictions on historical transaction logs overnight, where latency is irrelevant.

Deploy on a single ml.c5.xlarge instance with a real-time endpoint

Why wrong: A single instance may not handle 1000 req/s with low latency; also cost may be higher if overprovisioned.

Deploy separate real-time endpoints for each model on ml.m5.large instances

Why wrong: Separate endpoints increase cost and management overhead without performance benefit.

Pre-warm the endpoint by keeping a fixed number of additional instances

Why wrong: Pre-warming is manual and not cost-efficient.

Increase the scale-in cooldown period to avoid frequent downsizing

Why wrong: Increasing cooldown would delay scaling in but not help with scaling out during spikes.

Change the instance type to a larger one like ml.c5.xlarge to handle the spikes

Why wrong: Larger instances increase cost and may still not handle spikes without scaling out.

The output format 'parquet' is not supported by Glue

Why wrong: Parquet is supported.

The input partition path is incorrect because it includes the partition key

Why wrong: Including partition key in path is acceptable.

The schema contains a column with a reserved name

Why wrong: Error message does not indicate schema issue.

Manually update the endpoint to point to the previous model version

Why wrong: Manual rollback is not automatic and may take too long.

Create multiple endpoints and use Amazon Route 53 weighted routing to shift traffic

Why wrong: This adds complexity and does not natively support automatic rollback.

Use AWS CodeDeploy with Amazon EC2 instances behind an Elastic Load Balancer

Why wrong: CodeDeploy is for EC2/on-premises, not SageMaker endpoints.

Spigot transform

Why wrong: Spigot is used for writing sample data for debugging.

ResolveChoice transform

Why wrong: ResolveChoice handles data type ambiguity, not flattening.

ApplyMapping transform

Why wrong: ApplyMapping maps columns but does not flatten nested structures.

Increase the instance size

Why wrong: Larger instances may not address the root cause if CPU is not the bottleneck.

Disable auto-scaling and use a larger instance

Why wrong: This is a static solution; does not handle varying traffic efficiently.

Switch to GPU instances

Why wrong: GPU instances are for specialized workloads; not necessarily fixing scaling based on wrong metric.

The request payload size is too large.

Why wrong: Large payloads increase latency overall but are not specific to OverheadLatency.

The SageMaker endpoint is not in the same VPC as the client.

Why wrong: Network latency would affect the overall latency but not specifically OverheadLatency.

The model inference code is inefficient.

Why wrong: Inefficient inference code would increase ModelLatency, not OverheadLatency.

The role does not have permissions to write to the feature store.

Why wrong: While possible, the type mismatch is the direct cause.

The `OnlineStoreConfig` must include a `SecurityConfig`.

Why wrong: SecurityConfig is optional for online store.

The `EventTimeFeatureName` is incorrectly spelled.

Why wrong: Spelling error would cause creation failure, not ingestion.

Deploy the model on multiple endpoints behind an Application Load Balancer.

Why wrong: Adds complexity and latency.

Use scheduled scaling to increase the instance count during known peak hours.

Why wrong: Scheduled scaling cannot handle unexpected spikes.

Manually increase the instance count during peak hours.

Why wrong: Manual scaling is not automated.

Store IAM credentials in the container image as environment variables

Why wrong: Hardcoding credentials is insecure and violates best practices.

Attach an IAM instance profile to the underlying EC2 instance

Why wrong: SageMaker endpoints are managed services; you don't attach instance profiles directly.

Retrieve temporary credentials from AWS Secrets Manager within the container code

Why wrong: Secrets Manager is for secrets, but SageMaker can use execution role directly.

Label the dataset for fine-tuning

Why wrong: Labeling is needed only if fine-tuning; pre-trained models can be used directly.

Train the model from scratch on the company's data

Why wrong: JumpStart provides pre-trained models; fine-tuning is optional but not required for basic predictions.

Convert the model to ONNX format

Why wrong: ONNX conversion is not required; JumpStart models are already in a supported format.

Use AWS Glue to convert all timestamps to UTC, apply a mapping function to correct merchant category misspellings to a standard list, and drop records with missing fraud_label.

Why wrong: Dropping 5% of records may lose important fraud cases and introduce bias.

Use AWS Glue to convert timestamps to UTC, use a fuzzy matching algorithm to standardize merchant categories, and replace missing fraud_label with the mean value (0.05).

Why wrong: Mean imputation on a binary variable produces non-integer values, which are invalid for classification.

Use AWS Glue to convert timestamps to UTC, correct merchant categories by mapping known misspellings to correct names, and drop records with missing fraud_label.

Why wrong: Only mapping known misspellings may miss variants, and dropping missing labels causes data loss.

Use SageMaker Model Monitor to collect drift metrics, and have a data scientist manually analyze the metrics and trigger retraining via the SageMaker console

Why wrong: Manual process contradicts automatic requirement.

Schedule a daily SageMaker Pipeline that retrains the model and deploys it using a new endpoint, then updates the application to point to the new endpoint

Why wrong: Scheduled retraining doesn't react to drift; updating application endpoint may cause downtime if not careful.

Use SageMaker Model Monitor to publish drift metrics to Amazon CloudWatch, and create a CloudWatch alarm that triggers an AWS Lambda function to retrain and deploy the model

Why wrong: Lambda can trigger retraining, but deploying a new model with no downtime requires more orchestration; Lambda may not handle complex pipeline.

Use a NAT gateway to allow outbound traffic to the private PyPI mirror

Why wrong: NAT gateway provides internet access, which is not desired; it also requires internet routing.

Disable internet access for Studio and rely on SageMaker's default VPC configuration

Why wrong: Default VPC configuration may not have VPC endpoint to private PyPI; without it, packages cannot be reached.

Enable internet access for Studio and use a VPC endpoint to the private PyPI mirror

Why wrong: Enabling internet access would allow unrestricted internet, violating the restriction.

Apply data augmentation to the majority class by adding noise.

Why wrong: Data augmentation is not standard for tabular data and may introduce noise.

Use a weighted loss function during training to penalize misclassifications of the minority class.

Why wrong: Weighted loss is a training technique, not a data preparation step.

Apply random under-sampling to reduce the majority class to match the minority class size.

Why wrong: Under-sampling discards data and may lose important patterns.

Apply MinMaxScaler to the target variable

Why wrong: Scaling the target does not detect or mitigate bias; it only changes the scale.

Apply one-hot encoding to all categorical features

Why wrong: One-hot encoding is for categorical variables, not for bias detection.

Remove all features with a correlation above 0.8

Why wrong: High correlation may cause multicollinearity but does not directly address bias.

Concept drift; ignore the change as long as input distribution remains stable

Why wrong: Concept drift degrades model performance and must be addressed, not ignored.

Data drift; update the baseline statistics and continue monitoring

Why wrong: Updating baseline does not address concept drift; the model itself needs retraining.

Data drift; retrain the model with the latest training data

Why wrong: Data drift refers to change in input distribution P(x), which is not the case here.

Use AWS CodePipeline with a deployment action that uses AWS CloudFormation to update the endpoint. Add a manual approval step for rollback.

Why wrong: Manual rollback does not meet automatic rollback requirement, and CloudFormation adds overhead.

Create an AWS Lambda function that is triggered by Model Registry events, deploys the model to a staging endpoint, runs a canary test, and if successful, updates the production endpoint.

Why wrong: Lambda requires custom code for staging, canary, and promotion, adding complexity.

Use an Amazon EKS cluster with a custom inference container and use ArgoCD for automated deployments.

Why wrong: EKS and ArgoCD introduce significant complexity and deviate from SageMaker managed services.

Imputation

Why wrong: Imputation handles missing values, not dimensionality reduction.

Principal Component Analysis (PCA)

Why wrong: PCA reduces dimensionality but transforms features, losing interpretability.

StandardScaler

Why wrong: StandardScaler normalizes features but does not reduce dimensionality.