Hard Difficulty Questions

The request payload size is too large.

Why it fails: Large payloads increase latency overall but are not specific to OverheadLatency.

The SageMaker endpoint is not in the same VPC as the client.

Why it fails: Network latency would affect the overall latency but not specifically OverheadLatency.

The model inference code is inefficient.

Why it fails: Inefficient inference code would increase ModelLatency, not OverheadLatency.

Ridge regression

Why it fails: Ridge regression applies an L2 penalty that shrinks coefficients toward zero but never sets them exactly to zero, so no features are eliminated. It is tempting because it is regularised linear regression, and it is correct when handling multicollinearity, not when you need sparse feature selection.

Recursive Feature Elimination (RFE)

Why it fails: RFE repeatedly fits a model and prunes the weakest features by ranking, but its penalty does not shrink coefficients to exactly zero during training. It is tempting because it does perform feature selection, and it is correct when you want wrapper-based ranking with a chosen estimator, not embedded regularisation.

Principal Component Analysis (PCA)

Why it fails: PCA transforms features into uncorrelated principal components, producing new axes rather than selecting original features or shrinking coefficients to zero. It is tempting because it reduces dimensionality, and it is correct for compression or decorrelation, not for identifying which original features to keep in a linear model.

Use AWS Glue to convert all timestamps to UTC, apply a mapping function to correct merchant category misspellings to a standard list, and drop records with missing fraud_label.

Why it fails: Dropping 5% of records may lose important fraud cases and introduce bias.

Use AWS Glue to convert timestamps to UTC, use a fuzzy matching algorithm to standardize merchant categories, and replace missing fraud_label with the mean value (0.05).

Why it fails: Mean imputation on a binary variable produces non-integer values, which are invalid for classification.

Use AWS Glue to convert timestamps to UTC, correct merchant categories by mapping known misspellings to correct names, and drop records with missing fraud_label.

Why it fails: Only mapping known misspellings may miss variants, and dropping missing labels causes data loss.

Apply data augmentation to the majority class by adding noise.

Why it fails: Data augmentation is not standard for tabular data and may introduce noise.

Use a weighted loss function during training to penalize misclassifications of the minority class.

Why it fails: Weighted loss is a training technique, not a data preparation step.

Apply random under-sampling to reduce the majority class to match the minority class size.

Why it fails: Under-sampling discards data and may lose important patterns.

Apply MinMaxScaler to the target variable

Why it fails: Scaling the target does not detect or mitigate bias; it only changes the scale.

Apply one-hot encoding to all categorical features

Why it fails: One-hot encoding is for categorical variables, not for bias detection.

Remove all features with a correlation above 0.8

Why it fails: High correlation may cause multicollinearity but does not directly address bias.

Drop the 'merchant_id' column to avoid overfitting

Why it fails: Dropping merchant_id discards the predictive signal it carries, since merchant behaviour is a strong fraud indicator; overfitting should instead be handled by frequency encoding, grouping rare merchants, or regularisation. Dropping is tempting when cardinality is extreme, and would suit an identifier with no predictive value.

Apply label encoding and treat it as a numeric feature

Why it fails: Label encoding imposes an arbitrary ordinal relationship on merchant IDs, so the model treats ID 40,000 as numerically greater than ID 200, fabricating signal that does not exist. It suits genuinely ordered categories such as severity ratings, not nominal identifiers.

One-hot encode the 'merchant_id' column

Why it fails: One-hot encoding creates 50,000 sparse binary columns, so rare merchants get their own weights and the model memorises them, worsening overfitting. It suits low-cardinality categoricals where every level has ample observations, not this high-cardinality column.

Disable the existing endpoint to prevent stale predictions during retraining

Why it fails: Disabling the endpoint halts serving entirely, so the solution cannot detect drift on live traffic or retrain automatically; the endpoint must stay available while monitoring runs. It is tempting because pausing stale predictions sounds prudent, and it would be correct if the requirement were preventing outdated inference during a planned outage.

Manually compare the model's predictions against a holdout validation set each week

Why it fails: Weekly manual comparison is neither automated drift detection nor automatic retraining, and it cannot scale or trigger remediation. It is tempting because holdout evaluation genuinely measures model quality, and it would be correct for periodic offline validation rather than the continuous monitored pipeline the scenario demands.

Create a new pipeline version for each run.

Why it fails: Creating new versions does not help with reprocessing a single step.

Use SageMaker Model Monitor to detect drift and trigger retraining.

Why it fails: Model Monitor is for monitoring inference quality, not for pipeline step recovery.

Manually rerun the pipeline with updated parameters.

Why it fails: Manual rerun would repeat all steps, wasting time and resources.

Configure the pipeline to run on a schedule instead of on-demand

Why it fails: Scheduling changes only the trigger, not the execution logic; the pipeline still runs every step regardless of whether source data changed. It is tempting because EventBridge schedules suit time-driven recurring jobs, but here the requirement is data-driven caching via step-level caching configuration.

Use the Parameter step to pass previous execution ID

Why it fails: Passing a previous execution ID as a parameter merely supplies a value to the pipeline; it does not skip steps whose inputs are unchanged. This suits traceability or lineage tracking, whereas the requirement is caching configuration that reuses prior step outputs when inputs match.

Use Lambda step to check data changes before running

Why it fails: Using a Lambda step introduces custom logic outside SageMaker Pipelines' native capabilities for detecting input data changes. While Lambda is excellent for bespoke pre-processing, external API calls, or complex conditional logic, SageMaker Pipelines possess built-in mechanisms to cache step outputs and skip re-execution if upstream data or code remains unchanged. Implementing a manual check via Lambda would bypass this inherent optimisation, adding unnecessary complexity and cost. It would be suitable for checks against systems not natively integrated or for highly specific, non-standard conditions.

Use a KMS key for data encryption

Why it fails: A KMS key encrypts data at rest, such as volumes and artefacts, not traffic between containers during a training job. It is the correct choice for at-rest encryption of training data and output models, but inter-container traffic encryption needs the encrypt inter-container traffic setting enabled.

Configure the training job in VPC-only mode

Why it fails: VPC-only mode restricts network routing to a customer VPC, controlling where traffic flows rather than encrypting it between containers. It suits isolating training jobs within private networking, but the requirement for encrypted inter-container transmission is met by enabling the encrypt inter-container traffic setting.

Enable network isolation mode

Why it fails: Network isolation blocks all outbound network traffic from containers, including inter-container communication, rather than encrypting it; it suits preventing containers from reaching the internet or other resources. Inter-container traffic encryption requires enabling the encrypt inter-container traffic setting, which applies TLS to that channel.

Take a random sample of 10% of the rows from the entire dataset.

Why it fails: Random sampling does not prioritize recent data; distribution may still shift.

Use all historical data and let the model learn the temporal patterns.

Why it fails: Including old data can bias the model toward past behavior that is no longer relevant.

Downsample older data exponentially so that recent data is overrepresented.

Why it fails: This still includes some old data and may not fully capture recent shifts.

Use SageMaker Training steps with checkpointing

Why it fails: Training-step checkpointing saves model state during training so interrupted jobs resume, not preprocessed data reuse across executions. It is tempting because checkpointing genuinely suits long training runs recovering from failures, but the stem concerns skipping an unchanged preprocessing step, which pipeline step caching handles.

Use SageMaker Feature Store to store the preprocessed features

Why it fails: Feature Store persists engineered features for online and offline serving, but it does not automatically skip a pipeline's preprocessing step when source data is unchanged. It is tempting because Feature Store genuinely suits feature reuse across models and teams, yet the requirement is execution-level step caching, configured on the pipeline step itself.

Use SageMaker Data Wrangler for the preprocessing

Why it fails: Data Wrangler transforms data interactively and exports flows, but it does not cache step outputs across pipeline executions, so preprocessing reruns each time. It is tempting because Data Wrangler genuinely suits exploratory feature preparation, yet the requirement is step-level caching keyed on unchanged inputs, which SageMaker Pipelines caching configuration provides.

Provide the same KMS key for both model artifacts and inference data.

Why it fails: Using the same key is an option but not required; different keys can be used with proper permissions.

Use a customer-managed key (CMK) with the same key material.

Why it fails: Key material is irrelevant; permissions are key.

Configure the endpoint to use SSE-S3 instead of SSE-KMS.

Why it fails: Changing encryption type would require re-uploading artifacts.

Manually update the endpoint to point to the previous model version

Why it fails: Manual rollback requires a human to detect degradation and intervene, so it cannot satisfy the automatic requirement. It is tempting because updating an endpoint to a previous model version is a genuine rollback action, and it would be the right choice when an operator deliberately reverts after investigating an incident.

Create multiple endpoints and use Amazon Route 53 weighted routing to shift traffic

Why it fails: Route 53 weighted routing shifts traffic between separate endpoints but provides no health evaluation of model performance, so no automatic rollback triggers. It is tempting because weighted DNS routing genuinely supports gradual canary traffic shifting, and it would be correct when distributing load across independent, already-validated endpoints.

Use AWS CodeDeploy with Amazon EC2 instances behind an Elastic Load Balancer

Why it fails: CodeDeploy with EC2 behind a load balancer manages instance deployments, not SageMaker endpoint variant weights, so it cannot roll back the model. It is tempting because CodeDeploy does provide automatic rollback on CloudWatch alarms, and it would be correct for blue/green deployments of application code on EC2.

SageMaker Multi-Model Endpoint

Why it fails: Multi-Model Endpoint hosts many models behind one endpoint to cut hosting cost, and it lacks native traffic-splitting for canary rollouts. The option tempts because it serves multiple models concurrently, but the requirement here is a production variant with weighted traffic distribution, which is the production variant feature.

SageMaker Batch Transform

Why it fails: Batch Transform runs offline scoring jobs against stored datasets and provides no real-time endpoint, no multi-AZ hosting and no traffic splitting. The option tempts because it processes fraud data in bulk, but the scenario demands low-latency inference with canary deployments, which requires a real-time endpoint.

SageMaker Serverless Inference

Why it fails: Serverless Inference scales to zero and cannot span multiple Availability Zones, so it fails the high-availability requirement and offers no canary traffic splitting. The option tempts because it scales automatically with request volume, but that scaling is cold-start based and single-AZ only.

Accuracy

Why it fails: Accuracy is misleading on imbalanced data because predicting the majority class yields 99.9% while missing nearly all fraud. Recall, precision or F1 would be correct here. Accuracy suits balanced datasets where class distribution is roughly equal.

Precision

Why it fails: Precision measures how many flagged transactions are genuinely fraudulent, so a model catching 5% of fraud can still post high precision by flagging only obvious cases. It is tempting because precision matters when investigation capacity is limited; it would be correct if the priority were minimising false alarms rather than catching fraud.

F1-score

Why it fails: F1-score blends precision and recall into one number, so a model catching only 5% of fraud still scores poorly, but it obscures which dimension is failing. It is tempting because F1 balances both concerns on imbalanced data; it would suit a scenario where false positives and false negatives carry comparable cost.

Undersample the majority class to create a balanced dataset, then split.

Why it fails: Undersampling discards 99% of majority-class rows, destroying the signal needed to model the 1:100 ratio and leaving too few examples for 30 features. It is tempting because balanced classes suit accuracy metrics, and undersampling is valid when the majority class is genuinely redundant or tiny datasets make resampling costly.

Oversample the minority class using SMOTE on the entire dataset before splitting into train/validation sets.

Why it fails: SMOTE synthesises minority points before splitting, so interpolated neighbours straddle train and validation folds, leaking information and inflating validation scores. It is tempting because SMOTE directly rebalances classes, and it is correct when applied only to the training fold after the split.

Randomly oversample the minority class by duplicating rows, then perform stratified train/test split.

Why it fails: Duplicating minority rows before splitting lets identical copies land in both train and test sets, leaking information and inflating test metrics. Oversampling is valid, but must occur after the split, or via SMOTE on training folds only. Class weighting in XGBoost avoids duplication entirely.

The container entrypoint is missing the full path to the script

Why it fails: This would cause a startup error, not a timeout.

The S3Input S3CompressionType is set to "None" but the file is compressed

Why it fails: This would cause parsing errors, not timeout.

The IAM role does not have permission to write to the output bucket

Why it fails: Permission errors would appear before processing starts.

Convert all timestamps to UTC in the ETL script using Spark's from_utc_timestamp

Why it fails: from_utc_timestamp converts from UTC, not to UTC; the function name is misleading and can cause incorrect conversions.

Use AWS Glue's built-in transform to parse timestamps with timezone offsets

Why it fails: While Glue can parse timestamps, it does not automatically normalize to a common timezone for consistent hour extraction.

Use Python's datetime.strptime with tzlocal

Why it fails: tzlocal uses the system time zone, which is not reliable for multiple time zones.

Hash encoding using Apache Spark on Amazon EMR

Why it fails: Hash encoding may cause collisions and lose interpretability, though scalable.

One-hot encoding using SageMaker Processing with scikit-learn

Why it fails: One-hot encoding with 1000+ categories produces a very wide dataset, inefficient for training.

Label encoding using Pandas in a SageMaker notebook

Why it fails: Label encoding imposes arbitrary ordinal relationships, misleading tree-based models.