MLA-C01 Timed Hard Questions — 25 Difficult Questions

Question 1

A company's SageMaker real-time endpoint is experiencing high latency under load. The CloudWatch metrics show that the ModelLatency is acceptable, but the OverheadLatency is spiking. What is the most likely cause?

15 minute time limit — choose an answer to begin.

Accepted Answer

The endpoint is under-provisioned with insufficient instance count.. OverheadLatency measures the time spent on infrastructure overhead (e.g., request routing, network I/O, and container startup) rather than model inference. When the endpoint is under-provisioned with too few instances, requests queue up, causing the SageMaker front-end to wait for a free worker, which directly inflates OverheadLatency while ModelLatency (pure inference time) remains unaffected.

Answer

The request payload size is too large.

Answer

The SageMaker endpoint is not in the same VPC as the client.

Answer

The model inference code is inefficient.

Question 2

A data scientist observes that a linear regression model has many irrelevant features. They want to perform feature selection to improve generalization. Which method combines feature selection with model training using a penalty that can shrink coefficients to zero?

Accepted Answer

Lasso regression. Lasso regression (L1 regularization) adds a penalty equal to the absolute value of the coefficients, which can shrink some coefficients exactly to zero — effectively performing feature selection during training. This makes it the method that combines feature selection with model fitting in a single step.

Answer

Ridge regression

Answer

Recursive Feature Elimination (RFE)

Answer

Principal Component Analysis (PCA)

Question 3

A financial services company is building a fraud detection model using transactional data stored in Amazon S3. The data includes transaction_id, timestamp, amount, merchant_category, and fraud_label (0/1). The data is collected from multiple sources and has inconsistencies: timestamps are in different timezones (UTC and EST), merchant categories are sometimes misspelled (e.g., 'RESTAURANT', 'Restaurant', 'restaurant'), and the fraud_label is missing for about 5% of records. The data science team uses AWS Glue for ETL. They need to prepare a clean dataset for training. The final dataset must have consistent timestamps in UTC, standardized merchant categories, and no missing fraud labels. The team also wants to minimize data loss. Which set of actions should the team take?

Accepted Answer

Use AWS Glue to convert timestamps to UTC, use a mapping table to group similar merchant categories (e.g., all restaurant variants to 'Restaurant'), and impute missing fraud_label using mode (most frequent value).. It preserves data by imputing missing fraud labels using the mode (most frequent value), which is appropriate for a binary classification label where the majority class is likely 0. It also standardizes timestamps to UTC and uses a mapping table to group merchant category variants, ensuring consistency without data loss. Dropping records (as in A and C) would reduce the dataset size, and imputing with the mean (as in B) is invalid for a categorical label.

Answer

Use AWS Glue to convert all timestamps to UTC, apply a mapping function to correct merchant category misspellings to a standard list, and drop records with missing fraud_label.

Answer

Use AWS Glue to convert timestamps to UTC, use a fuzzy matching algorithm to standardize merchant categories, and replace missing fraud_label with the mean value (0.05).

Answer

Use AWS Glue to convert timestamps to UTC, correct merchant categories by mapping known misspellings to correct names, and drop records with missing fraud_label.

Question 4

A machine learning engineer is preparing a dataset for a binary classification model. The dataset has a severe class imbalance (95% class A, 5% class B). The engineer wants to use Amazon SageMaker to train the model. Which data preparation technique should the engineer apply to the training dataset to address the imbalance and improve model performance?

Accepted Answer

Apply Synthetic Minority Over-sampling Technique (SMOTE) to generate synthetic samples for the minority class.. SMOTE generates synthetic samples for the minority class by interpolating between existing minority instances, which directly addresses the severe class imbalance (95% class A, 5% class B) by creating a more balanced training dataset. This technique is particularly effective for tabular data in Amazon SageMaker, as it increases the representation of the minority class without simply duplicating existing samples, thereby reducing overfitting and improving the model's ability to learn decision boundaries for the minority class.

Answer

Apply data augmentation to the majority class by adding noise.

Answer

Use a weighted loss function during training to penalize misclassifications of the minority class.

Answer

Apply random under-sampling to reduce the majority class to match the minority class size.

Question 5

A machine learning engineer is using Amazon SageMaker Data Wrangler to prepare a dataset for a regression model. After applying a StandardScaler to numeric features, the target variable has a mean of 50 and standard deviation of 20. Which additional step should the engineer take to reduce model bias?

Accepted Answer

Use SageMaker Clarify to evaluate bias in the dataset. SageMaker Clarify is specifically designed to detect various types of bias in datasets and models, including regression bias. Even after standard scaling, the target variable's distribution (mean=50, std=20) may still contain systemic biases related to sensitive attributes (e.g., race, gender). SageMaker Clarify computes bias metrics such as Conditional Demographic Disparity in Labels (CDDL) and can identify whether the model's predictions are unfairly skewed across demographic groups, which is the direct step needed to reduce model bias.

Answer

Apply MinMaxScaler to the target variable

Answer

Apply one-hot encoding to all categorical features

Answer

Remove all features with a correlation above 0.8

Question 6

A company is building a fraud detection model on credit card transactions. The dataset contains a column 'merchant_id' with 50,000 unique values, many with low frequency. The team wants to avoid overfitting while preserving predictive signal. Which feature engineering approach is most appropriate?

Accepted Answer

Apply target encoding with smoothing based on global mean. Target encoding with smoothing replaces each category with a blend of the category's mean target value and the global mean, weighted by frequency. This preserves predictive signal from high-frequency merchants while reducing overfitting for low-frequency ones by shrinking their encoded values toward the global mean. It is a standard technique for high-cardinality categorical features.

Answer

Drop the 'merchant_id' column to avoid overfitting

Answer

Apply label encoding and treat it as a numeric feature

Answer

One-hot encode the 'merchant_id' column

Question 7

A healthcare company deploys a model to predict patient readmission risk. The model was trained on historical data and is now showing signs of concept drift. The team needs to implement a monitoring solution that can detect drift and automatically retrain the model when drift is detected. Which THREE steps should the team take to build this solution? (Choose THREE.)

Accepted Answer

Deploy SageMaker Model Monitor to track prediction quality over time. Option A is correct because SageMaker Model Monitor is the AWS service designed to continuously capture endpoint data and evaluate it against baselines, emitting CloudWatch metrics and alarms when data quality, model quality, bias, or drift violations occur, which is exactly what detecting concept drift requires. Option C is correct because Model Monitor's model quality monitoring depends on ground truth labels being joined to captured predictions; for readmission risk, those labels come from actual patient outcomes, so a labeling/feedback pipeline is essential to measure real drift. Option E is correct because the automated retraining requirement is met by triggering a SageMaker training job programmatically — for example, a Lambda function invoked by a CloudWatch alarm on the drift metric — which closes the detect-and-retrain loop. Option B is not appropriate because disabling the endpoint would halt predictions for clinicians and is unnecessary; retraining can occur while the existing endpoint continues serving, with a new model version deployed afterward. Option D is not appropriate because weekly manual comparison against a holdout set is neither automated nor a production drift-detection mechanism, and it would not satisfy the requirement to automatically retrain when drift is detected.

Answer

Disable the existing endpoint to prevent stale predictions during retraining

Answer

Manually compare the model's predictions against a holdout validation set each week

Question 8

An ML team uses SageMaker Pipelines to automate retraining. After a pipeline failure, they need to reprocess only the failed step without rerunning the entire pipeline. What should they do?

Accepted Answer

Use SageMaker Pipelines Cache with step-level caching.. SageMaker Pipelines Cache with step-level caching allows you to reuse outputs from previous successful runs of unchanged steps. When a pipeline fails, only the failed step and any downstream steps that depend on it need to be re-executed, because cached results from prior successful steps are automatically retrieved. This avoids rerunning the entire pipeline, saving time and compute resources.

Answer

Create a new pipeline version for each run.

Answer

Use SageMaker Model Monitor to detect drift and trigger retraining.

Answer

Manually rerun the pipeline with updated parameters.

Question 9

A data science team uses SageMaker Pipelines to orchestrate their ML workflow. They noticed that even when source data hasn't changed, the pipeline re-runs all steps, wasting compute time. What should they enable to avoid redundant runs?

Accepted Answer

Enable pipeline caching by setting the CacheConfig property for each step. SageMaker Pipelines supports step caching via the `CacheConfig` property. When enabled, the pipeline checks if the step's inputs (including source data, parameters, and code) have changed since the last successful run. If no changes are detected, the step is skipped and the previous output is reused, eliminating redundant compute.

Answer

Configure the pipeline to run on a schedule instead of on-demand

Answer

Use the Parameter step to pass previous execution ID

Answer

Use Lambda step to check data changes before running

Question 10

An organization needs to ensure that all data transmitted between containers in a SageMaker training job is encrypted. In the training job configuration, which setting should they enable?

Accepted Answer

Enable inter-container traffic encryption. SageMaker training jobs support inter-container traffic encryption, which ensures that data transmitted between containers (e.g., distributed training workers) is encrypted in transit. This setting uses TLS to protect the communication channel, meeting the organization's requirement for encrypted data transmission between containers.

Answer

Use a KMS key for data encryption

Answer

Configure the training job in VPC-only mode

Answer

Enable network isolation mode

Question 11

An e-commerce company uses Amazon SageMaker to train a model that predicts click-through rates. The training data includes a timestamp column 'click_time' and a categorical feature 'device_type' (8 values). They notice that the model's performance degrades over time because the data distribution shifts. They want to ensure the training data represents the most recent behavior. The data is stored in a daily partitioned S3 bucket (e.g., s3://bucket/data/2024-01-01/). The total dataset size is 500 GB. Which approach should they take to prepare the training data while minimizing bias and cost?

Accepted Answer

Select only the data from the last 30 days to train the model.. Selecting only the last 30 days of data directly addresses the data distribution shift by focusing on the most recent user behavior, which is critical for click-through rate prediction. This approach minimizes bias from outdated patterns and reduces training cost by using a smaller, relevant dataset (approximately 500 GB / 365 * 30 ≈ 41 GB). SageMaker training jobs benefit from this reduced volume through faster data loading and lower compute costs.

Answer

Take a random sample of 10% of the rows from the entire dataset.

Answer

Use all historical data and let the model learn the temporal patterns.

Answer

Downsample older data exponentially so that recent data is overrepresented.

Question 12

A data science team is using Amazon SageMaker Pipelines to orchestrate a multi-step workflow that includes data preprocessing, training, and model evaluation. They want to reuse the preprocessed data across multiple pipeline executions without re-running the preprocessing step if the source data hasn't changed. What should they configure?

Accepted Answer

Use SageMaker Processing steps with caching. SageMaker Processing steps support caching, which allows the pipeline to skip re-execution of the preprocessing step if the input data and pipeline parameters have not changed. This is achieved by configuring a `CacheConfig` with a caching key based on the input data source and step parameters, ensuring that the preprocessed data is reused across multiple pipeline executions without redundant computation.

Answer

Use SageMaker Training steps with checkpointing

Answer

Use SageMaker Feature Store to store the preprocessed features

Answer

Use SageMaker Data Wrangler for the preprocessing

Question 13

An ML team trained a model using SageMaker and stored the model artifacts in S3 with server-side encryption using AWS KMS (SSE-KMS). They need to deploy the model to a SageMaker endpoint that uses a different KMS key for inference data encryption. What must they do to ensure the endpoint can decrypt the model artifacts?

Accepted Answer

Grant the SageMaker execution role access to both KMS keys.. The SageMaker endpoint needs to decrypt the model artifacts stored with SSE-KMS using the original KMS key, and then re-encrypt the inference data with a different KMS key. The SageMaker execution role must have kms:Decrypt permission on the key used for the model artifacts and kms:Encrypt permission on the key used for inference data encryption. Without granting access to both keys, the endpoint cannot read the model artifacts or encrypt the output.

Answer

Provide the same KMS key for both model artifacts and inference data.

Answer

Use a customer-managed key (CMK) with the same key material.

Answer

Configure the endpoint to use SSE-S3 instead of SSE-KMS.

Question 14

A company deploys a machine learning model as a SageMaker real-time endpoint. They need to implement a mechanism to automatically roll back to the previous model version if performance degrades after a deployment. Which approach should they use?

Accepted Answer

Configure the SageMaker endpoint deployment with traffic shifting and set up CloudWatch alarms to trigger automatic rollback. SageMaker endpoints support deployment with traffic shifting (e.g., canary or linear patterns) via the 'DeploymentConfig' parameter, and you can attach CloudWatch alarms to the endpoint's variant metrics. If the alarm triggers (e.g., due to increased error rate or latency), SageMaker automatically rolls back the traffic to the previous model version, ensuring minimal manual intervention and fast recovery.

Answer

Manually update the endpoint to point to the previous model version

Answer

Create multiple endpoints and use Amazon Route 53 weighted routing to shift traffic

Answer

Use AWS CodeDeploy with Amazon EC2 instances behind an Elastic Load Balancer

Question 15

A financial services company needs to deploy a machine learning model for real-time fraud detection. The model must be highly available across multiple Availability Zones and must support automatic scaling based on request volume. The company also needs to perform canary deployments to test new model versions with a small percentage of traffic before full rollout. Which SageMaker feature should they use?

Accepted Answer

SageMaker real-time endpoint with production variants. SageMaker real-time endpoints with production variants enable canary deployments by routing a small percentage of traffic to a new model version while the majority goes to the current version. This feature also supports multi-AZ deployment for high availability and automatic scaling based on request volume via Application Auto Scaling, meeting all the stated requirements.

Answer

SageMaker Multi-Model Endpoint

Answer

SageMaker Batch Transform

Answer

SageMaker Serverless Inference

Question 16

A financial services firm is training a fraud detection model using SageMaker. The dataset is highly imbalanced (0.1% fraudulent transactions). The model currently achieves 99.9% accuracy but only catches 5% of fraud cases. Which metric should the team prioritize to evaluate model performance?

Accepted Answer

Recall. Recall is the proportion of actual fraud cases that the model correctly identifies, so it directly measures the model's ability to catch fraud. With only 5% of fraud cases caught, recall is extremely low, and improving it is the priority. Accuracy is misleading here because a model that predicts 'not fraud' for every transaction would still achieve 99.9% accuracy on a 0.1% fraud dataset.

Answer

Accuracy

Answer

Precision

Answer

F1-score

Question 17

A data scientist is training a binary classifier on a highly imbalanced dataset (1:100 class ratio). The dataset contains 500,000 rows and 30 features. The data is stored in S3 in Parquet format. The data scientist wants to use SageMaker's built-in XGBoost algorithm. Which data preparation technique should the data scientist apply to best address the class imbalance without causing data leakage?

Accepted Answer

Use the scale_pos_weight parameter in XGBoost to assign higher weight to the minority class.. The scale_pos_weight parameter in XGBoost directly adjusts the loss function to penalize misclassifications of the minority class more heavily, effectively handling class imbalance without modifying the dataset. This avoids data leakage because the weighting is applied during training only, not during preprocessing, and does not involve any synthetic data generation or resampling that could inadvertently expose test information.

Answer

Undersample the majority class to create a balanced dataset, then split.

Answer

Oversample the minority class using SMOTE on the entire dataset before splitting into train/validation sets.

Answer

Randomly oversample the minority class by duplicating rows, then perform stratified train/test split.

Question 18

Refer to the exhibit. A SageMaker Processing job configured as above fails with a timeout error. The input data is 100 GB of CSV files. The processing script performs standard data cleaning operations. What is the most likely cause?

Accepted Answer

The processing job does not have enough memory for the data volume. The SageMaker Processing job is configured with a single `ml.m5.large` instance, which has 8 GiB of memory. The input data is 100 GB of CSV files, and the processing script performs standard data cleaning operations that typically load the entire dataset into memory (e.g., using pandas). With only 8 GiB of RAM, the instance cannot hold 100 GB of data, causing the job to run out of memory and eventually fail with a timeout error as the OS kills the process or the job hangs.

Answer

The container entrypoint is missing the full path to the script

Answer

The S3Input S3CompressionType is set to "None" but the file is compressed

Answer

The IAM role does not have permission to write to the output bucket

Question 19

A data engineer is processing a large dataset in Amazon S3 with AWS Glue ETL. The dataset contains timestamps in multiple time zones. The engineer needs to create a feature for hour-of-day consistent across all records. Which approach ensures correctness?

Accepted Answer

Convert all timestamps to UTC during the ETL process, then extract hour. Converting all timestamps to UTC during the ETL process ensures a consistent time zone reference before extracting the hour-of-day feature. This avoids ambiguity from mixed time zones and aligns with best practices for machine learning feature engineering. AWS Glue ETL with Apache Spark provides built-in functions like `to_utc_timestamp()` to perform this conversion reliably.

Answer

Convert all timestamps to UTC in the ETL script using Spark's from_utc_timestamp

Answer

Use AWS Glue's built-in transform to parse timestamps with timezone offsets

Answer

Use Python's datetime.strptime with tzlocal

Question 20

A company is preparing a dataset with a categorical feature that has over 1000 unique values. They need to create features for a random forest model. Which feature engineering approach is most scalable and effective in AWS for high-cardinality categories?

Accepted Answer

Target encoding with smoothing using SageMaker Data Wrangler. Target encoding with smoothing in SageMaker Data Wrangler is the most scalable and effective approach because it replaces each high-cardinality category with the mean of the target variable, smoothed by a global prior to prevent overfitting. SageMaker Data Wrangler handles datasets with over 1000 unique values efficiently without exploding feature dimensions, unlike one-hot encoding, and avoids the ordinal bias of label encoding.

Answer

Hash encoding using Apache Spark on Amazon EMR

Answer

One-hot encoding using SageMaker Processing with scikit-learn

Answer

Label encoding using Pandas in a SageMaker notebook