MLA-C01 Practice Exam 2 — 20 Questions

Question 1

A team receives alerts that their SageMaker endpoint latency has increased significantly. They check CloudWatch metrics and see Invocations rising, but ModelLatency remains stable. Which metric should they investigate to find the source of the increased latency?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

OverheadLatency. OverheadLatency measures the time taken by the SageMaker infrastructure to handle requests before and after model inference, including request routing, authentication, and response processing. Since ModelLatency is stable but total endpoint latency has increased, the extra time must be in the overhead component, making OverheadLatency the correct metric to investigate.

Answer

ModelLatency

Answer

5XXError

Answer

4XXError

Question 2

An ML team is deploying a model using SageMaker. The model requires GPU inference and must be available in multiple AWS regions for low latency. The team has created a multi-model endpoint with GPU instances. After deployment, they notice high latency spikes when a new model is loaded. What is the most likely cause?

Accepted Answer

The team is using a multi-model endpoint, which loads models on demand; loading a model into GPU memory causes latency spikes.. A multi-model endpoint (MME) loads models on demand from Amazon S3 into the instance's memory. When a new model is requested and not already cached, SageMaker must download the model artifacts and load them into GPU memory, which is a time-consuming operation that causes a latency spike for the first inference request. This cold-start behavior is inherent to MMEs and explains the observed spikes.

Answer

The endpoint is configured with a single production variant, causing all traffic to overload one instance.

Answer

The endpoint is using the wrong instance type that lacks sufficient GPU memory.

Answer

The model is too large for the specified container memory, causing swap to disk.

Question 3

A company uses SageMaker endpoints with auto-scaling based on CPU utilization. During a flash sale, latency increases despite low CPU. What should be done?

Accepted Answer

Use a custom metric such as memory utilization or request count for auto-scaling. CPU utilization is a poor scaling metric for inference workloads that are I/O or memory-bound. During a flash sale, increased request concurrency can cause queuing and latency spikes even when CPU is low. Using a custom metric like request count per instance or memory utilization directly reflects the load on the inference endpoint, enabling the Application Auto Scaling target tracking policy to scale out proactively before latency degrades.

Answer

Increase the instance size

Answer

Disable auto-scaling and use a larger instance

Answer

Switch to GPU instances

Question 4

A company's SageMaker real-time endpoint is experiencing high latency under load. The CloudWatch metrics show that the ModelLatency is acceptable, but the OverheadLatency is spiking. What is the most likely cause?

Accepted Answer

The endpoint is under-provisioned with insufficient instance count.. OverheadLatency measures the time spent on infrastructure overhead (e.g., request routing, network I/O, and container startup) rather than model inference. When the endpoint is under-provisioned with too few instances, requests queue up, causing the SageMaker front-end to wait for a free worker, which directly inflates OverheadLatency while ModelLatency (pure inference time) remains unaffected.

Answer

The request payload size is too large.

Answer

The SageMaker endpoint is not in the same VPC as the client.

Answer

The model inference code is inefficient.

Question 5

A company uses SageMaker for training and inference. They have a model that retrains weekly. After each retraining, the model is evaluated on a held-out test set. If the evaluation metrics meet a threshold, the model is registered as 'Approved' in the SageMaker Model Registry. The team manually deploys the approved model to a production endpoint. They want to automate this deployment process to reduce manual errors. However, the deployment should only proceed if the new model passes a canary test in a staging environment. Which combination of AWS services should the team use to achieve this?

Accepted Answer

SageMaker Pipelines with a conditional deployment step that includes a canary test.. SageMaker Pipelines natively supports conditional execution steps, allowing you to add a canary test step that evaluates the new model in a staging environment before automatically promoting it to production. This directly addresses the requirement for automated deployment gated by a canary test, without needing external orchestration services.

Answer

AWS CodeDeploy with a blue/green deployment strategy.

Answer

AWS Lambda to deploy to staging, then automatically promote to production if staging tests pass.

Answer

Amazon EKS with a custom inference container and use ArgoCD for automated deployments.

Question 6

A data scientist needs to deploy a single ML model that will serve real-time predictions with low latency (under 10 ms) for a high-traffic web application. The model fits in memory and requires GPU acceleration. Which SageMaker inference option is MOST suitable?

Accepted Answer

Real-time endpoint on ml.g4dn instances. Real-time endpoints on GPU instances (ml.g4dn) provide low latency and GPU acceleration, ideal for high-traffic, latency-sensitive workloads.

Answer

Real-time endpoint on ml.m5 instances

Answer

Batch Transform

Answer

Serverless Inference

Question 7

A data scientist runs the exhibit AWS Glue ETL job. The job fails with a Spark stage failure error. What is the most likely cause?

Accepted Answer

The data type mapping in ApplyMapping is incorrect; "value" column contains non-numeric strings that cannot be cast to double.. The Spark stage failure error in an AWS Glue ETL job is most likely caused by a data type mismatch during the ApplyMapping transformation. When the 'value' column contains non-numeric strings that cannot be cast to double, Spark throws a stage failure because it cannot complete the required type conversion, leading to task failures and job termination.

Answer

The output path is missing.

Answer

The S3 bucket does not exist.

Answer

The job does not have enough memory.

Question 8

A data scientist has trained a binary classification model for fraud detection. The dataset is highly imbalanced (99% non-fraud, 1% fraud). After evaluation, the model shows an accuracy of 99%, but the recall for fraud cases is only 10%. Which metric should the data scientist prioritize to improve the model's performance for fraud detection?

Accepted Answer

F1-score. F1-score balances precision and recall, making it more informative than accuracy for imbalanced datasets. AUC-ROC is also used but F1 directly addresses the trade-off between false positives and false negatives. Precision alone does not capture recall, and Log loss does not directly indicate recall improvement.

Answer

Log loss

Answer

Precision

Answer

Area under the ROC curve (AUC-ROC)

Question 9

A team uses SageMaker Clarify to monitor bias drift on a deployed model. They have defined a baseline with training data and set up a monitoring schedule. After one month, they receive a violation report indicating that the post-training metrics have deviated from the baseline. What does this violation indicate?

Accepted Answer

The model's predictions relative to sensitive attributes have shifted compared to the training baseline. SageMaker Clarify bias drift monitoring compares predicted outcomes (post-training) against the baseline to detect changes in fairness metrics like disparate impact. It does not measure prediction accuracy or data quality.

Answer

The SHAP values for features have changed

Answer

The model's predictions are becoming less accurate

Answer

The distribution of input features has changed

Question 10

A data scientist creates a feature group as shown in the exhibit. When ingesting data with an 'age' column of integer values, the ingestion fails. What is the most likely cause?

Accepted Answer

The `age` feature type should be `Integral`, not `String`.. The feature group definition specifies the 'age' column as a `String` type, but the ingested data contains integer values. Amazon SageMaker Feature Store requires that the data types of ingested records match the schema defined in the feature group. When a mismatch occurs, such as providing an integer for a string field, the ingestion fails with a type conversion error.

Answer

The role does not have permissions to write to the feature store.

Answer

The `OnlineStoreConfig` must include a `SecurityConfig`.

Answer

The `EventTimeFeatureName` is incorrectly spelled.

Question 11

A team is using Amazon SageMaker Processing for data preprocessing. They have a Parquet dataset in Amazon S3. Which configuration will provide the most efficient reading of the dataset during processing?

Accepted Answer

Read the Parquet files directly using SparkSession.read.parquet. SageMaker Processing natively integrates with Apache Spark, and reading Parquet files directly via `SparkSession.read.parquet` leverages columnar storage, predicate pushdown, and compression (e.g., Snappy) to minimize I/O and deserialization overhead. This approach is far more efficient than text-based or format-conversion methods, as Parquet is optimized for analytical workloads and preserves schema information.

Answer

Read the Parquet files as text using SparkContext.textFile

Answer

Split the dataset into many small Parquet files (e.g., 1 MB each)

Answer

Convert the Parquet files to CSV before processing

Question 12

A financial services company is deploying a real-time fraud detection model using Amazon SageMaker. The model is a gradient boosting model (XGBoost) trained on historical transaction data. The inference endpoint uses an ml.m5.2xlarge instance with a single variant. Recently, the company has experienced a 3x increase in transaction volume during peak hours, causing inference latency to exceed the 200ms SLA. The data science team has already optimized the model by reducing the number of trees and feature set, but the latency remains high during spikes. The team considers using SageMaker's built-in scaling policies. They currently have a single endpoint with one production variant. The team wants to maintain low latency without over-provisioning resources. They have ruled out model changes. Which approach should the team take?

Accepted Answer

Configure an Application Auto Scaling target tracking scaling policy for the variant based on the 'SageMakerVariantInvocationsPerInstance' metric, with a target value that keeps the inference latency within the SLA.. SageMaker's built-in target tracking scaling policy using the 'SageMakerVariantInvocationsPerInstance' metric allows the endpoint to automatically adjust the instance count based on real-time invocation load. By setting a target value that correlates with the 200ms SLA, the policy dynamically scales out during traffic spikes and scales in during lulls, preventing over-provisioning while maintaining low latency. This approach directly addresses the 3x peak-hour volume increase without requiring manual intervention or model changes.

Answer

Deploy the model on multiple endpoints behind an Application Load Balancer.

Answer

Use scheduled scaling to increase the instance count during known peak hours.

Answer

Manually increase the instance count during peak hours.

Question 13

A data scientist needs to evaluate a binary classification model. The dataset is highly imbalanced (5% positive class). Which metric is MOST appropriate for assessing model performance?

Accepted Answer

AUC. AUC (Area Under the ROC Curve) is robust to class imbalance as it evaluates the model's ability to rank positive vs negative examples. Precision, recall, and F1 can be misleading if not threshold-optimized.

Answer

Precision

Answer

Accuracy

Answer

Recall

Question 14

A company wants to track the lineage of their ML models, including the training dataset, hyperparameters, and training job used to produce each model version. Which AWS service should they use?

Accepted Answer

SageMaker ML Lineage Tracking. SageMaker ML Lineage Tracking is the correct choice because it is purpose-built to record and query the provenance of ML models, capturing relationships between datasets, training jobs, hyperparameters, and model versions. It creates a directed acyclic graph (DAG) of entities (e.g., artifacts, actions, contexts) that allows you to trace how a specific model version was produced, which directly meets the requirement for lineage tracking.

Answer

Amazon DynamoDB

Answer

AWS Glue Data Catalog

Answer

Amazon S3 object tagging

Question 15

A financial services company deploys a fraud detection model on a SageMaker real-time endpoint. The inference logic includes a pre-processing step that requires access to a DynamoDB table for user metadata. The model container is a custom Docker image. How should the team grant the endpoint access to DynamoDB?

Accepted Answer

Create an IAM role with DynamoDB read access and assign it to the SageMaker endpoint as the execution role. SageMaker endpoints require an IAM execution role to be assigned at creation time. This role defines the permissions the endpoint's container has when making AWS API calls, such as reading from DynamoDB. By attaching a policy with DynamoDB read access to this execution role, the endpoint securely obtains temporary credentials via the AWS STS service, eliminating the need to hardcode or manage long-term credentials.

Answer

Store IAM credentials in the container image as environment variables

Answer

Attach an IAM instance profile to the underlying EC2 instance

Answer

Retrieve temporary credentials from AWS Secrets Manager within the container code

Question 16

A company wants to use a pre-trained NLP model from SageMaker JumpStart for sentiment analysis. Which step is required to make predictions?

Accepted Answer

Deploy the model to an endpoint. D is correct because SageMaker JumpStart provides pre-trained models that are ready for inference without additional training. To make predictions, you must deploy the model to a SageMaker endpoint, which creates a hosted inference endpoint that can accept input data and return sentiment analysis results.

Answer

Label the dataset for fine-tuning

Answer

Train the model from scratch on the company's data

Answer

Convert the model to ONNX format

Question 17

Refer to the exhibit. A data scientist ran a training job using a custom algorithm container. The job failed with the error shown. What is the most likely cause?

Accepted Answer

The algorithm script references an undefined variable or metric named 'loss'. The error 'Cannot evaluate expression: loss' indicates that the training script attempted to compute or log a variable named 'loss' that is not defined in the code. The training image access, S3 output path, and instance type are not related to this specific error.

Answer

The S3 output path is incorrect

Answer

The training image is not accessible

Answer

The instance type is insufficient

Question 18

A financial services company is building a fraud detection model using transactional data stored in Amazon S3. The data includes transaction_id, timestamp, amount, merchant_category, and fraud_label (0/1). The data is collected from multiple sources and has inconsistencies: timestamps are in different timezones (UTC and EST), merchant categories are sometimes misspelled (e.g., 'RESTAURANT', 'Restaurant', 'restaurant'), and the fraud_label is missing for about 5% of records. The data science team uses AWS Glue for ETL. They need to prepare a clean dataset for training. The final dataset must have consistent timestamps in UTC, standardized merchant categories, and no missing fraud labels. The team also wants to minimize data loss. Which set of actions should the team take?

Accepted Answer

Use AWS Glue to convert timestamps to UTC, use a mapping table to group similar merchant categories (e.g., all restaurant variants to 'Restaurant'), and impute missing fraud_label using mode (most frequent value).. It preserves data by imputing missing fraud labels using the mode (most frequent value), which is appropriate for a binary classification label where the majority class is likely 0. It also standardizes timestamps to UTC and uses a mapping table to group merchant category variants, ensuring consistency without data loss. Dropping records (as in A and C) would reduce the dataset size, and imputing with the mean (as in B) is invalid for a categorical label.

Answer

Use AWS Glue to convert all timestamps to UTC, apply a mapping function to correct merchant category misspellings to a standard list, and drop records with missing fraud_label.

Answer

Use AWS Glue to convert timestamps to UTC, use a fuzzy matching algorithm to standardize merchant categories, and replace missing fraud_label with the mean value (0.05).

Answer

Use AWS Glue to convert timestamps to UTC, correct merchant categories by mapping known misspellings to correct names, and drop records with missing fraud_label.

Question 19

A company has a SageMaker endpoint running a model that provides real-time recommendations. Recently, the model's accuracy has degraded due to data drift. The team wants to automatically retrain the model when a drift metric exceeds a threshold and deploy the new model without downtime. Which architecture should the team implement?

Accepted Answer

Use SageMaker Model Monitor to trigger an Amazon EventBridge event that starts a SageMaker Pipeline, which retrains the model, registers it in the Model Registry, and then updates the existing endpoint with a new production variant. It uses SageMaker Model Monitor to detect data drift and emit an EventBridge event, which triggers a SageMaker Pipeline to retrain the model, register it in the Model Registry, and then update the existing endpoint with a new production variant. This architecture enables automatic retraining and zero-downtime deployment by leveraging the endpoint's production variants for a blue/green deployment.

Answer

Use SageMaker Model Monitor to collect drift metrics, and have a data scientist manually analyze the metrics and trigger retraining via the SageMaker console

Answer

Schedule a daily SageMaker Pipeline that retrains the model and deploys it using a new endpoint, then updates the application to point to the new endpoint

Answer

Use SageMaker Model Monitor to publish drift metrics to Amazon CloudWatch, and create a CloudWatch alarm that triggers an AWS Lambda function to retrain and deploy the model

Question 20

A data scientist is performing text preprocessing for a sentiment analysis model. The dataset contains many stop words and rare words. Which combination of preprocessing steps will reduce dimensionality and improve model performance?

Accepted Answer

Tokenization, stop-word removal, and TF-IDF. Tokenization, stop-word removal, and TF-IDF vectorization reduce dimensionality by filtering common words and weighting important terms. Lemmatization can further reduce sparsity.

Answer

Remove all words shorter than 3 characters and apply label encoding

Answer

Only tokenization without any removal

Answer

Tokenization and one-hot encoding of words