MLA-C01 Practice Test 8 — 10 Questions

Question 1

A team is deploying a machine learning model using Amazon SageMaker. They need to serve predictions with sub-100ms latency for a real-time application. The model is a large ensemble that requires 4 GB of memory. The team expects traffic of 100 requests per second initially, but it may double during peak hours. Which instance type and deployment configuration should the team choose to minimize cost while meeting the latency requirement?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Deploy on one ml.c5.large instance with an Application Auto Scaling target tracking policy based on memory utilization. The ml.c5.large instance provides 4 GB of memory, which meets the model's requirement, and its compute-optimized nature ensures low-latency inference. Using Application Auto Scaling with a target tracking policy based on memory utilization allows the instance to scale out during traffic spikes (up to 200 requests per second) while minimizing cost by running a single instance during normal load.

Answer

Deploy on one ml.t2.medium instance with an Application Auto Scaling target tracking policy based on CPU utilization

Answer

Deploy on one ml.p3.2xlarge instance with provisioned concurrency

Answer

Deploy on two ml.m5.large instances behind a load balancer with manual scaling

Question 2

A data scientist is using SageMaker to train an XGBoost model for a regression problem. After training, they evaluate the model on a test set and get an RMSE of 10 and an R² of 0.85. Which additional metric would give the MOST insight into the model's average prediction error magnitude?

Accepted Answer

Mean Absolute Error (MAE). MAE (Mean Absolute Error) gives the average absolute prediction error, which is easy to interpret in the same units as the target. RMSE gives a similar but squared metric, and R² indicates variance explained, but MAE directly answers the average error magnitude.

Answer

AUC

Answer

Confusion matrix

Answer

F1 score

Question 3

A team uses SageMaker Pipelines to train and register a model. They want to conditionally run a hyperparameter tuning step only if the data quality check passes. Which pipeline step type should they use to branch the execution?

Accepted Answer

ConditionStep. The ConditionStep allows comparing values and branching to different steps. If data quality passes, the tuning step runs; otherwise, the pipeline stops or runs an alternative step. Other steps do not provide conditional branching.

Answer

TuningStep

Answer

TrainingStep

Answer

TransformStep

Question 4

A company uses SageMaker Model Monitor to track feature attribution drift with SHAP. They notice that the SHAP values have changed significantly for a feature, while the model performance remains stable. What is the MOST likely interpretation?

Accepted Answer

The model's behavior has changed, which may lead to future performance degradation and warrants investigation. A significant change in SHAP values indicates that the model's internal feature importance has shifted, even if overall performance metrics like accuracy or loss remain stable. This is a classic sign of concept drift or model behavior drift, where the model's decision boundary has changed for that feature, which can lead to future performance degradation as the drift accumulates. SageMaker Model Monitor tracks feature attribution drift separately from data drift, and a change in SHAP values without data drift suggests the model is relying on the feature differently, warranting investigation.

Answer

The ground truth labels are incorrect, causing the drift

Answer

Data drift has occurred for that feature

Answer

The model is learning equally from all features, so no action is needed

Question 5

A company wants to deploy a machine learning model that makes real-time predictions for a mobile app. The model is a deep neural network with a large model size (500 MB). Which SageMaker endpoint configuration is most cost-effective while meeting low-latency requirements?

Accepted Answer

Real-time endpoint with a single instance. A real-time endpoint with a single instance provides the lowest latency for a 500 MB deep neural network model, as it keeps the model loaded in memory and ready for inference without cold starts or multi-model overhead. This configuration is also cost-effective for consistent traffic patterns, as you pay for the instance uptime rather than per-invocation or for multiple model loads.

Answer

Multi-model endpoint

Answer

Serverless inference

Answer

Batch transform

Question 6

A company is training a large Transformer model on SageMaker and wants to use model parallelism to fit the model into memory. The model has 10 billion parameters. Which instance type is MOST cost-effective for this task while supporting SageMaker's model parallelism?

Accepted Answer

ml.p4d.24xlarge. The ml.p4d.24xlarge instances are optimized for large-scale distributed training with high memory and support SageMaker's model parallelism. ml.trn1 instances are designed for training with AWS Trainium, but they use a different chip architecture and may require specific SDKs. ml.g4dn instances are for inference and light training. ml.c5 instances are compute-optimized but lack GPU memory for large models.

Answer

ml.trn1.32xlarge

Answer

ml.c5.18xlarge

Answer

ml.g4dn.12xlarge

Question 7

A machine learning engineer uses Amazon SageMaker Data Wrangler to preprocess a dataset. After applying a transform, the engineer wants to export the data to a feature group in Amazon SageMaker Feature Store for reuse in training and inference. Which export option should they choose?

Accepted Answer

Export to Amazon SageMaker Feature Store. SageMaker Data Wrangler can export directly to a feature group in Feature Store, making the features available for both training (offline) and inference (online).

Answer

Export to Amazon DynamoDB

Answer

Export to Amazon S3 as CSV

Answer

Export to Amazon SageMaker Pipelines

Question 8

A company wants to deploy its trained model to edge devices such as cameras and IoT devices. The model must run efficiently with low latency and minimal memory footprint. Which THREE actions should the company take to prepare the model for edge deployment? (Choose THREE.)

Accepted Answer

Use SageMaker Edge Manager to package and manage the model on devices.. SageMaker Edge Manager is purpose-built to package, optimize, and manage machine learning models on edge devices. It provides model packaging, runtime monitoring, and over-the-air updates, ensuring the model runs efficiently with low latency and minimal memory footprint on resource-constrained hardware like cameras and IoT devices.

Answer

Increase the model's complexity to improve accuracy on edge devices.

Answer

Deploy the model directly as a SageMaker endpoint and have the edge devices call it over the internet.

Question 9

A data engineer is building a feature engineering pipeline in AWS Glue ETL to process streaming data from Amazon Kinesis. The data includes a nested JSON structure with arrays. The engineer needs to flatten the nested structures into a tabular format for machine learning. Which THREE approaches are valid for this task? (Choose 3.)

Accepted Answer

Use Python's json.loads in a map function. Python's json.loads can be used within a PySpark map function to parse nested JSON strings from streaming data in AWS Glue ETL. This allows you to extract and flatten nested fields into a tabular structure by iterating over each record and converting the JSON into a flat dictionary, which can then be mapped to DataFrame columns.

Answer

Use Athena's UNNEST function on the raw data

Answer

Use Amazon SageMaker Processing with scikit-learn

Question 10

A company needs to deploy a new model version to a SageMaker real-time endpoint. They want to route 5% of traffic to the new version initially to monitor for errors before full rollout. Which deployment strategy should they use?

Accepted Answer

Canary deployment with production variants. A canary deployment with production variants allows you to route a specific percentage of traffic (e.g., 5%) to the new model version by adjusting the `InitialVariantWeight` parameter in the production variant configuration. This enables gradual traffic shifting while monitoring errors, and you can later increase the weight to 100% for full rollout. SageMaker real-time endpoints support this natively by hosting multiple model variants behind the same endpoint.

Answer

Blue/green deployment

Answer

Shadow testing

Answer

Multi-model endpoint