MLA-C01 Practice Test 25 — 15 Questions

Question 1

A machine learning team at a retail company has deployed a product recommendation model using Amazon SageMaker. The model is updated weekly with new data. Recently, the team noticed that the model's accuracy on a holdout evaluation set has been declining over the past month. The data pipeline that feeds the training job has not changed. The team suspects data drift. They have SageMaker Model Monitor enabled on the inference endpoint and have set up Amazon CloudWatch metrics for feature distribution distances. Upon reviewing the CloudWatch dashboards, they see that the feature distribution distance metric for the most important feature 'product_category' has increased significantly. However, the team is unsure if this is the root cause. Which remediation step should the team take FIRST?

Accepted Answer

Investigate the data pipeline that feeds the training job to ensure consistent data collection and encoding of the 'product_category' feature. Before retraining the model or deploying a new endpoint, the team should investigate the source of the data drift by checking the input data pipeline. The data pipeline might have introduced a systematic error, such as a change in how 'product_category' is encoded or collected. Option A (retrain the model with more recent data) might not help if the data itself is corrupted. Option B (reduce the number of features) could ignore the problem. Option D (rebuild the endpoint) would not address the data drift. Therefore, the first step is to investigate the data pipeline.

Answer

Retrain the model using the most recent week of data and redeploy to the endpoint

Answer

Rebuild the SageMaker endpoint with a different instance type to improve performance

Answer

Reduce the number of features in the model by removing 'product_category'

Question 2

A data science team needs to track and compare multiple ML training runs, including hyperparameters, metrics, and output artifacts. Which TWO AWS services can be used together to meet this requirement? (Choose two.)

Accepted Answer

Amazon SageMaker Experiments. Amazon SageMaker Experiments provides experiment tracking and comparison. Amazon SageMaker Model Registry helps manage model versions and artifacts. SageMaker Studio notebooks alone lack tracking; S3 provides storage but no tracking; CloudWatch logs is for monitoring.

Answer

Amazon S3

Answer

Amazon SageMaker Studio notebooks

Answer

Amazon CloudWatch Logs

Question 3

A machine learning engineer observes that a SageMaker training job fails with the error shown in the exhibit. What is the most likely cause of the failure?

Accepted Answer

The SageMaker execution role does not have an IAM policy that grants read access to the S3 bucket containing the training data.. The error clearly states that the SageMaker execution role lacks the necessary permissions to download data. The role assigned to the training job must have S3 read access. Option A is correct. Option B is incorrect because the error is explicit about credentials. Option C is incorrect because the error is not about network. Option D is incorrect because the error does not mention data format.

Answer

The training data is stored in an unsupported format like Parquet.

Answer

The training job is using an incorrect AWS Region for the S3 bucket.

Answer

The VPC configuration prevents the training job from reaching the S3 bucket.

Question 4

A company wants to monitor its Amazon SageMaker real-time endpoint for data quality issues. Which TWO actions should the company take?

Accepted Answer

Create a baseline from the training data to compare against live data.. To monitor data quality with SageMaker Model Monitor, you need to enable data capture on the endpoint and create a baseline from the training data. The other options are not directly required for data quality monitoring.

Answer

Use SageMaker Debugger to analyze training jobs.

Answer

Set up an AWS Lambda function to preprocess incoming requests.

Answer

Configure Amazon S3 bucket notifications for model artifacts.

Question 5

An ML team at a financial services company has developed a fraud detection model using Amazon SageMaker. The model is currently deployed to a production endpoint with a single variant using the previous model version. The team wants to deploy a new model version with a canary deployment where 10% of traffic goes to the new version and 90% remains on the old version for 30 minutes before shifting all traffic to the new version if no issues are detected. Which step is essential to achieve this safe rollout?

Accepted Answer

Update the endpoint with a new production variant for the new model version and set the 'InitialVariantWeight' to 10 for the new variant and 90 for the old variant, specifying a 'BlueGreenUpdatePolicy' with a 'TrafficRoutingConfiguration' for canary.. Option C is correct because SageMaker canary deployments are configured by setting the 'BlueGreenUpdatePolicy' in an endpoint update. Option A is incorrect because SageMaker does not support A/B testing through the predictor directly. Option B is incorrect because SageMaker does not provide a built-in canary deployment mode via the built-in algorithms. Option D is incorrect because while the endpoint must be hosted on multiple instances, that alone does not enable canary routing.

Answer

Use the 'Deploy' method on the model object with the 'mode' parameter set to 'canary' within the built-in XGBoost algorithm container.

Answer

Ensure the endpoint is hosted on at least two instances to enable load balancing, then deploy the new model version as a separate variant and manually adjust the endpoint's DNS to split traffic.

Answer

Deploy the new model as a separate endpoint and use a SageMaker predictor to randomly route 10% of inference requests to the new endpoint.

Question 6

A machine learning engineer sees the above error in Amazon CloudWatch Logs for a SageMaker endpoint. What is the most likely cause?

Accepted Answer

The inference code in the Docker container outputs a different response format than expected by the endpoint.. The error indicates the model returned a response with an unexpected structure. The expected format was a JSON with a 'predictions' array, but the model output a single 'prediction' field. This mismatch is typically due to a bug in the inference code within the Docker container, not corruption, overload, or misconfiguration of data capture.

Answer

The model file is corrupted during deployment.

Answer

The data capture configuration is incorrectly set to capture only the response body.

Answer

The endpoint is overloaded and dropping requests.

Question 7

A machine learning team has deployed a model using Amazon SageMaker and wants to set up continuous monitoring for data drift. Which TWO actions are essential for ongoing data drift detection?

Accepted Answer

Enable data capture on the SageMaker endpoint to store inference data in Amazon S3.. Option A (Enable data capture) is essential because data capture collects inference requests and responses, which are required for monitoring. Option C (Configure Model Monitor to run hourly) is essential because Model Monitor analyzes the captured data against a baseline to detect drift. Option B is a prerequisite but not an ongoing action. Options D and E are unrelated to data drift monitoring.

Answer

Set up Amazon CloudWatch alarms on the endpoint's invocation latency metric.

Answer

Deploy a shadow endpoint to compare predictions from the current model and a challenger model.

Answer

Create a baseline from the training data to serve as a reference distribution.

Question 8

A company deployed a machine learning model on an Amazon SageMaker real-time endpoint. Over several weeks, they notice that inference latency has been gradually increasing, especially during peak business hours. The model and instance type have remained unchanged. What is the most likely cause of the increased latency?

Accepted Answer

The SageMaker endpoint auto scaling is not configured to scale out quickly enough under increasing traffic.. Option B is correct because the gradual increase in latency over time, especially during peak hours, suggests that the endpoint may not be scaling out adequately to handle increased traffic. Option A is incorrect because the model size has not changed. Option C is incorrect because data capture does not inherently cause latency increases over time. Option D is incorrect because batch processing is not used for real-time endpoints.

Answer

The inference script is not using batch processing.

Answer

The model size is too large for the instance type.

Answer

The endpoint has data capture enabled, causing additional overhead.

Question 9

A data science team deploys a TensorFlow model for real-time inference using the Amazon SageMaker model configuration shown. They observe high latency during the first few requests after deployment. Which TWO actions would reduce cold start latency? (Choose two.)

Accepted Answer

Configure a Production Variant with an initial instance count greater than zero. Options C and D are correct. Using Multi-Model Endpoints allows the endpoint to stay warm and reduces the time to load a model on demand. Setting an initial instance count greater than zero ensures that the endpoint always has at least one instance running, eliminating cold starts. Option A (adding another container) increases cold start latency. Option B (changing environment variable) does not affect model loading time. Option E (data capture) adds overhead without reducing cold start latency.

Answer

Enable data capture on the endpoint

Answer

Set the SAGEMAKER_PROGRAM environment variable to a more optimized entry point

Answer

Add a secondary container for model ensemble

Question 10

A data scientist is training a binary classification model using Amazon SageMaker. The dataset is highly imbalanced (95% negative class, 5% positive class). The model is evaluated on a held-out test set, and the F1 score is 0.12. The data scientist wants to improve the F1 score. Which two actions should the data scientist take? (Choose two.)

Accepted Answer

Apply SMOTE (Synthetic Minority Oversampling Technique) to the training data using a preprocessing script in SageMaker Processing.. Setting scale_pos_weight in XGBoost adjusts the loss function to penalize misclassification of the minority class more heavily, improving recall and F1. SMOTE oversamples the minority class to balance the dataset. Option C reduces model capacity, which may not help; D changes the metric but doesn't fix the problem; E increases threshold and likely reduces recall further.

Answer

Reduce the model complexity by decreasing the number of layers in a deep neural network.

Answer

Increase the decision threshold to reduce false positives.

Answer

Use recall as the primary evaluation metric instead of F1.

Question 11

Refer to the exhibit. A data scientist used a SageMaker training job with a custom Scikit-learn script. The training job failed with the error shown. What is the most likely cause of this failure?

Accepted Answer

The training script is reading the CSV file incorrectly, causing a shape mismatch.. The error 'Expected 2D array, got 1D array' indicates the input data is being interpreted as a single-dimensional array. In SageMaker, when reading a CSV file with one column, the default behavior may produce a 1D array. The script likely expects a 2D array. Option A is correct because the script is incorrectly reading or processing the CSV, causing a shape mismatch.

Answer

The InputDataConfig specifies ContentType text/csv but the actual file is not CSV.

Answer

The SageMaker training image is outdated and does not support Scikit-learn 1.0.

Answer

The training data contains missing values that need to be imputed.

Question 12

A financial services company is building a fraud detection model using historical transaction data stored in Amazon S3. The data includes features such as transaction amount, merchant category, time of day, and user location. The data scientist observes that the 'merchant_category' column is a text attribute with over 200 unique values. Additionally, the 'transaction_amount' column has a long-tail distribution with extreme outliers. The dataset is 200 GB in size, and the company wants to use Amazon SageMaker for model training. The data scientist needs to engineer features that capture the high-cardinality category and reduce the impact of outliers. What is the MOST efficient and effective approach to prepare this data?

Accepted Answer

Use Amazon EMR with Spark to apply ordinal encoding to merchant_category based on frequency, and log-transform the transaction_amount to reduce skewness.. Option B is correct because ordinal encoding based on frequency handles high-cardinality categorical features efficiently without exploding dimensionality, and log-transform is a standard technique to reduce skewness in long-tail distributions. Using Amazon EMR with Spark provides distributed processing for the 200 GB dataset, making it scalable and cost-effective compared to single-node alternatives.

Answer

Use AWS Glue ETL to apply one-hot encoding to merchant_category and min-max scaling to transaction_amount.

Answer

Use Amazon Athena to bin transaction_amount into 10 equal-width bins and replace merchant_category with its count encoding.

Answer

Use AWS Glue DataBrew to apply a one-hot encoding on merchant_category and a standard scaler on transaction_amount after removing outliers.

Question 13

A team is deploying a TensorFlow model on a SageMaker real-time endpoint with automatic scaling. They set the scaling policy to target an average CPU utilization of 50%. However, during traffic spikes, the endpoint experiences high latency and 503 errors. The instance type is ml.c5.large. What should the team do to resolve this while minimizing cost?

Accepted Answer

Add a scaling policy based on the number of concurrent requests per instance. Option D is correct because scaling based on CPU utilization alone is often insufficient for inference workloads where latency is the primary concern. By adding a scaling policy based on the number of concurrent requests per instance, the team can proactively scale out before CPU saturation occurs, reducing latency and eliminating 503 errors. SageMaker's automatic scaling supports multiple target tracking metrics, and using concurrent requests per instance aligns more closely with the actual demand on the model serving container.

Answer

Pre-warm the endpoint by keeping a fixed number of additional instances

Answer

Increase the scale-in cooldown period to avoid frequent downsizing

Answer

Change the instance type to a larger one like ml.c5.xlarge to handle the spikes

Question 14

An ML engineer is setting up monitoring for a SageMaker endpoint. Which THREE metrics should be monitored to detect performance issues? (Select THREE.)

Accepted Answer

Model latency. Model latency is a critical metric for detecting performance issues in a SageMaker endpoint because it directly measures the time taken to process inference requests. High latency can indicate resource bottlenecks, model inefficiency, or scaling problems, and it is essential for meeting service-level agreements (SLAs). Monitoring latency helps identify when the endpoint is underprovisioned or when the model itself has degraded in performance.

Answer

Invocations per second

Answer

DiskWriteBytes

Question 15

A company uses AWS Glue ETL jobs to transform data for machine learning. They have a dataset with a column 'income' that is heavily right-skewed. Which transformation should be applied to make the distribution more Gaussian-like?

Accepted Answer

Log transformation (natural log). A log transformation is appropriate for heavily right-skewed data because it compresses the long tail by applying a concave function, pulling extreme values closer to the mean and making the distribution more symmetric. In AWS Glue ETL, you can apply this using Spark SQL's `LOG` function or a Python UDF with `numpy.log`, which directly addresses the skewness to better approximate a Gaussian distribution for downstream ML models.

Answer

Standardization (z-score)

Answer

Min-max scaling to [0,1]

Answer

Equal-width binning