MLA-C01 Practice Test 38 — 15 Questions

Question 1

A data scientist needs to evaluate a binary classification model. The dataset is highly imbalanced (5% positive class). Which metric is MOST appropriate for assessing model performance?

Accepted Answer

AUC. AUC (Area Under the ROC Curve) evaluates the model's ability to rank positive instances above negatives across all classification thresholds, making it robust to class imbalance because it does not depend on a single threshold or on the majority class dominating the score. With only 5% positives, accuracy and threshold-dependent metrics can be misleading, but AUC remains a reliable discriminator measure.

Answer

Precision

Answer

Accuracy

Answer

Recall

Question 2

Which feature scaling method is most robust to outliers in the data?

Accepted Answer

Robust scaling. Robust scaling is the most robust to outliers because it uses the median and interquartile range (IQR) instead of the mean and standard deviation. The median and IQR are not influenced by extreme values, so the scaled features remain stable even when the data contains significant outliers.

Answer

Normalization (L2)

Answer

Standardization (Z-score)

Answer

Min-max scaling

Question 3

A machine learning engineer is evaluating a binary classification model for detecting fraudulent transactions. The dataset is highly imbalanced, and the cost of false negatives (missing a fraud) is very high. Which two evaluation metrics should the engineer consider? (Choose two.)

Accepted Answer

F1-score. Recall (Option C) is correct because it measures the proportion of actual fraud cases that the model successfully identifies (TP / (TP + FN)), directly addressing the scenario's high cost of false negatives—maximizing recall minimizes missed frauds. F1-score (Option A) is correct because it is the harmonic mean of precision and recall (2 × (Precision × Recall) / (Precision + Recall)), providing a balanced single metric that remains informative on highly imbalanced datasets where a model could otherwise achieve high recall by flagging everything. Accuracy (Option B) is not appropriate because with severe class imbalance a trivial model predicting 'no fraud' for all transactions can score very high accuracy while catching zero frauds. Precision (Option D) is relevant but not among the two marked correct; it focuses on false positives rather than the costly false negatives and is already incorporated into the F1-score. Mean absolute error (Option E) is a regression metric and does not apply to binary classification evaluation.

Answer

Accuracy

Answer

Precision

Answer

Mean absolute error

Question 4

A company wants to version and track ML models, with an approval workflow for promoting models from staging to production. Which SageMaker feature should they use?

Accepted Answer

SageMaker Model Registry. SageMaker Model Registry is the correct choice because it provides a centralized repository to catalog, version, and manage ML models, and it supports approval workflows (e.g., PendingApproval, Approved, Rejected) to promote models from staging to production. This directly addresses the requirement for version tracking and an approval gate for model promotion.

Answer

SageMaker Model Monitor

Answer

SageMaker Experiments

Answer

SageMaker Pipelines

Question 5

An organization wants to schedule a retraining pipeline to run every Sunday night. Which AWS service should they use to trigger the pipeline on a schedule?

Accepted Answer

Amazon EventBridge. Amazon EventBridge is the correct choice because it provides a scheduled event source using cron or rate expressions to trigger target services at specified times, such as every Sunday night. It can directly invoke an AWS Step Functions state machine or a Lambda function to start the retraining pipeline, making it the native scheduling service for event-driven workflows in AWS.

Answer

AWS Lambda

Answer

AWS Step Functions

Answer

Amazon SQS

Question 6

A company is using SageMaker Pipelines to automate a multi-step ML workflow. The pipeline includes data preprocessing, training, and model evaluation. The team wants to ensure that if the evaluation step fails, the pipeline stops and sends an alert to the operations team. Which SageMaker Pipelines feature should they use?

Accepted Answer

Use a Condition step to check the evaluation result and route to a Fail step if the result indicates failure. SageMaker Pipelines provides a built-in Condition step that evaluates a boolean expression (e.g., checking if evaluation metrics meet a threshold) and then routes execution to different steps. If the condition fails, you can direct the pipeline to a Fail step, which immediately stops the pipeline and marks it as failed. This is the native, event-driven way to halt a pipeline based on step output without relying on external services.

Answer

Configure an Amazon CloudWatch Events rule to monitor the pipeline execution status and stop it if the evaluation step fails

Answer

Register the model in the Model Registry only if evaluation passes, and configure the pipeline to stop if registration fails

Answer

Add a Lambda step after the evaluation step that checks the evaluation metrics and sends an SNS notification if the metrics are below a threshold

Question 7

A machine learning engineer is deploying a pre-trained NLP model on Amazon SageMaker for real-time inference. The model expects input sequences of variable length, and performance is critical. The engineer wants to minimize latency while handling the variable-length inputs efficiently. Which approach should the engineer choose?

Accepted Answer

Use dynamic batching with a custom inference script that groups requests by sequence length.. Dynamic batching with a custom inference script that groups requests by sequence length minimizes padding overhead and maximizes hardware utilization. By batching similar-length sequences together, the model avoids excessive padding to the maximum length in the batch, which reduces wasted computation and latency. This approach is particularly effective for variable-length NLP inputs on SageMaker, where the inference container can be customized to implement the grouping logic.

Answer

Reduce the model size by pruning and quantization.

Answer

Pad all input sequences to the maximum length in the batch.

Answer

Process each request individually to avoid padding overhead.

Question 8

An MLOps engineer is building an automated retraining pipeline for a fraud detection model. The model must be retrained weekly, and the new model should only be promoted to production if it meets predefined performance thresholds compared to the current model. Which combination of SageMaker capabilities should the engineer use?

Accepted Answer

Amazon SageMaker Pipelines and Amazon SageMaker Model Registry. Amazon SageMaker Pipelines provides the orchestration for the automated retraining workflow (including weekly scheduling and conditional logic), while SageMaker Model Registry enables versioning, approval, and promotion of models based on performance thresholds. Together, they allow the engineer to define a pipeline that trains a new model, evaluates it against the current production model, and only registers it for deployment if it meets the predefined criteria.

Answer

Amazon SageMaker Debugger and Amazon SageMaker Clarify

Answer

Amazon SageMaker Model Monitor and Amazon SageMaker Ground Truth

Answer

Amazon SageMaker Autopilot and Amazon SageMaker Experiments

Question 9

A company has a dataset of 2 billion records stored as text files in Amazon S3. The data is partitioned by year and month. The data science team wants to read only the last 6 months of data for model training using SageMaker. To minimize data scanned and reduce costs, which approach should the team use?

Accepted Answer

Use AWS Glue to create a catalog table with partitions, then query with Athena to create a filtered dataset in S3.. AWS Glue can crawl the S3 data to create a catalog table with partitions by year and month. Athena can then query only the partitions corresponding to the last 6 months, scanning minimal data and writing the filtered results back to S3 for SageMaker training. This approach leverages partition pruning to reduce costs and avoids loading or processing the full 2 billion records.

Answer

Use S3 Select to retrieve only the last 6 months of data by applying an SQL expression on each object.

Answer

Use SageMaker Processing with a script that lists all objects in the bucket and reads only those with the desired prefixes.

Answer

Use SageMaker Processing with Input Mode 'File' and specify the S3 prefix for the last 6 months.

Question 10

A company is deploying a large NLP model on SageMaker for real-time inference. They want to reduce inference latency and cost by optimizing the model for the target hardware. The model is trained in PyTorch. Which SageMaker feature should they use to compile the model for best performance on the chosen instance?

Accepted Answer

SageMaker Neo. SageMaker Neo is the correct choice because it is specifically designed to compile trained models (including PyTorch models) into an optimized binary for a target hardware instance, reducing inference latency and improving throughput. Neo applies hardware-specific optimizations such as operator fusion, memory layout tuning, and quantization, which directly address the need for best performance on the chosen SageMaker instance.

Answer

AWS Step Functions

Answer

Amazon Elastic Inference

Answer

SageMaker Triton Inference Server

Question 11

A financial services company deploys a fraud detection model on a SageMaker real-time endpoint. The inference logic includes a pre-processing step that requires access to a DynamoDB table for user metadata. The model container is a custom Docker image. How should the team grant the endpoint access to DynamoDB?

Accepted Answer

Create an IAM role with DynamoDB read access and assign it to the SageMaker endpoint as the execution role. SageMaker endpoints require an IAM execution role to be assigned at creation time. This role defines the permissions the endpoint's container has when making AWS API calls, such as reading from DynamoDB. By attaching a policy with DynamoDB read access to this execution role, the endpoint securely obtains temporary credentials via the AWS STS service, eliminating the need to hardcode or manage long-term credentials.

Answer

Store IAM credentials in the container image as environment variables

Answer

Attach an IAM instance profile to the underlying EC2 instance

Answer

Retrieve temporary credentials from AWS Secrets Manager within the container code

Question 12

A team has a large number of models that need to be deployed for batch inference weekly. They want to minimize cost and management overhead. Which approach is MOST efficient?

Accepted Answer

Use SageMaker Batch Transform with separate jobs for each model.. SageMaker Batch Transform is the most efficient approach for weekly batch inference because it automatically provisions and terminates compute resources for each job, minimizing cost and management overhead. Running separate jobs for each model allows independent scaling and avoids the complexity of managing persistent endpoints or multi-model hosting for batch workloads.

Answer

Use SageMaker Pipelines to run inference as part of the pipeline.

Answer

Create a single SageMaker endpoint for all models and update the model periodically.

Answer

Deploy each model to a separate SageMaker endpoint and delete after use.

Question 13

A machine learning engineer is configuring auto-scaling for a SageMaker real-time endpoint. The endpoint is expected to have steady traffic during business hours and low traffic at night. The engineer wants to minimize costs by scaling in during low traffic, but the model container has a long start-up time (about 5 minutes). Which scaling policy should the engineer use to prevent request drops during sudden traffic spikes?

Accepted Answer

Use a target tracking scaling policy based on average invocations per minute with a warm-up of 300 seconds.. Target tracking scaling policies in SageMaker automatically adjust capacity to maintain a target metric value, and the warm-up time of 300 seconds accounts for the 5-minute container start-up latency. This prevents request drops during sudden traffic spikes by ensuring new instances are fully initialized before they receive traffic, while still allowing the endpoint to scale in during low traffic to minimize costs.

Answer

Use a step scaling policy based on invocations per minute with a step that adds two instances at a time.

Answer

Use a scheduled scaling action to add instances before business hours and remove them after.

Answer

Use a simple scaling policy based on average CPU utilization with a cooldown period of 5 minutes.

Question 14

A company has a SageMaker endpoint running a model that provides real-time recommendations. Recently, the model's accuracy has degraded due to data drift. The team wants to automatically retrain the model when a drift metric exceeds a threshold and deploy the new model without downtime. Which architecture should the team implement?

Accepted Answer

Use SageMaker Model Monitor to trigger an Amazon EventBridge event that starts a SageMaker Pipeline, which retrains the model, registers it in the Model Registry, and then updates the existing endpoint with a new production variant. It uses SageMaker Model Monitor to detect data drift and emit an EventBridge event, which triggers a SageMaker Pipeline to retrain the model, register it in the Model Registry, and then update the existing endpoint with a new production variant. This architecture enables automatic retraining and zero-downtime deployment by leveraging the endpoint's production variants for a blue/green deployment.

Answer

Use SageMaker Model Monitor to collect drift metrics, and have a data scientist manually analyze the metrics and trigger retraining via the SageMaker console

Answer

Schedule a daily SageMaker Pipeline that retrains the model and deploys it using a new endpoint, then updates the application to point to the new endpoint

Answer

Use SageMaker Model Monitor to publish drift metrics to Amazon CloudWatch, and create a CloudWatch alarm that triggers an AWS Lambda function to retrain and deploy the model

Question 15

Refer to the exhibit. A team has configured data capture for a SageMaker endpoint. The endpoint is returning predictions but no captured data appears in the S3 bucket. What is the most likely cause?

Accepted Answer

The IAM role for the endpoint does not have s3:PutObject permission.. The most likely cause is that the IAM role associated with the SageMaker endpoint lacks the `s3:PutObject` permission. Without this permission, the endpoint can generate capture data internally but cannot write it to the specified S3 bucket, resulting in no captured data appearing even though predictions are returned successfully.

Answer

The InitialSamplingPercentage is too low.

Answer

The capture status is 'Configured' but not 'Running'.

Answer

The endpoint is not receiving any traffic.