MLA-C01 Practice Test 3 — 25 Questions

Question 1

A company is using an AWS Step Functions state machine to orchestrate a multi-step ML deployment. The workflow includes: training a model, evaluating it, registering the model, and deploying to a staging endpoint. They need to implement an approval gate before deploying to production. Which THREE components are necessary to achieve this? (Choose three.)

Accepted Answer

A task in the state machine that pauses and waits for manual approval via SNS or Lambda. Option B is correct because Step Functions can use a task with a callback pattern (`.waitForTaskToken`) to pause the workflow and wait for external manual approval. When combined with an SNS topic or Lambda function that sends a task success or failure signal back to Step Functions, this creates a reliable approval gate. This pattern allows the state machine to halt execution until a human approves or rejects the deployment, which is essential for production deployment control.

Answer

An AWS CodePipeline pipeline with approval stage

Answer

An Amazon SNS topic for notification of approval status

Question 2

A company has a model that receives low traffic but needs to handle sudden spikes. Which deployment option is most cost-effective?

Accepted Answer

SageMaker Serverless Inference. SageMaker Serverless Inference is the most cost-effective option for low-traffic models with sudden spikes because it automatically scales to zero when not in use and scales up instantly to handle bursts, charging only for the compute time consumed per inference request. This eliminates the cost of idle provisioned infrastructure, making it ideal for unpredictable or intermittent traffic patterns.

Answer

SageMaker Real-Time Endpoint with Auto Scaling

Answer

SageMaker Multi-Model Endpoint

Answer

SageMaker Batch Transform

Question 3

A company is deploying multiple models on a single endpoint to reduce costs. They need to update one model without affecting others. Which solution?

Accepted Answer

Use SageMaker Multi-Model Endpoint. SageMaker Multi-Model Endpoint (MME) allows hosting multiple models on a single endpoint, sharing the underlying compute instance. When you need to update one model, you can simply upload a new model artifact (e.g., a new `model.tar.gz`) to Amazon S3, and the endpoint will automatically load the updated version on subsequent inference requests without affecting the other models currently cached or in use.

Answer

Use multiple single-model endpoints behind an Application Load Balancer

Answer

Use SageMaker Batch Transform for some models

Answer

Use a SageMaker Endpoint with multiple production variants

Question 4

A team has a large number of models that need to be deployed for batch inference weekly. They want to minimize cost and management overhead. Which approach is MOST efficient?

Accepted Answer

Use SageMaker Batch Transform with separate jobs for each model.. SageMaker Batch Transform is the most efficient approach for weekly batch inference because it automatically provisions and terminates compute resources for each job, minimizing cost and management overhead. Running separate jobs for each model allows independent scaling and avoids the complexity of managing persistent endpoints or multi-model hosting for batch workloads.

Answer

Use SageMaker Pipelines to run inference as part of the pipeline.

Answer

Create a single SageMaker endpoint for all models and update the model periodically.

Answer

Deploy each model to a separate SageMaker endpoint and delete after use.

Question 5

Refer to the exhibit. A company configures a SageMaker Model Monitor Data Quality monitoring schedule as shown. The schedule runs every hour. However, the team notices that the monitoring job fails intermittently with an AccessDenied error when accessing the S3 bucket for output. The IAM role SageMakerMonitorRole has permissions to write to s3://my-bucket/monitor-output. What is the MOST likely cause of the failure?

Accepted Answer

The monitoring job runs in a VPC that does not have an S3 VPC endpoint, and the bucket policy denies requests from outside the VPC.. The intermittent AccessDenied error when SageMaker Model Monitor attempts to write to the S3 output bucket strongly indicates a network or policy restriction. If the monitoring job is configured to run inside a VPC (common for security compliance) and that VPC lacks an S3 VPC endpoint, traffic to S3 traverses the public internet. If the S3 bucket policy explicitly denies requests from outside the VPC (using a condition like `aws:SourceVpce` or `aws:SourceVpc`), then jobs running inside the VPC without an endpoint will be denied access intermittently, especially if the job's execution role is assumed from within the VPC.

Answer

The S3UploadMode is set to Continuous, which is only supported for batch transform jobs.

Answer

The cron expression is invalid; it should use rate(1 hour) instead.

Answer

The baseline constraints and statistics files are missing from the S3 bucket.

Question 6

Refer to the exhibit. A user is unable to invoke a SageMaker endpoint. The IAM policy shown is attached to the user. Which permission is missing to allow invocation?

Accepted Answer

sagemaker:InvokeEndpoint. To invoke a SageMaker endpoint, the user needs the `sagemaker:InvokeEndpoint` permission. The IAM policy shown lacks this action, which is required for making real-time inference requests to the endpoint. Without it, any attempt to call the endpoint via the SDK or CLI will fail with an access denied error.

Answer

sagemaker:DescribeEndpoint

Answer

sagemaker:CreateEndpoint

Answer

sagemaker:ListEndpoints

Question 7

Refer to the exhibit. A SageMaker Pipeline fails with 'Invalid output reference' at the TrainingStep. What is the most likely cause?

Accepted Answer

The TrainingStep expects a single artifact but TuningStep produces multiple. Option C is correct because in SageMaker Pipelines, a `TrainingStep` that expects a single artifact as input will fail with 'Invalid output reference' if the preceding `TuningStep` produces multiple artifacts (e.g., from multiple training jobs). The pipeline cannot resolve which specific artifact to pass, causing the error.

Answer

The TuningStep output name is misspelled

Answer

The pipeline role lacks permissions

Answer

The instance type is incompatible

Question 8

A data scientist runs this pipeline but the Train step fails with "ResourceLimitExceeded". What is the most likely cause?

Accepted Answer

The account has a limit of 0 for ml.p3.2xlarge instances.. The 'ResourceLimitExceeded' error indicates that the requested instance type (ml.p3.2xlarge) exceeds the account's service quota for that specific instance family. In AWS SageMaker, each account has a default limit of 0 for certain GPU instance types like ml.p3.2xlarge unless a quota increase has been requested and approved. This error occurs at the Train step because SageMaker attempts to launch the training job with an instance type that is not allowed by the current quota.

Answer

The volume size is too small for training.

Answer

The Preprocess step did not complete successfully.

Answer

The training image is not accessible.

Question 9

A company deploys a model on Amazon SageMaker for real-time inference. The inference latency is too high. The model is a large deep learning model. The company wants to reduce latency without significantly impacting accuracy. Which approach should the company consider?

Accepted Answer

Use SageMaker Inference Recommender to test different instance types and optimizations.. SageMaker Inference Recommender is designed specifically to automate load testing and benchmarking across various instance types and model optimizations (e.g., Elastic Inference, GPU acceleration, serialization formats). It provides latency and throughput metrics to identify the optimal configuration for reducing inference latency while maintaining accuracy, making it the correct choice for a large deep learning model with high latency.

Answer

Increase the batch size for inference.

Answer

Use a smaller instance type to reduce inference time.

Answer

Enable SageMaker Model Monitor to detect performance issues.

Question 10

A financial services company has a SageMaker pipeline that trains a fraud detection model daily. The pipeline consists of three steps: preprocessing (using a Spark script), training (XGBoost), and evaluation. The evaluation step calculates the F1 score and compares it to a threshold of 0.95. If the F1 score is below 0.95, the pipeline should fail and notify the team via email. The team implemented this using a Condition step that checks if the F1 score is greater than or equal to 0.95. If true, the pipeline proceeds to register the model; if false, the pipeline fails. However, the team notices that even when the F1 score is 0.94, the pipeline continues to the registration step. The evaluation script outputs the F1 score as a float with two decimal places in a JSON file. The Condition step uses the expression: $.evaluation.metrics.f1_score >= 0.95. What is the most likely cause of the issue?

Accepted Answer

The evaluation script outputs the F1 score as a string, and string comparison '0.94' >= '0.95' evaluates to true because it is lexicographically compared. The most likely cause is that the evaluation script outputs the F1 score as a string (e.g., "0.94") rather than a numeric value. In AWS SageMaker Pipelines, the Condition step evaluates expressions using JSONPath, and when comparing two values, if one is a string, the comparison is performed lexicographically (character by character). Lexicographically, the string "0.94" is considered greater than or equal to "0.95" because '9' > '5' after the decimal point, causing the condition to pass incorrectly.

Answer

The evaluation step must be split into two steps: one for evaluation and one for condition check

Answer

The Condition step cannot be used to check metric values; it can only check step status

Answer

The threshold should be set to 0.95 but the Condition step uses a less than or equal operator

Question 11

An e-commerce company uses Amazon SageMaker to train a model that predicts click-through rates. The training data includes a timestamp column 'click_time' and a categorical feature 'device_type' (8 values). They notice that the model's performance degrades over time because the data distribution shifts. They want to ensure the training data represents the most recent behavior. The data is stored in a daily partitioned S3 bucket (e.g., s3://bucket/data/2024-01-01/). The total dataset size is 500 GB. Which approach should they take to prepare the training data while minimizing bias and cost?

Accepted Answer

Select only the data from the last 30 days to train the model.. Option A is correct because selecting only the last 30 days of data directly addresses the data distribution shift by focusing on the most recent user behavior, which is critical for click-through rate prediction. This approach minimizes bias from outdated patterns and reduces training cost by using a smaller, relevant dataset (approximately 500 GB / 365 * 30 ≈ 41 GB). SageMaker training jobs benefit from this reduced volume through faster data loading and lower compute costs.

Answer

Take a random sample of 10% of the rows from the entire dataset.

Answer

Use all historical data and let the model learn the temporal patterns.

Answer

Downsample older data exponentially so that recent data is overrepresented.

Question 12

A company has trained a custom model using PyTorch on Amazon SageMaker. The model achieves high accuracy, but the inference latency on a real-time endpoint is above the required 100ms SLA. The model is a large neural network with many layers. The company wants to reduce latency without significantly impacting accuracy. Which approach should the machine learning engineer take?

Accepted Answer

Use SageMaker Neo to compile the model for the target hardware.. SageMaker Neo compiles trained models into an optimized binary for the target hardware (e.g., CPU, GPU, or Inferentia). It applies graph-level optimizations, operator fusion, and quantization-aware tuning to reduce inference latency while preserving model accuracy. This directly addresses the need to lower latency below 100ms without retraining or sacrificing significant accuracy.

Answer

Reduce the batch size used during inference.

Answer

Increase the instance size of the endpoint.

Answer

Implement a cache for frequent inference requests.

Question 13

A model has high training accuracy but low validation accuracy. Which action is least likely to reduce overfitting?

Accepted Answer

Increase model complexity. Increasing model complexity (e.g., adding more layers or parameters) makes the model more flexible, which typically exacerbates overfitting by allowing it to memorize noise in the training data. Since the goal is to reduce overfitting, this action is counterproductive and therefore the least likely to help.

Answer

Use dropout

Answer

Increase regularization strength

Answer

Add more training data

Question 14

A data scientist has trained a binary classification model for fraud detection. The dataset is highly imbalanced (99% non-fraud, 1% fraud). After evaluation, the model shows an accuracy of 99%, but the recall for fraud cases is only 10%. Which metric should the data scientist prioritize to improve the model's performance for fraud detection?

Accepted Answer

F1-score. F1-score balances precision and recall, making it more informative than accuracy for imbalanced datasets. AUC-ROC is also used but F1 directly addresses the trade-off between false positives and false negatives. Precision alone does not capture recall, and Log loss does not directly indicate recall improvement.

Answer

Log loss

Answer

Precision

Answer

Area under the ROC curve (AUC-ROC)

Question 15

A data scientist is preparing a large dataset for training a machine learning model. The dataset contains missing values in several columns. Which approach is the MOST efficient for handling missing values in a large dataset using AWS services?

Accepted Answer

Use Amazon SageMaker Data Wrangler to impute missing values using built-in transforms.. Amazon SageMaker Data Wrangler provides a visual interface and built-in transforms for handling missing values efficiently at scale, without writing custom code. Glue ETL is more code-heavy, and imputation with pandas is not scalable for large datasets. Removing all rows with missing values is not always optimal and may not be efficient.

Answer

Use AWS Glue ETL to write a custom Python script that imputes missing values with the mean.

Answer

Use pandas in a SageMaker notebook to impute missing values with the median.

Answer

Remove all rows with missing values from the dataset.

Question 16

A data scientist is preparing a dataset for a machine learning model that predicts customer churn. The dataset contains a column 'CustomerID' that is a unique identifier. What should the data scientist do with this column before training the model?

Accepted Answer

Remove the column from the feature set.. Option C is correct because 'CustomerID' is a unique identifier with no predictive power for churn. Including it as a feature would cause the model to memorize individual customers rather than learn generalizable patterns, leading to overfitting and poor performance on unseen data. In machine learning, such columns should be removed during data preparation to ensure the model learns from meaningful features.

Answer

Keep the column as a feature because it uniquely identifies each customer.

Answer

Use the column as the target variable.

Answer

Encode the column using one-hot encoding.

Question 17

A company uses AWS Glue to run ETL jobs that prepare data for machine learning. The data is stored in Amazon S3 in Parquet format. A data engineer notices that the Glue job is running slowly and consuming a lot of resources. What is the MOST cost-effective way to improve the performance of the Glue job?

Accepted Answer

Use the G.1X worker type, which provides more memory per worker compared to the Standard worker type.. Increasing the number of DPUs (Data Processing Units) in AWS Glue can improve parallelism and reduce job runtime, but it increases cost. Using G.1X worker type with more memory per worker can improve performance without increasing DPU count, offering better resource utilization. Switching to CSV may degrade performance. Using partition pruning on the source data can reduce data scanned but may not address resource consumption.

Answer

Use partition pruning on the source data to reduce the amount of data processed.

Answer

Switch the output format from Parquet to CSV to reduce processing overhead.

Answer

Use a larger instance type for the Glue job by increasing the number of DPUs.

Question 18

A machine learning team is building a model using a dataset that contains a mix of numerical and categorical features. The categorical features have high cardinality (e.g., zip code with thousands of unique values). The team wants to use Amazon SageMaker for training. Which technique should the team use to encode the high-cardinality categorical features effectively?

Accepted Answer

Apply target encoding (mean encoding) to the high-cardinality features.. For high-cardinality categorical features, target encoding (mean encoding) replaces each category with the mean of the target variable for that category, which captures information without creating a large number of dummy variables. One-hot encoding would create too many features. Label encoding implies ordinal relationships. Hash encoding can cause collisions.

Answer

Apply hash encoding to map categories to a fixed number of buckets.

Answer

Apply one-hot encoding to all categorical features.

Answer

Apply label encoding to assign integer values to each category.

Question 19

A retail company is preparing a dataset for a machine learning model to predict customer churn. The dataset includes customer_id, signup_date, last_purchase_date, total_purchases, average_order_value, and churn_label. The data scientist notices that the 'total_purchases' column has missing values for 15% of the records. The company wants to use AWS Glue for data preparation. Which approach should the data scientist take to handle the missing values while minimizing bias and preserving data integrity?

Accepted Answer

Use AWS Glue DynamicFrame to perform model-based imputation, predicting missing total_purchases using other features like average_order_value and signup_date.. Option C is correct because model-based imputation uses relationships between features (e.g., average_order_value and signup_date) to predict missing total_purchases values, minimizing bias compared to simple mean/median imputation. AWS Glue DynamicFrames support custom transformation logic, allowing you to implement a predictive model (e.g., using Spark MLlib) directly within the Glue ETL job. This approach preserves data integrity by leveraging existing data patterns rather than discarding records or introducing arbitrary constants.

Answer

Use AWS Glue DataBrew to fill missing values with the median of total_purchases.

Answer

Drop all records with missing total_purchases values.

Answer

Replace missing total_purchases with the mean of the non-missing values.

Question 20

A team is deploying a machine learning model for real-time fraud detection. The model must have inference latency under 10 ms and handle up to 1000 requests per second. The model is a gradient boosting model using XGBoost. Which SageMaker hosting configuration is MOST cost-effective while meeting the requirements?

Accepted Answer

Use a SageMaker Multi-Model Endpoint (MME) on an ml.c5.4xlarge instance with auto scaling. Option B is correct because a Multi-Model Endpoint (MME) on a single ml.c5.4xlarge instance allows multiple models to share the same endpoint, reducing cost while still meeting the latency (<10 ms) and throughput (1000 req/s) requirements. The ml.c5.4xlarge provides sufficient compute (16 vCPUs, 32 GB memory) for XGBoost inference, and auto scaling ensures capacity adjusts to handle peak load without over-provisioning.

Answer

Use SageMaker Batch Transform with multiple instances

Answer

Deploy on a single ml.c5.xlarge instance with a real-time endpoint

Answer

Deploy separate real-time endpoints for each model on ml.m5.large instances