MLA-C01 Practice Test 6 — 10 Questions

Question 1

A financial services company has a SageMaker pipeline that trains a fraud detection model daily. The pipeline consists of three steps: preprocessing (using a Spark script), training (XGBoost), and evaluation. The evaluation step calculates the F1 score and compares it to a threshold of 0.95. If the F1 score is below 0.95, the pipeline should fail and notify the team via email. The team implemented this using a Condition step that checks if the F1 score is greater than or equal to 0.95. If true, the pipeline proceeds to register the model; if false, the pipeline fails. However, the team notices that even when the F1 score is 0.94, the pipeline continues to the registration step. The evaluation script outputs the F1 score as a float with two decimal places in a JSON file. The Condition step uses the expression: $.evaluation.metrics.f1_score >= 0.95. What is the most likely cause of the issue?

Accepted Answer

The evaluation script outputs the F1 score as a string, and string comparison '0.94' >= '0.95' evaluates to true because it is lexicographically compared. The most likely cause is that the evaluation script outputs the F1 score as a string (e.g., "0.94") rather than a numeric value. In AWS SageMaker Pipelines, the Condition step evaluates expressions using JSONPath, and when comparing two values, if one is a string, the comparison is performed lexicographically (character by character). Lexicographically, the string "0.94" is considered greater than or equal to "0.95" because '9' > '5' after the decimal point, causing the condition to pass incorrectly.

Answer

The evaluation step must be split into two steps: one for evaluation and one for condition check

Answer

The Condition step cannot be used to check metric values; it can only check step status

Answer

The threshold should be set to 0.95 but the Condition step uses a less than or equal operator

Question 2

An e-commerce company uses Amazon SageMaker to train a model that predicts click-through rates. The training data includes a timestamp column 'click_time' and a categorical feature 'device_type' (8 values). They notice that the model's performance degrades over time because the data distribution shifts. They want to ensure the training data represents the most recent behavior. The data is stored in a daily partitioned S3 bucket (e.g., s3://bucket/data/2024-01-01/). The total dataset size is 500 GB. Which approach should they take to prepare the training data while minimizing bias and cost?

Accepted Answer

Select only the data from the last 30 days to train the model.. Option A is correct because selecting only the last 30 days of data directly addresses the data distribution shift by focusing on the most recent user behavior, which is critical for click-through rate prediction. This approach minimizes bias from outdated patterns and reduces training cost by using a smaller, relevant dataset (approximately 500 GB / 365 * 30 ≈ 41 GB). SageMaker training jobs benefit from this reduced volume through faster data loading and lower compute costs.

Answer

Take a random sample of 10% of the rows from the entire dataset.

Answer

Use all historical data and let the model learn the temporal patterns.

Answer

Downsample older data exponentially so that recent data is overrepresented.

Question 3

A company has trained a custom model using PyTorch on Amazon SageMaker. The model achieves high accuracy, but the inference latency on a real-time endpoint is above the required 100ms SLA. The model is a large neural network with many layers. The company wants to reduce latency without significantly impacting accuracy. Which approach should the machine learning engineer take?

Accepted Answer

Use SageMaker Neo to compile the model for the target hardware.. SageMaker Neo compiles trained models into an optimized binary for the target hardware (e.g., CPU, GPU, or Inferentia). It applies graph-level optimizations, operator fusion, and quantization-aware tuning to reduce inference latency while preserving model accuracy. This directly addresses the need to lower latency below 100ms without retraining or sacrificing significant accuracy.

Answer

Reduce the batch size used during inference.

Answer

Increase the instance size of the endpoint.

Answer

Implement a cache for frequent inference requests.

Question 4

A model has high training accuracy but low validation accuracy. Which action is least likely to reduce overfitting?

Accepted Answer

Increase model complexity. Increasing model complexity (e.g., adding more layers or parameters) makes the model more flexible, which typically exacerbates overfitting by allowing it to memorize noise in the training data. Since the goal is to reduce overfitting, this action is counterproductive and therefore the least likely to help.

Answer

Use dropout

Answer

Increase regularization strength

Answer

Add more training data

Question 5

A data scientist has trained a binary classification model for fraud detection. The dataset is highly imbalanced (99% non-fraud, 1% fraud). After evaluation, the model shows an accuracy of 99%, but the recall for fraud cases is only 10%. Which metric should the data scientist prioritize to improve the model's performance for fraud detection?

Accepted Answer

F1-score. F1-score balances precision and recall, making it more informative than accuracy for imbalanced datasets. AUC-ROC is also used but F1 directly addresses the trade-off between false positives and false negatives. Precision alone does not capture recall, and Log loss does not directly indicate recall improvement.

Answer

Log loss

Answer

Precision

Answer

Area under the ROC curve (AUC-ROC)

Question 6

A data scientist is preparing a large dataset for training a machine learning model. The dataset contains missing values in several columns. Which approach is the MOST efficient for handling missing values in a large dataset using AWS services?

Accepted Answer

Use Amazon SageMaker Data Wrangler to impute missing values using built-in transforms.. Amazon SageMaker Data Wrangler provides a visual interface and built-in transforms for handling missing values efficiently at scale, without writing custom code. Glue ETL is more code-heavy, and imputation with pandas is not scalable for large datasets. Removing all rows with missing values is not always optimal and may not be efficient.

Answer

Use AWS Glue ETL to write a custom Python script that imputes missing values with the mean.

Answer

Use pandas in a SageMaker notebook to impute missing values with the median.

Answer

Remove all rows with missing values from the dataset.

Question 7

A data scientist is preparing a dataset for a machine learning model that predicts customer churn. The dataset contains a column 'CustomerID' that is a unique identifier. What should the data scientist do with this column before training the model?

Accepted Answer

Remove the column from the feature set.. Option C is correct because 'CustomerID' is a unique identifier with no predictive power for churn. Including it as a feature would cause the model to memorize individual customers rather than learn generalizable patterns, leading to overfitting and poor performance on unseen data. In machine learning, such columns should be removed during data preparation to ensure the model learns from meaningful features.

Answer

Keep the column as a feature because it uniquely identifies each customer.

Answer

Use the column as the target variable.

Answer

Encode the column using one-hot encoding.

Question 8

A company uses AWS Glue to run ETL jobs that prepare data for machine learning. The data is stored in Amazon S3 in Parquet format. A data engineer notices that the Glue job is running slowly and consuming a lot of resources. What is the MOST cost-effective way to improve the performance of the Glue job?

Accepted Answer

Use the G.1X worker type, which provides more memory per worker compared to the Standard worker type.. Increasing the number of DPUs (Data Processing Units) in AWS Glue can improve parallelism and reduce job runtime, but it increases cost. Using G.1X worker type with more memory per worker can improve performance without increasing DPU count, offering better resource utilization. Switching to CSV may degrade performance. Using partition pruning on the source data can reduce data scanned but may not address resource consumption.

Answer

Use partition pruning on the source data to reduce the amount of data processed.

Answer

Switch the output format from Parquet to CSV to reduce processing overhead.

Answer

Use a larger instance type for the Glue job by increasing the number of DPUs.

Question 9

A machine learning team is building a model using a dataset that contains a mix of numerical and categorical features. The categorical features have high cardinality (e.g., zip code with thousands of unique values). The team wants to use Amazon SageMaker for training. Which technique should the team use to encode the high-cardinality categorical features effectively?

Accepted Answer

Apply target encoding (mean encoding) to the high-cardinality features.. For high-cardinality categorical features, target encoding (mean encoding) replaces each category with the mean of the target variable for that category, which captures information without creating a large number of dummy variables. One-hot encoding would create too many features. Label encoding implies ordinal relationships. Hash encoding can cause collisions.

Answer

Apply hash encoding to map categories to a fixed number of buckets.

Answer

Apply one-hot encoding to all categorical features.

Answer

Apply label encoding to assign integer values to each category.

Question 10

A retail company is preparing a dataset for a machine learning model to predict customer churn. The dataset includes customer_id, signup_date, last_purchase_date, total_purchases, average_order_value, and churn_label. The data scientist notices that the 'total_purchases' column has missing values for 15% of the records. The company wants to use AWS Glue for data preparation. Which approach should the data scientist take to handle the missing values while minimizing bias and preserving data integrity?

Accepted Answer

Use AWS Glue DynamicFrame to perform model-based imputation, predicting missing total_purchases using other features like average_order_value and signup_date.. Option C is correct because model-based imputation uses relationships between features (e.g., average_order_value and signup_date) to predict missing total_purchases values, minimizing bias compared to simple mean/median imputation. AWS Glue DynamicFrames support custom transformation logic, allowing you to implement a predictive model (e.g., using Spark MLlib) directly within the Glue ETL job. This approach preserves data integrity by leveraging existing data patterns rather than discarding records or introducing arbitrary constants.

Answer

Use AWS Glue DataBrew to fill missing values with the median of total_purchases.

Answer

Drop all records with missing total_purchases values.

Answer

Replace missing total_purchases with the mean of the non-missing values.