MLA-C01 ML Model Development Practice Test 11 — 15 Questions

Question 1

A machine learning engineer is preparing a training dataset in Amazon SageMaker for a binary classification model. The dataset is stored as a single CSV file in Amazon S3 and contains 12 categorical features with high cardinality (thousands of unique values each). The engineer wants to avoid the curse of dimensionality and reduce training time while preserving predictive power. Which preprocessing approach should be used with the SageMaker built-in XGBoost algorithm?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use target encoding (mean encoding) for the categorical features, computed within cross-validation folds to prevent leakage.. High-cardinality categorical features require an encoding that avoids exponential feature expansion. Target encoding summarizes each category by its relationship to the target, reducing dimensionality and training time. Performing it within cross-validation folds prevents data leakage, ensuring that validation metrics remain honest. This approach is compatible with the SageMaker built-in XGBoost algorithm, which accepts numerical features and can benefit from the reduced feature space.

Answer

Convert the categorical features to integer codes using LabelEncoder and pass them directly to XGBoost.

Answer

Hash the categorical features into a fixed number of buckets using a hashing trick, then one-hot encode the hashed values.

Answer

Apply one-hot encoding to all categorical features before training.

Question 2

A machine learning engineer is preparing a training dataset in Amazon SageMaker for a binary classification model. The dataset is stored as a single CSV file in Amazon S3 and contains 12 categorical features with high cardinality (thousands of unique values each). The engineer wants to avoid the curse of dimensionality and reduce training time while preserving predictive power. Which preprocessing approach should be used with the SageMaker built-in XGBoost algorithm?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Create an IAM role with a policy that allows s3:GetObject on the specific S3 bucket and attach it to the SageMaker training job.. The correct approach is to create an IAM role with a policy that grants s3:GetObject permission on the specific S3 bucket and attach it to the SageMaker training job. This follows the principle of least privilege and allows the training job to access the data securely without embedding credentials. It is the standard method for granting SageMaker training jobs access to S3 data.

Answer

Store AWS credentials (access key and secret key) in the training script and use them to access S3 directly.

Answer

Make the S3 bucket public and allow the training job to read the data without authentication.

Answer

Generate a presigned URL for each object in the S3 bucket and pass the URLs as hyperparameters to the training job.

Question 3

A machine learning engineer is preparing a dataset for training a SageMaker model. The dataset contains missing values in several numerical features. The engineer wants to handle these missing values during the training pipeline. Which two methods are valid ways to handle missing values in SageMaker? (Choose two.)

Accepted Answer

Use SageMaker Data Wrangler to impute missing values with the mean or median.. SageMaker Data Wrangler offers built-in transformations to impute missing values, and the built-in XGBoost algorithm natively handles missing values by learning default directions. Both are valid methods for dealing with missing data in a SageMaker training pipeline. The other services are not designed for data preprocessing.

Answer

Use SageMaker Clarify to replace missing values with the mode of each column.

Answer

Use SageMaker Debugger to automatically fill missing values during training.

Answer

Configure SageMaker Automatic Model Tuning to impute missing values via hyperparameter search.

Question 4

A machine learning engineer is training a SageMaker job with the TensorFlow estimator and wants to automatically capture model training metadata such as loss curves and accuracy for later comparison, without writing any custom code. Which SageMaker feature should they enable?

Accepted Answer

SageMaker Experiments. SageMaker Experiments automatically tracks training parameters, metrics, and artifacts when a training job is launched within an experiment context. It requires no custom code to capture loss and accuracy, and it provides a UI and API to compare runs. Debugger, Model Monitor, and Clarify serve different purposes and do not provide automatic scalar metric logging for experiment comparison.

Answer

SageMaker Model Monitor

Answer

SageMaker Clarify

Answer

SageMaker Debugger

Question 5

A financial services company is training a fraud detection model using SageMaker. The dataset is highly imbalanced, with only 0.2% fraudulent transactions. The team wants to optimize the model for recall at a fixed precision of 90%. They are using the SageMaker built-in XGBoost algorithm with binary:logistic objective. Which evaluation metric should they monitor during training and hyperparameter tuning?

Accepted Answer

Precision-Recall AUC (PR-AUC). For imbalanced classification with a precision constraint, the precision-recall curve is the right tool. PR-AUC summarizes performance across thresholds and emphasizes the minority class. The team can use the PR curve to find the threshold where precision is 90%, then read off recall. This directly aligns with their goal of maximizing recall at that precision, unlike ROC-AUC or single-threshold metrics such as F1 or MCC.

Answer

F1 score

Answer

Area Under the ROC Curve (AUC)

Answer

Matthews Correlation Coefficient (MCC)

Question 6

A data scientist is using SageMaker to train a deep learning model with the PyTorch estimator. They want to log custom scalar metrics such as validation accuracy and loss during training so they can monitor the job in SageMaker. Which approach should they use to emit these metrics from the training script?

Accepted Answer

Print the metrics to stdout in a consistent format and define regex patterns in the estimator's metric_definitions parameter.. For SageMaker training jobs, custom metrics are captured by parsing the job's logs. The PyTorch estimator supports metric_definitions, a list of name and regex pairs. Printing metrics to stdout in a consistent format lets SageMaker extract them for monitoring and tuning. Direct API calls or file writes are not the supported mechanism for estimator metric capture.

Answer

Use the sagemaker_metrics API to write metrics to a local file and pass its path to the estimator's metric_definitions.

Answer

Call the SageMaker Metrics API directly from the training container to publish each metric.

Answer

Store metrics in Amazon CloudWatch Logs using the PutMetricData API and then reference them in the estimator.

Question 7

A machine learning engineer is training a deep learning model on SageMaker using the PyTorch estimator. The training job fails with an error indicating that the GPU memory is exhausted. The engineer wants to reduce memory usage without changing the model architecture. Which SageMaker feature should the engineer use?

Accepted Answer

SageMaker model parallelism. SageMaker model parallelism partitions a large model across multiple GPUs, reducing the memory required on each GPU. This directly addresses GPU memory exhaustion without changing the model architecture. Data parallelism replicates the model, which does not help with memory constraints, and the other services are for monitoring or tuning, not memory reduction.

Answer

SageMaker data parallelism

Answer

SageMaker Debugger

Answer

SageMaker Automatic Model Tuning

Question 8

A machine learning engineer is using SageMaker Pipelines to automate the training and deployment of a model. The pipeline includes a processing step for feature engineering, a training step, and a model registration step. The engineer wants to ensure that the pipeline is reproducible and that the model artifacts are versioned. Which two actions should be taken? (Choose two.)

Accepted Answer

Store the training dataset in Amazon S3 with versioning enabled and reference the specific version in the pipeline.. Reproducibility requires pinning inputs, and versioning requires a registry for artifacts. Enabling S3 versioning and referencing a specific version ensures the training data is immutable and traceable. Registering the model in the SageMaker Model Registry creates a versioned entry that links to the artifacts and metadata. Together, these actions provide a reproducible pipeline and a versioned model catalog.

Answer

Configure the training step to use the latest SageMaker training image without specifying a version.

Answer

Enable pipeline caching to reuse previous step outputs.

Answer

Use SageMaker Experiments to log the pipeline execution parameters and metrics.

Question 9

A machine learning engineer is preparing a training dataset in Amazon SageMaker for a binary classification model. The dataset is stored as a single CSV file in Amazon S3 and contains 12 categorical features with high cardinality (thousands of unique values each). The engineer wants to avoid the curse of dimensionality and reduce training time while preserving predictive power. Which preprocessing approach should be used with the SageMaker built-in XGBoost algorithm?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use a framework-provided dataset or iterator that reads from the path in SM_CHANNEL_TRAIN and yields batches lazily during training.. SageMaker downloads the contents of each training channel to a local path and exposes that path through the SM_CHANNEL_TRAIN environment variable. Using a framework-native dataset or iterator that reads lazily from that path streams batches on demand, so memory stays bounded regardless of dataset size. Direct S3 reads, EFS mounts, and volume resizing do not address in-memory loading.

Answer

Set the estimator's volume_size parameter large enough to hold the dataset and enable cacheutils to copy the data into the container's memory before training starts.

Answer

Read the dataset directly from the Amazon S3 URI passed in the SM_CHANNEL_TRAIN environment variable using the AWS SDK for Python (Boto3) inside the training loop.

Answer

Configure the estimator with an Amazon EFS file system and mount the dataset into the container, then use memory-mapped file access for all training samples.

Question 10

A machine learning engineer is using SageMaker Debugger to monitor a training job and wants to detect issues early. The engineer wants to receive alerts when the training job is likely to fail due to vanishing gradients and when the loss is not decreasing. Which two actions should the engineer take to achieve this? (Choose two.)

Accepted Answer

Configure a built-in rule for vanishing gradients, such as the VanishingGradient rule.. The VanishingGradient and LossNotDecreasing built-in rules in SageMaker Debugger are designed to automatically detect the specified training issues and trigger alerts. They are easy to configure and provide real-time monitoring, which is exactly what the engineer needs. Custom rules or other services would require more effort or are not suited for this purpose.

Answer

Use SageMaker Clarify to detect bias in the gradients.

Answer

Set up a custom rule using a Python script that analyzes gradients and loss.

Answer

Configure the training job to save all tensors to Amazon S3 for manual analysis.

Question 11

A company is training a deep learning model with SageMaker and wants to reduce training time by using pipe mode instead of file mode for a large dataset stored as TFRecord files in Amazon S3. After switching the estimator's input mode to Pipe, the training job fails immediately with a dataset format error. The data scientist confirms the files are valid TFRecords and that the same script works with File mode. What is the most likely cause?

Accepted Answer

The training script is still trying to read files from the local filesystem path instead of consuming the named pipe provided by SageMaker.. Pipe mode exposes training data as a named pipe rather than as files on disk. A script written for File mode typically lists files and opens them by path, which fails when SM_CHANNEL_TRAIN points to a FIFO. To use Pipe mode, the script must consume the stream sequentially, for example with a framework reader designed for pipes. Format conversion and Region settings are unrelated to this error.

Answer

The S3 bucket and the training job are in different AWS Regions, so Pipe mode cannot stream the objects across Regions.

Answer

Pipe mode only supports RecordIO-encoded data for built-in algorithms, so TFRecord files must be converted to RecordIO before use.

Answer

The estimator is missing the enable_network_isolation parameter, which is required for Pipe mode to establish the streaming connection to S3.

Question 12

A machine learning engineer is using a SageMaker training job with a custom training script. They need to save the trained model artifacts to Amazon S3 so that the model can be deployed later. Which parameter in the SageMaker estimator should they configure to specify the S3 location for model artifacts?

Accepted Answer

output_path. The output_path parameter in the SageMaker estimator defines the S3 URI where training job artifacts, including the final model, are stored. It allows control over the bucket and prefix, which is essential for governance and deployment. Other parameters like model_dir, code_location, and dependencies serve different purposes in the training lifecycle.

Answer

code_location

Answer

model_dir

Answer

dependencies

Question 13

A machine learning engineer is using SageMaker to train a model and wants to automatically stop a training job when the validation loss has not improved for 10 consecutive epochs, while still saving the best model artifacts. The engineer is using the SageMaker training toolkit in a custom container. Which combination of actions should the engineer take?

Accepted Answer

Implement early stopping in the training script by tracking validation loss and calling the SageMaker training toolkit's save_model or exiting the loop when the patience threshold is reached.. Early stopping based on validation loss patience is logic that belongs in the training script when using a custom container. The script should evaluate validation loss each epoch, track the best value, and stop after 10 epochs without improvement, saving the best model artifacts. Estimator time limits, tuning strategies, and Debugger rules do not implement this per-epoch behavior.

Answer

Configure the estimator's stopping condition with the MaxRuntimeInSeconds parameter and set the checkpoint_s3_uri to save intermediate models.

Answer

Use SageMaker Automatic Model Tuning with the Bayesian strategy and set the max_parallel_jobs parameter to 1 to enable early stopping across trials.

Answer

Enable SageMaker Debugger with the vanishing_gradient rule and configure a stop condition on the rule to halt training when validation loss plateaus.

Question 14

A machine learning engineer is deploying a model to a SageMaker endpoint for real-time inference. The model must return predictions within 100 milliseconds for 95% of requests. The engineer wants to monitor the endpoint's latency and automatically roll back if latency exceeds the threshold. Which combination of SageMaker features should be used?

Accepted Answer

SageMaker deployment guardrails with a blue/green deployment and CloudWatch alarms for latency.. SageMaker deployment guardrails are designed to safely update endpoints with automatic rollback. By associating a CloudWatch alarm that monitors model latency, the guardrail can trigger a rollback if the alarm state changes to ALARM. This provides the required automatic protection. Other features like Model Monitor, Inference Recommender, or Clarify do not offer this latency-based rollback capability out of the box.

Answer

SageMaker Model Monitor with a custom monitoring schedule and AWS Lambda for rollback.

Answer

SageMaker Clarify for bias detection and AWS Step Functions for rollback orchestration.

Answer

SageMaker Inference Recommender to select the optimal instance type and automatic scaling.

Question 15

A company is training a deep learning model using SageMaker and wants to reduce the time spent on data loading from Amazon S3 during training. The training dataset consists of many small files. Which approach is MOST effective to accelerate data loading?

Accepted Answer

Package the small files into a few large files (e.g., TFRecord or RecordIO) and use File mode.. Many small files cause high S3 request overhead. Consolidating them into a few large files in a format like TFRecord or RecordIO reduces the number of requests and allows efficient sequential reads. File mode then downloads these large files quickly to local storage, significantly speeding up data loading compared to streaming many small files or adding instances.

Answer

Use SageMaker Pipe mode to stream data directly from S3.

Answer

Increase the number of training instances in the cluster.

Answer

Enable SageMaker Debugger to monitor data loading metrics.