MLS-C01 Mock Exam 73 — 25 Questions

Question 1

A team is training a deep learning model with the SageMaker built-in Object Detection algorithm on a custom dataset and observes that validation mean average precision plateaus early while training loss keeps decreasing. The team wants to reduce overfitting without changing the algorithm. (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable early stopping on the validation mean average precision metric so training halts when that metric stops improving.. Overfitting in the built-in Object Detection algorithm is countered by increasing effective data diversity and by stopping training at the right time. Augmentation hyperparameters such as random flip, crop, and color jitter broaden the input distribution, while early stopping on validation mean average precision halts training once the metric stops improving.

Answer

Increase the number of training epochs while keeping all other hyperparameters unchanged.

Answer

Reduce the mini-batch size to one so that gradient updates are computed per individual image.

Answer

Remove the validation dataset from training and evaluate only on the training set to confirm the model is learning.

Question 2

A company runs an Amazon Kinesis Data Firehose delivery stream that buffers incoming records for 300 seconds and 5 MB before delivering them to Amazon S3. Downstream ML feature pipelines read the S3 objects every 15 minutes. Analysts report that the freshest features are sometimes almost 20 minutes old. The team wants to reduce end-to-end latency without changing the downstream consumers or switching to a different ingestion service. What should the data engineer do?

Accepted Answer

Change the Firehose buffer interval to 60 seconds and buffer size to 1 MB, and configure S3 event notifications to invoke the downstream processing as objects arrive.. Two independent delays combine here: the Firehose buffer holds records up to 300 seconds before writing to Amazon S3, and the consumers poll on a fixed 15-minute schedule. Reducing the buffer interval and size shortens the first delay, while S3 event notifications eliminate the second by pushing work as soon as objects land. Together they cut end-to-end feature latency substantially.

Answer

Change the Firehose delivery stream's buffer interval to 60 seconds and buffer size to 1 MB, and enable dynamic partitioning on the destination.

Answer

Enable Amazon S3 Transfer Acceleration on the destination bucket and increase the Firehose delivery stream's shard count.

Answer

Increase the Firehose buffer size to 128 MB and keep the 300-second interval so that each delivery contains a complete batch.

Question 3

A company wants to deploy a trained SageMaker model to production with the ability to shift a small percentage of live traffic to a new model version for testing. The company needs to monitor the new model's performance without affecting the majority of users. Which deployment strategy should be used?

Accepted Answer

SageMaker endpoint with production variants and traffic distribution. Using production variants on a SageMaker endpoint allows you to deploy multiple models and assign traffic weights. By setting a low weight for the new variant, you can test its performance on a small slice of live traffic while the existing variant serves the majority, enabling safe experimentation.

Answer

SageMaker Serverless Inference

Answer

SageMaker Batch Transform with a random split

Answer

SageMaker multi-model endpoint

Question 4

A data scientist is performing EDA on a dataset where the target is binary and one predictor is a categorical column with 1,200 distinct levels, most appearing fewer than ten times. The scientist wants a numeric representation that preserves information while keeping the feature dimensionality manageable for a downstream linear model. Which encoding approach is most appropriate?

Accepted Answer

Apply target (mean) encoding with out-of-fold computation and smoothing, mapping each level to a regularized statistic.. Out-of-fold target encoding with smoothing reduces the high-cardinality categorical column to a single numeric feature while retaining the level-to-target relationship. Computing the statistic out of fold prevents target leakage into training rows, and smoothing shrinks rare-level values toward the global mean so that levels with very few observations do not produce wildly unstable encodings that a linear model would overfit.

Answer

One-hot encode all 1,200 levels and rely on L1 regularization to zero out the uninformative columns.

Answer

Hash the categorical values into 50 buckets and one-hot encode the resulting bucket identifiers.

Answer

Use frequency encoding, replacing each level with its count in the dataset.

Question 5

A machine learning engineer is using Amazon SageMaker to train a model on a large dataset stored in Amazon S3. The dataset is in CSV format and is 1 TB in size. The training job is taking a long time to start because the data is being downloaded to the training instance's Amazon EBS volume. The engineer wants to reduce the startup time and improve training efficiency. Which solution should the engineer implement?

Accepted Answer

Convert the dataset to the RecordIO format and use Pipe mode in the SageMaker training job.. Pipe mode in SageMaker streams training data directly from Amazon S3 to the training algorithm, bypassing the need to download the entire dataset to the EBS volume. This drastically reduces startup time, especially for large datasets like 1 TB. Converting to RecordIO format is recommended because it supports efficient streaming and is compatible with Pipe mode. Other options either do not address the download bottleneck or are unrelated to performance.

Answer

Use SageMaker's File mode with a larger instance type that has higher network bandwidth.

Answer

Enable SageMaker's network isolation and use a VPC endpoint to accelerate the S3 download.

Answer

Increase the size of the Amazon EBS volume attached to the training instance to accommodate the dataset faster.

Question 6

A data engineering team is preparing a training dataset stored as 12,000 small gzip-compressed CSV objects (about 40 KB each) in an Amazon S3 prefix. A SageMaker training job reading these objects directly spends most of its time on per-object GET requests rather than on model computation. The team must reduce I/O overhead without changing the data content. Which approach should the team take?

Accepted Answer

Convert the objects into a single RecordIO-Web (application/x-recordio-protobuf) file and point the training job's channel at that file.. Packing thousands of tiny objects into a sharded container format such as RecordIO-Web lets SageMaker stream large sequential blocks, eliminating the per-object request overhead that dominates runtime. Storage-class changes, transfer acceleration, and more workers all leave the fundamental small-file access pattern intact, so they do not address the measured bottleneck.

Answer

Enable S3 Transfer Acceleration on the bucket and keep the 12,000 objects as they are.

Answer

Increase the training instance count so that more workers read the small objects in parallel.

Answer

Change the objects' storage class from S3 Standard to S3 Intelligent-Tiering before training.

Question 7

A company is running a hyperparameter tuning job on Amazon SageMaker to optimize a neural network. The tuning job is using Bayesian optimization and has completed 20 training jobs. The best model so far has an accuracy of 92%. The company wants to reduce the cost of the tuning job without significantly impacting the final model performance. Which action should a machine learning engineer take?

Accepted Answer

Enable early stopping in the hyperparameter tuning job to stop poorly performing training jobs early.. Enabling early stopping in a SageMaker hyperparameter tuning job allows the tuner to automatically stop training jobs that are not improving on the objective metric. This reduces the number of epochs and compute resources spent on unpromising trials, lowering cost. The best-performing jobs continue, so the final model quality is maintained. This is the recommended approach to optimize cost without sacrificing performance.

Answer

Change the objective metric to a less resource-intensive metric like training loss instead of validation accuracy.

Answer

Reduce the maximum number of training jobs for the tuning job to 10.

Answer

Use a smaller instance type for the training jobs to reduce cost per job.

Question 8

A data scientist is using Amazon SageMaker to train a model with the built-in K-Means algorithm. The data scientist wants to determine the optimal number of clusters for a dataset. Which approach should the data scientist use?

Accepted Answer

Use the elbow method by plotting the within-cluster sum of squares against the number of clusters.. The elbow method is a widely used heuristic for determining the optimal number of clusters in K-Means. It plots the within-cluster sum of squares against k and looks for an elbow, where adding more clusters yields diminishing returns. Other options are either incorrect (silhouette should be maximized, AIC not applicable) or arbitrary (feature count).

Answer

Use the silhouette score and select the k with the lowest score.

Answer

Use the Akaike information criterion (AIC) and select the k with the highest AIC value.

Answer

Set the number of clusters to the number of features in the dataset.

Question 9

A data scientist is training a SageMaker built-in k-means model to segment customers using a feature set that includes annual income in dollars and a normalized engagement score between 0 and 1. The resulting clusters are almost entirely explained by income, and engagement barely influences assignment. The scientist wants engagement to contribute meaningfully to the distance calculation without discarding the income feature. Which approach should they take?

Accepted Answer

Scale or normalize the features so each contributes comparably to the distance metric.. k-means assigns points by Euclidean distance, so raw units determine influence: a feature spanning thousands of units dominates one spanning a single unit. Standardizing or min-max scaling the inputs equalizes their contribution, allowing engagement to affect assignments while retaining income. Changing k, altering the distance metric, or removing income do not address the scale disparity directly.

Answer

Increase the number of clusters k until engagement separates the segments.

Answer

Reduce the feature set to only the engagement score before training.

Answer

Switch the distance metric to cosine similarity in the training job.

Question 10

A machine learning engineer is training a gradient boosting model with Amazon SageMaker on a tabular dataset where the target is highly skewed, with a small number of extreme high values. The engineer notices that the model's predictions are pulled toward the extreme values and wants to reduce the influence of those outliers during training without discarding any records. The engineer is using the SageMaker built-in XGBoost algorithm. Which approach should the engineer take?

Accepted Answer

Use the pseudo_huber or fair objective, which grows linearly rather than quadratically for large residuals.. Robust objectives such as pseudo_huber and fair change the loss from quadratic to linear growth for large residuals, which is exactly what reduces the leverage of a few extreme target values during boosting. They preserve all training rows, unlike trimming or winsorizing, and they act directly on the gradient rather than indirectly through tree depth or global regularization.

Answer

Set the lambda and alpha regularization parameters to very high values to shrink all leaf weights.

Answer

Increase the max_depth parameter so trees can isolate the extreme values into their own leaves.

Answer

Set the objective parameter to reg:squarederror and rely on the default learning rate.

Question 11

A data scientist is building a time-series forecasting model using Amazon SageMaker. The dataset contains daily sales data for 500 products over two years, with strong weekly and yearly seasonality. The data scientist wants to use the built-in DeepAR algorithm. Which data format is required for the training input?

Accepted Answer

A JSON Lines file where each line is a JSON object containing a start timestamp, a target array, and optional dynamic features.. DeepAR expects training data in JSON Lines format, where each line represents a single time series with a 'start' timestamp, a 'target' array of values, and optional 'dynamic_feat' for covariates. This format allows the algorithm to process multiple related time series and learn cross-series patterns. The data scientist must convert the dataset into this format before training.

Answer

A CSV file where each row represents a time step and columns are the target and related features.

Answer

A RecordIO protobuf file with each record containing a sequence of floats and a label.

Answer

A Parquet file with columns for item_id, timestamp, and sales, sorted by timestamp.

Question 12

A machine learning engineer is using Amazon SageMaker to train a model with the built-in Object2Vec algorithm to learn embeddings for a recommendation system. The training data consists of user-item interaction pairs. The engineer wants to evaluate the quality of the learned embeddings. Which metric or method is most appropriate for assessing the embeddings' usefulness for downstream tasks?

Accepted Answer

Use the embeddings as features in a downstream classifier and evaluate its performance on a held-out test set.. The most reliable way to assess embedding quality is to evaluate their performance in a downstream task. For recommendation systems, this could involve using the embeddings to predict user-item interactions and measuring metrics like recall@k or NDCG. This approach directly tests whether the embeddings capture meaningful patterns that improve task outcomes, rather than relying on intrinsic metrics that may not correlate with practical utility.

Answer

Compute the average cosine similarity between all pairs of embeddings to ensure they are diverse.

Answer

Monitor the training loss curve and ensure it converges to a low value.

Answer

Visualize the embeddings using t-SNE and check for clear cluster separation.

Question 13

A company has deployed a real-time inference endpoint on Amazon SageMaker using a custom container. The endpoint is experiencing high latency and occasional timeouts during peak traffic. The model is CPU-bound and the container is configured with one CPU core and 2 GB of memory. The company wants to reduce latency and improve throughput without changing the model. Which action should a machine learning engineer take?

Accepted Answer

Increase the number of CPU cores allocated to the container and enable multi-threaded inference in the model server.. For a CPU-bound model, the best way to reduce latency and improve throughput is to allocate more CPU resources and enable multi-threaded inference. This allows the container to handle multiple requests concurrently. Auto scaling increases capacity but not per-request speed, GPU instances are unnecessary for CPU-bound models, and serverless may add cold start latency.

Answer

Enable auto scaling for the endpoint based on the InvocationsPerInstance metric.

Answer

Configure the endpoint to use a GPU instance type and enable GPU acceleration in the container.

Answer

Deploy the model to a SageMaker Serverless Inference endpoint to automatically scale with traffic.

Question 14

A data scientist is using Amazon SageMaker to train a linear learner model for a regression task. The training data is stored in Amazon S3 as CSV files. The scientist wants to ensure the model does not overfit and also wants to automatically find the best hyperparameters. Which combination of SageMaker features should be used?

Accepted Answer

Use SageMaker automatic model tuning (hyperparameter tuning job) with the linear learner algorithm and specify a range for the L1 or L2 regularization parameter.. To automatically find the best hyperparameters while controlling overfitting, SageMaker automatic model tuning is the right service. By defining a hyperparameter range for the regularization parameter of the linear learner algorithm, the tuning job explores combinations and selects the one that optimizes the objective metric, effectively balancing fit and complexity. Other options either lack automatic search or do not directly address overfitting through regularization.

Answer

Use SageMaker Debugger to monitor training and automatically adjust hyperparameters.

Answer

Use the built-in linear learner with default hyperparameters and enable early stopping.

Answer

Use SageMaker Experiments to track different manual training runs and pick the best one.

Question 15

A data scientist has a trained scikit-learn model artifact in Amazon S3 and needs a fully managed HTTPS endpoint that returns predictions in real time with automatic patching of the underlying infrastructure. The team has no Docker expertise and wants the fewest possible steps. Which SageMaker deployment approach should the data scientist use?

Accepted Answer

Use the SageMaker scikit-learn built-in framework container by calling the model deploy method with the framework version and the S3 model artifact path, letting SageMaker create the model, endpoint configuration, and endpoint.. SageMaker ships managed framework containers for scikit-learn, so an existing model artifact can be deployed by supplying the framework version and artifact path to the deploy method. SageMaker then creates the model, endpoint configuration, and endpoint, and handles infrastructure patching. This meets the managed, real-time, low-effort requirements without Docker work or custom serving code.

Answer

Build a custom inference container with the SageMaker Inference Toolkit, push it to Amazon ECR, and create a model that references the image URI and the model artifact in S3.

Answer

Package the model into an AWS Lambda function using the scikit-learn Python library and expose it through an Amazon API Gateway REST API with a proxy integration.

Answer

Create a SageMaker batch transform job with the model artifact and invoke it on demand each time a prediction is requested, using the job output as the response.

Question 16

A data scientist is using Amazon SageMaker to train a model on a large dataset. The training job is taking longer than expected, and the data scientist suspects that the input data pipeline is the bottleneck. The dataset consists of many small files stored in Amazon S3. The data scientist wants to improve the training throughput by optimizing data input. Which action should the data scientist take?

Accepted Answer

Combine the small files into larger files (e.g., TFRecord or RecordIO format) before training.. Combining many small files into larger files in an optimized format like TFRecord or RecordIO reduces the overhead of file operations and allows for more efficient sequential reads, which directly improves data input throughput. This is a best practice for training on large datasets with many small files.

Answer

Use a larger instance type with more CPU cores to speed up data loading.

Answer

Use the SageMaker Pipe mode with the built-in algorithm to stream data directly from S3.

Answer

Increase the number of epochs to allow the model to learn more from the data.

Question 17

A machine learning platform team must give a SageMaker training job access to training data in Amazon S3 and to a secret stored in AWS Secrets Manager that holds database credentials. Security policy forbids long-term access keys on the training instances and requires that the permissions be scoped to only the required resources. The team will pass an execution role to the training job. Which two actions should the team take to meet these requirements? (Choose two.)

Accepted Answer

Attach an IAM policy to the SageMaker execution role that grants s3:GetObject on the specific training data prefix and secretsmanager:GetSecretValue on the specific secret ARN.. SageMaker training jobs obtain credentials by assuming an execution role whose trust policy permits the SageMaker service principal. Passing that role ARN at job creation lets the container receive temporary, automatically rotated credentials. Scoping the attached policy to the exact S3 prefix and the specific secret ARN satisfies least privilege, while no static keys are ever placed on the instances.

Answer

Enable network isolation on the training job and retrieve the secret from the container using the AWS SDK default credential chain.

Answer

Grant the training job access by attaching the AdministratorAccess managed policy to the execution role.

Answer

Store an IAM user's access key and secret key as environment variables in the training job definition.

Question 18

A machine learning engineer is training a linear regression model in Amazon SageMaker using the built-in Linear Learner algorithm. The dataset contains a highly skewed target variable with extreme outliers. The engineer wants to reduce the influence of these outliers on the model without removing any data points. Which of the following approaches should the engineer take?

Accepted Answer

Apply a logarithmic transformation to the target variable before training.. The correct approach is to apply a logarithmic transformation to the target variable. Linear Learner minimizes squared error, which is highly sensitive to outliers. By compressing the range of the target, the transformation reduces the impact of extreme values on the loss, leading to a more robust model. Other options either do not address the data distribution or are inappropriate for regression.

Answer

Increase the number of epochs in the training job.

Answer

Set the 'predictor_type' hyperparameter to 'binary_classifier'.

Answer

Use a larger instance type for training.

Question 19

A machine learning engineer is analyzing a dataset with a highly skewed numerical feature. The engineer wants to reduce the skewness to make the feature more suitable for a linear model. After applying a transformation, the engineer wants to verify the effectiveness of the transformation. Which combination of techniques should the engineer use to assess the skewness before and after the transformation?

Accepted Answer

Compute the skewness coefficient and create a histogram of the feature before and after the transformation.. The skewness coefficient is a direct numerical measure of asymmetry; values greater than zero indicate right skew, less than zero indicate left skew. Histograms visually show the distribution's shape. By comparing skewness values and histograms before and after transformation, the engineer can objectively determine if skewness was reduced. This combination is standard for assessing transformations like log or Box-Cox.

Answer

Calculate the Pearson correlation with the target variable and create a scatter plot of the feature versus the target.

Answer

Perform a Kolmogorov-Smirnov test against a uniform distribution and calculate the interquartile range.

Answer

Calculate the mean and standard deviation, and create a Q-Q plot against a normal distribution.

Question 20

A data scientist is analyzing a dataset with two numerical features, 'age' and 'income', and suspects a non-linear relationship. The scientist computes the Pearson correlation coefficient and finds it to be near zero. However, a scatter plot reveals a clear U-shaped pattern. Which statement best explains this discrepancy?

Accepted Answer

Pearson correlation only detects linear relationships, so a non-linear pattern can result in a near-zero coefficient.. Pearson correlation quantifies linear association only. When a relationship is non-linear, such as U-shaped, the linear correlation can be near zero even though the variables are strongly dependent. The scatter plot provides a more complete view. Therefore, the discrepancy is explained by the limitation of Pearson correlation to detect non-linear patterns.

Answer

The scatter plot is misleading because it suggests a relationship that the correlation coefficient correctly refutes.

Answer

The Pearson correlation coefficient is incorrectly computed because it requires normally distributed variables.

Answer

Pearson correlation is robust to outliers, so the near-zero value must be due to a calculation error.