MLA-C01 Practice Test 25 — 15 Questions

Question 1

A company is building a binary classifier for credit default prediction. The dataset is highly imbalanced (98% no default). They want to maximize recall for the minority class while maintaining reasonable precision. Which metric should be optimized during hyperparameter tuning?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

F1 score. F1 score balances precision and recall, making it suitable for imbalanced datasets when both metrics are important. Other options are less appropriate because accuracy is misleading due to imbalance, precision ignores recall, and AUC-ROC does not directly optimize recall at a decision threshold.

Answer

AUC-ROC

Answer

Accuracy

Answer

Precision

Question 2

A machine learning team notices an increase in 5XXError count for a SageMaker endpoint. They want to set up automated remediation. Which THREE actions should they take? (Select THREE)

Accepted Answer

Add an SNS topic as the alarm action. Adding an SNS topic as the alarm action enables automated notifications when the CloudWatch Alarm triggers on the 5XXError metric. This allows the team to receive alerts and trigger downstream remediation workflows, such as invoking a Lambda function, without manual intervention.

Answer

Increase the endpoint instance count manually

Answer

Enable detailed monitoring on the endpoint

Question 3

A company uses Amazon Kinesis Data Streams to ingest real-time user interactions and wants to store the data in Amazon S3 for historical analysis. They need to transform the data (e.g., add timestamps, filter records) before storage. Which approach is MOST cost-effective?

Accepted Answer

Use Kinesis Data Firehose with AWS Lambda transformation. The most cost-effective because Kinesis Data Firehose natively integrates with AWS Lambda for near-real-time, per-record transformations, and directly writes to S3 without requiring a separate compute resource to manage. This serverless approach minimizes operational overhead and cost compared to running continuous ETL jobs or managing a custom consumer application.

Answer

Use AWS Glue ETL to read from the stream in micro-batches and write to S3

Answer

Use Kinesis Data Analytics for SQL transformations and then output to Firehose

Answer

Use Kinesis Client Library (KCL) to consume the stream, transform, and write to S3

Question 4

A data scientist needs to version and manage multiple models for a team of five. The team frequently experiments with different algorithms and hyperparameters. They need a centralized registry to store, deploy, and compare model versions. Which AWS service should the data scientist use?

Accepted Answer

Use Amazon SageMaker Model Registry.. Amazon SageMaker Model Registry is the correct choice because it provides a centralized repository specifically designed for cataloging, versioning, approving, and deploying machine learning models. It integrates natively with SageMaker pipelines and endpoints, enabling the team to compare model versions, manage metadata (e.g., hyperparameters, metrics), and promote models through stages (e.g., from staging to production) with approval workflows.

Answer

Store each model artifact in Amazon S3 with manual versioning in the key name.

Answer

Use AWS Config to track model version changes.

Answer

Use AWS CodeArtifact to store model packages.

Question 5

A company wants to monitor their machine learning model for bias over time. Which THREE AWS services or features can they use to achieve this? (Choose THREE.)

Accepted Answer

Amazon SageMaker Clarify. Amazon SageMaker Clarify is correct because it provides built-in capabilities to detect bias in machine learning models both before training (pre-training bias) and after deployment (post-training bias). It can generate bias reports for training data and predictions, making it a direct tool for monitoring bias over time.

Answer

Amazon SageMaker Experiments

Answer

AWS CloudTrail

Question 6

A data scientist is preparing a dataset for training a binary classification model. The dataset has 100,000 rows and 50 features. The target variable is imbalanced, with only 5% positive cases. Which technique should the data scientist apply to address the class imbalance BEFORE training?

Accepted Answer

Random oversampling of the minority class. Random oversampling of the minority class (Option B) directly addresses the class imbalance by duplicating examples from the positive class until the class distribution is more balanced. This prevents the binary classification model from being biased toward the majority class, which is critical when only 5% of the 100,000 rows are positive cases. Oversampling is applied before training to ensure the model sees sufficient minority examples during learning.

Answer

Principal Component Analysis (PCA) dimensionality reduction

Answer

Standard scaling of numerical features

Answer

One-hot encoding of categorical variables

Question 7

A company uses SageMaker endpoints with auto-scaling. The endpoint is experiencing high latency during peak hours. The metrics show CPU utilization is low but memory is high. What is the most likely cause?

Accepted Answer

The auto-scaling policy is based on CPU utilization, which does not trigger scaling.. The auto-scaling policy is based on CPU utilization, which remains low during the memory-bound issue. Since the scaling trigger is not met, the endpoint does not add more instances to handle the increased load, leading to high latency. Memory pressure without CPU spikes indicates the bottleneck is memory, not compute, so a CPU-based metric fails to scale appropriately.

Answer

The model is not optimized for inference, causing memory leaks.

Answer

The instance type has insufficient network bandwidth.

Answer

The endpoint is deployed in a VPC without a NAT gateway.

Question 8

A company uses SageMaker Model Registry to manage model versions. They want to enforce that only models with an 'Approved' status can be deployed to production endpoints. How can they enforce this?

Accepted Answer

Set IAM policies with a condition on sagemaker:ModelVersionStatus. SageMaker Model Registry supports approval workflows. By using IAM policies that conditionally allow deployment only when the model version status is 'Approved', the company can enforce governance.

Answer

Use AWS Lambda to check the model status during deployment

Answer

Use SageMaker Pipelines to deploy only approved models

Answer

Configure SCPs to block deployment of unapproved models

Question 9

A machine learning team deploys a model for loan approval. They want to monitor data drift on the real-time endpoint using SageMaker Model Monitor. Which set of actions should they take to set up data quality monitoring?

Accepted Answer

Enable data capture on the endpoint, generate a baseline from training data, create a data quality monitoring schedule, and set up a CloudWatch Alarm on violations. SageMaker Model Monitor requires a baseline from training data, then schedules monitoring jobs that compare live endpoint captures against that baseline. Alerts are sent via CloudWatch Alarms.

Answer

Use SageMaker Clarify to detect data drift on the endpoint

Answer

Create a model quality monitoring schedule directly on the endpoint without any baseline

Answer

Enable data capture and rely on SageMaker Model Monitor to automatically infer drift without a baseline

Question 10

An ML engineer runs the CLI command shown in the exhibit. However, the training job fails immediately with an error: 'Unable to assume role'. What is the most likely cause?

Accepted Answer

The IAM role's trust policy does not grant SageMaker permission to assume the role.. The 'Unable to assume role' error indicates that SageMaker cannot assume the IAM role specified in the CLI command. This is a trust policy issue: the role's trust policy must include SageMaker as a trusted service (i.e., `"Service": "sagemaker.amazonaws.com"`). Without this, SageMaker is not authorized to assume the role, regardless of the role's permissions.

Answer

The IAM role 'SageMakerExecutionRole' does not have permission to create the training job.

Answer

The training image in ECR does not exist.

Answer

The S3 bucket 'my-bucket' does not exist.

Question 11

A company uses SageMaker Model Monitor for data quality. They notice that monitoring jobs are failing intermittently with constraint violations. Upon review, they see that some features have different data types in production compared to the baseline (e.g., string instead of integer). Which type of drift is this?

Accepted Answer

Schema drift. Schema drift occurs when the structure or data types of features in production data differ from the baseline used during model training. In this scenario, a feature that was an integer in the baseline is now a string in production, which is a classic example of schema drift. SageMaker Model Monitor detects this by comparing the inferred schema of production data against the baseline schema, flagging any type mismatches as constraint violations.

Answer

Concept drift

Answer

Statistical drift

Answer

Bias drift

Question 12

An ML engineer needs to orchestrate a multi-step workflow that includes data preprocessing on Spark, model training on SageMaker, and deployment to a production endpoint. They require tight integration with other AWS services and the ability to add custom logic. Which AWS service should they use alongside SageMaker?

Accepted Answer

AWS Step Functions. AWS Step Functions is the correct choice because it provides a serverless workflow orchestration service that can coordinate multi-step ML pipelines involving Spark on AWS Glue or EMR, SageMaker training jobs, and endpoint deployments. It offers tight integration with over 200 AWS services via direct SDK integrations, supports custom logic through Lambda functions, and includes built-in error handling, retries, and parallel execution — making it ideal for complex, heterogeneous ML workflows that extend beyond SageMaker's native capabilities.

Answer

AWS CloudFormation

Answer

SageMaker Pipelines

Answer

Amazon EventBridge

Question 13

A team uses SageMaker Pipelines with a Condition step to decide whether to register a model based on evaluation metrics. They want to also store the evaluation results for lineage tracking. Which step should they use to record the metrics?

Accepted Answer

RegisterModel step. The RegisterModel step in SageMaker Pipelines is designed to create a model version in the SageMaker Model Registry, and it can accept metadata such as evaluation metrics via the `InferenceSpecification` or by passing a metrics dictionary. This allows the team to store evaluation results alongside the model for lineage tracking, fulfilling the requirement to record metrics after a Condition step approves registration.

Answer

Condition step

Answer

Training step

Answer

Processing step

Question 14

A company wants to deploy a machine learning model using infrastructure as code to ensure reproducibility. They need to define the SageMaker Studio domain, user profiles, and the endpoint configuration. Which tool should they use?

Accepted Answer

AWS CloudFormation or AWS CDK. AWS CloudFormation and AWS CDK are infrastructure-as-code (IaC) tools that allow you to define, provision, and manage AWS resources declaratively. For this use case, they can model the entire SageMaker Studio domain, user profiles, and endpoint configuration in templates or code, ensuring reproducibility and version control. This aligns directly with the requirement to deploy ML infrastructure as code.

Answer

SageMaker Pipelines

Answer

AWS Step Functions

Answer

SageMaker Studio

Question 15

A data science team needs to deploy a trained PyTorch model for real-time inference with sub-100ms latency. The model fits on a single GPU. Which SageMaker inference option is MOST cost-effective while meeting the latency requirement?

Accepted Answer

SageMaker real-time endpoint on ml.g4dn.xlarge. SageMaker real-time endpoints provide dedicated, persistent instances that can handle synchronous inference with sub-100ms latency. The ml.g4dn.xlarge instance includes a single NVIDIA T4 GPU, which is sufficient for the model size and offers the lowest cost among GPU instances that meet the latency requirement. This option balances performance and cost for real-time, low-latency inference.

Answer

SageMaker Batch Transform

Answer

SageMaker Async Inference

Answer

SageMaker Serverless Inference