Fundamentals of AI and ML practice questions

The training data is stored in the wrong S3 bucket

Why it fails: S3 permissions or path issues do not cause container start errors.

The container image does not have the correct entry point

Why it fails: Missing entry point would cause a different error.

The GPU drivers are outdated

Why it fails: Outdated GPU drivers would cause a different error, but SageMaker manages drivers.

SageMaker Training Compiler

Why it fails: Compiler accelerates training, does not handle early stopping.

SageMaker Debugger

Why it fails: Debugger monitors training and can stop jobs based on rules, but early stopping is more directly supported by built-in algorithms.

SageMaker Experiments

Why it fails: Experiments track iterations, but do not stop jobs.

Increase the number of epochs

Why it fails: If loss is not decreasing, more epochs won't help.

Increase the learning rate

Why it fails: Increasing learning rate may cause divergence.

Decrease the batch size

Why it fails: Decreasing batch size can introduce noise but may not directly address plateauing loss.

The number of layers in the model

Why it fails: Layer count describes model architecture, not hardware capability; it does not determine whether CPU or GPU compute is needed. It is tempting because deeper models feel compute-heavy, but the deciding factor is whether training parallelises across many matrix operations, which GPUs accelerate.

The AWS Region

Why it fails: Region selection governs data residency, latency and service availability, not the CPU-versus-GPU decision. Instance type choice turns on whether the training workload needs parallel matrix computation, which GPUs accelerate, versus sequential or small-scale work suited to CPUs. Region would matter for compliance or proximity requirements instead.

The choice of hyperparameter optimizer

Why it fails: The optimiser choice is a training configuration decision independent of instance hardware; both CPU and GPU instances run the same optimisers. It is tempting because optimisers affect training speed, but the CPU-versus-GPU decision hinges on workload parallelism and dataset size, not the optimiser selected.

The file name contains spaces

Why it fails: File names with spaces do not cause Access Denied; they might cause a different error if not properly URL-encoded.

The policy does not grant 's3:ListBucket' permission

Why it fails: s3:ListBucket is not required to read a specific object; only s3:GetObject is needed for direct access.

The policy allows 's3:PutObject' which is not needed

Why it fails: Allowing s3:PutObject does not prevent read operations; the absence of s3:GetObject would cause the error, but this option does not state that.

The number of epochs is insufficient

Why it fails: Insufficient epochs typically result in training loss still decreasing for both training and validation sets, not the pattern of training loss decreasing while validation loss increases.

The dataset is too small

Why it fails: A small dataset may contribute to overfitting, but the direct symptom shown in the loss curves is overfitting itself, not the root cause of dataset size.

The learning rate is too low

Why it fails: A low learning rate usually causes slow convergence with both losses decreasing slowly, not the diverging pattern of training loss decreasing and validation loss increasing.

Switch to batch inference and run predictions every 100 milliseconds.

Why it fails: Batch inference increases latency and is not suitable for real-time fraud detection.

Reduce the maximum tree depth and retrain the model.

Why it fails: Reducing tree depth may reduce latency but can also decrease model accuracy.

Apply post-training pruning to remove redundant trees.

Why it fails: Pruning reduces model size but may harm accuracy and is not the best way to reduce latency for a single prediction.

To tune hyperparameters

Why it fails: Hyperparameter tuning uses the validation set, not the test set.

To balance class distributions

Why it fails: Class balancing is achieved through resampling, class weights or synthetic data generation, not through partitioning rows into three sets. It tempts because imbalance is a real preprocessing concern, but splitting preserves whatever distribution exists rather than altering it.

To prevent overfitting during training

Why it fails: Overfitting is countered by regularisation, early stopping and dropout; the split only detects it by revealing a gap between training and validation performance. It tempts because held-out data exposes overfitting, but prevention happens during training, not through partitioning.

SageMaker Experiments

Why it fails: SageMaker Experiments tracks runs, parameters and metrics across training jobs; it does not expose model-internal feature importance scores. It would be correct for comparing and organising multiple training trials. Viewing XGBoost feature importance requires the model's built-in interpretation output, not experiment tracking metadata.

SageMaker Autopilot

Why it fails: Autopilot automates model selection and hyperparameter tuning to produce a deployable model; it does not surface feature importance for an already-trained XGBoost model. It would be correct when starting from raw tabular data without a chosen algorithm. Feature importance comes from the trained model's interpretation output.

SageMaker Model Monitor

Why it fails: Model Monitor detects data drift and quality deviations on deployed endpoints; it does not report feature importance from a trained XGBoost model. It would be correct for ongoing production monitoring of inference data. Feature importance is a property of the trained model itself, not of runtime monitoring.

K-Means

Why it fails: K-Means is an unsupervised clustering algorithm, not suitable for regression tasks.

BlazingText

Why it fails: BlazingText is an NLP-focused algorithm for text classification and word embeddings, not general regression.

Linear Learner

Why it fails: This option duplicates option B. Although the text is identical, it is not the intended correct answer and should not be selected.

Increase learning rate

Why it fails: Raising the learning rate increases step size, which worsens divergence and overfitting rather than reducing it. It is tempting because a higher rate can escape a plateau, and it would be the correct choice when training stalls early due to an overly small rate, not when the model overfits.

Add more layers to the model

Why it fails: Adding layers increases model capacity, giving the network more parameters to memorise training data, which deepens overfitting. It is tempting because extra depth can improve fit, and it would be the correct choice when the model underfits and training loss stays high.

Reduce the number of epochs

Why it fails: Cutting epochs stops training early, but the model has already memorised the training set, so overfitting persists; regularisation is needed instead. It is tempting because fewer epochs limit exposure to data, and it would be the correct choice when the model overfits only after many epochs of continued training.

SageMaker Experiments

Why it fails: Experiments tracks and compares runs, logging metrics, parameters and artefacts for analysis, but it does not evaluate validation performance during training or terminate the job. It would be correct for organising and comparing many training attempts, not for halting a run when validation loss plateaus.

SageMaker Debugger

Why it fails: Debugger captures tensors and training metrics for analysis, and its built-in rules can raise alerts on conditions such as overfitting, yet it does not itself stop a job when validation performance plateaus. It suits diagnosing training anomalies, whereas early stopping is configured through the training job's stopping condition.

SageMaker Model Monitor

Why it fails: Model Monitor watches a deployed endpoint for data drift and quality issues after training completes, so it cannot intervene mid-job. It would be the right choice for production monitoring of inference traffic, not for terminating a training run whose validation metric has stopped improving.

Reduce the number of layers in the model

Why it fails: Removing layers shrinks the model's capacity, directly lowering accuracy, which the stem forbids. It is tempting because a smaller model needs less memory, but the failure is data-side memory pressure; the fix is streaming or sharding the S3 input rather than altering the architecture.

Increase the batch size

Why it fails: A larger batch holds more activations and gradients simultaneously, increasing peak GPU memory and worsening the OutOfMemoryError. It is tempting because bigger batches speed up throughput, but the correct remedy reduces per-step memory — for example smaller batches with gradient accumulation, or Pipe mode input.

Use a smaller instance type with less memory

Why it fails: Choosing an instance with less memory reduces available GPU RAM, so the same tensors overflow sooner and the job fails faster. It is tempting as a cost-saving move, but the correct approach provisions an instance with sufficient memory or applies memory-reducing techniques such as gradient checkpointing.

Deploy behind an Application Load Balancer with multiple ml.m5.xlarge EC2 instances running the model

Why it fails: An Application Load Balancer distributes HTTP traffic but provides no model hosting, autoscaling, or GPU management, and bursty traffic would leave instances idle or overwhelmed. It would suit routing to self-managed inference servers, not SageMaker's managed real-time endpoints with autoscaling.

Use a SageMaker multi-model endpoint with ml.m5.large instances to cache multiple models

Why it fails: Multi-model endpoints load models on demand onto ml.m5.large instances, which cap at 16 GiB but cannot hold an 8 GB ensemble resident per request with sub-100ms cold-start loading. It suits hosting many small models behind one endpoint, not a single large latency-critical model.

Use SageMaker asynchronous inference with a large batch size

Why it fails: Asynchronous inference queues requests and returns results via Amazon S3, so responses are not synchronous; sub-100ms latency is unattainable. It is designed for large payloads and long processing times, such as near-real-time batch scoring where minutes of delay are acceptable.

Automated labeling using pre-trained models

Why it fails: Ground Truth's automated labelling uses pre-trained models only for image and text classification, not object detection bounding boxes, so it cannot produce the required output. It is tempting because automated labelling genuinely accelerates classification datasets, where pre-trained model predictions are the correct strategy.

Active learning with manual verification

Why it fails: Active learning is a Ground Truth workflow that selects uncertain samples for human review, not a standalone labelling strategy; the two strategies are automated data labelling and human labelling. It is tempting because it genuinely reduces labelling cost, but it presupposes an existing model and verified labels.

Fully automated labeling via AWS Lambda

Why it fails: Ground Truth offers no Lambda-driven fully automated labelling pipeline; Lambda functions only run custom pre- and post-annotation logic. It is tempting because Lambda genuinely extends Ground Truth workflows, but that is for transforming input data and consolidating output, not for generating labels without human annotators.

The bucket policy requires encryption in transit

Why it fails: Encryption-in-transit requirements produce TLS or KMS-related errors, not AccessDenied on GetObject, and the exhibit's policy governs the role, not the bucket. It is tempting because bucket policies can deny access, but here the role's own policy is the constraint.

The training job is using the wrong AWS region

Why it fails: A wrong region yields endpoint or NoSuchBucket errors, not AccessDenied, because the bucket name simply would not resolve there. It is tempting since region mismatches do break S3 calls, but the exhibit shows the role's policy itself is the limiting factor.

The policy does not include the s3:PutObject action

Why it fails: PutObject governs writes, while the failing operation is reading files, which requires s3:GetObject. It is tempting because a missing action in the policy does cause AccessDenied, but adding PutObject would not enable reads; the absent GetObject permission is the cause.

Scale the endpoint to a larger instance type, such as ml.r5.large

Why it fails: A larger instance type adds memory and compute, but the described pattern — latency and memory errors growing after hours of running — points to a memory leak in the inference code, which resizing cannot fix. Larger instances suit genuinely undersized, steady-state workloads.

Enable auto-scaling to add instances during high load

Why it fails: Auto-scaling adds instances to handle traffic spikes, yet the symptom is per-instance memory growth over time, not increased request volume. Adding replicas leaves each leaking container unaffected. Auto-scaling is correct for variable load with stable per-instance memory.

Use SageMaker Model Monitor to detect data drift

Why it fails: Model Monitor detects data and model quality drift in incoming requests; it does not address container memory growth or latency degradation. It is tempting because monitoring sounds relevant to endpoint health, but drift detection suits accuracy degradation, not resource exhaustion.

Accuracy

Why it fails: With 1% fraud, a model predicting every transaction as legitimate scores 99% accuracy while catching no fraud, so the metric cannot distinguish useful classifiers from trivial ones. Accuracy suits roughly balanced class distributions; imbalance demands precision-recall or F1 measures.

Recall

Why it fails: Recall alone is maximised by flagging every transaction as fraud, producing overwhelming false positives that make the alerts unusable for investigators. Recall suits screening where missing a positive is catastrophic and false alarms are tolerable; imbalanced fraud detection needs precision and recall balanced together.

Precision

Why it fails: Precision alone is maximised by flagging only the single most suspicious transaction, missing almost all fraud and defeating the detection purpose. Precision suits ranking where reviewer capacity is tightly limited; imbalanced fraud detection needs recall captured alongside it, typically via F1 or PR-AUC.

Increase the number of training epochs

Why it fails: More epochs means additional full passes over the training data, so the job runs longer, not shorter. It is tempting because extra epochs can improve model accuracy, and it would be the right lever when the goal is convergence rather than speed.

Use a smaller instance type

Why it fails: A smaller instance type provides fewer vCPUs and less memory, so each epoch takes longer and training time rises. It is tempting as a cost-saving measure, and would be correct when the priority is reducing spend on a job that is not time-critical.

Enable spot training

Why it fails: Spot training cuts cost by using spare capacity, but it does not accelerate the job and can interrupt or pause it, extending wall-clock time. It is tempting because spot capacity is cheaper, and would be right when minimising cost matters more than completion speed.

Always split data 80/20 for training and testing

Why it fails: A fixed 80/20 split is arbitrary; the ratio should reflect dataset size, class balance and whether validation or cross-validation is needed. It is tempting as a common default, yet stratified or time-based splits are often required instead.

One-hot encoding for ordinal categories

Why it fails: One-hot encoding creates unordered binary columns, destroying the inherent order that ordinal categories carry. It is tempting because it is the standard treatment for nominal features, but ordinal data should instead be encoded with integer ranks preserving sequence.

Drop duplicate records only if they are manual entry errors

Why it fails: Duplicates should be removed based on their effect on the model, not only when traced to manual entry errors; legitimate repeated observations can still bias training. Conditional removal is tempting because it avoids discarding genuine data, yet systematic deduplication is the practice.