Picking the wrong machine learning model is like using a sledgehammer to crack a nut—you waste time, money, and compute power, and you still get a mess. For the MLA-C01 exam, you need to know how to choose the right model from SageMaker’s built-in toolkit and how to train it effectively. This chapter strips away the jargon and shows you exactly how model selection and training work, step by step.
Jump to a section
A simple way to picture Model Selection and Training Frameworks on SageMaker
Your kitchen counter, covered in flour and scattered notes. You want to bake a cake for a friend’s birthday, but you have never baked one before. You search through a big box of recipe cards. Each card is a different recipe—chocolate sponge, lemon drizzle, vegan banana bread. Some recipes come with pre-mixed flour bags (the mix is already measured out). Others give you plain flour, eggs, and sugar and say ‘mix these yourself’. You choose the chocolate sponge card because it looks easy and your friend loves chocolate.
This is exactly how Amazon SageMaker works when you pick a machine learning model. SageMaker keeps a ‘recipe box’ of built-in algorithms—pre-written mathematical instructions for common tasks like predicting house prices or sorting photos by subject. You pick the one that fits your problem, just like you pick a cake recipe that fits the occasion. You then ‘train’ the model by giving it your data (your ingredients). SageMaker handles the ovens, timers, and mixing bowls behind the scenes. If a pre-made recipe does not suit you, you can bring your own recipe (your own code) and SageMaker will still help with the baking. The model you choose is the recipe; the training run is the actual baking process. Getting this right means you end up with a cake that rises, not a sad, flat pancake.
Machine learning (ML) sounds complicated, but at its heart it is just a computer learning a pattern from examples. Imagine showing a computer thousands of photos of cats and dogs until it learns to tell them apart. That ‘learning’ process is called training, and the ‘pattern’ it learns is called a model. A model is just a mathematical structure that, once trained, can make predictions on new data.
Amazon SageMaker is a service that helps you do all of this without needing to manage the underlying computer hardware yourself. It abstracts away the messy details—servers, networking, storage—so you can focus on the data and the algorithm. SageMaker is part of Amazon Web Services (AWS), which is a massive collection of cloud computing services offered by Amazon.
SageMaker offers built-in algorithms. An algorithm is a step-by-step method for solving a problem. A built-in algorithm is one that AWS has already written and optimised for you. You do not have to code it from scratch. For example, if you want to predict a number (like next week’s sales), you might use a built-in algorithm called Linear Learner. If you want to sort images into categories (like ‘cat’ or ‘dog’), you might use Image Classification. Each built-in algorithm is designed for a specific type of problem:
Linear Learner: predicts a numeric value (regression) or a category (classification) when the relationship between inputs is roughly linear (a straight-line pattern).
XGBoost: a powerful algorithm for both regression and classification, especially good with tabular data (data in rows and columns, like a spreadsheet).
Image Classification: identifies objects or scenes in images.
Object Detection: finds and labels multiple objects within an image, drawing boxes around them.
Sequence-to-Sequence: converts one sequence to another, such as translating English sentences to French.
BlazingText: works with text, for tasks like sentiment analysis (deciding if a review is positive or negative) or word meaning.
Each of these algorithms expects data in a specific format, often a CSV file (comma-separated values, like a simple spreadsheet) or Apache Parquet (a more efficient columnar storage format). SageMaker can read data from an Amazon S3 bucket, which is like a cloud-based hard drive. You point SageMaker to the S3 bucket where your data lives, tell it which algorithm to use, and it sets up the compute resources, runs the training, and stores the resulting model.
Training is the process of feeding the algorithm your labelled data (data where the correct answer is known, such as a picture labelled ‘cat’) so the algorithm adjusts its internal numbers (called parameters) to minimise its prediction errors. This adjustment happens over many ‘epochs’, where one epoch is a full pass through your entire dataset. The algorithm uses a ‘loss function’, which is a mathematical score that measures how wrong its predictions are. The goal is to make this loss score as low as possible.
Why does this matter instead of manual coding? In the old days, a data scientist would write a custom algorithm from scratch, manage their own server, and manually adjust parameters—a process that could take weeks. SageMaker automates almost all of that. You write a few lines of code (or use the AWS console, a point-and-click web interface) to start a ‘training job’. A training job is a single run of the algorithm on your data. SageMaker spins up a server (called an instance, usually an EC2 instance), runs the training, and shuts down the instance when done. You only pay for the time you use.
For the MLA-C01 exam, you need to know which built-in algorithm fits which scenario. The exam will describe a business problem (like ‘predict customer churn from a table of customer data’) and ask you to pick the right algorithm. They will often include tempting distractor answers, like using a deep learning image algorithm for a simple table problem. The key is to match the algorithm’s strengths to the data type and the goal. Tabular data and structured prediction usually point to XGBoost or Linear Learner. Image or text data point to the specialised algorithms.
Training also involves splitting your data into three subsets: training set (used to adjust the model), validation set (used to check performance during training and avoid overfitting), and test set (used only at the end to evaluate how well the model generalises to unseen data). Overfitting means the model memorises the training data too well but fails on new data—like a student who only memorises past exam answers but cannot answer a new question. SageMaker has built-in features to help with this, such as early stopping (halting training when performance on the validation set stops improving) and automatic model tuning (which the exam calls hyperparameter tuning). Hyperparameters are settings you choose before training starts, such as how fast the algorithm learns (learning rate) or how many trees to build in XGBoost. Tuning these is like adjusting the dials on a radio to get the clearest signal.
Identify the problem type and data format
Determine whether your task is classification (predicting a category), regression (predicting a number), object detection (finding objects in images), or something else. Note if your data is tabular (rows and columns), images, text, or sequences. This step is the most critical because it dictates which built-in algorithm to use.
Choose a built-in algorithm from SageMaker
Select the algorithm that matches your problem from SageMaker’s catalogue. For tabular classification or regression, choose XGBoost or Linear Learner. For image tasks, choose Image Classification or Object Detection. For text, choose BlazingText. Do not overcomplicate—built-in algorithms are designed to work out of the box.
Prepare and upload data to S3
Clean your data (remove missing values, reformat columns) and split it into training, validation, and test sets. Upload these files to an Amazon S3 bucket. SageMaker reads data only from S3, so this step is mandatory.
Configure and start the training job
Use the SageMaker console, SDK, or CLI to define the training job. Specify the algorithm or container image, the S3 input channels for your data, the instance type (e.g., ml.m5.large), the output S3 path for the model, and any hyperparameters. Then start the job. SageMaker provisions the hardware, loads the algorithm, and runs training.
Evaluate model metrics and deploy
After the job completes, check the metrics logged (e.g., accuracy, AUC, loss). If performance is good, deploy the model to a SageMaker endpoint to serve real-time predictions. If not, adjust hyperparameters or try a different algorithm and retrain.
Let us walk through a realistic day in the life of an IT professional using SageMaker for model selection and training. Sarah is a machine learning engineer at a retail company called ShopFast. The business wants to predict which products are likely to sell out next week so they can restock in time.
Sarah starts with the existing data. She has a CSV file in Amazon S3 with columns for product ID, past sales, price, season, and whether the product sold out last month (a ‘yes’ or ‘no’ value). This is tabular data with a binary outcome (sold out or not). Sarah thinks: ‘This is a classification problem, and the data is a simple table. XGBoost is the natural choice.’
She opens the AWS Management Console, navigates to Amazon SageMaker, and creates a notebook instance. A notebook instance is like a virtual computer with Jupyter Notebook pre-installed, which is an interactive coding environment. She writes a short Python script that uses the SageMaker Python SDK (a software development kit—a set of tools and libraries) to start a training job. She specifies:
The S3 location of her training data.
The algorithm: XGBoost. SageMaker provides a pre-built Docker container image for XGBoost (a container is a lightweight, portable package of software and dependencies).
The instance type: ml.m5.large, a general-purpose server with enough memory for her modest dataset.
The output location for the trained model.
She also configures a validation dataset by splitting her data. She uses 80% for training and 20% for validation. She sets the number of rounds (how many times the algorithm iterates) to 100 and enables early stopping to prevent overfitting.
Sarah clicks ‘Create training job’. Behind the scenes, SageMaker provisions the ml.m5.large instance, pulls the XGBoost Docker image, copies her data from S3 into the instance’s local storage, and runs the training algorithm. She can see logs in real time from the CloudWatch service (AWS’s logging system). After 15 minutes, the job completes successfully. The trained model artifact (the file containing all the learned parameters) is saved to her specified S3 bucket.
Now she wants to know: did XGBoost perform well? She reviews the metrics: accuracy (percentage of correct predictions) and AUC (Area Under the ROC Curve, a measure of how well the model separates classes). Both look good. She then deploys the model to a SageMaker endpoint—a managed service that hosts the model and serves predictions in real time. The endpoint is like a tiny web server that the ShopFast inventory system can call with new product data to get a ‘sell out yes/no’ prediction.
The next week, the business team asks for a different feature: can the model also predict the exact number of units that will sell (not just whether it will sell out)? That is a regression problem (predicting a number), not classification. Sarah goes back to the console, selects the Linear Learner built-in algorithm instead, and retrains with the same data but a different target column (units sold). She follows the same steps. Each algorithm maps to a distinct business question, and Sarah knows which to pick because she understands the problem type.
Key actions an IT professional performs in this scenario:
Assess the business problem: classification, regression, or something else (like object detection on images).
Identify the data format: tabular, image, text, or sequence.
Pick the matching built-in algorithm from the SageMaker catalogue.
Configure training parameters and data splits.
Start the training job and monitor its progress.
Evaluate the model’s metrics and decide if it is good enough.
Deploy the model for inference (making predictions).
The main skill is pattern matching: given a description of a problem, know which algorithm is the correct fit. SageMaker handles all the heavy lifting of provisioning hardware, installing dependencies, and cleaning up afterwards.
The MLA-C01 exam tests your ability to select the correct built-in algorithm and training framework for a given scenario. Expect scenario-based multiple-choice questions. They present a sentence or two describing a business problem and ask ‘Which SageMaker built-in algorithm should the engineer use?’ The correct answer is almost always the algorithm that matches both the type of data (tabular, image, text) and the task (classification, regression, object detection, translation).
Here are the exact concepts the exam loves:
Data type vs. algorithm mapping: Tabular data points to XGBoost or Linear Learner. Image classification points to Image Classification or Object Detection. Text classification points to BlazingText. Sequence-to-sequence problems (like translation or summarisation) point to Sequence-to-Sequence.
Distinguishing classification from regression: If the output is a category (yes/no, dog/cat), it is classification. If the output is a number (price, temperature), it is regression. Linear Learner can do both, but XGBoost is typically preferred for tabular data with mixed feature types.
Built-in algorithm vs. bring your own container (BYOC): The exam will test when to use a built-in algorithm (fits the task, saves time) versus when to bring custom code (e.g., using a proprietary or novel algorithm not in the SageMaker catalogue). Do not assume every problem needs a custom solution. Built-in algorithms are often the correct, efficient choice.
Training job configuration: You should know the components of a training job: the algorithm or container image, the input data channel (pointing to S3), the instance type, the output path, and hyperparameters. The exam may ask what is required to start a training job. The minimum is: an algorithm (or container), training data in S3, and an output S3 bucket path.
SageMaker’s automatic model tuning (AMT): The exam asks about tuning hyperparameters. They want you to know that this is a separate feature that runs multiple training jobs with different hyperparameter values to find the best combination. It is not part of a single training job by default.
Overfitting and validation: Expect questions about splitting data into train, validation, and test sets. A trap is suggesting you use the test set for tuning—you must use the validation set for that. The test set is only for final evaluation.
Common trap patterns the exam uses:
They describe a problem with text data but list ‘Linear Learner’ as an option. Linear Learner expects numeric features and cannot handle raw text directly. The correct answer would be BlazingText or a text-based algorithm.
They describe a regression problem (predicting a numeric value) and include ‘XGBoost’ and ‘Linear Learner’. Both can work, but if the data is simple and linear (straight-line relationships), Linear Learner is simpler; if there are complex interactions, XGBoost is better. The exam often chooses the more specific correct match.
They ask about a training job but include a trick where the data is in an on-premises database. SageMaker cannot read directly from a local database—you must copy the data to S3 first.
They phrase a question about hyperparameters and suggest changing them in the model artifact after training. Hyperparameters must be set before training starts; you cannot change them after.
Key terms to memorise:
Training job: a single run of an algorithm on data.
Algorithm: the pre-built code SageMaker provides (XGBoost, Linear Learner, etc.).
Container image: a packaged environment that contains the algorithm code and dependencies.
Instance type: the hardware (CPU, memory) used for training (e.g., ml.m5.large).
Input data channel: how you tell SageMaker where your data lives in S3.
Hyperparameter: a setting you choose before training (e.g., learning rate, number of trees).
Metric: a score that measures model performance (e.g., accuracy, AUC).
Model artifact: the output file containing the trained model’s parameters, stored in S3.
Expect to see at least three to four questions on this topic in the exam. The questions are straightforward if you have memorised the algorithm-to-problem mapping. Use elimination: if the problem involves images, remove table-based algorithms immediately. If it involves structured tabular data with missing values, XGBoost is often the safer choice because it handles missing values natively.
SageMaker built-in algorithms are pre-optimised code packages that handle specific data types and tasks—use them instead of writing custom code when possible.
Match the algorithm to the data type: XGBoost for tabular data, Image Classification for pictures, BlazingText for text, and Sequence-to-Sequence for translation or summarisation.
A training job requires three things: an algorithm (or container image), training data in S3, and an output S3 bucket path for the model artifact.
Hyperparameters are set before training and control the learning process; they cannot be changed after the job starts.
Always split your data into training, validation, and test sets; use the validation set for tuning and the test set only for final evaluation.
SageMaker automatic model tuning runs multiple training jobs with different hyperparameter combinations to find the best model configuration.
These come up on the exam all the time. Here's how to tell them apart.
XGBoost
Handles missing values automatically during training
Good for complex, non-linear relationships in tabular data
Uses an ensemble of decision trees (a series of if-then rules)
Linear Learner
Cannot handle missing values; data must be pre-filled
Best for simple, linear relationships (straight-line patterns)
Uses a single linear equation to make predictions
Image Classification
Assigns one label to the entire image (e.g., ‘cat’)
Output is a single category
Built for images with one main subject
Object Detection
Finds and labels multiple objects within an image, including their bounding boxes
Output is a list of objects with locations
Built for images with multiple distinct items (e.g., cars and pedestrians)
Built-in Algorithm
Pre-written by AWS, no coding required
Fully managed: SageMaker handles dependencies and updates
Limited to the algorithms AWS provides
Bring Your Own Container (BYOC)
You write and package your own code in a Docker container
Full control over dependencies and algorithm customisation
Required for proprietary or niche algorithms not in SageMaker catalogue
Mistake
I must always bring my own custom code because built-in algorithms are too simple.
Correct
SageMaker built-in algorithms are highly optimised and cover most common use cases. They are often the best choice because they are pre-tuned and require fewer lines of code.
Beginners think custom code makes them look more skilled, but the exam rewards efficiency. Built-in algorithms are robust and tested at scale.
Mistake
Once I start a training job, I can change the hyperparameters by editing the model file afterwards.
Correct
Hyperparameters must be set before the training job begins. They control the training process itself. After training, the model is fixed and you cannot retroactively adjust hyperparameters.
This is a common trap because beginners confuse parameters (which the model learns during training) with hyperparameters (which you set beforehand).
Mistake
SageMaker can train on data stored anywhere, like a local hard drive or a Google Drive folder.
Correct
SageMaker trains exclusively on data stored in Amazon S3. You must first upload your data to an S3 bucket and then point the training job to that location.
Many newcomers assume cloud services can magically access any file. SageMaker only has access to resources within AWS, and S3 is the standard storage layer.
Mistake
If I use XGBoost for a regression problem, I must also write custom code to make it work because it is a classification algorithm.
Correct
XGBoost supports both classification and regression natively. You just set the objective parameter to ‘reg:squarederror’ for regression or ‘binary:logistic’ for classification.
People mistakenly think algorithms are purpose-built for only one task. In reality, many algorithms have flexible modes. The exam tests your awareness of these settings.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
A built-in algorithm is pre-written code provided by AWS that you can use without any setup. A custom container allows you to bring your own code and dependencies, which is useful for proprietary or novel algorithms. Use built-in first because they are simpler and already optimised.
No, SageMaker cannot read directly from your laptop. You must upload your data to an Amazon S3 bucket first. Once it is in S3, you can point the training job to that location.
Use Linear Learner when the relationship between features and the target is roughly linear (simple, straight-line pattern). Use XGBoost when data has complex interactions, missing values, or mixed feature types (numbers and categories). XGBoost is more flexible and often performs better on real-world tabular data.
If you do not specify a validation set, SageMaker may still use a default split (e.g., 70-30) depending on the algorithm. But it is best to split manually. Without a validation set, you risk overfitting and cannot accurately tune hyperparameters.
No. Automatic model tuning (AMT) is a separate feature that runs multiple training jobs, each with different hyperparameter values, to find the best combination. A single training job uses one set of hyperparameters. AMT is for optimisation, not for initial training.
No, you cannot change the instance type or any other configuration once the training job is running. You must stop the job and start a new one with the desired instance type.
You've finished Model Selection and Training Frameworks on SageMaker. Continue through the MLA-C01 study guide to build a complete picture of the exam.
Done with this chapter?