Exam domain 4.0 (Model Deployment and Inference) asks you to understand how to put a trained machine learning model to work in the real world. The core problem is simple: once you have built a model that can make predictions, how do you make those predictions available to users or other systems, and which method should you choose for different business needs?
Jump to a section
A simple way to picture Model Deployment Strategies: Real-Time, Batch, and Serverless Inference
A professional kitchen has three very different ways to serve food, each designed for a specific type of customer demand.
First, the à la carte counter is the real-time deployment. A customer walks up, orders a burger, and the chef cooks it right then, on the spot. The customer waits a few minutes, and the burger is served immediately. This kitchen keeps a chef stationed there at all times, ready to cook any single order instantly. The kitchen is always on, always waiting, and it costs money for the chef's time even when no one is ordering.
Second, the catering service is batch deployment. A school calls ahead and orders 300 boxed lunches for tomorrow's field trip. The kitchen doesn't cook one box at a time all day. Instead, it waits until all the orders are in, then fires up the ovens and cooks everything in one big batch. It takes an hour to prepare all 300 lunches, but then they are delivered together. This is hugely efficient when you have a large, predictable amount of work that doesn't need to be instant.
Third, the food truck is serverless deployment. The truck only starts its engine when an order comes in through an app. If no one orders for an hour, the truck is parked, using no fuel. If suddenly 50 orders arrive at once, the truck automatically scales up its fryers and grills to handle the surge, then powers down again when the rush is over. It only costs money when it is actually cooking.
A trained machine learning model is basically a very smart calculator. You give it input data (like a photo of a cat), and it outputs a prediction (like 'cat, 98% confidence'). But the model itself is just a file sitting on disk. To make it useful, you need to 'deploy' it, meaning you put it on a server (a computer that runs continuously) and create a way for other programs to send it data and get back predictions. This is called 'inference', the act of running data through a deployed model to get a result.
Amazon SageMaker is the main tool you use on AWS to do this. It is a fully managed service, meaning AWS handles the hardware, the operating system, and the networking. You just upload your model file and choose how you want it to serve predictions. SageMaker gives you three main strategies: real-time inference, batch transform, and serverless inference. Each one is designed for a different pattern of demand.
Real-time inference means you deploy a model that is always running and waiting for requests. You create an endpoint, which is a permanent URL that other applications can call at any moment. When a request arrives, the model processes it and sends back a response in milliseconds or seconds. This is used for things like a fraud detection system that must check a credit card transaction before it is authorised. The model cannot schedule the work for later, it must answer right now. The downside is cost: because the server must be on 24/7, you pay for the compute time even when no one is making predictions.
Batch transform is the opposite. You do not keep the model running all the time. Instead, you give SageMaker a large file containing many input records (like millions of customer records) and tell it to process them all at once. SageMaker spins up temporary servers, runs every record through the model, collects all the predictions into an output file, and then shuts the servers down. This is perfect for tasks like generating nightly product recommendations for a retail website or processing an entire month of sensor data. It is much cheaper for large volumes because you only pay for the time the servers are actually running.
Serverless inference is a middle ground. You do not manage any servers at all. You simply define your model and AWS automatically scales the infrastructure up and down in response to demand. If there are zero requests, there is zero cost. If a sudden spike occurs, AWS instantly provisions more capacity. You pay only for the number of requests you process and the duration they took. The trade-off is that there is a 'cold start' delay the first time a request comes in after a period of inactivity, so it is not ideal for ultra-low-latency applications.
When you deploy with SageMaker real-time, you have to choose an instance type (the size of the server, like an 'ml.m5.large' with 2 CPUs and 8 GB of RAM). For batch transforms, you can also choose the instance type and the number of concurrent workers. For serverless, AWS handles the hardware selection automatically, but you can set a maximum concurrency limit to control costs.
These three strategies replace the old way of doing things, where a developer would have to manually install the model on a physical server, configure a web server, and write code to handle load balancing and failover. SageMaker automates all of that. For the MLA-C01 exam, you need to be able to select the correct deployment strategy given a business requirement. If the question says 'needs responses in under 100 milliseconds', that is real-time. If it says 'process 10 million records once a day', that is batch. If it says 'unpredictable traffic with no usage for hours', that is serverless.
Prepare the Model Artifact
You must package your trained model file (e.g., a .pkl, .pt, or .h5 file) into a tar.gz archive. SageMaker expects this single compressed file as input. This step is identical for all three deployment strategies.
Choose the Deployment Strategy
Based on your business requirements (latency, volume, cost), you select either real-time, batch, or serverless. This decision defines which SageMaker API you will call: create_endpoint, create_transform_job, or create_endpoint_config for serverless.
Create the Endpoint Configuration (Real-Time / Serverless)
For real-time, you specify the instance type (e.g., ml.m5.large) and initial instance count. For serverless, you specify the maximum concurrency and memory size, but not the instance type. This configuration tells SageMaker what hardware to provision.
Deploy the Model to an Endpoint
Using the SageMaker SDK, you deploy the model artifact to the configured endpoint. SageMaker spins up the required compute, loads the model, and exposes a REST API URL. This completes the real-time or serverless deployment.
Run the Batch Transform Job
For batch deployment, you skip the endpoint step. Instead, you specify the input data location in S3, the output path, and the instance type. SageMaker creates a temporary cluster, processes all data, writes predictions to S3, and then terminates the cluster.
Monitor and Update
After deployment, you monitor the endpoint or job using Amazon CloudWatch. For real-time endpoints, you may set up auto-scaling rules. For batch jobs, you check the output file for completeness. All strategies require periodic model updates to maintain accuracy.
Imagine you work for a bank that has built a machine learning model to detect fraudulent credit card transactions. The bank processes tens of thousands of transactions every minute, and the model must check each one in under a second before the payment is approved. As the IT professional, you would deploy this model using a SageMaker real-time endpoint.
Your first step is to package the model into a SageMaker-compatible format, which usually means compressing it into a tar.gz file along with any inference code. You then upload this file to an Amazon S3 bucket (a cloud storage service). Next, you use the SageMaker console or SDK to create an endpoint configuration, where you specify the instance type, such as an ml.c5.xlarge, and the number of initial instances (say, 2 for redundancy). SageMaker then provisions those servers, deploys your model, and gives you a URL that your bank's transaction system can call with every transaction. - You set up auto-scaling policies so that during Black Friday sales, SageMaker automatically adds more instances to handle the increased load. - You configure monitoring using Amazon CloudWatch to track latency and error rates. - You enable data capture so that you can log every prediction for auditing and retraining purposes.
Now consider a second scenario: the bank also has a model that analyses customer transaction history to identify patterns for a monthly marketing campaign. This is a much larger dataset, containing every transaction from the last month, and it does not need to be processed instantly. For this, you use batch transform. You write a script that places the transaction data in a CSV file in S3, then you launch a SageMaker batch transform job. SageMaker spins up a cluster of servers, processes the file row by row, and writes the predictions to a new S3 file. You then import that file into the marketing department's database. The total cost for this batch job might be a few dollars, whereas a real-time endpoint running for the same month would cost hundreds of dollars.
Finally, consider a third model: a chatbot that answers customer questions about their balance. Usage is highly uneven, with almost no requests at 3 AM and a massive rush during lunch hour. Deploying a real-time endpoint for this would waste money during quiet hours. Instead, you use serverless inference. You define the model in SageMaker Serverless, set the maximum concurrency to 50 (so it never spins up more than 50 simultaneous instances), and forget about it. When a customer asks a question at 3 AM, the first request might take 2-3 seconds due to the cold start, but subsequent requests are fast. The cost is pennies per month because you only pay for actual usage.
The MLA-C01 exam tests your ability to distinguish between these three deployment strategies with scenario-based questions. You will be given a business requirement and asked to pick the correct SageMaker feature. The exam writers love to set traps by mixing up the characteristics of each strategy.
One common question pattern is: 'A company needs to process one million images every Sunday night. The processing is not time-sensitive, and cost optimisation is critical. Which deployment strategy should they use?' The correct answer is batch transform. The trap is that some candidates choose real-time because it sounds more 'standard', but real-time is inappropriate for infrequent, large-volume processing.
Another frequent pattern is: 'An application expects variable traffic, including periods of complete inactivity. Latency of up to 5 seconds is acceptable. What is the most cost-effective option?' The correct answer is serverless inference. The trap is that candidates might choose a real-time endpoint with auto-scaling, but that still incurs a base cost for the running instances, whereas serverless charges nothing during idle periods. - You must memorise the exact SageMaker feature names: 'SageMaker real-time endpoints', 'SageMaker batch transform', and 'SageMaker serverless inference'. - Know that real-time endpoints require choosing an instance type and can be deployed with 'SageMaker hosting services'. - Understand that batch transform uses a 'TransformJob' and does not create a persistent endpoint. - Remember that serverless inference does not support GPU instances and has a maximum timeout (typically 15 minutes per invocation). - The exam loves to test 'cold start' latency as a disadvantage of serverless inference. - They also test 'payload size limits' for real-time endpoints (usually 5 MB) versus batch transform (which can handle terabytes).
Traps you will see include questions where the scenario mentions 'real-time' but the data volume is huge (batch is still better), or where the scenario mentions 'serverless' but the latency requirement is under 100ms (cold start makes it unsuitable). Another common trap is asking which strategy is 'fully managed' – all three are fully managed, so that is a distractor. The exam often tests the trade-off between cost and latency. You must be able to articulate that real-time gives you the lowest latency at the highest cost, batch gives you the highest throughput at the lowest cost, and serverless gives you moderate latency with cost that scales to zero.
Real-time inference uses a persistent endpoint that is always on and is ideal for low-latency, single-request predictions like fraud detection or chatbots with instant responses.
Batch transform processes a large dataset all at once and is the cheapest option for high-volume, time-flexible workloads like monthly report generation.
Serverless inference scales to zero when not in use, making it the most cost-effective choice for unpredictable or spiky traffic with tolerant latency requirements.
Cold start latency is the primary disadvantage of serverless inference and can make it unsuitable for sub-second response needs.
You must choose an instance type for real-time endpoints and batch transforms, but serverless inference handles compute provisioning automatically.
The correct deployment strategy is determined by three factors: latency requirement, data volume, and cost optimisation goals.
These come up on the exam all the time. Here's how to tell them apart.
Real-Time Endpoint
Always running, constant cost even when idle
Millisecond latency per request
Best for online applications needing instant responses
Batch Transform
Runs only when you start a job, cost scales to zero when idle
Minutes to hours latency for full dataset
Best for offline processing of large datasets
Real-Time Endpoint
You choose the instance type (e.g., ml.m5.large)
No cold start, immediate response
Pay for provisioned compute time regardless of usage
Serverless Inference
AWS chooses the compute automatically
Possible cold start of 2-10 seconds after idle period
Pay only for actual requests and duration
Batch Transform
Processes whole files (CSV, JSON) at once
Does not expose a real-time API endpoint
Cheapest for high-volume, infrequent processing
Serverless Inference
Processes individual single requests
Exposes a real-time API endpoint
Cost-effective for variable/sporadic traffic patterns
Mistake
Real-time inference is always better than batch because it is faster.
Correct
Real-time is faster for single requests, but batch is much more cost-effective and efficient for processing large volumes of data at once. They solve different problems.
Beginners tend to think 'faster' automatically means 'better', ignoring the massive cost difference for bulk processing workloads.
Mistake
Serverless inference is just a name for the same thing as a real-time endpoint.
Correct
Serverless inference is a separate service that scales to zero when idle, while a real-time endpoint always has at least one running server. They have different pricing models and cold start behaviour.
Both produce predictions, so novices conflate them, but the underlying infrastructure and cost model are entirely different.
Mistake
If I use batch transform, I can call it like an API to get one prediction at a time.
Correct
Batch transform is designed to process a whole file of records at once, not individual requests on demand. For API-style requests, you need a real-time or serverless endpoint.
The term 'transform' is confusing. Beginners assume it means 'transform a single input', but it actually means 'transform a whole batch'.
Mistake
The 'cold start' problem only affects the very first request to a serverless endpoint.
Correct
Cold starts can happen anytime after a period of inactivity, not just the first ever invocation. If no request comes for 10 minutes, the next request may trigger a new cold start.
AWS documentation emphasises the first invocation, but in practice, the idle timeout (often minutes) means cold starts recur frequently for sporadic traffic.
Mistake
All three deployment strategies require you to choose an instance type manually.
Correct
Only real-time endpoints and batch transform jobs require you to choose an instance type. Serverless inference manages the underlying compute automatically, though you can set a maximum concurrency limit.
Beginners assume that because real-time needs manual instance selection, all SageMaker deployments must have the same requirement.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
A SageMaker endpoint is a permanent URL that your model lives at. Other applications send data to this URL and get predictions back instantly, like a waiter bringing your food when you order at a restaurant.
No. You only pay for the compute time while the batch transform job is actively processing data. Once the job finishes, the servers are shut down and you stop being charged.
Real-time endpoints have no cold start latency and can support very low response times (under 100 milliseconds). Serverless may take several seconds to start up after being idle, making it unsuitable for time-sensitive applications.
Currently, SageMaker serverless inference does not support GPU instances. If your model requires a GPU (like for image processing), you must use a real-time endpoint or batch transform with a GPU instance type.
You create a new endpoint configuration pointing to the updated model artifact, then use the 'UpdateEndpoint' API call. SageMaker performs a rolling update to avoid downtime.
The maximum payload size for a request to a SageMaker real-time endpoint is 5 MB. If you need to send larger files, you should use batch transform or pre-process the data into smaller chunks.
You've finished Model Deployment Strategies: Real-Time, Batch, and Serverless Inference. Continue through the MLA-C01 study guide to build a complete picture of the exam.
Done with this chapter?