Exam objective 4.2 asks you to master how machine learning models move from a data scientist's notebook into the real world and stay healthy there. This chapter solves the problem of making a model useful and reliable for a business, which is the entire point of becoming a data engineer. For the PDE exam, understanding deployment and monitoring is critical because it separates a theoretical project from a production system that actually drives decisions.
Jump to a section
A simple way to picture ML Model Deployment and Monitoring
A restaurant kitchen is the heart of any successful dining experience.
When a chef develops a new signature dish, they spend weeks perfecting it in a test kitchen. They tweak ingredients, adjust cooking times, and taste-test constantly. This is like training a machine learning (ML) model using historical data in a development environment. The dish works perfectly in the test kitchen. Now, the chef must 'deploy' it to the real restaurant kitchen for the dinner rush. Deploying the dish means writing the recipe into the official kitchen system, training all the line cooks on the exact steps, and ensuring the necessary ingredients are stocked in the right quantities. The chef also sets up a 'monitoring' station. A senior sous-chef is assigned to taste every tenth plate that goes out. They check for quality, consistency, and any deviation from the perfect recipe. If the dish is too salty or over-cooked, the sous-chef raises an alarm, and the chef can immediately adjust the cooking process or the ingredient supplier. Without this monitoring, one bad batch of ingredients could ruin dozens of orders and the restaurant's reputation. The chef also keeps the old recipe version in a book, ready to revert if the new dish fails. This entire system—the tested recipe, the trained staff, the quality checks, the rollback plan—mirrors deploying a model to a Vertex AI endpoint, monitoring its live predictions, and managing versions in a model registry.
The deployed dish is never static. Customer feedback and seasonal ingredient changes require constant minor adjustments to the recipe, just as a model needs retraining on new data to stay accurate.
Before we dive into the cloud, let us define the core problem. You have trained an ML model. A model is a mathematical representation that has learned patterns from data to make predictions. For example, a model might predict whether a customer will cancel their subscription. The model exists as a file on a computer. To be useful, this file needs to be turned into a live service that other applications can call with new data. This process is called deployment.
In the old way of working (which the PDE exam expects you to replace), a data scientist would train a model and then hand a file to an engineering team. The engineers would have to set up servers, install libraries, write custom code to load the model, and build an API (Application Programming Interface) to accept requests. An API is a bridge that lets two pieces of software talk to each other. This manual process was slow, error-prone, and hard to scale. Each model deployment was a unique, fragile snowflake.
Google Cloud's Vertex AI solves this by providing a managed service. Managed means Google handles the underlying infrastructure—the servers, the networking, the security patches—so you do not have to. You upload your model to a central repository called the Model Registry. This is like a library for models. It keeps track of every version of your model, its training data, its performance metrics, and its current status (for example, 'staging' or 'production'). Instead of manually re-creating the wheel for every model, you simply point to the version in the registry and ask Vertex AI to create an endpoint.
An endpoint is a stable web address that applications can send data to get predictions back. For instance, an e-commerce website's checkout page can send customer data to your model endpoint, and the endpoint returns a probability of churn. Vertex AI automatically scales the underlying servers up and down based on traffic. If no one uses the model at 3 AM, the service may reduce to zero instances to save money. This is called autoscaling. During a big sale event with millions of requests, Vertex AI spins up hundreds of servers instantly to handle the load.
Once the model is deployed, you cannot just walk away. Models will degrade over time. This is called model drift. Drift happens for two main reasons:
Data drift: The incoming data changes. For example, after a pandemic, customer behaviour shifted completely. A model trained on pre-pandemic data becomes inaccurate.
Concept drift: The relationship between the data and the prediction changes. For example, the pattern that 'more logins equals higher loyalty' may reverse if users are forced to log in too often.
To catch drift, you set up continuous monitoring. Vertex AI Model Monitoring compares the distributions of incoming features (the input data) and the models predictions against the baseline data used during training. A distribution is a statistical summary of data, like the average age or the common ranges for spending amounts. If the live distribution strays too far from the baseline, the monitoring tool sends an alert.
The monitoring dashboard also tracks prediction skew (difference between training data and real data) and prediction drift (change in outputs over time). A data engineer's job is to set up these alerts, decide thresholds for what counts as 'too far,' and define an action plan when drift is detected. The action plan often involves retraining the model on newer data and then deploying the new version using a process called a canary deployment. In a canary deployment, you send a small fraction of traffic (say, 5%) to the new model version while the old model handles 95% of requests. You compare the performance. If the new model is stable and accurate, you gradually shift more traffic to it. If it fails, traffic rolls back to the old version automatically. This is called a rollback.
All of this—model storage, versioning, endpoint creation, scaling, monitoring, and traffic splitting—is managed through the same Vertex AI console or API, giving a single source of truth for model lifecycle management.
Prepare the model for serving
Export the trained model in a format compatible with Vertex AI, such as TensorFlow SavedModel or PyTorch TorchScript. Package it in a Docker container if using custom dependencies. This step converts the model into a deployable artefact.
Upload model to Vertex AI Model Registry
Use the Vertex AI console or API to upload the model artefact along with metadata like training dataset, version notes, and evaluation metrics. This makes the model discoverable and version-controlled.
Create a Vertex AI Endpoint
Define an endpoint by specifying its name, region, and machine type. This sets up a stable URL where applications will send requests for predictions. The underlying infrastructure is provisioned automatically.
Deploy the model to the endpoint
Associate the model version from the registry with the endpoint. Configure autoscaling parameters (min/max replicas), traffic splitting rules, and deployment strategy (canary or blue/green). Vertex AI then serves the model.
Enable and configure model monitoring
Turn on Vertex AI Model Monitoring for the deployed model. Provide a baseline dataset (usually the training data) and set alert thresholds for feature skew and prediction drift. Define notification channels for alerts.
Monitor alerts and retrain
When a monitoring alert triggers, investigate the cause by examining data distributions and business context. If drift is confirmed, coordinate with the data science team to retrain the model. Upload the new version to the registry and perform a canary deployment.
Imagine you work as a data engineer for a large health insurance company called 'CareGuard'. The company has built an ML model that predicts which patients are at high risk of a hospital readmission within 30 days of discharge. The purpose is to enable nurses to proactively call those patients with extra support, reducing readmission rates and costs.
Your first task is to take the trained model from the data science team and deploy it to production. You start by opening the Vertex AI Model Registry. The data scientist has already uploaded the model there, along with its training metadata. You inspect the model version, check the performance metrics (for example, precision of 0.85 and recall of 0.78), and label it as 'approved-for-production'.
Next, you create a Vertex AI endpoint. You give it a sensible name like 'readmission-predictor-v1' and select a machine type that balances speed and cost. You choose a region close to your data centre for low latency, say 'us-central1'. You deploy the model to this endpoint, which takes about 10 minutes as Vertex AI spins up the necessary infrastructure. You enable autoscaling with a minimum of 1 instance and a maximum of 10 instances. You also set up an SLA (Service Level Agreement) target of 200ms response time for 99% of requests.
Now the endpoint is live. The hospital's discharge management system starts sending patient data (age, diagnoses, previous visits, medication list) to the endpoint. The model returns a risk score between 0 and 1. A risk score above 0.7 triggers a notification to a nurse.
You must now set up monitoring. You navigate to the Vertex AI Model Monitoring dashboard. You select the 'readmission-predictor-v1' model and enable monitoring for both input features and prediction outputs. You define a baseline dataset from the original training data. You set alert thresholds: if the distribution of a key feature like 'patient age' shifts more than 15% from the baseline, or if the average predicted risk score drifts more than 0.05, you want a notification. You configure alerts to go to a Google Cloud Monitoring channel that sends a message to your team's chat room.
Three months later, a notification comes in. The average predicted risk score has dropped from 0.4 to 0.25. This is suspicious. Upon investigation, you discover that the hospital changed its discharge protocol six weeks ago, sending healthier patients home earlier. The model was never retrained on this new population. You take a snapshot of the new patient data from the last four weeks and send it to the data science team for retraining. They produce a new model version. You upload it to the model registry as 'readmission-predictor-v2'. You then perform a canary deployment by sending 10% of live traffic to the new version. After 48 hours with no degradation in performance and no increase in response latency, you shift 100% of traffic to the new version and retire v1.
This whole workflow—deploying, monitoring, detecting drift, retraining, and redeploying—is exactly what the PDE exam expects you to architect and explain.
The PDE exam will test your understanding of the 'how' and 'why' of Vertex AI model deployment and monitoring, not just the definition. Question types are typically scenario-based. You will be given a business problem (e.g., 'a bank wants to deploy a fraud detection model with zero downtime and must monitor for data drift') and asked to choose the correct architectural approach.
Specific concepts the exam loves:
Model Registry vs. Bucket storage: A common trap is offering a Cloud Storage bucket as the place to store model versions. While a bucket can hold a model file, the Model Registry is the correct answer because it stores metadata, version history, and can trigger deployments. The exam expects you to know the registry is the single source of truth.
Vertex AI Endpoint vs. Compute Engine VM: A trap often presents deploying a model on a custom Compute Engine virtual machine as a cheaper or simpler option. The correct answer is always Vertex AI Endpoint because it is a managed service with built-in scaling, monitoring, and traffic splitting. Custom VMs require too much manual work.
Autoscaling parameters: They may ask what happens if you set 'min replica count' to 0. The answer: traffic can cause a cold start delay while a new instance boots up. Setting it to 1 avoids this but costs more. Know the trade-off.
Model Monitoring alerts: The exam differentiates between 'skew detection' (comparing live data distribution to training data) and 'drift detection' (comparing live data to recent historical data). They will give you a scenario and ask which metric to use.
Canary vs. Blue/Green deployment: A canary deployment shifts incremental traffic (e.g., 5%, then 50%, then 100%). A blue/green deployment runs two identical environments and switches all traffic at once. The exam will ask which method minimises risk. Answer: canary, because you test on real traffic slowly.
Retraining triggers: Questions test whether retraining should be triggered automatically by an alert or manually after investigation. The correct answer in a monitored environment is: the alert triggers a notification, and a human (or automated pipeline) evaluates whether to retrain. Never automatically retrain without verification in a high-stakes production system.
Online vs. Batch prediction: They love to ask when to use each. Online prediction provides a real-time response for a single request (e.g., a credit card transaction). Batch prediction processes a large set of data asynchronously (e.g., running predictions on all customers at midnight). Know the differences and the use cases.
Trap to watch: The exam might describe a situation where the model is accurate but the business outcomes are poor (e.g., high readmission predictions but no reduction in readmissions). This points to a business logic problem, not a model problem. The correct response is not to redeploy the model, but to re-evaluate the threshold or the intervention process. The exam tests your ability to distinguish between model metrics and business impact.
Vertex AI Model Registry is the central catalogue for storing, versioning, and managing ML model artefacts and their associated metadata.
A Vertex AI Endpoint provides a managed, scalable HTTPS endpoint that autoscales based on request traffic, with no manual server management.
Model monitoring in Vertex AI detects data skew and prediction drift by comparing live feature distributions against a training baseline.
Canary deployments reduce risk by sending a small percentage of live traffic to a new model version and gradually increasing it if performance remains stable.
Autoscaling with a minimum of 1 replica avoids cold-start latency but incurs a continuous cost even when no requests are made.
The correct action when a drift alert fires is to investigate the root cause and retrain the model on newer data, not to blindly revert to a previous version.
Online predictions serve a single request in real time, while batch predictions process large datasets asynchronously, often on a schedule.
Blueprint deployment strategies like A/B testing and canary releases are essential for safe model updates and rollback management.
Vertex AI Monitoring alerts are integrated with Google Cloud Monitoring, allowing engineers to receive notifications via email, SMS, or chat platforms.
A model can be accurate on its training metrics yet cause poor business outcomes if the prediction threshold is misaligned with the business goal.
These come up on the exam all the time. Here's how to tell them apart.
Online Prediction
Returns predictions for a single input in real time (e.g., a credit card transaction).
Requires a persistent endpoint and low latency (milliseconds to seconds).
Best for use cases needing immediate decision-making, like fraud detection.
Batch Prediction
Processes a large set of inputs asynchronously, returning results later.
Can be run on a schedule or triggered by an event (e.g., nightly scoring of all customers).
Cheaper per prediction because it uses pre-emptible or low-cost compute resources.
Canary Deployment
Routes a small percentage of live traffic to the new model version.
Monitors new version's performance over time before increasing traffic.
Allows gradual rollback with minimal impact if problems occur.
Blue/Green Deployment
Runs two complete identical environments (blue and green).
Switches all traffic from the old to the new environment at once.
Requires double the infrastructure cost during the switch, but offers instant rollback by flipping back.
Model Skew
Compares live input data distribution to training data distribution.
Reveals if the production data differs significantly from what the model was trained on.
Often detected early as soon as production begins if data collection methods change.
Model Drift
Compares current live data distribution to a recent window of live data.
Indicates a gradual or sudden change in the underlying data patterns over time.
More relevant for long-running models where the world changes seasonally or due to external events.
Model Registry
Stores model artefacts plus rich metadata (version, metrics, training pipeline).
Integrates directly with Vertex AI deployment and monitoring tools.
Supports model versioning, approval workflows, and automatic deployment triggers.
Cloud Storage Bucket
Only stores raw files (model binaries, serialised objects) as flat objects.
No built-in versioning, metadata management, or integration with serving endpoints.
Suitable for backing up model files or sharing them manually, but not for production lifecycle management.
Mistake
Once a model is deployed and accurate, my job is done. I do not need to monitor because the data scientist validated it.
Correct
Models degrade over time due to data drift and concept drift. Continuous monitoring is essential to detect when accuracy drops and trigger retraining.
Beginners often treat model deployment as a hand-off and a finish line, forgetting that data is dynamic like a river, not a static pond.
Mistake
Deploying a model to Vertex AI Endpoint automatically makes it highly available and fault-tolerant without any configuration.
Correct
You must explicitly configure min and max replica counts, health checks, and traffic splitting for high availability. Default settings may have only one replica, which is a single point of failure.
Cloud services often seem 'magically resilient' to new users, but the responsibility for redundancy still falls on the engineer's configuration choices.
Mistake
Model Monitoring tools only check for prediction accuracy, so if the model is still accurate, no alert is needed.
Correct
Vertex AI Model Monitoring checks for feature distribution skew and drift, not just accuracy. The model can still make accurate predictions on drifted data, but the predictions may no longer be relevant to the business problem.
People confuse a model's statistical accuracy with its business utility. A model can be 90% accurate predicting customer churn for the wrong customer segment, making the output useless.
Mistake
If I deploy a model using a pre-built container, I cannot customise the serving infrastructure or use my own dependencies like TensorFlow or PyTorch versions.
Correct
You can use custom containers in Vertex AI. You build a Docker container with your exact environment (libraries, system tools, model file) and deploy that. The managed endpoint still handles scaling and monitoring.
Beginners think managed services lock them into rigid templates. In reality, custom containers offer full flexibility while still providing managed benefits.
Mistake
Model registry is just a fancy name for a folder in Cloud Storage where I store model files.
Correct
The registry stores metadata (training data source, hyperparameters, evaluation metrics, version history) alongside the model artifact. It also integrates with deployment pipelines and allows associating models with endpoints.
People see the word 'registry' and think of a simple file listing, missing the rich metadata and lifecycle management capabilities that differentiate it from a bucket.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Not necessarily. Vertex AI supports pre-built containers for popular frameworks like TensorFlow, PyTorch, and scikit-learn. You only need Docker if you require custom libraries, dependencies, or system configurations.
If you set the minimum replica count to 0, Vertex AI scales the endpoint down to zero instances, so you stop paying for compute. The next request will trigger a cold start, which may take a few minutes as a new instance boots up.
Yes. Vertex AI Endpoints support traffic splitting, so you can route a percentage of requests to one model version and the rest to another version. This enables A/B testing and canary deployments.
Skew compares the distribution of live prediction requests against the distribution of the training data. Drift compares the distribution of recent live requests against a previous window of live data. Both detect changes, but skew reveals immediate deviation from training, while drift tracks gradual evolution.
Use traffic splitting. Redirect 100% of traffic back to the older model version in the registry that is already deployed to the endpoint. The rollback is instant because the old version is still running or can be redeployed if scaled down.
No. It incurs charges based on the number of monitored features, the frequency of logging, and the volume of monitoring data processed. You pay for the underlying Cloud Logging and Cloud Monitoring resources it uses.
A cold start occurs when an endpoint receives a request but no instance is currently running (because the minimum replica count was 0). The system must provision a new instance, which adds latency of several seconds to minutes depending on the container size.
You've finished ML Model Deployment and Monitoring. Continue through the PDE study guide to build a complete picture of the exam.
Done with this chapter?