# Cost Optimization, Security, and Compliance for ML Workloads

> Chapter 16 of the Courseiva GOOGLE-PMLE curriculum — https://courseiva.com/learn/google-pmle/cost-optimization-and-security

**Official objective:** 7.2 — Optimize costs and ensure security and compliance for ML resources on Google Cloud.

## Introduction

Cost optimisation, security, and compliance for ML workloads is the practice of running machine learning models on Google Cloud in the most cost-effective, safe, and legally sound way possible. For the PMLE exam, you must understand how to minimise cloud bills, protect your data and models from unauthorised access, and meet regulations like GDPR or HIPAA—all without slowing down your ML projects.

## The Personal Chef vs Meal Prep Delivery Analogy

3 friends share a house. They spend £600 a month total on food. Friend A buys premium ingredients and cooks every meal fresh, wasting about £150 in unused vegetables and herbs. Friend B orders takeaways every night, paying £12 per meal plus £3 delivery fees. Friend C joins a meal-prep subscription: £8 per portion, exactly what they eat, no waste, no delivery fee. They learn that Friend A’s approach is like running custom GPU instances that sit idle 60 percent of the time. Friend B’s habit is like paying for on-demand ML training without committed-use discounts. Friend C’s subscription mirrors preemptible VMs and spot instances: you get the same food (compute power) for less because you accept that occasionally the meal might not arrive (the VM might be reclaimed). The friends also install a lock on their shared fridge (Identity and Access Management) so only the three of them can take food. They agree a rule that nobody brings in outside food without a safety inspection (compliance policy: data must stay in the approved region). To save money further, Friend C starts batch-cooking twice a week instead of daily, analogous to using batch prediction jobs to reduce per-request compute costs. The trio saves £200 a month, secures their kitchen from strangers, and follows a written house agreement (audit log and compliance report) that satisfies the landlord’s rules (regulatory requirements). This is cost optimisation, security, and compliance for ML workloads in a nutshell: use the right compute pricing model, control access, and obey rules—while getting the same delicious model training done.

## Core explanation

Cost optimisation, security, and compliance for ML workloads means you design, deploy, and manage machine learning systems on Google Cloud so they are inexpensive to run, protected from attackers, and aligned with legal or industry rules. This is not an optional nice-to-have: if you train a model that costs £10,000 per month when you could run it for £3,000, you waste budget. If you leave a dataset publicly accessible, a malicious actor could steal sensitive customer data. If you store personal health information in a region not approved under HIPAA, your company could face severe fines. 

Let us break down each part. 

Cost optimisation for ML workloads

You pay for compute resources like CPU, GPU, and TPU, storage of datasets and models, and networking to move data. In Google Cloud, you can choose different pricing models. Preemptible VMs and spot VMs are short-lived instances that Google can reclaim to give to other customers, but they cost 60 to 91 percent less than standard VMs. They are perfect for batch training jobs that can be interrupted and resumed. Committed use contracts let you commit to one or three years of usage in exchange for up to 70 percent discount on GPU and TPU resources. Sustained use discounts automatically apply if you run an instance for more than 25 percent of a month. You can also use managed services like Vertex AI Training and Prediction, which abstract away the underlying machines and only charge you for the time your job actually runs. Autoscaling inference endpoints (the live model that serves predictions) ensures you only pay for the compute needed to handle current request traffic, not for peak capacity 24/7. 

Security for ML workloads

Security controls who can see, modify, or delete your ML assets. Identity and Access Management (IAM) lets you grant granular permissions: a data engineer might get read access to the training bucket (a storage container) but not permission to deploy a model to production. VPC Service Controls create a secure perimeter around your resources so that data cannot be exfiltrated to the internet unless explicitly allowed. Cloud Key Management Service (Cloud KMS) manages encryption keys that protect your data both at rest (stored on disk) and in transit (moving across the network). You can also use confidential VMs that encrypt data while it is being processed, so even Google cannot see it. For ML models specifically, you should apply least privilege (give the minimum permissions necessary) and use service accounts (special identities for applications) instead of sharing user passwords. 

Compliance for ML workloads

Compliance means following laws, industry standards, and company policies. Examples: if you handle credit card data, you must follow PCI DSS rules; if you process health data for US patients, you must follow HIPAA; if you operate in Europe, GDPR governs how you store and process personal data. Google Cloud provides compliance reports, certifications, and tools like Assured Workloads that enforce data residency (keeping data in a specific country). You can set up organization policies to prevent resources from being created outside approved regions. Cloud Audit Logs record every action taken on your ML resources: who accessed what, when, and from where. These logs are essential for proving compliance to regulators. You should also enable Data Loss Prevention (DLP) to scan your datasets for sensitive information like credit card numbers or passport IDs and redact or mask them before training a model. 

Why all three together matter

If you optimise cost but ignore security, you could lose millions in a data breach. If you secure everything but ignore compliance, you could face legal penalties. If you comply with rules but overspend, your ML project becomes unsustainable. The PMLE exam expects you to recommend solutions that balance all three, using the right Google Cloud services and configurations.

## Real-world context

Imagine you are the sole ML engineer at a mid-sized e-commerce company called ShopSmart. The company wants to build a recommendation model that suggests products to customers based on their browsing history. You need to train the model, deploy it as a live API, and then continuously improve it. Here is exactly what you do step by step. 

First, you assess the data. The dataset contains 500 GB of customer click logs, including IP addresses and purchase history. Because ShopSmart has customers in the EU, GDPR applies: you must not process personal data outside the EEA without explicit consent. You use Google Cloud’s Data Loss Prevention (DLP) API to scan the dataset and identify any personal identifiers like email addresses and phone numbers. DLP redacts those fields automatically before you copy the data into a BigQuery table in the europe-west1 region (located in Belgium). You then set an organization policy that prevents any ML resource from being created in a region outside the EU. This is compliance. 

Second, you optimise training cost. The model uses a GPU-intensive neural network that takes 40 hours to train. Running on a standard NVIDIA Tesla T4 GPU would cost £1.50 per hour, so £60 per training run. But you are not in a rush; the model is for production use next quarter. You switch to preemptible VMs, costing £0.22 per hour, so £8.80 per run. However, preemptible VMs can be terminated at any time, so you implement checkpointing: every 15 minutes, the training script saves the current model weights to Cloud Storage. If the VM is preempted, the next training job resumes from the last checkpoint. This is cost optimisation. 

Third, you secure the deployment. The live model endpoint needs to accept prediction requests from the public website. You do not want just anyone to call it because each prediction costs money and could expose model internals. You deploy the model on Vertex AI Prediction and enable IAM authentication. The website’s backend service gets a service account with the roles/aiplatform.user role, allowing it to call the endpoint. No other service or person can invoke your model. You also enable Cloud Logging and Cloud Monitoring to track invocation rates, latency, and errors. This is security. 

Fourth, you monitor costs. After two weeks, you notice the model is serving 30,000 predictions per hour, but the endpoint is using one node that never scales down. You switch to autoscaling with a minimum of 1 node and a maximum of 4 nodes. Now during quiet hours (midnight to 6am) the endpoint runs on 1 node, and during peak hours it scales up. Your monthly bill drops from £2,100 to £700. You set up a budget alert in Google Cloud Billing so that if the ML costs exceed £1,000 in a month, you get an email. 

Fifth, you prepare for an audit. The compliance officer asks for evidence that customer data stayed in the EU and that only authorised employees touched the model. You export Cloud Audit Logs from the last six months: logs show the data transfer from the raw bucket to the BigQuery table only happened within europe-west1, and that the only IAM principal that invoked the model endpoint was the website’s service account. You store these logs in a bucket with retention set to 3 years. The compliance officer is satisfied. 

What you did in this scenario: you complied with GDPR, used preemptible VMs to save 85 percent on training, secured the model endpoint with IAM, set up autoscaling to reduce inference costs, and maintained audit logs for compliance. This is exactly what a PMLE candidate needs to be able to design in an exam scenario.

## Exam focus

The PMLE exam tests cost optimisation, security, and compliance for ML workloads in multiple question formats, mainly scenario-based multiple-choice questions where you must choose the best Google Cloud service or configuration. Here is what you need to know. 

First, the exam heavily tests your ability to pick the right cost model for a given ML workload. You will see questions like: 
- 'A team needs to train a model that takes 80 hours on GPUs. The job can be interrupted and resumed. Which compute option is the most cost-effective?' The correct answer is preemptible VMs or spot VMs. Trap: they list standard VMs with a committed use discount. Standard VMs are still more expensive for interruptible jobs. 
- 'An ML pipeline runs predictions for 1 hour every day at 3am. What is the cheapest way to run the compute?' Answer: create a VM with 'stop on idle' or use a Batch job that starts only during that window. Trap: they suggest a constantly running low-cost instance, which still runs 24/7 and costs more. 

Second, security questions focus on IAM roles and VPC Service Controls. Common question: 'A data scientist needs to read training data from BigQuery but must not be able to export the data to their personal laptop.' The correct answer: grant the roles/bigquery.dataViewer role and restrict access via VPC Service Controls so data cannot be egressed outside the organisation. Trap: they suggest using Cloud DLP to redact data—that helps with compliance but does not prevent export. 

Third, compliance questions often involve data residency and encryption. For example: 'A healthcare company must store patient data only within the United States. Which Google Cloud capability enforces this?' Answer: Organization policies with a location restriction. Trap: they mention using Cloud KMS to encrypt data—encryption does not enforce location. 

Fourth, you need to know which services have specific compliance certifications. The exam may ask: 'Which Google Cloud service is certified under HIPAA for ML workloads?' Vertex AI is HIPAA-eligible when configured properly (with data residency and logging enabled). Trap: they list Cloud Storage alone, but Cloud Storage as a service is HIPAA-eligible only when you sign a Business Associate Agreement and enable appropriate controls. 

Fifth, the exam likes to test least privilege. You will see: 'A CI/CD pipeline needs to deploy a new version of a model to Vertex AI. What is the minimum IAM role for the service account?' Answer: roles/aiplatform.user (covers deploying models and endpoints). Trap: they suggest roles/aiplatform.admin, which gives unnecessary permissions like deleting other models. 

Key definitions to memorise: 
- Preemptible VM: lasts up to 24 hours, can be terminated at any time, 60-91% cheaper. 
- Spot VM: similar but no maximum runtime, can be terminated. 
- Committed use discount: one or three year commitment for up to 70% discount on GPU/TPU. 
- VPC Service Controls: prevent data exfiltration from specified services. 
- Cloud Audit Logs: log every API call for compliance. 
- Organization policies: enforce rules like location restrictions across the whole company. 
- Vertex AI Experiments and Model Registry: track model versions and lineage for auditability. 

Finally, watch for questions where cost optimisation conflicts with security or compliance. Example: they might suggest using a single public dataset accessible to everyone to save on storage costs—that is never acceptable because it violates security. The correct answer always prioritises security and compliance over cost, but chooses the most cost-effective option that still meets those requirements.

## Step by step

1. **Identify workload type and constraints** — Determine if your ML workload is batch training, real-time inference, or periodic prediction. Also note the regulatory requirements (e.g., GDPR, HIPAA) and performance needs (e.g., latency < 200ms). This step decides which compute option and pricing model you can use.
2. **Choose the compute pricing model** — For interruptible batch training, select preemptible or spot VMs for maximum savings. For steady-state training with predictable usage, sign a committed use contract for GPU/TPU resources. For inference, choose Vertex AI Prediction with autoscaling to pay only for the compute you use.
3. **Configure identity and access management (IAM)** — Create service accounts for your ML jobs and deployers. Assign the minimum roles required: for a data scientist reading data, use roles/bigquery.dataViewer; for a pipeline deploying models, use roles/aiplatform.user. Avoid using personal accounts or broad roles like owner.
4. **Implement security perimeters and encryption** — Set up VPC Service Controls to prevent data from leaving your approved network. Enable Cloud KMS to encrypt datasets, models, and training scripts. For highly sensitive workloads, use confidential VMs to encrypt data in use. Ensure all connections to APIs use TLS.
5. **Set compliance policies and audit logging** — Use organization policies to restrict resource creation to compliant regions. Enable Cloud Audit Logs with Admin Read, Data Read, and Data Write categories for all ML services. Export logs to a Cloud Storage bucket with retention policies (e.g., 3 years) for auditor review.
6. **Monitor and optimise continuously** — Set budget alerts in Google Cloud Billing to notify you if ML costs exceed thresholds. Review Cloud Monitoring dashboards for inference endpoint utilisation. If a node is under-utilised, reduce the minimum number of nodes or switch to a cheaper machine type.

## Comparisons

### Preemptible VM vs Committed Use Discount

**Preemptible VM:**
- Costs 60-91% less than standard on-demand VMs
- Has a maximum runtime of 24 hours
- Best for batch training jobs that can resume from checkpoints

**Committed Use Discount:**
- Costs up to 70% less than standard on-demand for GPU/TPU
- Requires a 1 or 3 year commitment
- Best for steady, predictable training workloads

### IAM Roles vs VPC Service Controls

**IAM Roles:**
- Controls who can perform actions on resources
- Example: roles/bigquery.dataViewer allows querying data
- Does not prevent data exfiltration (e.g., copying data to external project)

**VPC Service Controls:**
- Controls where data can be sent (network perimeter)
- Example: blocks API calls to resources outside the perimeter
- Does not authenticate users; relies on IAM for that

### Cloud KMS vs Data Loss Prevention (DLP)

**Cloud KMS:**
- Manages encryption keys for data at rest and in transit
- Does not inspect or redact content
- Used to protect confidentiality against unauthorised decryption

**Data Loss Prevention (DLP):**
- Scans datasets for sensitive information like credit card numbers
- Can redact or mask found data automatically
- Used to meet compliance requirements like GDPR or PCI DSS

### Vertex AI Prediction (autoscaling) vs Vertex AI Batch Prediction

**Vertex AI Prediction (autoscaling):**
- Serves predictions in real time with low latency
- Costs based on number of nodes * hours, even if idle at minimum
- Best for user-facing applications like product recommendations

**Vertex AI Batch Prediction:**
- Processes large batches of inputs offline
- Costs only for the compute time the job runs
- Best for periodic scoring like monthly customer churn analysis

## Diagram

_Decision flow for cost optimisation (choosing compute pricing model) and security/compliance measures for ML workloads on Google Cloud._

```mermaid
graph LR
    A[Identify ML Workload] --> B{Is it interruptible?}
    B -- Yes --> C[Preemptible or Spot VM]
    B -- No --> D{Steady usage?}
    D -- Yes --> E[Committed Use Discount]
    D -- No --> F[On-demand VM / Autoscaling]
    C --> G[Training Cost Optimised]
    E --> G
    F --> H[Inference Cost Optimised]
    I[Set IAM Roles] --> J[Secure Resources]
    J --> K[Enable VPC Service Controls]
    K --> L[Prevent Data Exfiltration]
    M[Set Org Policies] --> N[Enforce Data Residency]
    N --> O[Compliance Achieved]
    P[Enable Audit Logs] --> O
```

## Common misconceptions

- **Misconception:** Preemptible VMs are the same as Spot VMs **Reality:** Both are cheaper instances that can be reclaimed, but Preemptible VMs have a maximum lifetime of 24 hours, while Spot VMs have no fixed maximum runtime. Preemptible VMs are a subset of Spot VMs. (Exam materials sometimes use the terms interchangeably, but Google Cloud documentation distinguishes them. The PMLE exam may specifically ask about one or the other.)
- **Misconception:** If you encrypt data with Cloud KMS, compliance is automatically satisfied. **Reality:** Encryption protects confidentiality, but compliance also requires data residency (where data is stored), access controls (IAM), audit logging, and adherence to specific regulations like HIPAA or GDPR. Encryption is just one piece. (Beginners equate 'encrypted' with 'secure and compliant', but regulations mandate many more controls.)
- **Misconception:** Cost optimisation means always using the cheapest possible resources. **Reality:** Cost optimisation means minimising cost while meeting performance, security, and compliance requirements. For a real-time fraud detection model that needs responses in under 100 milliseconds, preemptible VMs are unsuitable because they might be terminated during a transaction. (People think 'cheapest' is always best, but ML workloads have latency, reliability, and location constraints.)
- **Misconception:** If you use an organisation policy to restrict resource creation to a region, that is enough for GDPR compliance. **Reality:** An organisation policy prevents new resources from being created outside a region, but it does not prevent data from being moved to another region after creation. You need VPC Service Controls and data residency enforcement on storage services too. (GDPR demands that personal data stays within the EU/EEA at all times, not just at creation time.)
- **Misconception:** Cloud Audit Logs are optional and only needed when an incident happens. **Reality:** Audit logs are critical for both security (detecting unauthorised access) and compliance (proving to auditors what actions occurred). They should be enabled proactively and exported to a secure, immutable location. (Newcomers think logging is a post-incident forensic tool, not a continuous compliance requirement.)

## Key takeaways

- Cost optimisation for ML workloads on Google Cloud relies on choosing the right pricing model: preemptible/spot VMs for interruptible training, committed use discounts for steady GPU/TPU workloads, and autoscaling for inference endpoints.
- Security for ML workloads means applying least privilege with IAM roles, preventing data exfiltration with VPC Service Controls, and using encryption (Cloud KMS) for data at rest and in transit.
- Compliance for ML workloads involves enforcing data residency with organisation policies, enabling Cloud Audit Logs for traceability, and using services that meet certification requirements like HIPAA or PCI DSS.
- Vertex AI is a managed service that simplifies cost optimisation (autoscaling, pay-per-use) and security (IAM integration, VPC-SC support) but must be configured correctly for compliance.
- The PMLE exam will test your ability to recommend the most cost-effective solution that still meets security and compliance constraints, not just the cheapest option available.
- Cloud DLP should be applied to datasets before training to redact sensitive personal data, helping meet compliance requirements while also reducing the risk of exposing that data through model outputs.

## FAQ

**What is the difference between preemptible VMs and spot VMs in Google Cloud?**

Preemptible VMs can run for up to 24 hours and are a type of spot VM. Spot VMs have no maximum runtime but can also be terminated at any time. Both are cheaper than standard VMs but are not suitable for workloads that must run uninterrupted.

**How do I prevent a data scientist from exporting training data to their laptop?**

Use VPC Service Controls to create a perimeter around your BigQuery or Cloud Storage resources. This prevents data from leaving the approved project and network, even if the user has legitimate read access via IAM.

**What is the cheapest way to run a model that serves predictions once a day for 30 minutes?**

Use a batch prediction job on Vertex AI, which runs on demand and only charges for the compute time used. Alternatively, use a preemptible VM that starts via Cloud Scheduler and shuts down after the job completes.

**Do I need a Business Associate Agreement (BAA) to use Vertex AI with health data?**

Yes. To use any Google Cloud service under HIPAA, you must sign a BAA with Google. Vertex AI is HIPAA-eligible when you configure it with data residency and audit logging.

**What is a committed use discount and when should I use it?**

A committed use discount is a contract where you commit to spending a certain amount on GPU or TPU resources for 1 or 3 years in exchange for up to 70% discount. Use it when you have steady, predictable ML training needs, such as daily retraining of a core model.

**How does autoscaling help reduce costs for a deployed ML model?**

Autoscaling dynamically adjusts the number of compute nodes serving your model based on incoming request traffic. During low traffic periods, it reduces nodes to save money; during spikes, it adds nodes to handle the load.

## Check your understanding

1. **A team needs to train a model for 12 hours, and the job can be interrupted and restarted. Which compute pricing option is the most cost-effective?**

   Answer: Preemptible VMs (or spot VMs), because they offer the lowest cost for interruptible workloads.

2. **What Google Cloud capability prevents a data scientist from exporting BigQuery data to a personal Google Cloud project?**

   Answer: VPC Service Controls, which create a security perimeter that blocks data from leaving the approved project or network.

3. **True or false: If you encrypt your data with Cloud KMS, you automatically meet GDPR data residency requirements.**

   Answer: False. Encryption protects confidentiality but does not enforce where data is stored; you need organisation policies to restrict resource creation to specific regions.

4. **A healthcare company needs to run an ML model on patient data. What must they sign with Google Cloud first?**

   Answer: A Business Associate Agreement (BAA), which is required to use HIPAA-eligible Google Cloud services.

5. **How does autoscaling reduce costs for a deployed prediction endpoint?**

   Answer: It dynamically adjusts the number of compute nodes based on traffic, so you do not pay for idle nodes during low-traffic periods.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/google-pmle/cost-optimization-and-security