# Networking, Connectivity, and Load Balancing

> Chapter 7 of the Courseiva CLOUD-PLUS curriculum — https://courseiva.com/learn/cloud-plus/networking-connectivity

**Official objective:** 2.4 — Explain networking concepts for cloud environments

## Introduction

How does a cloud application handle millions of users at once without crashing? The answer lies in networking, connectivity, and load balancing, the invisible infrastructure that directs traffic between users and cloud resources. For the CV0-004 exam, you must understand how networks are designed in the cloud, how different connectivity options work, and how load balancers distribute traffic to keep applications fast and reliable.

## The Package Delivery Network Analogy

A small business's delivery van is the single connection point for all their outgoing packages. The business sends dozens of parcels each day through one van, which travels a fixed route to the local sorting facility. This works fine until a holiday rush hits and the van is overwhelmed, causing delays and lost shipments. The business solves this by hiring a regional carrier that uses a central hub. Now, each package is handed to the nearest carrier driver, who takes it to the hub. At the hub, packages from hundreds of drivers are sorted and loaded onto long-haul trucks heading toward specific cities. Those trucks deliver to local hubs, where smaller vans complete the final drop-off. This system is a precise map for cloud networking. The delivery van is a virtual machine (VM) or container. The fixed route is a static route. The overwhelmed van during the rush is network congestion or a single point of failure. The regional carrier and its hub represent a cloud provider's data centre network with switches and routers. The sorting process at the hub is packet switching, where data packets are examined and forwarded toward their destination. The long-haul trucks are high-bandwidth fibre connections between data centres (the cloud backbone). The final local vans are the local area network (LAN) inside the data centre's rack. This layered, hub-and-spoke structure—where packages move from local collection to central sorting to long-haul transport to local delivery—is exactly how data packets travel across a cloud provider's global network, ensuring reliable connectivity and load balancing.

## Core explanation

Networking in a cloud environment is the system of connections that allows data to move between computers, storage, and users. Think of it as the road network for data. Without it, your cloud resources would be isolated islands. The foundation of cloud networking is the virtual private cloud (VPC). A VPC is your own private section of the cloud provider's data centre. Inside this VPC, you can create subnets, which are smaller segments of the network. Each subnet can have its own set of rules about what traffic is allowed in and out. Subnets are like neighbourhoods within a city, each with its own local traffic laws.

For data to travel between these subnets and to the internet, you need routers and switches. In the cloud, these are virtualised. A virtual router is a software-based device that decides where data packets should go based on their destination IP addresses. An IP address is a unique number assigned to every device on a network, like a home address for your computer. Virtual switches connect devices within the same subnet, allowing them to communicate directly.

Now, connectivity is about how you link your cloud network to the outside world. There are several key options:

- **Internet Gateway (IGW):** This is like the front door of your VPC. It allows resources inside the VPC (like web servers) to communicate with the internet. Without an IGW, your cloud servers are completely private.
- **NAT Gateway (Network Address Translation):** This allows private resources (like databases) to initiate outbound traffic to the internet (to download updates) but prevents the internet from initiating connections to them. It's like a one-way door that lets you go out but stops strangers from coming in.
- **VPN (Virtual Private Network):** A VPN creates an encrypted tunnel between your on-premises network (your office) and your cloud VPC over the public internet. Think of it as a secure, private road through a busy public city.
- **Direct Connect or Cloud Interconnect:** This is a dedicated, physical fibre connection between your data centre and the cloud provider. It bypasses the public internet entirely, offering higher speed and more reliability. It is like building a private highway directly from your warehouse to the cloud's loading dock.
- **Peering:** This connects two VPCs directly, so they can communicate as if they were on the same network. It is used for sharing resources between different projects or accounts within the same cloud provider.

Load balancing is the last critical component. A load balancer sits in front of a group of servers (called a backend pool or target group). When a user sends a request to your application, the request first hits the load balancer. The load balancer then decides which server should handle that request, based on algorithms. Common algorithms include:

- **Round Robin:** Sends requests to each server in turn, like dealing cards to players.
- **Least Connections:** Sends new requests to the server currently handling the fewest active connections.
- **IP Hash:** Routes requests from the same user IP address to the same server, ensuring session persistence.

Why is load balancing essential? Without it, a single server could get overwhelmed and crash. A load balancer also automatically detects if a server has failed and stops sending traffic to it, routing all requests to healthy servers instead. This is called health checking. The load balancer periodically sends a test request to each server. If a server does not respond correctly, it is marked as unhealthy and removed from the pool until it recovers.

In the cloud, you can also use autoscaling alongside load balancing. Autoscaling automatically adds or removes servers based on demand. When traffic spikes, the autoscaler launches new virtual machines and registers them with the load balancer. When traffic drops, it shuts down unneeded machines. This combination of load balancing and autoscaling is how cloud applications maintain performance during peak times while controlling costs. Together, networking, connectivity, and load balancing form the invisible architecture that makes cloud computing scalable, resilient, and performant.

## Real-world context

An IT professional managing a cloud environment for an e-commerce company regularly uses networking, connectivity, and load balancing to keep the online shop running during sales events. Here is a concrete scenario: The company's application runs on a set of virtual machines (VMs) inside a VPC. The web servers are in a public subnet with an internet gateway, so customers can reach them. The database servers are in a private subnet with no internet gateway, to protect sensitive customer data.

For connectivity, the company's office network is connected to the cloud VPC via a site-to-site VPN. This allows employees to access internal tools (like the admin dashboard) securely from the office. Additionally, the company uses a Direct Connect circuit for high-bandwidth data transfers between its on-premises legacy system and the cloud database, ensuring fast and reliable syncing of inventory data.

During Black Friday, traffic to the website spikes to ten times normal levels. The IT professional has configured an Application Load Balancer (ALB) to distribute incoming HTTP traffic across a pool of web servers. The load balancer uses the round-robin algorithm, but with sticky sessions enabled (using cookies) so that a user's shopping cart data is consistently handled by the same server during a session.

To handle the spike, the IT professional also set up an autoscaling group. As CPU utilisation on the web servers rises above 70%, the autoscaler automatically launches three new VMs. These VMs are configured with the same application code via a launch template. The autoscaling group registers each new VM with the load balancer, which immediately starts sending traffic to them. When the sale ends and traffic drops, the autoscaler terminates the extra VMs.

What does the IT professional do when a server fails? The health checks on the load balancer detect that one server is not responding to pings. The load balancer automatically removes that server from the target group and stops sending new connections to it. The IT professional receives an alert via Cloud Monitoring. They then:

- Check the logs of the failed VM to diagnose the issue.
- Reboot the VM or replace it by terminating it and letting the autoscaler launch a fresh one.
- Adjust the health check settings if the failure was due to a false positive.
- Test the failover by intentionally stopping a healthy server to confirm the load balancer redirects traffic correctly.

This scenario shows that networking, load balancing, and autoscaling are not static configurations. They are actively managed to respond to failures, traffic changes, and business needs. The IT professional must understand how to configure security groups (firewalls) to allow traffic through the load balancer, how to set up DNS routing (like using a global server load balancer to direct users to the nearest region), and how to monitor network performance using tools like Cloud Logging and metrics dashboards.

## Exam focus

The CV0-004 exam tests your understanding of networking, connectivity, and load balancing through scenario-based multiple-choice questions and drag-and-drop activities. You will not be asked to configure a real load balancer, but you will be expected to know the purpose of each component, how they interact, and when to choose one option over another. The exam focuses on these core concepts:

- **Load Balancer Types:** You must distinguish between an Application Load Balancer (ALB, Layer 7), a Network Load Balancer (NLB, Layer 4), and a Classic Load Balancer (deprecated but still tested). The key differentiator is the OSI layer they operate on: ALB uses HTTP/HTTPS headers, NLB uses TCP/UDP and handles millions of requests per second with ultra-low latency. Expect questions that say 'An application needs to route traffic based on the URL path. Which load balancer?' The answer is ALB.
- **Connectivity Options:** You must know when to use a VPN versus Direct Connect. A classic trap: 'Your company needs a high-bandwidth, low-latency connection with consistent performance. Which option?' The answer is Direct Connect (or Cloud Interconnect). Another trap: 'Which option is encrypted and uses the public internet?' The answer is VPN. They love to ask about the trade-off between cost and reliability.
- **Network Segmentation:** Questions on subnets, public vs private, and why you separate tiers. The correct answer pattern is always 'to reduce the attack surface by limiting direct internet access to databases.'
- **IP Addressing:** CIDR notation, private IP ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16), and the difference between a public and elastic IP. A common trap: 'What happens when a VM is stopped and started?' The public IP changes unless you attached an elastic IP.
- **DNS and Routing:** You need to know that Route 53 or similar DNS services can be used for global load balancing through latency-based routing or geolocation routing. Questions may ask: 'How do you direct users to the nearest region?' The answer is a global load balancer or DNS routing policy.
- **High Availability Design:** The exam asks about fault tolerance. The correct approach is to deploy resources in at least two Availability Zones (AZs) within a region. A trap question might suggest spanning regions for high availability—this is actually for disaster recovery, not just high availability. Know the difference.
- **Security Groups and ACLs:** Security groups are stateful firewalls at the instance level. Network ACLs are stateless and operate at the subnet level. Memorise the stateful vs stateless distinction. A trap: 'You need to block a specific source IP. Which should you modify?' The answer is a Network ACL (inbound rule), because a security group only allows you to allow, not block, specific IPs.
- **Common Trap Patterns:** The exam often gives a scenario with a single load balancer and asks what fails if that load balancer goes down. The correct answer is 'the application becomes unreachable' unless a second load balancer in another AZ is configured. Another trap: 'An autoscaling group launches new instances, but traffic still fails. Why?' The answer is typically that the new instances are not registered with the load balancer, or the security group does not allow traffic from the load balancer. You must also be prepared to identify the correct order of steps for setting up load balancing: create a target group, register targets, configure a listener, associate with the load balancer, then test health checks.

Overall, memorise the definitions and use cases for each connectivity type, the two load balancer types (ALB and NLB), and the core difference between security groups and network ACLs. Practise with scenario questions that force you to choose between VPN and Direct Connect, or between ALB and NLB.

## Step by step

1. **Create a Virtual Private Cloud (VPC)** — You define a logically isolated network within the cloud provider's infrastructure, specifying an IP address range (e.g., 10.0.0.0/16). This VPC will contain all your cloud resources, acting as the private network boundary for your application.
2. **Create Subnets and an Internet Gateway** — Divide the VPC into public subnets (for web servers) and private subnets (for databases). Attach an internet gateway to the public subnets to allow internet traffic in. Configure a route table that directs traffic destined for 0.0.0.0/0 (any internet address) to the internet gateway.
3. **Deploy a Load Balancer and Target Group** — Create an Application Load Balancer (ALB) in the public subnets, listening on port 80 (HTTP) or 443 (HTTPS). Define a target group that contains the private IP addresses of your web servers. This target group is what the load balancer will forward traffic to.
4. **Register Backend Servers and Configure Health Checks** — Register each web server instance with the target group by specifying its private IP. Configure a health check path (e.g., /health) that the load balancer requests periodically. If a server does not respond with a success code (like 200) within a timeout, it is marked unhealthy and removed from rotation.
5. **Set Up Autoscaling with the Load Balancer** — Create an autoscaling group that automatically launches new virtual machine instances based on CPU utilisation or other metrics. Configure the autoscaling group to automatically register new instances with the load balancer's target group. When traffic drops, the autoscaler terminates extra instances and deregisters them.
6. **Establish Connectivity to On-Premises (Optional)** — If you need to connect your office network to the cloud VPC, create a VPN connection using a virtual private gateway, or order a Direct Connect circuit. Configure the on-premises router to establish the encrypted tunnel (VPN) or direct fibre link (Direct Connect). Update route tables in the VPC to route traffic to the on-premises network.
7. **Test the Full Stack** — Simulate traffic by sending requests to the load balancer's public DNS name. Verify that requests are distributed across healthy servers. Simulate a server failure by stopping one instance and checking that the load balancer redirects all traffic to the remaining healthy servers. Monitor network performance using cloud monitoring tools.

## Comparisons

### Application Load Balancer (ALB) vs Network Load Balancer (NLB)

**Application Load Balancer (ALB):**
- Operates at Layer 7 (HTTP/HTTPS), routing based on URL paths, headers, or cookies.
- Best suited for microservices, container orchestration (like Kubernetes), and web applications.
- Supports advanced features like SSL termination, WebSockets, and HTTP/2.

**Network Load Balancer (NLB):**
- Operates at Layer 4 (TCP/UDP), routing based on port and protocol.
- Best suited for extreme performance needs, millions of requests per second, and real-time applications.
- Provides ultra-low latency and can handle static IP addresses (elastic IP) for whitelisting.

### VPN (Virtual Private Network) vs Direct Connect (Cloud Interconnect)

**VPN (Virtual Private Network):**
- Uses the public internet to create an encrypted tunnel between your network and the cloud.
- Lower cost but subject to internet congestion, latency, and jitter.
- Setup is quick and can be deployed in minutes using cloud provider VPN services.

**Direct Connect (Cloud Interconnect):**
- Provides a dedicated, physical fibre connection that bypasses the public internet.
- Higher cost but offers consistent low latency, high bandwidth, and more reliable throughput.
- Setup requires physical fibre installation and carrier coordination, often taking weeks to months.

### Security Group vs Network ACL (Access Control List)

**Security Group:**
- Stateful: if you allow inbound traffic on a port, the return outbound traffic is automatically allowed.
- Operates at the instance level (applied to a virtual machine or network interface).
- Only supports 'allow' rules; you cannot explicitly deny traffic from a specific IP (you just omit it).

**Network ACL (Access Control List):**
- Stateless: you must define separate inbound and outbound rules for both directions of traffic.
- Operates at the subnet level (applied to the entire subnet, affecting all instances inside).
- Supports both 'allow' and 'deny' rules, giving you fine-grained control to block specific IPs.

### Public Subnet vs Private Subnet

**Public Subnet:**
- Has a route to an internet gateway, allowing direct internet inbound and outbound access.
- Typically used for web servers, load balancers, and bastion hosts that need internet connectivity.
- Resources automatically receive a public IP unless configured otherwise.

**Private Subnet:**
- Has no route to an internet gateway; it is isolated from the internet.
- Typically used for database servers, application servers, and sensitive data workloads.
- Resources can access the internet through a NAT gateway for outbound only (e.g., to download updates).

## Diagram

_This diagram shows how user traffic flows from the internet through an internet gateway to a load balancer deployed across two availability zones, then to web servers in public subnets, and finally to databases in private subnets._

```mermaid
flowchart TD
    A[User Internet] --> B[Internet Gateway]
    B --> C[Public Subnet - AZ1]
    B --> D[Public Subnet - AZ2]
    C --> E[Load Balancer ALB]
    D --> E
    E --> F[Target Group with Web Servers]
    E --> G[Target Group with Web Servers]
    F --> H[Private Subnet - AZ1 - App Servers]
    G --> I[Private Subnet - AZ2 - App Servers]
    H --> J[Private Subnet - AZ1 - Database]
    I --> K[Private Subnet - AZ2 - Database]
    J --> L[Database Cluster]
    K --> L
```

## Common misconceptions

- **Misconception:** A load balancer distributes traffic equally to all servers at all times, ensuring every server gets the same number of connections. **Reality:** A load balancer uses algorithms like round robin or least connections; round robin gives equal turns, but least connections sends traffic to the server with the fewest current connections, which does not guarantee equal total load. (People assume 'balance' means equal share by default, but the actual distribution depends on the chosen algorithm and may not be perfectly equal over time.)
- **Misconception:** A VPN and Direct Connect both provide the same level of security and reliability because they both connect your network to the cloud. **Reality:** A VPN runs over the public internet and is subject to internet congestion and latency, while Direct Connect is a dedicated private line that bypasses the internet entirely, offering higher reliability and consistent performance. (Both provide a connection, so beginners conflate them, but they are fundamentally different in terms of performance, cost, and use case.)
- **Misconception:** A virtual private cloud (VPC) is completely isolated from the internet by default, and you cannot connect it without special permissions. **Reality:** A VPC is isolated by default, but you can connect it to the internet by attaching an internet gateway to its public subnets. It is not permanently isolated; connectivity is configurable. (The word 'private' suggests total isolation, but in practice you deliberately open parts of the VPC to the internet for web servers.)
- **Misconception:** Security groups and network ACLs are interchangeable and do the same job; you only need one of them. **Reality:** Security groups are stateful (allow return traffic automatically) and operate at the instance level, while network ACLs are stateless (must define both inbound and outbound rules) and operate at the subnet level. They are complementary and both are commonly used together. (Both are firewalls, so beginners think they serve the same purpose, but the stateful/stateless difference and the scope (instance vs subnet) are critical for exam questions.)
- **Misconception:** When using autoscaling with a load balancer, the load balancer automatically scales itself to handle more traffic without any configuration. **Reality:** The load balancer itself does not autoscale in the same way as compute instances; you must deploy the load balancer in multiple availability zones and potentially use multiple load balancers for very high traffic. Autoscaling only adds compute instances, not load balancer capacity. (Beginners often assume 'autoscaling' applies to all components, but load balancers have fixed throughput limits unless designed for high availability with multiple nodes.)

## Key takeaways

- A virtual private cloud (VPC) is your private network in the cloud, subdivided into subnets that can be public or private.
- An internet gateway allows public internet traffic into a VPC; a NAT gateway allows private instances to initiate outbound traffic but blocks inbound internet requests.
- Load balancers distribute incoming traffic across multiple servers using algorithms such as round robin or least connections.
- Application Load Balancers (ALB) route traffic based on Layer 7 information (HTTP headers, URL paths); Network Load Balancers (NLB) route based on Layer 4 (TCP/UDP) and offer extreme performance.
- Health checks allow a load balancer to automatically stop sending traffic to a failed server and redirect requests to healthy ones.
- Direct Connect (or Cloud Interconnect) is a dedicated private line that offers higher reliability and consistent performance compared to a VPN, which runs over the public internet.
- Security groups are stateful firewalls that control traffic at the instance level; network ACLs are stateless and control traffic at the subnet level.
- Deploying resources across at least two availability zones ensures high availability by protecting against single data centre failures.

## FAQ

**What is the difference between a public subnet and a private subnet in a VPC?**

A public subnet has a route to an internet gateway, so resources inside it (like web servers) can receive traffic from the internet. A private subnet does not have that route; it is isolated from the internet, typically used for databases or application servers that should not be accessed directly online.

**Do I need both a load balancer and autoscaling together?**

No, but they work best together. A load balancer distributes traffic across servers, and autoscaling automatically adjusts the number of servers. Using both allows you to handle traffic spikes by adding servers that are immediately registered with the load balancer, while scaling down to save costs when traffic falls.

**Can a load balancer handle HTTPS traffic?**

Yes. An Application Load Balancer can terminate HTTPS connections, which means it decrypts the traffic, inspects it, and then forwards it to the backend servers over HTTP or HTTPS. This offloads the encryption work from your servers and centralises certificate management.

**What is a health check and why does it matter?**

A health check is a periodic test (like a ping or an HTTP request) that a load balancer sends to each server to see if it is running correctly. If a server fails the health check, the load balancer stops sending traffic to it, which automatically routes users to healthy servers and prevents them from seeing errors.

**What is the difference between a Network Load Balancer and an Application Load Balancer?**

A Network Load Balancer (NLB) operates at Layer 4 (TCP/UDP) and can handle millions of requests per second with very low latency. An Application Load Balancer (ALB) operates at Layer 7 (HTTP/HTTPS) and can route based on URL paths, headers, or cookies, making it suitable for microservices and containerised applications.

**Is Direct Connect faster than a VPN?**

Yes. Direct Connect provides a dedicated, private fibre connection that bypasses the public internet, offering lower latency, higher bandwidth, and more consistent performance. A VPN uses the public internet, which is shared and subject to congestion. Direct Connect is also more expensive.

## Check your understanding

1. **Which type of load balancer would you choose to route traffic based on the URL path in the request, such as /api vs /images?**

   Answer: An Application Load Balancer (ALB), because it operates at Layer 7 and can inspect HTTP headers and paths to make routing decisions.

2. **What is the primary difference between a security group and a network ACL in terms of state?**

   Answer: A security group is stateful (return traffic is automatically allowed), while a network ACL is stateless (you must define both inbound and outbound rules explicitly).

3. **Your company needs a private, high-bandwidth connection between its on-premises data centre and the cloud, with consistent latency. Which connectivity option should you choose?**

   Answer: Direct Connect (or Cloud Interconnect), because it provides a dedicated physical line that bypasses the public internet for reliable performance.

4. **Why would you deploy cloud resources in at least two Availability Zones?**

   Answer: To achieve high availability; if one data centre (AZ) fails, the application continues running in the other AZ, and the load balancer routes traffic only to healthy instances.

5. **What happens when an autoscaling group launches a new VM in an environment with a load balancer?**

   Answer: The new VM is automatically registered with the load balancer's target group, so the load balancer starts sending incoming traffic to it as soon as it passes health checks.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/cloud-plus/networking-connectivity