# DNS for Service Discovery and External Name Resolution

> Chapter 17 of the Courseiva CNCF-CKAD curriculum — https://courseiva.com/learn/cncf-ckad/dns-service-discovery

**Official objective:** 6.4 — Use Kubernetes DNS for service discovery and external name resolution

## Introduction

The DNS for Service Discovery and External Name Resolution exam objective is all about helping your applications find each other and reach external services easily, without hardcoding IP addresses. For CKAD, this is essential because Kubernetes clusters are dynamic — services can be created, moved, or scaled up at any moment, and you need to know how to configure DNS so your apps can always connect to the right place. Mastering this saves you from hours of debugging connectivity problems in production.

## The Hotel Concierge Analogy

A hotel concierge is the person at the front desk who knows exactly where every room, restaurant, and service is located inside the hotel. When a guest asks, 'Where is the swimming pool?' the concierge doesn't have to guess or wander around — she consults her mental map and instantly replies, 'It is on the second floor, room 205.' She also knows about places outside the hotel: if a guest wants to visit a famous museum across town, the concierge looks up the museum's street address and gives directions. 

Now imagine hundreds of guests arriving every second, each asking for a different service. The concierge cannot memorise every single room number or external address, so she uses a big directory book. When a guest asks for 'the pool', she flips to the right page and reads the room number. When a guest asks for 'the museum', she checks the external directory and gives the street address. This is exactly how Kubernetes DNS works inside a cluster. Each service in the cluster gets a 'room number' (the IP address), and the DNS system acts like the concierge — it translates human-friendly names like 'my-database-service' into the correct IP address so applications can find each other without memorising numbers. And for resources outside the cluster, like an external API, DNS provides a way to look up the external address too. Without this concierge, every application would need to know the exact IP address of every other service it depends on, which is impossible in a cluster where services are constantly being created, moved, and deleted.

## Core explanation

Let's start with the big idea: in a Kubernetes cluster, your applications (called Pods) need to talk to each other. A Pod is like a tiny virtual computer that runs your application code. But Pods come and go — they can be created, moved to another node, or deleted when you update your app. If your application uses a fixed IP address to reach another service, that connection will break as soon as the target Pod is replaced by a new one with a different IP address. This is the problem DNS (Domain Name System) solves. DNS is a system that translates friendly names, like 'my-app-service', into machine-readable IP addresses. Think of it as the phonebook of the internet — you look up a person's name, and you get their phone number. In Kubernetes, DNS gives each service a stable name that never changes, even if the underlying Pods get new IP addresses.

Kubernetes has a built-in DNS service called CoreDNS (or historically, kube-dns). CoreDNS runs as a set of Pods inside the cluster and automatically watches for any Services you create. A Service is a stable endpoint that groups one or more Pods together. When you create a Service named 'backend' in the 'production' namespace, CoreDNS registers that name. Any other Pod in the cluster can then reach that service using the DNS name 'backend.production.svc.cluster.local'. This is called service discovery — applications discovering each other using DNS names.

Let's break down that DNS name piece by piece:

- 'backend' is the name of the Service you created.
- 'production' is the Kubernetes namespace — a way to organise resources. If you don't specify a namespace, it defaults to 'default'.
- 'svc' stands for service. This part tells you it is a Kubernetes Service.
- 'cluster.local' is the cluster domain — a suffix that is the same for all services inside that cluster.

If you have a Pod in the same namespace as the service, you can shorten the name to just 'backend'. Kubernetes DNS will automatically append the namespace, 'svc', and 'cluster.local'. So a simple 'curl http://backend' from a Pod in the same namespace will work. This is convenient for everyday use.

Now, what about external names? Sometimes your application needs to reach a resource outside the cluster — for example, an external database hosted on Amazon RDS or a third-party API. You can create a special type of Service called an ExternalName Service. This Service does not point to any Pods inside the cluster. Instead, it returns a CNAME record — a DNS record that maps one name to another. For example, if you create an ExternalName Service called 'external-db' that maps to 'mydb.amazonaws.com', then any Pod that looks up 'external-db' will get the IP address of 'mydb.amazonaws.com'. This keeps external dependencies hidden behind a stable local DNS name.

There is also the concept of Headless Services. A Headless Service is created by setting the 'clusterIP' field to 'None'. When you do this, DNS does not return a single virtual IP address. Instead, it returns all the IP addresses of the Pods matching the service's selector. This is useful for stateful applications like databases where you want to discover individual Pods, not load-balanced traffic.

Another important detail is the Pod's 'hostname' and 'subdomain'. You can set these in a Pod's YAML specification. If a Pod has a subdomain that matches an existing Service name, DNS will create an additional record: 'hostname.subdomain.namespace.svc.cluster.local'. This allows you to address individual Pods by their hostname. This is commonly used with StatefulSets, where each Pod gets a stable identity and DNS name.

Let's look at a concrete example. Imagine you create a Service called 'orders' in the namespace 'myapp'. A Pod in the same namespace can reach it with 'curl http://orders'. A Pod in a different namespace called 'api' would use 'curl http://orders.myapp.svc.cluster.local'. CoreDNS handles the lookup by checking its internal registry of Services.

Why does this matter? In the real world, applications are made of many microservices — small independent pieces that each handle a specific function (like orders, payments, and notifications). Without DNS, each microservice would need to know the exact IP address and port of every other service it calls. If a service scales up and gets a new Pod with a new IP, every caller would break. DNS gives you a level of indirection: your code always uses a name, and DNS always resolves that name to the current correct IP address. This is the heartbeat of a microservices architecture.

## Real-world context

Imagine you are working for an e-commerce company that runs its platform on Kubernetes. The platform is made up of several microservices: a 'frontend' service that serves the website, an 'orders' service that handles purchases, a 'payments' service that processes transactions, and a 'notifications' service that sends email confirmations. Each service runs as a Deployment with multiple replicas for reliability.

Your job is to ensure that the 'frontend' service can talk to the 'orders' service, and that the 'orders' service can call the 'payments' service and the 'notifications' service. Without Kubernetes DNS, you would have to manually find the IP addresses of each service and hardcode them into configuration files. Every time a service scales up or down, you would need to update those IPs. This is not just tedious — it is dangerous. A single mistake could bring down the entire checkout flow.

Using Kubernetes DNS, you create a Service for each microservice:

- Create a Service named 'frontend' for the frontend Pods.
- Create a Service named 'orders' for the orders Pods.
- Create a Service named 'payments' for the payments Pods.
- Create a Service named 'notifications' for the notifications Pods.

Then in the frontend application code, you configure it to call 'http://orders' (if they are in the same namespace) or 'http://orders.default.svc.cluster.local' (if in different namespaces). The application never needs to know the actual IP addresses. If the orders service is updated and its Pods are recreated, the DNS name 'orders' will instantly resolve to the new Pod IPs.

Now, step back to a specific real-world task. Your team is migrating from an old monolithic application to microservices. The old system used a single database hosted outside Kubernetes. You need the 'orders' service to connect to this external database. Instead of hardcoding the external database's hostname in every Pod, you create an ExternalName Service:

```yaml
apiVersion: v1
kind: Service
metadata:
  name: legacy-db
spec:
  type: ExternalName
  externalName: mycompany.legacydb.com
```

Now, the 'orders' service can connect to 'legacy-db' (or 'legacy-db.default.svc.cluster.local'), and DNS will resolve it to 'mycompany.legacydb.com'. If the database provider changes the hostname later, you only update this one Service definition, not every Pod.

Another common scenario: debugging DNS issues. Suppose the 'payments' service reports that it cannot reach 'notifications'. You would check whether the 'notifications' Service exists and whether its DNS name is correct. You can test from a Pod using tools like 'nslookup' or 'dig':

- 'kubectl run -it --rm debug --image=busybox -- sh'
- Inside the container, run 'nslookup notifications'

If the DNS lookup fails, you check whether the Service exists ('kubectl get svc notifications') and whether CoreDNS is running ('kubectl get pods -n kube-system | grep coredns'). This step-by-step troubleshooting is a core part of working with Kubernetes DNS in production.

Finally, consider a scenario where you need individual Pod identities. You deploy a StatefulSet for a database called 'mysql' with three replicas. Each replica gets a stable hostname: 'mysql-0', 'mysql-1', and 'mysql-2'. You create a Headless Service called 'mysql' with 'clusterIP: None'. Now, DNS will return A records for each Pod: 'mysql-0.mysql.default.svc.cluster.local' resolves to the IP of the first Pod, etc. Your application can connect to a specific replica by its hostname, which is critical for database clusters that need to know which node is the primary and which are replicas.

All of these examples are what IT professionals actually do daily: they design and configure Kubernetes DNS for service discovery, manage ExternalName Services for external dependencies, and troubleshoot DNS resolution when applications cannot connect. It is a foundational skill that saves teams from brittle, hardcoded configurations.

## Exam focus

The CKAD exam tests your ability to configure and troubleshoot DNS for service discovery and external name resolution in a terminal-based environment. You will have a Kubernetes cluster running, and you must complete tasks using kubectl commands and YAML files. The exam is hands-on, not multiple-choice, so you need to know the exact syntax and behaviour.

Here are the specific concepts that the CKAD exam loves to test:

- Creating a Service and verifying that a Pod can resolve its DNS name. You may be asked to create a simple Service for an existing Deployment and then run a 'curl' command from another Pod to test connectivity.
- Using ExternalName Services. You will be asked to create an ExternalName Service that maps to an external hostname, such as 'example.com' or an external database URL. You need to know the YAML structure: 'type: ExternalName' and 'externalName' field.
- Headless Services. You may need to create a Service with 'clusterIP: None' (headless). Then you must verify that DNS returns multiple A records pointing to the Pod IPs, rather than a single virtual IP.
- DNS naming conventions. You must know the full DNS name format: '<service>.<namespace>.svc.cluster.local'. Commands like 'kubectl run' with '--rm -it' to start a temporary pod for testing DNS lookups are common (e.g., 'kubectl run test --image=busybox -it --rm -- nslookup my-service').
- StatefulSets and Pod DNS. You may be asked to create a StatefulSet with a Headless Service, then verify that each Pod has a unique DNS name in the format '<pod-name>.<service>.<namespace>.svc.cluster.local'.
- Namespace scoping. A common trap: you create a Service in namespace 'ns1', then try to reach it from a Pod in namespace 'ns2' using just the service name. It will fail unless you use the full DNS name including the namespace. The exam expects you to know that DNS names are fully qualified with namespace when crossing namespace boundaries.

Common traps and how to avoid them:

- Trap: Forgetting to include 'cluster.local' when using fully qualified domain names. In practice, Kubernetes DNS often works without it, but the exam may test whether you know the full format. Always use the full format when explicitly writing DNS names in configuration or commands.
- Trap: Confusing 'type: ExternalName' with 'type: ClusterIP'. ExternalName does not create a cluster IP. It only creates a DNS CNAME record. If you try to connect to it via IP, it will not work.
- Trap: Assuming DNS works immediately after creating a Service. In reality, there is a short propagation delay (a few seconds). The exam may test your patience — you might need to wait or re-run a command.
- Trap: Not using the correct 'selector' in a Service to match Pod labels. If the service cannot find matching Pods, DNS will still create the record, but it will not point to any Pod IPs. The exam often hides a mismatch in labels.
- Trap: Creating a Headless Service but forgetting to set 'clusterIP: None'. If you omit the field, you get a regular ClusterIP service instead.

Key definitions to memorise:
- Service: a stable endpoint that directs traffic to a set of Pods based on label selectors.
- CoreDNS: the default DNS server in Kubernetes that runs as a Deployment in the kube-system namespace.
- ExternalName Service: a Service that maps to an external DNS name instead of Pods.
- Headless Service: a Service without a cluster IP, used for direct Pod-to-Pod discovery.
- FQDN (Fully Qualified Domain Name): the complete DNS name ending with a dot, e.g., 'my-svc.my-ns.svc.cluster.local.'
- A record: a DNS record that maps a hostname to an IPv4 address.
- CNAME record: a DNS record that maps a hostname to another hostname.
- Namespace: a virtual cluster partition for organising resources.
- kube-dns: the older name for the DNS service; still used in some documentation, but CoreDNS is the current standard.

The exam expects you to write YAML quickly and accurately. Practise writing Service definitions from memory. You should be able to create a Service YAML file in less than two minutes. Also, memorise the kubectl command to run a temporary pod for DNS testing: 'kubectl run tmp --image=busybox -it --rm -- nslookup <service-name>'.

## Step by step

1. **Create a Deployment with a label** — First, define a Deployment that runs your application Pods with a specific label, for example 'app: my-app'. This label will be used by the Service to select the Pods. Without the correct label, the Service will not find any Pods to route traffic to.
2. **Expose the Deployment as a Service** — Create a Service YAML that sets 'selector.matchLabels' to match the label you defined in the Deployment (e.g., 'app: my-app'). Specify the port your app listens on. Kubernetes DNS will automatically register this service's name (e.g., 'my-service') and create a DNS A record pointing to its ClusterIP.
3. **Verify the DNS record is created** — From a test Pod within the same namespace, run 'nslookup my-service' or 'curl http://my-service:port'. If the DNS record exists, the lookup succeeds. This step confirms the service is discoverable. If it fails, check that the Service exists and that the Pods are running with the correct labels.
4. **Test cross-namespace DNS resolution** — Create a second namespace and a Pod in that namespace. From that Pod, run 'nslookup my-service.default.svc.cluster.local' (replacing 'default' with the namespace of your Service). This step validates that you can reach services across namespaces using the full DNS name.
5. **Create an ExternalName Service** — Define a Service with 'type: ExternalName' and set 'externalName' to an external hostname (e.g., 'api.example.com'). No Pods are needed. Then from any Pod, test that 'nslookup my-external-service' returns a CNAME record pointing to 'api.example.com'. This allows your application to use a local name for external resources.
6. **Create a Headless Service for StatefulSet** — Define a Service with 'clusterIP: None' and a StatefulSet that uses it as its service name. Deploy three replicas. Then verify that each Pod gets a unique DNS name: '<pod-name>.<service-name>.<namespace>.svc.cluster.local'. Run 'nslookup <service-name>' to see all Pod IPs listed. This is how stateful applications discover individual instances.

## Comparisons

### ClusterIP Service vs Headless Service

**ClusterIP Service:**
- Creates a virtual IP (ClusterIP) that load balances traffic to Pods.
- DNS returns a single A record pointing to the ClusterIP.
- Used for stateless applications where you do not care about individual Pod identity.

**Headless Service:**
- Does not create a virtual IP (clusterIP: None).
- DNS returns multiple A records, one per Pod IP.
- Used for stateful applications like databases where each Pod needs a stable identity.

### ClusterIP Service vs ExternalName Service

**ClusterIP Service:**
- Points to Pods inside the cluster via label selectors.
- Has a ClusterIP that can be used to reach the service.
- DNS returns an A record with the ClusterIP.

**ExternalName Service:**
- Points to an external hostname, not Pods inside the cluster.
- Has no ClusterIP at all.
- DNS returns a CNAME record pointing to the external hostname.

### Short DNS Name (e.g., 'orders') vs Fully Qualified DNS Name (e.g., 'orders.default.svc.cluster.local')

**Short DNS Name (e.g., 'orders'):**
- Only works within the same namespace.
- Automatically appended by Kubernetes DNS.
- Convenient for within-namespace communication.

**Fully Qualified DNS Name (e.g., 'orders.default.svc.cluster.local'):**
- Works across any namespace and from outside the cluster.
- Must be explicitly written by the user.
- Required when the Pod's DNS search domains do not include the target namespace.

### Service Discovery via DNS vs Service Discovery via Environment Variables

**Service Discovery via DNS:**
- Dynamic: resolves to the current IPs at lookup time.
- Works across namespaces with full DNS names.
- Is the recommended approach for production.

**Service Discovery via Environment Variables:**
- Static: IPs are injected when the Pod starts, and do not update if the service changes.
- Only works for services created before the Pod (ordering dependency).
- Considered legacy; DNS is preferred.

## Diagram

_Flow chart showing how an application Pod uses CoreDNS to resolve a Service name to a ClusterIP or an external CNAME record._

```mermaid
flowchart TD
    A[Application Pod] -->|Looks up 'orders'| B[CoreDNS Pod]
    B -->|Returns ClusterIP| C[ClusterIP Service 'orders']
    C -->|Forwards traffic| D[Pod 1 (IP 10.0.0.1)]
    C -->|Forwards traffic| E[Pod 2 (IP 10.0.0.2)]
    A -->|Looks up 'external-api'| F[CoreDNS Pod]
    F -->|Returns CNAME| G[ExternalName Service 'external-api']
    G -->|Resolves to| H[api.example.com]
```

## Common misconceptions

- **Misconception:** DNS only works for services in the same namespace; you cannot reach services in other namespaces at all. **Reality:** You can reach services in other namespaces by using the fully qualified DNS name: '<service-name>.<namespace>.svc.cluster.local'. This is a common pattern in microservices deployments. (Beginners often only learn the short form (just the service name) and think DNS is namespace-restricted. They do not realise the full DNS format is designed for cross-namespace communication.)
- **Misconception:** An ExternalName service can be accessed by its cluster IP just like a regular ClusterIP service. **Reality:** An ExternalName service does not have a cluster IP. It only provides a DNS CNAME record. You must use the DNS name to reach the external resource; connecting by IP is impossible. (The word 'Service' is overloaded. Beginners assume all services have IP addresses, but ExternalName is a special case that only affects DNS.)
- **Misconception:** If a Service is created, DNS is updated instantly, and any Pod can resolve it immediately. **Reality:** There is a small propagation delay (usually a few seconds) because CoreDNS must pick up the change. In practice, this is almost instant, but during the exam you might need to wait a moment before testing. (New users expect immediate consistency, but distributed systems have eventual consistency. The exam environment may have slight delays, leading to false test failures.)
- **Misconception:** You must use the full 'cluster.local' suffix in all DNS lookups, even within the same namespace. **Reality:** Inside the same namespace, just the service name works (e.g., 'my-service'). The 'cluster.local' suffix is only needed when resolving from a different namespace or when the system is not automatically appending the suffix. (Documentation often shows the full FQDN, making beginners think it is mandatory everywhere. In practice, Kubernetes DNS automatically appends the namespace and suffix.)
- **Misconception:** A Headless Service cannot be used for load balancing; it is useless for production. **Reality:** Headless Services are intentionally used for stateful workloads where you want to discover individual Pod IPs directly, such as databases (Cassandra, MySQL) or when you have your own load-balancing logic. (The term 'headless' sounds broken or incomplete. Beginners think it is less capable, but it is a deliberate design choice for specific use cases.)
- **Misconception:** Kubernetes DNS only supports A records (IPv4) and cannot handle CNAME records. **Reality:** Kubernetes DNS handles both A records (for cluster IPs and headless Pod IPs) and CNAME records (for ExternalName services). (Most introductory examples focus on A records. Beginners are not exposed to CNAME behaviour until they encounter ExternalName services.)

## Key takeaways

- Kubernetes DNS translates service names into IP addresses, so your applications can find each other without hardcoding IPs.
- The full DNS name format for a service is '<service-name>.<namespace>.svc.cluster.local'.
- An ExternalName Service creates a DNS CNAME record to an external hostname, not a cluster IP.
- A Headless Service (clusterIP: None) returns multiple A records for individual Pod IPs instead of a single virtual IP.
- CoreDNS is the default DNS server in Kubernetes that automatically registers new Services and Pods.
- When resolving a service from a different namespace, you must use the full DNS name including the namespace.
- DNS is not instant — there is a short propagation delay after creating or updating a Service.
- You can test DNS resolution from a Pod using tools like 'nslookup' or 'dig' from an interactive busybox container.

## FAQ

**What is the difference between a regular Service and a Headless Service in Kubernetes DNS?**

A regular Service (with a ClusterIP) creates a single DNS A record pointing to a virtual IP that load balances traffic to Pods. A Headless Service (clusterIP: None) creates multiple A records, each pointing directly to a Pod IP, which is useful for stateful applications that need to talk to specific instances.

**How do I make a service discoverable in another namespace?**

Use the fully qualified domain name: '<service-name>.<namespace>.svc.cluster.local'. For example, if you have a service 'orders' in namespace 'prod', a Pod in namespace 'dev' should connect to 'orders.prod.svc.cluster.local'.

**Why does my curl command fail even though the Service exists?**

Check that the Service's label selector matches the Pods' labels. Use 'kubectl describe service <name>' to see if the Endpoints list is empty. Also verify you are using the correct DNS name (including namespace if needed) and that the Pods are running.

**Can I use an ExternalName Service to point to a local IP address?**

Technically the externalName field expects a hostname, not an IP. However, you can use a hostname that resolves to the IP (e.g., an internal DNS name). For pointing to a specific external IP without DNS, consider using a Service type Endpoints or a regular Service with a custom Endpoint slice instead.

**How do I test DNS resolution from within a Pod?**

Run 'kubectl run -it --rm test --image=busybox -- nslookup <service-name>'. This creates a temporary Pod, runs the DNS lookup, and deletes the Pod after. Alternatively, use 'dig' if the image includes it.

**What is the default DNS policy for a Pod?**

The default DNS policy is 'ClusterFirst', which means the Pod uses the cluster's DNS server (CoreDNS) to resolve any DNS queries. If a query is not a cluster-local name (i.e., it does not match the cluster domain), CoreDNS forwards it to the external DNS server configured on the node.

**Does DNS work for Pods that are not part of any Service?**

By default, a Pod without a Service does not get a DNS record from CoreDNS. However, if the Pod has a 'hostname' and 'subdomain' set, and there is a Service with that subdomain name, DNS will create an A record for that Pod. StatefulSets handle this automatically.

**How do I change the DNS domain from 'cluster.local' to something else?**

You can modify the CoreDNS ConfigMap (usually in the kube-system namespace) to change the 'clusterDomain' setting. However, this is a cluster-wide configuration change and is not typically required for the CKAD exam.

## Check your understanding

1. **A Pod in the 'frontend' namespace needs to reach a service named 'api' in the 'backend' namespace. What DNS name should it use?**

   Answer: The Pod should use 'api.backend.svc.cluster.local' to resolve the service across namespaces.

2. **You create a Service with 'type: ExternalName' and 'externalName: example.com'. What type of DNS record does this create?**

   Answer: It creates a CNAME record that maps the service name to 'example.com'.

3. **What field must you set to 'None' to create a Headless Service?**

   Answer: You must set 'clusterIP: None' in the Service specification.

4. **When a Pod is in the same namespace as a Service, can it use just the service name (e.g., 'orders') to reach it?**

   Answer: Yes, because Kubernetes DNS automatically appends the namespace, 'svc', and 'cluster.local' when the Pod is in the same namespace.

5. **Why would you use a Headless Service instead of a regular ClusterIP Service for a StatefulSet?**

   Answer: A Headless Service provides direct DNS records for each Pod (e.g., 'mysql-0.mysql.default.svc.cluster.local'), which is needed for stateful applications that require stable network identities.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/cncf-ckad/dns-service-discovery