Kubernetes - DNS Resolution

You can follow this post with the interactive scenario on Katacoda.

We are going to explore how DNS resolution in Kubernetes works. First we create a namespace, then we create a pod and expose it via service. Afterwards, a second pod is used to perform DNS queries against the Kubernetes DNS Resolver to get the IP address of the service.

Namespace

We create a namespace that we will use to bind our resources.

kubectl create namespace dev

Next, we run a pod via imperative command.

Running some Pod

kubectl run my-app --image nginx --namespace dev --port 80

Describe the Pod

kubectl describe pod my-app -n dev

Show Output

Name:         my-app
Namespace:    dev
Priority:     0
Node:         raspberrypi/192.168.1.41
Start Time:   Fri, 25 Jun 2021 01:21:16 +0100
Labels:       run=my-app
Annotations:  <none>
Status:       Running
IP:           10.42.0.126
IPs:
  IP:  10.42.0.126
Containers:
  my-app:
    Container ID:   containerd://086772d833ec67917a98ef43561d6f18779f086daa5b93a3390474a6aa707160
    Image:          nginx
    Image ID:       docker.io/library/nginx@sha256:47ae43cdfc7064d28800bc42e79a429540c7c80168e8c8952778c0d5af1c09db
    Port:           80/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Fri, 25 Jun 2021 01:21:20 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-fnb62 (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-fnb62:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  30s   default-scheduler  Successfully assigned dev/my-app to raspberrypi
  Normal  Pulling    29s   kubelet            Pulling image "nginx"
  Normal  Pulled     28s   kubelet            Successfully pulled image "nginx" in 1.369183373s
  Normal  Created    28s   kubelet            Created container my-app
  Normal  Started    27s   kubelet            Started container my-app

Note the label that has been set by Kubernetes. run=my-app By default, Kubernetes will set labels that match the resource name. For resources started from a run it will have the form run=<resource-name>.

Now we can expose the pod. This will create a service matching the pods label.

Create Service

kubectl expose pod my-app --namespace dev

Check The Service

kubectl describe service my-app -n dev

Show Output

Name:              my-app
Namespace:         dev
Labels:            run=my-app
Annotations:       <none>
Selector:          run=my-app
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.43.52.98
IPs:               10.43.52.98
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.42.0.126:80
Session Affinity:  None
Events:            <none>

Note how the service selector is matching the label run=my-app. That means it will match the pod we have previously deployed.

Now we can deploy another pod from which we query the Kubernetes DNS Resolver.

Run dnsutils Pod

We run this pod in interactive mode and attach stdin so that we can use nslookup and dig from within the container to query the Kubernetes DNS Resolver.

kubectl run dnsutils --namespace dev --image tutum/dnsutils -ti -- bash

Making DNS Queries

nslookup resolves the service ok

nslookup my-app

Show Output

Server:         10.43.0.10
Address:        10.43.0.10#53

Name:   my-app.dev.svc.cluster.local
Address: 10.43.52.98

But dig doesn't find the service, why?

dig my-app

Show Output

; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>> my-app
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 51094
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 4c23b7d697ed3587 (echoed)
;; QUESTION SECTION:
;my-app.                                IN      A

;; AUTHORITY SECTION:
.                       13      IN      SOA     a.root-servers.net. nstld.verisign-grs.com. 2021062402 1800 900 604800 86400

;; Query time: 0 msec
;; SERVER: 10.43.0.10#53(10.43.0.10)
;; WHEN: Fri Jun 25 01:41:30 UTC 2021
;; MSG SIZE  rcvd: 122

/etc/resolv.conf

In order to understand why dig doesn't find the service, let's take a look at /etc/resolv.conf

cat /etc/resolv.conf

Show Output

search dev.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.43.0.10
options ndots:5

This file contains a line with the following format.

search <namespace>.svc.cluster.local svc.cluster.local cluster.local

That means, when providing an incomplete part of the fully qualified domain name (FQDN), this file can be used to complete the query. However, dig doesn't do it by default. We can use the +search flag in order to enable it.

dig +search my-app

Show Output

; <<>> DiG 9.11.5-P4-5.1+deb10u5-Debian <<>> +search my-app
;; global options: +cmd
;; Got answer:
;; WARNING: .local is reserved for Multicast DNS
;; You are currently testing what happens when an mDNS query is leaked to DNS
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 39376
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: de26c4eaa4e53026 (echoed)
;; QUESTION SECTION:
;my-app.dev.svc.cluster.local.  IN      A

;; ANSWER SECTION:
my-app.dev.svc.cluster.local. 5 IN      A       10.43.52.98

;; Query time: 0 msec
;; SERVER: 10.43.0.10#53(10.43.0.10)
;; WHEN: Fri Jun 25 01:42:34 UTC 2021
;; MSG SIZE  rcvd: 113

Now the service-name has been correctly resolved.

We can get the same service without +search flag when using the FQDN. The +short flag isn't required, but it will reduce the output to only the IP address.

$ dig +short my-app.dev.svc.cluster.local
10.43.52.98

However, the benefit of using the search method it that queries will automatically resolve to resources within the same namespace. This can be useful to apply the same configuration to different environments, such as production and development.

Resources in different namespaces always need to be looked up by the FQDN.

The same way the search entry in resolv.conf completes the query with the default name space, it will complete any part of the FQDN from left to right. So in the below example, it will resolve to the local cluster.

$ dig +short +search my-app.dev
10.43.52.98

Tags:

Kubernetes Dns Tutorial

Websites To Learn Programming For Free

C# For Beginners - Lesson 6: Strings

Laravel Fluent Str - 48 Methods [🚀 10 MINUTES!]

How to Safely Render Markdown From a React Component

Your Essential Guide To Map Built-in Object In Javascript

Get Prometheus Metrics from a Express.js app

Operating Systems: Introduction

How to cook up a powerful React async component using hooks (and no Suspense)

Kubernetes - DNS Resolution

Tags:

Related Posts