Kubernetes Service and Ingress-Part 2
ClusterIP and Kubernetes DNS
ClusterIP is the default service type and it is used for making your application reachable/accessible only from within the cluster; meaning that only the pods within the cluster can access it. This is achieved by exposing the service on a virtual IP address which exists on each worker node. This IP address is not announced nor shared with anywhere external to the cluster.
How is the virtual IP address implemented ?
The service virtual IP is assigned from a pool which is configured by setting the -—service-cluster-ip-range parameter in the kube-apiserver. Amazon EKS allows you to configure this parameter during the cluster creation process as an optional step. If not configured explicitly then Amazon EKS provisions either 10.100.0.0/16 or 172.20.0.0/16.
The service virtual IP address is configured on each worker node by a component called kube-proxy which is a process that runs on all the worker nodes. As mentioned earlier this virtual IP is not announced to anywhere exteranl to the Kubernetes cluster. That is the exact reason why other pods in the same cluster can access this service virtual IP however an external client/user cannot.
A visual representation of the ClusterIP is shown below. The client, which is the pod on the far left, sends a request to the service. The request then gets load balanced and forwarded to one of the pods backing that service.
How exactly is the load balancing/forwarding implemented ?
The load balancing and forwarding logic is based on the kube-proxy mode. Kube-proxy has different implementation modes (iptables, ipvs and others). Amazon EKS by default has kube-proxy implemented in iptables mode. This can be verified as shown below on the Amazon EKS cluster.
$kubectl get cm kube-proxy-config -n kube-system -o yaml | grep mode
mode: "iptables"When a pod sends a request to the service virtual IP, the packet is processed by the iptables rules on that worker node which are configured by the kube-proxy. A pod backing that service is chosen based on random by default or session affinity if configured; and then destination network address translation (DNAT) is applied. Basically, the destination IP address field in the IP header of the packet is changed from the service virtual IP address to the selected pod IP address.
What if the source and destination is on the same node ?
It is worth noting that the load balancing/forwarding logic (iptables) implemented is very simple. Meaning even when the respective ClusterIP service has pod(s) on the same worker node where the request is generated on, that request may still get forwarded to a pod on a different worker node. For more details on how to modify/enhance this default behaviour, please read the Service Internal Traffic Policy section in the official Kubernetes documentation. Keep in mind that there are trade-offs of implementing it.
How the list of endpoints (pods) automatically gets updated and how requests get forwarded to the available endpoints
I used the below manifest to create a deployment with one replica, and a service.
apiVersion: apps/v1
kind: Deployment
metadata:
name: test
spec:
replicas: 1
selector:
matchLabels:
app: demo
template:
metadata:
labels:
app: demo
spec:
containers:
- image: dumlutimuralp/networktest
name: test
---
apiVersion: v1
kind: Service
metadata:
name: test
spec:
selector:
app: demo
ports:
- protocol: TCP
port: 80The container image is based on Network-MultiTool which runs a web server using NGINX.
Let’ s identify the pods with label app: demolabel and their IP addresses.
% kubectl get pods --selector app=demo -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
test-587558b9c4-fhwwn 1/1 Running 0 176m 192.168.143.160 i-06a35fb3dc233a45f <none> <none>
Since the deployment has one replica there is only one pod in the output above.
Let’ s now examine the service test.
% kubectl describe service test
Name: test
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=demo
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.100.39.24
IPs: 10.100.39.24
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 192.168.143.160:80
Session Affinity: None
Events: <none>Kubernetes DNS (CoreDNS is the most common one) created an A record for this service which points out to the service virtual IP of the service. The format of the DNS name for the service is <service-name>.<namespace-name>.svc.<cluster-domain-name>. In this case it is test.default.svc.cluster.local.
In the output shown above,
Selector(app=demo) , the service basically identifies and selects the pods which have this label.IP(10.100.39.24) is the service virtual IP.Port(http 80) , name and value of the port which the service listens on.TargetPort(80) , the port which the application listens on the pod. We did not define theportparameter in the deployment manifest explicitly. In that case it automatically inherits the port defined for the service, which is80.Endpoints, the list of the endpoints (pods) backing the service. There is a single endpoint in this list since there is a single pod inproddetaildeployment. IP you see here is that single pod' s IP.
What about IP Family Policy , IP Families and IPs fields in the above output ? These fields are all related to dual-stack environments. For more information please have a look at IPv4/IPv6 Dual Stack of the Kubernetes documentation and Services section of the Kubernetes API guide.
How can we verify the way endpoints get picked ?
I already highlighted, in the What is a label selector section of Part 1 of this series, that there is a separate endpoints object defined for each service. The endpoints controller component continuously checks for the pods that have the label matching the selector defined in the service. It then automatically adds/removes the new/terminated pod IP addresses to the list in the endpoints object. You can verify the endpoints of any service by describing the endpoints object directly (rather than the service object). I will verify that now.
% kubectl describe endpoints test
Name: test
Namespace: default
Labels: <none>
Annotations: endpoints.kubernetes.io/last-change-trigger-time: 2025-06-17T14:22:45Z
Subsets:
Addresses: 192.168.143.160
NotReadyAddresses: <none>
Ports:
Name Port Protocol
---- ---- --------
<unset> 80 TCP
Events: <none>There is a single pod backing this service hence there is a single endpoint in the test endpoints object. The IP address you see in the addresses is the IP of the pod whose label matches the selector configured in the service. (app=demo)
Get Dumlu Timuralp’s stories in your inbox
Join Medium for free to get updates from this writer.
Scale out the deployment test to three pods.
% kubectl scale deployment test --replicas=3
deployment.apps/test scaled
% kubectl describe service test
Name: test
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=demo
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.100.39.24
IPs: 10.100.39.24
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 192.168.143.160:80,192.168.143.162:80,192.168.143.163:80
Session Affinity: None
Events: <none>Note : It may take a few minutes until the READY column displays 3/3.
Verify the new pods.
% kubectl get pods --selector app=demo -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
test-587558b9c4-22nfn 1/1 Running 0 16h 192.168.143.163 i-06a35fb3dc233a45f <none> <none>
test-587558b9c4-b5z75 1/1 Running 0 16h 192.168.143.162 i-06a35fb3dc233a45f <none> <none>
test-587558b9c4-fhwwn 1/1 Running 0 19h 192.168.143.160 i-06a35fb3dc233a45f <none> <none>Verify the updated list of endpoints.
% kubectl describe service test
Name: test
Namespace: default
Labels: <none>
Annotations: <none>
Selector: app=demo
Type: ClusterIP
IP Family Policy: SingleStack
IP Families: IPv4
IP: 10.100.39.24
IPs: 10.100.39.24
Port: <unset> 80/TCP
TargetPort: 80/TCP
Endpoints: 192.168.143.160:80,192.168.143.162:80,192.168.143.163:80
Session Affinity: None
Events: <none>As you can see the Endpoints field in the above output shows three IP addresses which belong to the pods (previous output) that are part of the test deployment.
Will nowdemonstrate the following request and forwarding scenario shown in the figure below.
Let’ s deploy a new pod as the client and generate a curl request for the test service.
% kubectl run client --rm --image=curlimages/curl -it -- sh
If you don't see a command prompt, try pressing enter.
~ $
~ $ curl test
Praqma Network MultiTool (with NGINX) - test-587558b9c4-b5z75 - 192.168.143.162/32
~ $ exit
Session ended, resume using 'kubectl attach client -c client -i -t' command when the pod is running
pod "client" deletedAs you can see in the above output, the client request got forwarded to the pod with the IP 192.168.143.162.
How is typing only “test” in the curl request enough to resolve to the DNS name of the actual service ?
The reason is that kubelet process on the worker node automatically injects the search domains into each pod. You can have a look at this in any pod.
% kubectl run client - rm - image=curlimages/curl -it - sh
If you don't see a command prompt, try pressing enter.
~ $ cat /etc/resolv.conf
search default.svc.cluster.local svc.cluster.local cluster.local eu-west-2.compute.internal
nameserver 10.100.0.10
options ndots:5
~ $ exit
Session ended, resume using 'kubectl attach client -c client -i -t' command when the pod is running
pod "client" deletedHow does the actual DNS resolution work ?
In the output just above, notice the Kubernetes DNS server IP address (nameserver) 10.100.0.10 is also from the IP pool of 10.100.0.0. That actually should give you a hint that even the Kubernetes DNS implementation itself is based on a Kubernetes service. When the client pod sends a DNS request for the testservice, the DNS request hits the respective virtual IP of the kube-dnsservice and then gets forwarded to one of the pods which is running the DNS service. You can verify this by examining the respective service for Kubernetes DNS and also the pods backing the Kubernetes DNS service, both shown below. You can use the commands kubectl get services -n kube-system and kubectl describe service kube-dns -n kube-system.
How about pinging the test service ? Would that work ?
Let' s try to ping the test service. Hint, you will not be able to successfully ping the service and you will have to perform CTRL + C to terminate it. I will test this by exec’ ing into any of the pods in the test deployment. (since the client pod uses curl image and it does not have ping utility)
% kubectl exec -it pod/test-587558b9c4-22nfn -- sh
/ #
/ #
/ # ping test
PING test.default.svc.cluster.local (10.100.39.24) 56(84) bytes of data.
^C
--- test.default.svc.cluster.local ping statistics ---
26 packets transmitted, 0 received, 100% packet loss, time 26022ms
/ # exit
command terminated with exit code 1Since the service virtual IP address is implemented as a virtual IP on each node, not shared nor announced to anywhere, it does not respond to any ICMP requests. This is expected behaviour. In the output above, the IP address, which the DNS name resolves to, is the same one which is assigned to the test service (10.100.39.24).
Service endpoint updates
I will now send 10 curl requests each second, from the client pod, and while I send these requests I will scale in the test deployment from 3 to 1 pods back again. During this process let’ s see if we notice any failures.
% kubectl run client --rm --image=curlimages/curl -it -- sh
~$ watch -n 0.1 curl test
As you see above, it is hard to notice any failures, which indicates that the list of the endpoints gets updated immediately and we start seeing only one pod serving all the requests.
