456

I am quite confused about the roles of Ingress and Load Balancer in Kubernetes.

As far as I understand Ingress is used to map incoming traffic from the internet to the services running in the cluster.

The role of load balancer is to forward traffic to a host. In that regard how does ingress differ from load balancer? Also what is the concept of load balancer inside kubernetes as compared to Amazon ELB and ALB?

0

13 Answers 13

334

Load Balancer: A kubernetes LoadBalancer service is a service that points to external load balancers that are NOT in your kubernetes cluster, but exist elsewhere. They can work with your pods, assuming that your pods are externally routable. Google and AWS provide this capability natively. In terms of Amazon, this maps directly with ELB and kubernetes when running in AWS can automatically provision and configure an ELB instance for each LoadBalancer service deployed.

Ingress: An ingress is really just a set of rules to pass to a controller that is listening for them. You can deploy a bunch of ingress rules, but nothing will happen unless you have a controller that can process them. A LoadBalancer service could listen for ingress rules, if it is configured to do so.

You can also create a NodePort service, which has an externally routable IP outside the cluster, but points to a pod that exists within your cluster. This could be an Ingress Controller.

An Ingress Controller is simply a pod that is configured to interpret ingress rules. One of the most popular ingress controllers supported by kubernetes is nginx. In terms of Amazon, ALB can be used as an ingress controller.

For an example, this nginx controller is able to ingest ingress rules you have defined and translate them to an nginx.conf file that it loads and starts in its pod.

Let's for instance say you defined an ingress as follows:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
   ingress.kubernetes.io/rewrite-target: /
 name: web-ingress
spec:
  rules:
  - host: kubernetes.foo.bar
    http:
      paths:
      - backend:
          serviceName: appsvc
          servicePort: 80
        path: /app

If you then inspect your nginx controller pod you'll see the following rule defined in /etc/nginx.conf:

server {
    server_name kubernetes.foo.bar;
    listen 80;
    listen [::]:80;
    set $proxy_upstream_name "-";
    location ~* ^/web2\/?(?<baseuri>.*) {
        set $proxy_upstream_name "apps-web2svc-8080";
        port_in_redirect off;

        client_max_body_size                    "1m";

        proxy_set_header Host                   $best_http_host;

        # Pass the extracted client certificate to the backend

        # Allow websocket connections
        proxy_set_header                        Upgrade           $http_upgrade;
        proxy_set_header                        Connection        $connection_upgrade;

        proxy_set_header X-Real-IP              $the_real_ip;
        proxy_set_header X-Forwarded-For        $the_x_forwarded_for;
        proxy_set_header X-Forwarded-Host       $best_http_host;
        proxy_set_header X-Forwarded-Port       $pass_port;
        proxy_set_header X-Forwarded-Proto      $pass_access_scheme;
        proxy_set_header X-Original-URI         $request_uri;
        proxy_set_header X-Scheme               $pass_access_scheme;

        # mitigate HTTPoxy Vulnerability
        # https://www.nginx.com/blog/mitigating-the-httpoxy-vulnerability-with-nginx/
        proxy_set_header Proxy                  "";

        # Custom headers

        proxy_connect_timeout                   5s;
        proxy_send_timeout                      60s;
        proxy_read_timeout                      60s;

        proxy_redirect                          off;
        proxy_buffering                         off;
        proxy_buffer_size                       "4k";
        proxy_buffers                           4 "4k";

        proxy_http_version                      1.1;

        proxy_cookie_domain                     off;
        proxy_cookie_path                       off;

    rewrite /app/(.*) /$1 break;
    rewrite /app / break;
    proxy_pass http://apps-appsvc-8080;
    
    }

Nginx has just created a rule to route http://kubernetes.foo.bar/app to point to the service appsvc in your cluster.

Here is an example of how to implement a kubernetes cluster with an nginx ingress controller.

3
  • 2
    I think last line proxy_pass apps-appsvc-8080; should be rather on port 80 instead. Also, often a LB like ELB is put in front of an Ingress Controller, why?
    – toto'
    Oct 19, 2021 at 19:21
  • If I understand correctly, load balancers act on layer 4 and ingress works on layer 7. And a load balancer can have multiple ingress May 26, 2022 at 14:46
  • When you have read all the answers and think you understand the difference, check this out: AWS Load Balancer Controller add-on (docs.aws.amazon.com/eks/latest/userguide/…): "provisions Application Load Balancers when you create a Kubernetes Ingress"
    – milan
    Oct 12, 2022 at 22:57
159

I found this very interesting article which explains the differences between NodePort, LoadBalancer and Ingress.

From the content present in the article:

LoadBalancer:

A LoadBalancer service is the standard way to expose a service to the internet. On GKE, this will spin up a Network Load Balancer that will give you a single IP address that will forward all traffic to your service.

If you want to directly expose a service, this is the default method. All traffic on the port you specify will be forwarded to the service. There is no filtering, no routing, etc. This means you can send almost any kind of traffic to it, like HTTP, TCP, UDP, Websockets, gRPC, or whatever.

The big downside is that each service you expose with a LoadBalancer will get its own IP address, and you have to pay for a LoadBalancer per exposed service, which can get expensive!

Ingress:

Ingress is actually NOT a type of service. Instead, it sits in front of multiple services and act as a “smart router” or entrypoint into your cluster.

You can do a lot of different things with an Ingress, and there are many types of Ingress controllers that have different capabilities.

The default GKE ingress controller will spin up a HTTP(S) Load Balancer for you. This will let you do both path based and subdomain based routing to backend services. For example, you can send everything on foo.yourdomain.example to the foo service, and everything under the yourdomain.example/bar/ path to the bar service.

Ingress is probably the most powerful way to expose your services, but can also be the most complicated. There are many types of Ingress controllers, from the Google Cloud Load Balancer, Nginx, Contour, Istio, and more. There are also plugins for Ingress controllers, like the cert-manager, that can automatically provision SSL certificates for your services.

Ingress is the most useful if you want to expose multiple services under the same IP address, and these services all use the same L7 protocol (typically HTTP). You only pay for one load balancer if you are using the native GCP integration, and because Ingress is “smart” you can get a lot of features out of the box (like SSL, Auth, Routing, etc)

1
  • 2
    What confused me was that the default GKE ingress controller spins up an HTTP(S) Load Balancer. This shouldn't be confused with a K8's LoadBalancer which is a type of service...
    – orrymr
    Jan 21, 2023 at 13:44
53

There are 4 ways to allow pods in your cluster to receive external traffic:
1.) Pod using HostNetworking: true and (Allows 1 pod per node to listen directly to ports on the host node. Minikube, bare metal, and rasberry pi's sometimes go this route which can allow the host node to listen on port 80/443 allow not using a load balancer or advanced cloud load balancer configurations, it also bypasses Kubernetes Services which can be useful for avoiding SNAT/achieving similar effect of externalTrafficPolicy: Local in scenarios where it's not supported like on AWS.)
2.) NodePort Service
3.) LoadBalancer Service (Which builds on NodePort Service)
4.) Ingress Controller + Ingress Objects (Which builds upon the above)

Lets say you have 10 websites hosted in your cluster and you want to expose them all to external traffic.
*If you use type LoadBalancer Service you'll spawn 10 HA Cloud load balancers (each costs money)
*If you use type Ingress Controller you'll spawn 1 HA Cloud load balancer(saves money), and it'll point to an Ingress Controller running in your cluster.

An Ingress Controller is:

  • A service of type Load Balancer backed by a deployment of pods running in your cluster.
  • Each pod does 2 things:
    1. Acts as a Layer 7 Load Balancer running inside your cluster. (Comes in many flavors Nginx is popular)
    2. Dynamically configures itself based on Ingress Objects in your cluster
      (Ingress Objects can be thought of as declarative configuration snippits of a Layer 7 Load Balancer.)

The L7 LB/Ingress Controller inside your cluster Load Balances / reverse proxies traffic to Cluster IP Services inside your Cluster, it can also terminate HTTPS if you have a Kubernetes Secret of type TLS cert, and Ingress object that references it.)

enter image description here

35

In simple words, load balancer distributes the requests among multiple backend services (of same type) whereas ingress is more like an API gateway (reverse proxy) which routes the request to a specific backend service based on, for instance, the URL.

0
33

TL:DR

  1. Ingress sits between the public network (Internet) and the Kubernetes services that publicly expose our Api's implementation.
  2. Ingress is capable to provide Load Balancing, SSL termination, and name-based virtual hosting.
  3. Ingress capabilities allows to securely expose multiple API's or Applications from a single domain name.

Let's start with practical use case: you have multiple Apis backed by service implementation packages (ASIP for clariy and brevity) to deploy under one single domain name. As you are a cutting edge developer, you implemented a micro-services architecture that requires separate deployments for each ASIP so they can be upgraded or scaled individually. Of course,these ASIPs are encapsulated in individual docker container and available to Kubernetes (K8s) from container repository.

Let's say now that you want to deploy this on Google's GKE K8s. To implement sustained availability, each ASIP instance (replica) is deployed on different nodes (VM) where each VM has it's own cloud internal IP address. Each ASIP deployment is configured in an aptly name "deployment.yaml" file where you declaratively specify, among other things, the number of replicas of the given ASIP K8s should deploy.

The next step is to expose the API to the ouside world and funnel requests to one of the deployed ASIP instance. Since we have many replicas of the same ASIP running on different nodes, we need something that will distribute the request among those replicas. To resolve this, we can create and apply a "service.yaml" file that will configure a K8s service (KServ) that will be externally exposed and accessible through an IP address. This KServ will take charge of the API's request distribution among it's configured ASIPs. Note that a KServ will be automatically reconfigured by the K8s master when an ASIP's node fail and is restarted. Internal IP address are never reused in such case and the KServ must be advised of the new ASIP's deployment location.

But we have other Api service packages that shall be exposed on the same domain name. Spinning a new KServ will create a new external IP address and we won't be able to expose it on the same domain name. Well, this is where Ingress comes in.

Ingress sit's between the Internet and all the KServices we expose to to the outside world. Ingress is capable to provide load balancing, SSL termination and name-based virtual hosting. The latter capacity is able to route an incoming request to the right service by analysing it's URL. Of Course, Ingress must be configured and applied with an... "ingress.yaml" file that will specify the rewrites and the routes required to send a request to the right KServ.

Internet -> Ingress -> K8s Services -> Replicas

So, with the right ingress, KServices and ASIPs configuration, we can securely expose many API's using the same domain name.

0
27

Short Version:

In Kubernetes, Object definitions define desired state, while controllers watch Object definitions to achieve that state.

Ingress:

  • "Ingress" Object that does little on its own, but defines L7 Load Balancing rules
  • "Ingress Controller" that watches state of Ingress Objects to create L7 LB configuration based on rules defined in the Ingress Objects

LoadBalancer:

  • "Service" Object of type "LoadBalancer" that allows a service to be attached to a LoadBalancer
  • "Load Balancer Controller" that creates Load Balancers based on rules defined in the Service Object

Ingress

Ingress Object:

A kubernetes Object that does not do anything on its own because an Ingress Controller is not included by default. An Ingress Object just describes a way to route Layer 7 traffic into your cluster by specifying things like the request path, request domain, and target kubernetes service, while adding a service object may actually creates services because a service controller is included in Kubernetes by default.

Ingress Controller:

A Kubernetes Deployment/DaemonSet + Service which:

  1. listens on specific ports (usually 80 and 443) for web traffic
  2. Watches for the creation, modification, or deletion of Ingress Resources
  3. Creates internal L7 routing rules based on desired state indicated by Ingress Objects

For example, the Nginx Ingress Controller may:

  1. Use a service to listen on port 80 and 443 for incoming traffic
  2. Watch for creation of Ingress Objects and convert the desired state into new server{} sections which are dynamically placed into nginx.conf

LoadBalancer

Load Balancer Controller:

Load Balancer Controllers may be configured in platforms such as AWS and GKE and provide a way to assign external IPs through the creation of external load balancers. This functionality can be used by:

  1. Deploying the Load Balancer Controller if it wasn't deployed yet
  2. Setting a service to type "LoadBalancer"
  3. Setting appropriate annotations in the Service to configure the Load Balancer

Service Type:

When the service type is set to LoadBalancer and there is a Cloud provided Load Balancer Controller present, the service is exposed externally using a cloud provider's load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created thereby assigning a service external IPs and or DNS.

Relationships

Ingress Controller Services are often provisioned as LoadBalancer type, so that http and https requests can be proxied / routed to specific internal services through an external ip.

However, a LoadBalancer is not strictly needed for this. Since, through the use of hostNetwork or hostPort you can technically bind a port on the host to a service (allowing you to visit it via the hosts external ip:port). Though officially this is not recommended as it uses up ports on the actual node.

References

https://kubernetes.io/docs/concepts/configuration/overview/#services

https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/

https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/#external-load-balancer-providers

https://kubernetes.io/docs/concepts/services-networking/ingress/ https://kubernetes.io/docs/concepts/architecture/cloud-controller/ https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.3/

18
Feature Ingress Load Balancer
Protocal HTTP level (Network layer 7) Network layer 4
Additional Features cookie-based session affinity, Ingress rules, Resource backends, Path types Only balance the load
Dependency Ingress controller need to be running. Different Kubernetes environments use different implementations of the controller, but several don’t provide a default controller at all. No dependency, Built-in support with K8
YAML manifest There is separate API for it. apiVersion: networking.k8s.io/v1 type: LoadBalancer
How it work Client connected to one of the pods through Ingress controller. The client first performed a DNS lookup of example.com, and the DNS server (or the local operating system) returned the IP of the Ingress controller. The client then sent an HTTP request to the Ingress controller and specified example.com in the Host header. From that header, the controller determined which service the client is trying to access, looked up the pod IPs through the Endpoints object associated with the service, and forwarded the client’s request to one of the pods. The load balancer redirects traffic to the node port across all the nodes. Clients connect to the service through the load balancer’s IP.

I hight recommend reading NodePort vs LoadBalancer vs Ingress? for Knowledge ++

0
6

Loadbalancer service: is a layer 4 proxy (TCP, UDP,..). It runs outside Kubernetes cluster-network. It doesn't have features of layer 7: CircuitBreaker, measure number of request, request's latency, failure, routing,...

ingress: is a layer 7 proxy (http, https, gRPC,..). It is an application in a pod inside Kubernetes cluster-network. If the ingress pod dies, Kubernetes will restart it or move it to other node in the cluster.

2

Kubernetes LoadBalancer and Ingress are both used to manage traffic in a Kubernetes cluster, but they serve different purposes.

A LoadBalancer service is used to expose a Kubernetes service externally by creating a cloud provider-specific load balancer. It is typically used for exposing a service to the internet or to a private network outside the Kubernetes cluster. LoadBalancer provides a single IP address that can be used to access the service.

On the other hand, Ingress is a Kubernetes object that manages external access to the services in a Kubernetes cluster, usually HTTP. It acts as a reverse proxy and allows you to define routing rules to map incoming requests to the appropriate Kubernetes services. Ingress can be used to route traffic based on the URL path, host name, or other criteria.

In summary, LoadBalancer is used to provide external access to a Kubernetes service, while Ingress is used to manage and route traffic to multiple services within a Kubernetes cluster.

0

Pods have their own IP:PORT but it is dynamic in nature and changes if deleted or redeployed.

Services are assigned ClusterIPor NodePort(a port in a VM where service resource is created) which can be mapped to set of pods or other backend [see: headless services]

  • To access correct Pod use ClusterIP(from within clutser)

  • NodePort can be used to access pods from outside of the cluster

LoadBalancer[External/Internal]: provided by cloud providers points to either ClusterIP or NodePort. You can access the service by using LB's IP

LB ~> SERVICE(ClusterIP or NodePort) ~> POD

Ingress resource is an entrypoint to cluster.A LB can listen to ingress rules and can route to specific service.[see this example]

LB(Ingress-managed) ~> SERVICE(ClusterIP or NodePort) ~> POD

0

In order to understand this, I think it is good to ask, "why do we need ingress, and why a load balancer is not sufficient"

The best answer I found in the book Kubernetes in Action, Marko Lukša December 2017.

Understanding why Ingresses are needed

One important reason is that each LoadBalancer service requires its own load balancer with its own public IP address, whereas an Ingress only requires one, even when providing access to dozens of services. When a client sends an HTTP request to the Ingress, the host and path in the request determine which service the request is forwarded to

-1

Since ingress controller is running inside the cluster with clusterrole permission to query and list services, it has the ability to discover services which are not exposed outside. This help to route requests from external sources to the correct service based on the ingress rules.

-1

Ingress routes traffic internally to multiple services based on L7 rules like (header, uri etc)

Load Balancers routes external traffic to a single service.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.