14

Running Kubernetes on GKE

Installed Nginx controller with latest stable release by using helm.

Everythings works well, except adding the whitelist-source-range annotation results in that I'm completely locked out from my service.

Ingress config

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: staging-ingress
  namespace: staging
  annotations:
    kubernetes.io/ingress.class: nginx
    ingress.kubernetes.io/whitelist-source-range: "x.x.x.x, y.y.y.y"
spec:
  rules:
    - host: staging.com
      http:
        paths:
        - path: /
          backend:
            serviceName:staging-service
            servicePort: 80

I connected to the controller pod and checked the nginx config and found this:

# Deny for staging.com/
geo $the_real_ip $deny_5b3266e9d666401cb7ac676a73d8d5ae {
    default 1;

    x.x.x.x 0;
    y.y.y.y 0;
}

It looks like he is locking me out instead of whitelist this IP's. But it also locking out all other addresses... I get 403 by going from staging.com host.

1
  • Just to clarify, this config will lock every request coming from an IP that's not x.x.x.x or y.y.y.y. And you are sending requests coming from those IP's that get 403 responses? Commented Dec 28, 2017 at 0:44

3 Answers 3

18

Yes. However, I figured out by myself. Your service has to be enabled externalTrafficPolicy: Local. That means that the actual client IP should be used instead of the internal cluster IP.

To accomplish this run kubectl patch svc nginx-ingress-controller -p '{"spec":{"externalTrafficPolicy":"Local"}}'

2
  • 2
    If this doesn't solve the entire problem, also restart your nginx controller pods. After that it should work. Commented Sep 20, 2018 at 12:40
  • Also note, if your nginx pod is running on different nodes than the destination services, this may break. You can solve this by setting it to run as a DaemonSet rather than Deployment, through a similar kubectl patch command.
    – parkerfath
    Commented Jun 27, 2023 at 17:23
10

Your nginx controller service has to be set as externalTrafficPolicy: Local. That means that the actual client IP will be used instead of cluster's internal IP.

You need to get the real service name from kubectl get svc command. The service is something like:

NAME                                          TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)                      AGE
nobby-leopard-nginx-ingress-controller        LoadBalancer   10.0.139.37    40.83.166.29   80:31223/TCP,443:30766/TCP   2d

nobby-leopard-nginx-ingress-controller is the service name you want to use.

To finish this, run kubectl patch svc nobby-leopardnginx-ingress-controller -p '{"spec":{"externalTrafficPolicy":"Local"}}'

When you setting up a new nginx controller, you can use the command below:

helm install stable/nginx-ingress \
  --namespace kube-system \
  --set controller.service.externalTrafficPolicy=Local

to have a nginx ingress controller accept whitelist after installing.

Therefore if you configure your nginx controller with a values.yaml file please add this to it:

controller:
  service:
    externalTrafficPolicy: "Local"
1
  • 1
    Which header would nginx whitelist against? Commented Jul 2, 2021 at 20:54
2

You don't need externalTrafficPolicy: Local, just set real_ip_header (via server-snippet) and use X-Forwarded-For header to retrieve and set client ip, or some other field, for example cloudflare is providing CF-Connecting-IP. http://nginx.org/en/docs/http/ngx_http_realip_module.html

1
  • Sorry, but no. I checked this and you still need to set externalTrafficPolicy: Local. Commented Oct 12, 2023 at 14:00

Not the answer you're looking for? Browse other questions tagged or ask your own question.