6

I am looking at how to make OpenVPN client work on a pod's container, I explain what I do, but you can skip all my explanation and offer your solution directly, I don't care replacing all the below with your steps if it works, I want to make my container to use a VPN (ExpressVPN for example) in a way that both external and internal networking works.

I have a docker image that is an OpenVPN Client, it works fine with the command:

docker run --rm -it --cap-add=NET_ADMIN --device=/dev/net/tun my-app /bin/bash

The docker image had an entry point bash script:

curl https://vpnvendor/configurations.zip -o /app/configurations.zip
mkdir -p /app/open_vpn/ip_vanish/config
unzip /app/configurations.zip -d /app/open_vpn/config
printf "username\npassword\n" > /app/open_vpn/vpn-auth.conf
cd /app/open_vpn/config
openvpn --config ./config.ovpn --auth-user-pass /app/open_vpn/vpn-auth.conf

It works fine, but when I deploy it as a container in a K8S Pod, it breaks, it is understandable, K8S clusters need internal network communication between the nodes, so the VPN breaks it ... how do I make it work? the Google search was frustrating, none of the solutions worked and there were just a few, there is one with similar issue: OpenVPN-Client Pod on K8s - Local network unreachable But did not understand it very well, please help.

Since IPVanish is well known, let's take their ovpn example, I use other vendor but had access to an IPVanish account and it does not work either:

client
dev tun
proto udp
remote lon-a52.ipvanish.com 443
resolv-retry infinite
nobind
persist-key
persist-tun
persist-remote-ip
ca ca.ipvanish.com.crt
verify-x509-name lon-a52.ipvanish.com name
auth-user-pass
comp-lzo
verb 3
auth SHA256
cipher AES-256-CBC
keysize 256
tls-cipher TLS-DHE-RSA-WITH-AES-256-CBC-SHA:TLS-DHE-DSS-WITH-AES-256-CBC-SHA:TLS-RSA-WITH-AES-256-CBC-SHA

I accept responses in Golang or YAML it does not matter, although I use go-client, my code for pod creation is:

podObj := &v1.Pod{
        ObjectMeta: metav1.ObjectMeta{
            Name:      "mypod",
            Namespace: "default",
        },
        Spec: v1.PodSpec{
            Containers: []v1.Container{
                {
                    Name:            "worker1",
                    Image:           "192.168.1.138:5000/myimage",
                    ImagePullPolicy: v1.PullAlways,
                    Stdin: true,
                    TTY:   true,
                    /* Trying to simulate --device=/dev/net/tun I copied the below, but it does not work
// https://garunski.medium.com/openvpn-and-minikube-25511099f8de
                    VolumeMounts: []v1.VolumeMount{
                        {
                            ReadOnly:  true,
                            Name:      "dev-tun",
                            MountPath: "/dev/net/tun",
                        },
                    },*/
                    SecurityContext: &v1.SecurityContext{
                        // Taken from https://caveofcode.com/how-to-setup-a-vpn-connection-from-inside-a-pod-in-kubernetes/
                        Privileged: boolPtr(true),
                        Capabilities: &v1.Capabilities{
                            Add: []v1.Capability{
                                "NET_ADMIN",
                            },
                        },
                    },
                },
            },
            NodeName: "worker-node01",
        },
    }
clientset.CoreV1().Pods("default").Create(context.Background(), podObj, metav1.CreateOptions{})

I can add the NET_ADMIN capability, but I need also to give access to the /dev/net/tun device and that's the problem, but even If I find a way, it will break internal networking.

Update one

I made external networking work, by adding the following two lines in my docker's entry point:

# Taken from https://caveofcode.com/how-to-setup-a-vpn-connection-from-inside-a-pod-in-kubernetes/
mknod /dev/net/tun c 10 200
chmod 600 /dev/net/tun
9
  • Can you provide full details of the errors in your question?
    – gohm'c
    Nov 24, 2021 at 0:37
  • @gohm'c My issue is simple, I just did not have internet connection, neither internal nor external, Now I made an improvement, I made external networking work, but not internal. See the Updates please
    – Melardev
    Nov 24, 2021 at 0:55
  • 1
    Hi, @Melardev . Now that you've made external connection work, I suppose you are at the same point as the person from the linked post. Try to configure two routes: one for Kubernetes service subnet and one for pod subnet. Usually, these are 10.43.0.0/16 and 10.42.0.0/16 respectively, but it is better to check that. For example, using this line route 10.42.0.0 255.255.0.0 net_gateway you are telling that to reach any IP in 10.42.0.0/16, the packet should go via the default gateway and NOT VPN tunnel. In other words, all traffic goes through the VPN except for IPs in 10.42.0.0/16.
    – anemyte
    Nov 24, 2021 at 7:16
  • @anemyte Hi, thanks but still not resolved, with kubectl get pod -o wide (is there a better command?) I see the internal IP for the pod is 192.168.87.228, so in my .ovpn file I added route 192.168.0.0 255.255.0.0 net_gatewayand also your two lines above (10.43.0.0 and 10.42.0.0 just in case), as a test I had a http server on the host on port 3002, trying curl 192.168.1.138 does not work with VPN, without VPN works this is the output for route pre and post: gist.github.com/melardev/9a1c9653ac41835625469ed1e5b6b77b
    – Melardev
    Nov 24, 2021 at 8:55
  • @Melardev please add the curl command that you used along with traceroute example and I'll look into this later.
    – anemyte
    Nov 24, 2021 at 9:26

2 Answers 2

7

Here is a minimal example of a pod with OpenVPN client. I used kylemanna/openvpn as a server and to generate a basic client config. I only added two routes to the generated config to make it working. See below:

apiVersion: v1
kind: Pod
metadata:
  name: ovpn
  namespace: default
spec:
  containers:
    - name: ovpn
      image: debian:buster
      args:
        - bash
        - -c
        # install OpenVPN and curl; use curl in an endless loop to print external IP
        - apt update && apt install -y openvpn curl && cd /config && openvpn client & while sleep 5; do echo $(date; curl --silent ifconfig.me/ip); done
      volumeMounts:
        - mountPath: /dev/net/tun
          readOnly: true
          name: tun-device
        - mountPath: /config
          name: config
      securityContext:
        capabilities:
          add: ["NET_ADMIN"]
  volumes:
    - name: tun-device
      hostPath:
        path: /dev/net/tun
    - name: config
      secret:
        secretName: ovpn-config
---
apiVersion: v1
kind: Secret
metadata:
  name: ovpn-config
  namespace: default
stringData:
  client: |

    # A sample config generated by https://github.com/kylemanna/docker-openvpn server
    client
    nobind
    dev tun

    # Remote server params
    remote PASTE.SERVER.IP.HERE 1194 udp

    # Push all traffic through the VPN
    redirect-gateway def1
    # except these two k8s subnets
    route 10.43.0.0 255.255.0.0 net_gateway
    route 10.42.0.0 255.255.0.0 net_gateway

    # Below goes irrelevant TLS config
    remote-cert-tls server
    <key>
    -----BEGIN PRIVATE KEY-----
    -----END PRIVATE KEY-----
    </key>
    <cert>
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
    </cert>
    <ca>
    -----BEGIN CERTIFICATE-----
    -----END CERTIFICATE-----
    </ca>
    key-direction 1
    <tls-auth>
    #
    # 2048 bit OpenVPN static key
    #
    -----BEGIN OpenVPN Static key V1-----
    -----END OpenVPN Static key V1-----
    </tls-auth>
5
  • thanks for the help, this is the solution I was looking for.
    – Melardev
    Nov 24, 2021 at 20:00
  • I need to redirect my application to a local ingress, but this doesn't work I'm getting error 504. Any idea what I should add? Feb 18, 2022 at 19:16
  • @RedaDrissi All I can say with this amount of data that it is probably not working :) I suggest you ask a question and bring the details on what you have, what you want to achieve, and what you tried so far. Then we'll see how it can be solved.
    – anemyte
    Feb 18, 2022 at 19:24
  • @anemyte thanks but I solved it through adding my pod IP range to LOCAL_NETWORK not ideal since it adds a security issue, but good enough for my use case. Feb 19, 2022 at 9:36
  • Hi @Melardev and anemyte I came across this question and it's similar to my problem, link below. Can you please take a look at it, as I've tried almost anything I can think of and can see that you guys have a better understanding of VPN networking than I do. I'd be extremely grateful :) stackoverflow.com/questions/72350518/…
    – Erokos
    May 23, 2022 at 14:53
-4

Try Tailscale. https://tailscale.com/ It's much simpler. And they have a cool free-tier

2
  • Thanks for the suggestion, I did not know about, but it is not exactly what I was looking for
    – Melardev
    Nov 25, 2021 at 9:38
  • I have it running in a k8s cluster. It makes a VPN gateway to the cluster so I see the pods on my local machine as if they were on my local network. It's a 10 mn install and it just ...works. There's an openVPN comparison here : tailscale.com/kb/1170/tailscale-vs-openvpn
    – Piero
    Nov 25, 2021 at 12:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.