Depending on how I need to use the volume, I have the following 3 options.
First, you can create the named volume directly and use it as an external volume in compose, or as a named volume in a docker run
or docker service create
command.
# create a reusable volume
$ docker volume create --driver local \
--opt type=nfs \
--opt o=nfsvers=4,addr=nfs.example.com,rw \
--opt device=:/path/to/dir \
foo
Next, there is the --mount
syntax that works from docker run
and docker service create
. This is a rather long option, and when you are embedded a comma delimited option within another comma delimited option, you need to pass some quotes (escaped so the shell doesn't remove them) to the command being run. I tend to use this for a one-off container that needs to access NFS (e.g. a utility container to setup NFS directories):
# or from the docker run command
$ docker run -it --rm \
--mount type=volume,dst=/container/path,volume-driver=local,volume-opt=type=nfs,\"volume-opt=o=nfsvers=4,addr=nfs.example.com\",volume-opt=device=:/host/path \
foo
# or to create a service
$ docker service create \
--mount type=volume,dst=/container/path,volume-driver=local,volume-opt=type=nfs,\"volume-opt=o=nfsvers=4,addr=nfs.example.com\",volume-opt=device=:/host/path \
foo
Lastly, you can define the named volume inside your compose file. One important note when doing this, the name volume only gets created once, and not updated with any changes. So if you ever need to modify the named volume you'll want to give it a new name.
# inside a docker-compose file
...
services:
example-app:
volumes:
- "nfs-data:/data"
...
volumes:
nfs-data:
driver: local
driver_opts:
type: nfs
o: nfsvers=4,addr=nfs.example.com,rw
device: ":/path/to/dir"
...
In each of these examples:
- Type is set to
nfs
, not nfs4
. This is because docker provides some nice functionality on the addr
field, but only for the nfs
type.
- The
o
are the options that gets passed to the mount syscall. One difference between the mount syscall and the mount command in Linux is the device has the portion before the :
moved into an addr
option.
nfsvers
is used to set the NFS version. This avoids delays as the OS tries other NFS versions first.
addr
may be a DNS name when you use type=nfs
, rather than only an IP address. Very useful if you have multiple VPC's with different NFS servers using the same DNS name, or if you want to adjust the NFS server in the future without updating every volume mount.
- Other options like
rw
(read-write) can be passed to the o
option.
- The
device
field is the path on the remote NFS server. The leading colon is required. This is an artifact of how the mount command moves the IP address to the addr
field for the syscall. This directory must exist on the remote host prior to the volume being mounted into a container.
- In the
--mount
syntax, the dst
field is the path inside the container. For named volumes, you set this path on the right side of the volume mount (in the short syntax) on your docker run -v
command.
If you get permission issues accessing a remote NFS volume, a common cause I've encountered is containers running as root, with the NFS server set to root squash (changing all root access to the nobody user). You either need to configure your containers to run as a well known non-root UID that has access to the directories on the NFS server, or disable root squash on the NFS server.