76

I'm developing something that needs Prometheus to persist its data between restarts. Having followed the instructions

$ docker volume create a-new-volume
$ docker run \
    --publish 9090:9090 \
    --volume a-new-volume:/prometheus-data \
    --volume "$(pwd)"/prometheus.yml:/etc/prometheus/prometheus.yml \
    prom/prometheus

I have a valid prometheus.yml in the right directory on the host machine and it's being read by Prometheus from within the container. I'm just scraping a couple of HTTP endpoints for testing purposes at the moment.

But when I restart the container it's empty, no data from the previous run. What am I missing from my docker run ... command to persist the data into the a-new-volume volume?

3

2 Answers 2

115

Use the default data dir, which is /prometheus. To do that, use this line instead of what you have in your command:

...
--volume a-new-volume:/prometheus \
...

Found here: https://github.com/prometheus/prometheus/blob/master/Dockerfile

Surprisingly is not mentioned in the image docs

5
  • 1
    when I do this, container failed to start. I realized it was because permissions of directory on host machine.
    – ibrahim
    Jul 2, 2020 at 11:24
  • 29
    To save someone the trouble to figure out the permission: if you are using bind mount to mount a host directory: chown the folder to nobody.
    – yegle
    Aug 2, 2020 at 6:08
  • 23
    Alternatively chown 65534:65534 <volume-dir>
    – Sathesh
    Jan 11, 2021 at 7:26
  • 7
    There is a pull request for adding this info to the official installation docs: github.com/prometheus/prometheus/pull/8747
    – wobmene
    Apr 21, 2021 at 19:43
  • That PR finally just got merged! (after more than 2 years)
    – Matt
    Jul 26 at 10:56
19

I had the same issue a today, but I was using a docker composer file. So wrapping up all what was in comments of other answers and what worked for me. In case setting up the Prometheus docker via yaml compose file...

First create a folder for the volume on the host machine, e.g.:

$ mkdir /tmp/prometheus

Then change the folder owner to nobody, like (use sudo if needed):

$ chown 65534:65534 /tmp/prometheus

Then add volume to the yaml configuration file:

prometheus:
  image: prom/prometheus
  container_name: prometheus
  ports:
    - 9090:9090
  volumes:
    - /tmp/prometheus:/prometheus
    - ./prometheus.yml:/etc/prometheus/prometheus.yml

That should do it.

Not the answer you're looking for? Browse other questions tagged or ask your own question.