1

I am running a Ceph Cluster (pacific) consisting of 3 nodes. I have 3 Monitor Daemons started (one on each node).

Just for some test I wanted to remove node3 out of the list of monitors with the command

sudo ceph mon remove node3

The mon disappeared in the Dashboard and also when I call ceph status command:

$ sudo ceph status
  cluster:
    id:     xxxxx-yyyyy-zzzzzzz
    health: HEALTH_WARN
            1 failed cephadm daemon(s)
 
  services:
    mon: 2 daemons, quorum node1,node2 (age 53m)
    mgr: node1.gafnwm(active, since 102m), standbys: node2.jppwxo
    osd: 3 osds: 3 up (since 15m), 3 in (since 96m)

But now I found no way to restart or add the monitor again into my cluster. I tried:

$ sudo ceph orch daemon add mon node3:10.0.0.4
Error EINVAL: name mon.node3 already in use

What does this mean? Why is the name mon.node3 already in use even if I can't see it?

5
  • 3
    If you run cephadm ls on that node you will see the previous daemon. Remove it with cephadm rm-daemon --name mon.<YOUR_MON>. If that worked you'll most likely be able to redeploy the mon again.
    – eblock
    Oct 8, 2021 at 6:39
  • 1
    The mon was listed in the 'cephadm ls' resultlist. I removed it with sudo cephadm rm-daemon --name mon.node3 --fsid xxxx-yyyy-zzz --force and the daemon disapeard in the cephadm ls. In the web dashboard it is still listed (strange) and still I am not able to add it. I got the same error message Error EINVAL: name mon.node3 already in use
    – Ralph
    Oct 8, 2021 at 7:01
  • 1
    I tried a reboot of the node and than the mon disappeared also in the web dashboard and now I was able to add the mon again ! Thanks a lot for your help.
    – Ralph
    Oct 8, 2021 at 7:06
  • 1
    Sometimes cephadm needs some time to update its information, most operations are delayed, so it could also have worked just with some waiting instead of reboot, but great that it worked that way, too.
    – eblock
    Oct 8, 2021 at 7:25
  • 1
    I suppose you could speed up the update if you restart mgr daemon. Or I've just been lucky.
    – ogurets
    May 24 at 23:04

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.