31

I am trying to cluster rabbit using chef

Here is my error. I shutdown all of rabbit on the second node.

rabbitmqctl join_cluster --ram rabbit@ip-10-158-xxx-xxx

Error: mnesia_unexpectedly_running

So..what is the deal? I tried this from http://agiletesting.blogspot.com/2010/05/rabbitmq-clustering-in-ubuntu.html which is to remove /var/lib/rabbitmq/mnesia. No go. And on what server doen it not be be running? All?

Thanks

6 Answers 6

48

Make sure you call sudo rabbitmqctl stop_app before issuing the cluster command. That seemed to be the problem for me.

Then make sure you call sudo rabbitmqctl start_app to begin it again :)

1
  • 1
    Worth noting that I had to do both steps separately. Using sudo rabbitmqctl force_reset did not work. Mar 30, 2015 at 22:15
5

To call out a very important piece of @Itai Ganot's comment that's not immediately intuitive: you need to run rabbitmqctl stop_app on all nodes but one. If you run rabbitmqctl stop_app on all nodes, you will get a different message: Error: mnesia_not_running.

In short:

  • Error: mnesia_unexpectedly_running means "you need to run rabbitmqctl stop_app on this node
  • Error: mnesia_not_running means "you need to run rabbitmqctl start_app on the node you're trying to cluster with"
3

You need to copy the cookie from the node you trying to connect

Let us use an example with 2 nodes: rabbit@node1 and rabbit@node2

  1. Go to rabbit@node1 and copy the cookie from cat /var/lib/rabbitmq/.erlang.cookie
  2. Go to rabbit@node2 remove the current cookie and paste the new one.
  3. Execute the following commands on the same node

    /usr/sbin/rabbitmqctl stop_app
    /usr/sbin/rabbitmqctl reset
    /usr/sbin/rabbitmqctl cluster rabbit@node1
    

That should do it.

The same procedure is documented here

2
  • For Windows (In elevated command prompt): 1. rabbitmqctl stop_app 2. rabbitmqctl reset 3. rabbitmqctl start_app
    – Ganesh R.
    May 29, 2014 at 10:46
  • Link in answer is dead - "This site can’t be reached | DNS_PROBE_FINISHED_NXDOMAIN".
    – Pang
    Jul 23, 2020 at 7:31
2

I also received the same error while trying to create a cluster of two rabbitmq servers.

As far as I know, the real process to create a cluster is a little bit different than the ones described in the other answers but is the one which eventually worked for me:

After you have two or more nodes running rabbitmq-server, choose one server and that's the one you don't touch - let's call it rabbitmaster, in all the other nodes, follow these steps:

1. Make sure the rabbitmq-server is not running.
2. # su - rabbitmq
3. $ rabbitmq-server -detached
4. $ rabbitmqctl cluster_status
5. $ rabbitmqctl stop_app
6. $ rabbitmqctl join_cluster rabbit@rabbitmaster
7. $ rabbitmqctl start_app

Now you can run rabbitmqctl cluster_status and see both the node you just configured and the rabbitmq master.

More information can be found in RabbitMQ Official site.

0

I dont like to answer my own qustion but clustering rabbin usig chef was a true pain. Issue with rabbit was resolved by not using the -N option with bootstrapping. Rabbit does not like the hostname to be changed.

So..following the docs on the rabbit site for clustering worked as expected. If you use the -N option with chef...major issues will arise.

1
  • Or edit the rabbitmq slaves' /etc/hosts file with entries for the nodes and assign an elastic IP to all members. Changing the hostname, or using the -N option, isn't really the issue here, you just need to work around it or override the cookbook defaults.
    – Sergio
    Aug 1, 2014 at 20:30
0

This means the app on the current node you are trying to link is running. Stop the app on the current node before joining it to the master node.

sudo rabbitmqctl stop_app rabbitmqctl join_cluster --ram rabbit@ip-10-158-xxx-xxx. then rabbitmqctl start_app.

Also make sure the app on the master node is running. Otherwise you'll get Error: mnesia_not_running error.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.