86

I've installed MongoDB v4.0 for the most amazing feature of it Transaction in Nodejs with mongodb 3.1 as a driver.

When I try to use a transaction session I've faced this error:

MongoError: Transaction numbers are only allowed on a replica set member or mongos.

What's that and how can I get rid of it?

0

13 Answers 13

72

Transactions are undoubtedly the most exciting new feature in MongoDB 4.0. But unfortunately, most tools for installing and running MongoDB start a standalone server as opposed to a replica set. If you try to start a session on a standalone server, you'll get this error.

In order to use transactions, you need a MongoDB replica set, and starting a replica set locally for development is an involved process. The new run-rs npm module makes starting replica sets easy. Running run-rs is all you need to start a replica set, run-rs will even install the correct version of MongoDB for you.

Run-rs has no outside dependencies except Node.js and npm. You do not need to have Docker, homebrew, APT, Python, or even MongoDB installed.

Install run-rs globally with npm's -g flag. You can also list run-rs in your package.json file's devDependencies.

npm install run-rs -g

Next, run run-rs with the --version flag. Run-rs will download MongoDB v4.0.0 for you. Don't worry, it won't overwrite your existing MongoDB install.

run-rs -v 4.0.0 --shell

Then use replicaSet=rs in your connection string.

You find more details about it here.

13
  • 1
    Also useful is the bitnami/mongodb docker image, which allows you to setup a replicaSet with docker-compose: github.com/bitnami/…
    – mltsy
    Commented Feb 4, 2019 at 18:35
  • 9
    From the doc, it seems running an existing (or a new) mongod instance with replica-set is rather trivial : docs.mongodb.com/manual/tutorial/… Commented Feb 18, 2019 at 12:53
  • 16
    Any idea why the ability to use transactions required MongoDB replica set ? (In my case I have MongoDb instance that run on docker for tests and cannot run transactions due this following this limitation)
    – Yohan
    Commented Mar 26, 2020 at 22:05
  • 8
    Why is replica set required? It makes no sense to me and seems this was a blunder on the mongodb team. Commented Dec 1, 2021 at 18:18
  • 2
    @MarcelFalliere It looks so trivial but it's designed in such way that implementing it in Docker is going to be painful. And it seems from another answer they already broke one Docker solution. Maybe they hate Docker, hate transactions or lure us to use replicas? It makes no sense and goes against whatever principles we created so far...
    – Gherman
    Commented Jul 19, 2022 at 23:11
36

I got the solution, and it's just three lines configuration inside the MongoDB config file.

After switching from MongoDB atlas and installing MongoDB v 4.4.0 on my CentOS 7 VPS with WHM, I faced that issue also.

the run-rs solution does not work for me, but I managed to solve this issue without any third-party tool, following these steps:

1. turn off mongod.

the most efficient way is by entering the MongoDB shell with the command mongo checkout the method

db.shutdownServer()

You will be no ability to use the MongoDB server. For me, the shutdown process took too long, and then I killed the process with the command:

systemctl stop -f mongod

if you killed the mongod process,s probably you will need to run mongod --dbpath /var/db --repair

The var/db should point to your database directory.

2. setting replicaSet configuration.

for the replicaSet settings step, check out the /etc/mongod.conf file, look for the replication value line, and you should add the following lines as below:

replication:
   oplogSizeMB: <int>
   replSetName: <string>
   enableMajorityReadConcern: <boolean>

use the replSetName value on the next step.

an example of those settings:

   oplogSizeMB: 2000
   replSetName: rs0
   enableMajorityReadConcern: false

3. add your connection string URL.

add the value of replSetName to your connection URL &replicaSet=--YourReplicationSetName--

if you used the name rs0 from our example, then you should add to your DB connection URL query replicaSet=rs0

4. turn on mongod again

enter the command: systemctl start mongod

5. Access your replicaSet database

enter MongoDB shell with the command mongo, enter the command rs.initiate() now you should be in your replicaSet database.

7
  • 2
    Best solution for local development in my opinion. It's really easy to do and works without any third-party software.
    – Nick
    Commented Aug 4, 2021 at 7:43
  • What value we need to give on replSetName? can you say some example
    – Kannan T
    Commented Jan 3, 2022 at 13:29
  • 1
    In case your Mongo Service cannot start after restart, follow askubuntu.com/questions/884541/cant-start-mongodb-service Commented Feb 16, 2022 at 11:22
  • 1
    If using MongoDB version 5 and above, exclude the enableMajorityReadConcern: true line for the /etc/mongod.conf file. Check your Mongo installation version with mongo --version
    – Looi
    Commented May 21, 2022 at 10:44
  • 1
    Link to the relevant docs: mongodb.com/docs/manual/reference/configuration-options/…
    – Jamie Syme
    Commented Feb 1, 2023 at 17:00
27

For those who wants to develop against of the dockerized MongoDB instance, here is the single-file docker-compose.yaml solution based on the official MongoDB docker image:

version: '3.8'

services:
  mongodb:
    image: mongo:5
    command: --replSet rs0
    ports:
      - "28017:27017"
    environment:
      MONGO_INITDB_DATABASE: attachment-api-local-dev
    healthcheck:
      # use 'mongo' instead of 'mongosh' if MongoDB version is lower than 5
      test: mongosh --eval 'db.runCommand("ping").ok' localhost:27017/test --quiet
      interval: 2s
      timeout: 3s
      retries: 5

  mongo-init:
    image: mongo:5
    restart: "no"
    depends_on:
      mongodb:
        condition: service_healthy
    command: >
      mongo --host mongodb:27017 --eval
      '
      rs.initiate( {
         _id : "rs0",
         members: [
            { _id: 0, host: "localhost:27017" }
         ]
      })
      '

A cleaner solution would be to replace the mongo-init container with an init-script shared as a volume:

volumes:
  - ./mongo/init/:/docker-entrypoint-initdb.d/:ro

where /mongo/init/0. mongo-init.js is:

rs.initiate()
// wait a bit for RS configuration to complete
sleep(3000)

A much easier solution is to just use Bitnami MongoDB image with MONGODB_REPLICA_SET_MODE env variable:

services:
  mongodb:
    image: bitnami/mongodb:5.0
    ports:
      - "27017:27017"
    environment:
      MONGODB_REPLICA_SET_MODE: primary
      ALLOW_EMPTY_PASSWORD: 'yes'

But the Bitnami image only supports linux/amd64 architecture and in practice it will hang forever even if you try to run it with Rosetta Docker host. So, if you have Mac on Apple Silicon, your the only option is to use the official mongo image unless bitnami/40937 is resolved

4
  • 2
    Apparently, the MONGODB_REPLICA_SET_KEY environment variable is also required. Without it, the Mongo process refused to start in the container with an error indicating the variable is mandatory. I just set it to something like MONGODB_REPLICA_SET_KEY: 123456
    – Boaz
    Commented Dec 13, 2022 at 17:52
  • @Boaz never had such a problem with Mongo 5. You probably use 6+
    – Kirill
    Commented Dec 13, 2022 at 18:00
  • 1
    Actually, I use 4.2 :)
    – Boaz
    Commented Dec 14, 2022 at 15:02
  • 1
    Thanks, the bitnami service seems to work fine for overleaf docker-compose. Commented Mar 22 at 16:24
13

Possible solution for local development using docker

Create Dockerfile

FROM mongo:4.4.7
RUN echo "rs.initiate();" > /docker-entrypoint-initdb.d/replica-init.js
CMD [ "--replSet", "rs" ]

Build this Dockerfile

docker build ./ -t mongodb:4.7-replset

Run this created image

docker run --name mongodb-replset -p 27017:27017 -d mongodb:4.7-replset

Connect to database using this URI

mongodb://localhost:27017/myDB
4
  • 1
    The suggested solution does not work for MongoDB 5.0.5 because of "MongoServerError: command replSetInitiate requires authentication" (docker-entrypoint.sh does not invoke mongo with user and password, also it does not print the error, you can only see it if mongo is replaced with mongosh). Aside from that you need CMD ["--replSet", "rs", "--keyFile", "/etc/mongodb.key"] Commented Jan 3, 2022 at 17:56
  • Even with this work around RUN echo 'rs.initiate();' > /docker-entrypoint-initdb.d/replica-init.script RUN echo 'mongosh -u "$MONGO_INITDB_ROOT_USERNAME" -p "$MONGO_INITDB_ROOT_PASSWORD" < /docker-entrypoint-initdb.d/replica-init.script' > /docker-entrypoint-initdb.d/replica-init.sh we get MongoServerError: This node was not started with the replSet option. It is simply because docker-entrypoint.sh is not designed to run replicas. Actually, it looks like it is designed not to run them. See # remove "--auth" and "--replSet" for our initial startup in the source code Commented Jan 3, 2022 at 18:16
  • Here is related breaking change: github.com/docker-library/mongo/issues/… Commented Jan 4, 2022 at 1:45
  • If anyone interested see scripts/docker-entrypoint.sh.patch and Dockerfile in github.com/thenewboston-developers/Node/pull/69 for workaround/solution Commented Jan 5, 2022 at 18:10
8

I faced the same issue recently. In my case it's because I'm connecting to a remote Mongo server with a different version than my local development environment.

To quickly solve the issue, I added the following param to my connection string:

?retryWrites=false

1
  • 5
    This looks rather suspicious. Are you sure it doesn't just make all errors silent instead of fixing them? Why is it supposed to work?
    – Gherman
    Commented Jul 19, 2022 at 23:14
8

When running MongoDB on a Linux Machine, you can simply use replication by updating connection string via editing service file

/usr/lib/mongod.service   or   /lib/systemd/system/mongod.service

and update it with following

ExecStart=/usr/bin/mongod --config "/etc/mongod.conf" --replSet rs0

where --config "/etc/mongod.conf" is pointing to your MongoDB Configuration file and --replSet rs0 is telling it to use replication with the name of rs0

and then restart

sudo systemctl daemon-reload     //<--To reload service units
sudo systemctl restart mongod    //<--To Restart MongoDB Server

and then initiate replication through your mongod instance in terminal

$ mongosh
$ rs.initiate()
4
  • 1
    This was the only that work for me. I have a little question, if I start a connection with mongoose like this: mongodb://localhost:27017/database_name do I need to add something relate with replicaSet. I mean, I've started to make some test and this works perfect only with this without replicaSet in the URL, but I don't know if this could cause problems later.
    – Luis
    Commented May 14, 2023 at 23:35
  • @Luis It would be better if you tell mongoose or any other connection by including ?replicaSet=rs0. Whenever I connect through PHP or any other connection, I always do so. Commented May 16, 2023 at 3:48
  • Something strange happened today. I was presented with the replica set error again (which I had solved a long time ago). I don't know why, but that configuration was deleted. I had to add it back. Is this related to mongo updates?
    – Luis
    Commented Jun 24, 2023 at 22:05
  • @Luis yes it can be highly related to your Linux auto updates. If you have not configured your update settings then mongodb will probably get updated and so its following files. I have witnessed the same for PHP and Nginx etc Commented Jun 26, 2023 at 6:11
5

For those who are looking for a proper docker-compose.yml for testing with MongoDB replica set initialized, try this, which also works with the latest official MongoDB image (7):

version: '2'

services:
  mongodb:
    image: mongo:7
    container_name: mongodb
    ports:
      - '27017:27017'
    command: mongod --replSet rs0
    healthcheck:
      test: |
        mongosh --eval "try { rs.status().ok } catch (e) { rs.initiate({ _id: 'rs0', members: [{ _id: 0, host: 'localhost:27017' }] }).ok }"
      start_period: 0s
      interval: 500ms
      timeout: 5s
      retries: 5

This will initialize a replica set on one of the first health checks.

1
  • 1
    What an ingenious solution!!! This is the solution you are looking for, guys! Thanks, @nikitaeverywhere :)
    – times29
    Commented Mar 24 at 8:31
3

In order to use transactions, you need a MongoDB replica set, and starting a replica set locally for development is an involved process.

You can use the run-rs npm module. Zero-config MongoDB runner. Starts a replica set with no non-Node dependencies, not even MongoDB.

Or you can simply create an account in MongoDB Atlas which gives you a limited resource MongoDB cluster and so you can run/test your application.

MongoDB Atlas

2

The easiest answer for this issue is -

  1. Stop Mongod Service
  2. Change the mongo config file to create replica set
  3. restart mongo service
  4. reinitiate replica set

Commands for the same -

  1. sudo systemctl stop mongod
  2. edit /etc/mongod.conf
    replication:
       replSetName: rs0
  1. Start mongod service again
    sudo systemctl start mongod
  1. Run mongo shell
    $mongosh
  1. initiate a replica set in mongo shell
    rs.initiate()
1

Works for mongo:5.0.5-focal image.

Dockerfile:

FROM mongo:5.0.5-focal AS rs-mongo

# Make MongoDB a replica set to support transactions. Based on https://stackoverflow.com/a/68621185/1952977
RUN apt-get update && apt-get install patch

# How to create scripts/docker-entrypoint.sh.patch
# 1. Download the original file:
#    wget https://github.com/docker-library/mongo/raw/master/5.0/docker-entrypoint.sh
#    ( wget https://github.com/docker-library/mongo/raw/b5c0cd58cb5626fee4d963ce05ba4d9026deb265/5.0/docker-entrypoint.sh )
# 2. Make a copy of it:
#    cp docker-entrypoint.sh docker-entrypoint-patched.sh
# 3. Add required modifications to docker-entrypoint-patched.sh
# 4. Create patch:
#    diff -u docker-entrypoint.sh docker-entrypoint-patched.sh > scripts/docker-entrypoint.sh.patch
# 5. Clean up:
#    rm docker-entrypoint.sh docker-entrypoint-patched.sh
COPY scripts/docker-entrypoint.sh.patch .
RUN patch /usr/local/bin/docker-entrypoint.sh docker-entrypoint.sh.patch
RUN mkdir -p /etc/mongo-key && chown mongodb:mongodb /etc/mongo-key

CMD ["--replSet", "rs", "--keyFile", "/etc/mongo-key/mongodb.key"]

scripts/docker-entrypoint.sh.patch:

--- docker-entrypoint.sh    2022-01-04 15:35:19.594435819 +0300
+++ docker-entrypoint-patched.sh    2022-01-06 10:16:26.285394681 +0300
@@ -288,6 +288,10 @@
    fi

    if [ -n "$shouldPerformInitdb" ]; then
+
+     openssl rand -base64 756 > /etc/mongo-key/mongodb.key
+    chmod 400 /etc/mongo-key/mongodb.key
+
        mongodHackedArgs=( "$@" )
        if _parse_config "$@"; then
            _mongod_hack_ensure_arg_val --config "$tempConfigFile" "${mongodHackedArgs[@]}"
@@ -408,7 +412,14 @@
        set -- "$@" --bind_ip_all
    fi

-   unset "${!MONGO_INITDB_@}"
+  echo 'Initiating replica set'
+  "$@" --logpath "/proc/$$/fd/1" --fork
+  echo 'rs.initiate({"_id":"rs","members":[{"_id":0,"host":"127.0.0.1:27017"}]});' | mongosh -u "$MONGO_INITDB_ROOT_USERNAME" -p "$MONGO_INITDB_ROOT_PASSWORD"
+  "$@" --logpath "/proc/$$/fd/1" --shutdown
+  echo 'Done initiating replica set'
+
+  unset "${!MONGO_INITDB_@}"
+
 fi

 rm -f "$jsonConfigFile" "$tempConfigFile"

docker-compose.yml:

version: '3.9'

services:
  mongo:
    image: rs-mongo:current
    restart: always
    env_file:
      - .env
    ports:
      - 127.0.0.1:27017:27017
    volumes:
      - mongo-db:/data/db
      - mongo-configdb:/data/configdb
      - mongo-key:/etc/mongo-key

volumes:
  mongo-db:
    driver: local
  mongo-configdb:
    driver: local
  mongo-key:
    driver: local

UPDATED: 6th of Jan, 2022

0

I've been fighting against this issue for weeks. I let you my conclusion. In order to be able to use transactions on a sharded cluster, you need to run at least MongoDB 4.2 on your cluster. If the cluster is not sharded, from 4.0. I was using a library that has as a sub-dependency mongodb NodeJS driver. This driver from version 3.3.x fails against the sharded MongoDB cluster with version 4.0.4. The solution for me was to update my cluster to 4.2 version.

src: https://www.bmc.com/blogs/mongodb-transactions/

0

The error is because you are using MongoDB sessions and it is not configured on your system.

run this to install run-rs :-

npm install run-rs -g

run:-

run-rs -v 4.0.0 --shell

You should see the below output. Please be patient since MongoDB 4.0.0 is about 70MB.

$ run-rs -v 4.0.0 --shell
Downloading MongoDB 4.0.0
Copied MongoDB 4.0.0 to '/home/node/lib/node_modules/run-rs/4.0.0'
Purging database...
Running '/home/node/lib/node_modules/run-rs/4.0.0/mongod'
Starting replica set...
Started replica set on "mongodb://localhost:27017,localhost:27018,localhost:27019"
Running mongo shell: /home/node/lib/node_modules/run-rs/4.0.0/mongo
rs:PRIMARY>

You now have a replica set running MongoDB 4.0.0 locally. Run rs.status() to verify the replica set is running.

NOTE:- Your nodejs version should be $gte v3.1.0

-1

com.mongodb.MongoQueryException: Query failed with error code 20 and error message 'Transaction numbers are only allowed on a replica set member or mongos'

I am facing this issue on SpringBoot microservice with mongoDB using with Transaction

Solution:

Transactions are not allowed on local running mongo, Remove Transactional annotation from service method on local run

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.