5

I've searched over the web but couldn't find my answer anywhere. I'm trying to run an API web service, using NestJS framework.

I'm running docker-compose that spins up the API server, a MongoDB instance, and a mongocryptd instance to allow Client-Side Field Level Encryption on my app.

I'm able to connect to the MongoDB instance, but not to the mongocryptd instance.

Docker-Compose file:

version: "3.7"
services:
  api:
    build:
      context: .
      dockerfile: Dockerfile
      labels:
        env: dev
      args:
        APP: appname
        APP_PORT: 3000
    ports:
      - "3000:3000"
    command: ["sh", "-c", "npm run start:app:dev"]
    volumes:
      - .:/app

  mongodb:
    build:
      context: .
      dockerfile: docker/MongoEP-Dockerfile
      labels:
        env: dev
      args:
        MONGO_PACKAGE: mongodb-enterprise
        MONGO_REPO: repo.mongodb.com
    image: mongo-enterprise:4.2.5
    command: ["--auth"]
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: usr
      MONGO_INITDB_ROOT_PASSWORD: pwd
    ports:
      - "27017:27017"
    volumes: ["/private/var/services/mongodb:/data/db"]

  mongocryptd:
    build:
      context: .
      dockerfile: docker/MongoEP-Dockerfile
      labels:
        env: dev
      args:
        MONGO_PACKAGE: mongodb-enterprise
        MONGO_REPO: repo.mongodb.com
    image: mongo-enterprise:4.2.5
    entrypoint: mongocryptd
    restart: always
    ports:
      - "27020:27020"
    volumes: ["/private/var/services/mongodb:/data/db"]

The used dockerfile is mongo's official dockerfile, but supplied with args to build an enterprise version of the image which includes the enterprise features.

When trying to connect to the database from the app, I'm running:

MongooseModule.forRoot(`mongodb://usr:pwd@mongodb:27017`, {
      useNewUrlParser: true,
      useUnifiedTopology: true,
      useFindAndModify: false,
      retryAttempts: 2,
      autoEncryption: {
        keyVaultNamespace,
        kmsProviders,
        extraOptions: {
          mongocryptdURI: `mongodb://mongocryptd:27020`,
          mongocryptdBypassSpawn: true
        }
      } as any
    })

** This is the NestJS version of supplying the configs. it's similar to mongoose - the first argument is the URI and the second is the settings object


Without the autoEncryption options, I'm able to connect without any problems. That means that my database address is correct. With the autoEncyption options, I'm getting MongooseServerSelectionError: connect ECONNREFUSED 172.25.0.4:27020 (mongocryptd address). That means that the IP is correct (DNS resolved), but the connection is refused. As I showed before, the port (27020) is being published by the docker-compose file, and I even tried to add an EXPOSE step in the build itself.

BUT when I map the network of the containers to host (network_mode: "host"), the application is able to connect without any problems (changing the connections DNS to localhost:27017 and 27020 of course). So that must mean it's a docker-related problem.

Additional things I've tried && a recap of what I tried:

  • Attach a volume to replace /etc/mongod.conf.orig with the following network configurations:
net:
  port: 27017
  bindIp: 0.0.0.0
  bindIpAll: true
  • Instead of attaching a volume, replacing it ^ at the build step before launching the mongo service.
  • I also tried changing the bindIp to the specific application IP that was given by the docker network.
  • All types of connection strings with & without user credentials, auth source, and default database.
  • Port 27020 is published in docker-compose & exposed on docker file.

I ran out of ideas. Any help is appreciated! :)

EDIT:

After more debugging, I can see that mongod is running with --bind_ip_all by default so changing the conf file shouldn't have an effect. Tried also running mongocryptd with mongods docker-entrypoint.sh entrypoint instead of overriding it.

6
  • can you share your updated dockerfile? I am facing this issue as well. thanks!
    – Ben Luk
    Commented Sep 12, 2020 at 0:07
  • @BenLuk just add these lines to install the mongocrypt process on the same container - ``` RUN wget -qO - mongodb.org/static/pgp/server-4.2.asc | apt-key add - \ && echo "deb repo.mongodb.com/apt/debian stretch/mongodb-enterprise/4.2 main" | tee /etc/apt/sources.list.d/mongodb-enterprise.list RUN apt-get update \ && apt-get install --no-install-recommends -y mongodb-enterprise-cryptd \ && rm -rf /var/lib/apt/lists/* ```
    – Asaf Kfir
    Commented Sep 21, 2020 at 7:39
  • @AsafKfir Have you succeed exposing port 27020(mongocryptd) from container to the host? --bind_ip_all only works for mongod
    – hksfho
    Commented Aug 4, 2022 at 14:34
  • @hksfho Nope. , I just gave up and installed the mongodb-enterprise-cryptd package inside the app container. It works flawlessly but this is not the optimal solution.
    – Asaf Kfir
    Commented Aug 6, 2022 at 17:23
  • @AsafKfir you can check my answer
    – hksfho
    Commented Aug 9, 2022 at 10:49

3 Answers 3

0
  1. Verify mongocryptd is running (ps awwxu, etc.)
  2. Verify you can connect to it from bash on the same container where it is running using mongo.
  3. Verify you can connect to it from host system using mongo.
  4. Check mongocryptd logs (it's basically a mongod with some extra functionality).
9
  • 1. I can verify it is running under the root user using ps awwxu. 2. I'm getting a failed, connection refused error - that's because i'm not running the mongod process on that container, just the mongocryptd. I've tried running both mongod & mongocryptd from the same container (run mongo regular image, open a shell, run mongocryptd), but I'm getting the same connection refused error. 3. Again, no mongod is running 4. The logs are empty. I can see just start up logs [listener] Listening on 127.0.0.1 & [listener] waiting for connections on port 27020
    – Asaf Kfir
    Commented Apr 20, 2020 at 4:40
  • I've re-run both mongocryptd and mongo from the same container and scanned the ports with Nmap. Both 27017 & 27020 are stated as open. I'm not getting a connection refused when using mongo CLI, and I'm able to connect from the app if I omit the option to use mongocryptd in the connection settings (autoEncryption key from the example above). When I re-add the encryption key, it tries to connect to mongocryptd and i'm getting the same ECONNREFUSED error. Again, when using the host network mode, everything works.
    – Asaf Kfir
    Commented Apr 20, 2020 at 5:34
  • Work through the steps, mongo shell can connect directly to a mongocryptd process and at least run ismaster on it.
    – D. SM
    Commented Apr 20, 2020 at 8:31
  • 1
    I've done that. They are configured with default configurations (and I've tried passing the conf file settings explained above as well). Currently, I couldn't solve this problem. My best guess is the reason mongocryptd doesn't have an option to bind to an other IP other than localhost (doesn't accept it in the .conf file and there is no option to pass it as a command argument). After 3 days, I just gave up and installed the mongodb-enterprise-cryptd package inside the app container. It works flawlessly but this is not the optimal solution.
    – Asaf Kfir
    Commented Apr 21, 2020 at 11:40
  • 1
    You are right, mongocryptd does not appear to recognize --bind_ip.
    – D. SM
    Commented Apr 21, 2020 at 23:33
0

I had similar issues with mongocryptd, but I was installing php node instead of npm. Tried different solutions but didn't managed to succeed. Even tried similar Asaf Kfir docker-composer.yaml with pre-installed mongodb-enterprise-cryptd lib, but had same issue. Keep in mind, in php node via Dockerfile I already been able to install libmongocrypt-dev and mongodb-enterprise-cryptd. (I will leave php Dockerfile below)

Those three containers I managed to link under the same ip address, tested with ncat and I was able to reach. But when I tried to run tests from php node, it start throwing:

MongoDB\Driver\Exception\BulkWriteException: Bulk write failed due to previous MongoDB\Driver\Exception\RuntimeException: key vault error: Invalid reply to find command.

and had this issue for two weeks. Basically didn't know how to resolve it.

P.S. remember these words: The automatic feature of field level encryption is only available in MongoDB Enterprise 4.2 or later

At that time my docker-compose.yaml file looked like that:

version: '3'
services:

    #PHP Service
    php:
        image: local-base-php
        container_name: app
        restart: unless-stopped
        tty: true
        ports:
            - "27017:27017"
            - "27020:27020"
        environment:
            SERVICE_NAME: app
            SERVICE_TAGS: dev
        working_dir: /var/www
        volumes:
            - ./:/var/www/projects
            - ./php/local.ini:/usr/local/etc/php/conf.d/local.ini

    #MongoDB Service
    mongodb:
        image: local-mongo-db
        container_name: mongodb
        restart: unless-stopped
        tty: true
        environment:
            MONGO_INITDB_DATABASE: test
            MONGO_INITDB_USERNAME: root
            MONGO_INITDB_PASSWORD: rootpassword
        network_mode: service:php
        volumes: ["/tmp/mongodb:/data/db"]

    #MongoDB Service
    mongocryptd:
        image: local-mongocryptd
        container_name: mongocryptd
        entrypoint: mongocryptd
        restart: unless-stopped
        tty: true
        network_mode: service:php
        volumes: ["/tmp/mongodb:/data/db"]

volumes:
    dbdata:
        driver: local

Mongo images where build from here: https://docs.mongodb.com/manual/tutorial/install-mongodb-enterprise-with-docker/

I managed to solve this issue like this: Instead of building mongo-enterprise image I accidentally build up with official mongo image mongo:4.2 and everything worked well. I don't know why mongo says that enterprise is needed for encryption. Because for me mongo-enterprise encryption didn't worked. The original mongo:4.2 image worked perfectly.

working docker-compose.yaml:

version: '3'
services:

    #PHP Service
    php:
        image: local-base-php
        container_name: app
        restart: always
        tty: true
        ports:
            - "27017:27017"
        environment:
            SERVICE_NAME: app
            SERVICE_TAGS: dev
        working_dir: /var/www
        volumes:
            - ./:/var/www/projects
            - ./php/local.ini:/usr/local/etc/php/conf.d/local.ini

    #MongoDB Service
    mongodb:
        image: mongo:4.2
        container_name: mongodb
        restart: always
        tty: true
        environment:
            MONGO_INITDB_DATABASE: test
            MONGO_INITDB_USERNAME: root
            MONGO_INITDB_PASSWORD: rootpassword
        network_mode: service:php

php node Dockerfile:

FROM php:7.4

RUN apt-get update && apt-get install -y zip unzip libzip-dev git mercurial zlib1g-dev libicu-dev libcurl4-gnutls-dev libssl-dev libssh2-1-dev libgmp-dev libpng-dev uuid-dev

RUN cd /tmp && git clone https://github.com/php/pecl-networking-ssh2 && cd /tmp/pecl-networking-ssh2 \
    && phpize && ./configure && make && make install \
    && echo "extension=ssh2.so" > /usr/local/etc/php/conf.d/ext-ssh2.ini \
    && rm -rf /tmp/ssh2

RUN docker-php-ext-configure gmp

RUN docker-php-ext-install zip json pdo pdo_mysql curl opcache bcmath sockets gmp gd

RUN docker-php-ext-install -j$(nproc) intl

RUN pecl install uuid pcov redis mongodb

RUN docker-php-ext-enable uuid pcov redis mongodb

RUN curl -sS https://get.symfony.com/cli/installer | bash -s -- --install-dir /usr/local/bin

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

RUN curl -L -sS "https://github.com/splitsh/lite/releases/download/v1.0.1/lite_linux_amd64.tar.gz" | tar xvz -C /usr/local/bin

RUN apt-get update

RUN apt-get install -y curl gpg wget

RUN sh -c 'curl -s https://www.mongodb.org/static/pgp/libmongocrypt.asc | gpg --dearmor >/etc/apt/trusted.gpg.d/libmongocrypt.gpg'

RUN echo "deb https://libmongocrypt.s3.amazonaws.com/apt/ubuntu bionic/libmongocrypt/1.0 universe" | tee /etc/apt/sources.list.d/libmongocrypt.list

RUN wget -qO - mongodb.org/static/pgp/server-4.2.asc | apt-key add -

RUN echo "deb http://repo.mongodb.com/apt/debian stretch/mongodb-enterprise/4.2 main" | tee /etc/apt/sources.list.d/mongodb-enterprise.list

RUN apt-get update

RUN apt-get install -y libmongocrypt-dev && apt-get install  --no-install-recommends -y mongodb-enterprise-cryptd

I hope I helped. Cheers!

UPDATE: My tests are running without libmongocrypt-dev lib, so I guess you only need mongodb-enterprise-cryptd.

0

I decided to use socat forwarding traffic

Dockerfile:

# Build stage
FROM ubuntu:focal

ENV ENTRY_FILE=docker-entrypoint.sh
ENV MONGODB_PATH=/usr/src/mongodb
ENV ENTRY_POINT=$MONGODB_PATH/$ENTRY_FILE

RUN apt-get update && apt-get install sudo
RUN sudo apt-get install -y curl telnet vim socat libcurl4 libgssapi-krb5-2 libldap-2.4-2 libwrap0 libsasl2-2 libsasl2-modules libsasl2-modules-gssapi-mit snmp openssl liblzma5

RUN curl -k -o mongodb.tgz "https://downloads.mongodb.com/linux/mongodb-linux-$(arch)-enterprise-ubuntu2004-5.0.10.tgz"
RUN tar -xf mongodb.tgz --strip-components=1

RUN sudo ln -s $MONGODB_PATH/bin/* /usr/local/bin/
RUN sudo mkdir -p /data/db
RUN sudo mkdir -p /data/log
RUN sudo chown `whoami` /data/db
RUN sudo chown `whoami` /data/log

COPY ./$ENTRY_FILE $ENTRY_POINT
RUN chmod +x $ENTRY_POINT
ENTRYPOINT $ENTRY_POINT

docker-entrypoint.sh:

#!/bin/sh

socat -d -d TCP-LISTEN:27017,fork,bind=$(hostname -I | awk '{print $1}') TCP:127.0.0.1:17017 &
socat -d -d TCP-LISTEN:27018,fork,bind=$(hostname -I | awk '{print $1}') TCP:127.0.0.1:17018 &
socat -d -d TCP-LISTEN:27019,fork,bind=$(hostname -I | awk '{print $1}') TCP:127.0.0.1:17019 &
socat -d -d TCP-LISTEN:27020,fork,bind=$(hostname -I | awk '{print $1}') TCP:127.0.0.1:17020 &

./bin/mongod --port 17017 --dbpath /data/db --logpath /data/log/mongod.log &
./bin/mongocryptd --port 17020 --logpath /data/log/mongocryptd.log

$(hostname -I | awk '{print $1}') is the remote ip(my docker host ip, 172.2.x.x), you can change to your another container ip.

docker-compose.yml:

version: "3.8"

networks:
    ABC:
        external: false
        name: ABC

services:
    mongo-database:
        container_name: MongoDB
        privileged: true
        build:
            context: .docker/db
            dockerfile: Dockerfile
        volumes:
            - .mongo/db:/data/db
            - .mongo/log:/data/log
        ports:
            - 27017-27020:27017-27020
        networks:
            - ABC

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.