13

Here's what I mean. How can I write a docker-compose.yaml file so that when one of the services is "built", it first runs another service?

I'll try to be even more specific. I'm trying to build a Java application using the database library JOOQ, which wants to connect to a database at build time in order to generate Java classes that correspond to database tables. I would like some combination of Dockerfile(s) and docker-compose.yaml file, such that the following happens in roughly the following order.

  1. A "database" service is started (in my case, using a postgres image).
  2. The database is initialized with SQL scripts in my development repo.
  3. The build phase of my "web" service runs, which uses a Dockerfile in the same repo, which invokes the Gradle build, which tells JOOQ to connect to the database service started in step 1. This generates Java files, compiles them, and does everything else for building the container.
  4. The "web" service is started, connecting to the database service started in step 1.

Here's the docker-compose.yaml file I'm working with

version: '3.7' 
services:
  postgres:
    image: postgres:10.5-alpine
    restart: always
    ports:
      - "6432:5432"
    environment:
      POSTGRES_DB: flashtools
      POSTGRES_USER: flashtools
      POSTGRES_PASSWORD: flashtools
    volumes:
      - ./src/main/scripts/01_init.sql:/docker-entrypoint- 
initdb.d/01_init.sql
  web:
    build: .
    network_mode: host
    depends_on:
      - postgres
    ports:
      - "8080:8080"

I'll flesh this out with more details, but hopefully what I'm asking is pretty straightforward.

7
  • Any issues with your compose file? Commented Oct 13, 2018 at 4:07
  • that's is why we have depends_on. If a service A depends on B, then B is ran first Commented Oct 13, 2018 at 13:03
  • 3
    No. That is not why we have depends_on. Yes, if a service A depends on B, then B runs (launches a container) before A is run (launches a container), but that says nothing about when Docker builds the image for A. The issue with my compose file is that irrespective of depends_on for running the services, the building_of A occurs before the _running of B. That's a problem if building A requires running B first. I suspect there's no way to do this in Docker Compose. Commented Oct 15, 2018 at 16:52
  • As I said, I suspect it's not possible. I'll hold this question open a little while, then if no one chimes in with a better answer, I'll close it out myself. Commented Oct 15, 2018 at 17:21
  • @DavidVentimiglia hey, how did you solve your problem finally? Commented Jun 9, 2020 at 20:17

3 Answers 3

3

This should now (technically) be possible:

since 2021-04-06 in version 1.29.0 (release note link)

Using the service_completed_successfully condition (which is a "Long Form" depends_on option):

Long syntax:

The long form syntax enables the configuration of additional fields that can't be expressed in the short form.

  • condition: condition under which dependency is considered satisfied
    • service_completed_successfully: specifies that a dependency is expected to run to successful completion before starting a dependent service.

(source - emphasis added)

Long Syntax Example:

depends_on:
  service_name:
    condition: service_completed_successfully

Update your docker-compose.yml file:

You'll need to make the following changes:

  • bump your compose version up to 3.8
  • add the new service_completed_successfully condition to your depends_on
version: "3.8"
services:
  postgres:
    image: postgres:10.5-alpine
    restart: always
    ports:
      - "6432:5432"
    environment:
      POSTGRES_DB: flashtools
      POSTGRES_USER: flashtools
      POSTGRES_PASSWORD: flashtools
    volumes:
      - ./src/main/scripts/01_init.sql:/docker-entrypoint-initdb.d/01_init.sql
  web:
    build: .
    network_mode: host
    depends_on:
      postgres:
        condition: service_completed_successfully
    ports:
      - "8080:8080"

You might also need to use a multi-stage Dockerfile:

(multi-stage build info)

Multi-stage builds are typically used to slim down your final Docker image, but it can also be used to run scripts and create "artifacts" needed before finishing your build.

It's hard to suggest this without seeing your exact Dockerfile, but I thought it was worth mentioning that you may need to also use this.

Sign up to request clarification or add additional context in comments.

3 Comments

I don't think this will work, for three reasons: the PostgreSQL container will stay running forever so it will never "complete"; in general image builds don't respect depends_on: (you can expect every build to happen before any container starts); and in general Dockerfiles can't contact other containers.
I implemented this solution and it worked! Thanks! The idea is to have a one-off run container that is polling for the healthcheck of the target service (host).
when using multi-stage build, how can you force web to wait for postgres to be built? with your solution it fails at "load metadata..." because postgres image doesnt exist yet
2

I had the same issue and solved it in a hacky way...

In my case the web service is Haskell, which builds with the stack build command, then executes with the stack exec APP_NAME. There's also a shorthand like stack build --exec APP_NAME.

Also there is an option to build only the dependencies with stack build --only-dependencies, so the Docker build cache them.

So I changed my Dockerfile like this:

COPY ./package.yaml /app/package.yaml        # Copying the package file
RUN stack build --only-dependencies          # Build dependencies only

COPY . /app                                  # Copying the rest of the files

EXPOSE 3000

CMD ["stack", "build", "--exec", "APP_NAME"] # Build the application itself

so the final build runs only when the service is started.

I don't know Java enough, but I guess there must be a similar option there too.

Also some hacking on the depends_on part is also needed: https://docs.docker.com/compose/startup-order/

Comments

0

In my case, the web service entrypoint is a command like node server.js.

It will work fine if the database is already available at the time it is called, but this is not the case when docker-compose is used.

One way to get around is to write code in your server.js so that it retries until the database is ready.

In my server.js:

connectDB()
.then(db => {
    /* Use db here...*/

    app.listen(3000);
});

async function connectDB() {
    var retry = 5;
    while (retry-- > 0 && !(dbReady()) {
        await sleep(1000);
    }
    return useDB();
}

function dbReady() {
    /* returns true iff DB available */
}

async function useDB() {
    /* returns DB instance */
}

function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

1 Comment

You might find this useful for your usecase: github.com/vishnubob/wait-for-it

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.