16

While trying to debug a RUN statements in my Dockerfile, I attempted to redirect output to a file in a bound volume (./mongo/log).

To my surprise I was unable to create files via the RUN command, or to pipe the output of another command to a file using the redirection/appending (>,>>) operators. I was however able to perform the said task by logging in the running container via docker exec -ti mycontainer /bin/sh and issuing the command from there.

Why is this behaviour happening? How can I touch file in the Dockerfile / redirect output to a file or to the console from which the Dockerfile is run?

Here is my Dockerfile:

FROM mongo:3.4

#Installing NodeJS
  RUN apt-get update && \
    apt-get install -y curl && \
    curl -sL https://deb.nodesource.com/setup_6.x | bash - && \
    apt-get install -y nodejs


#Setting Up Mongo
  WORKDIR /var/www/smq
  COPY ./mongo-setup.js mongo-setup.js

  ##for testing
  RUN touch /var/log/node.log && /
      node --help 2>&1 > /var/log/node.log

  ##this was the command to debug
  #RUN node mongo-setup.js > /var/log/mongo-setup.log 2> /var/log/mongo-setup.error.log

Here an excerpt from my docker-compose.yml:

mongodb:
    build:
      context: ./
      dockerfile: ./mongodb-dockerfile
    container_name: smqmongodb
    volumes:
     - /var/lib/mongodb/data
     - ./mongo/log/:/var/log/
     - ../.config:/var/www/.config
4
  • You should read about caching intermediate build steps: docs.docker.com/engine/userguide/eng-image/… . It's working exactly as it's designed to work - you seem to want it to work in a different way. While it's possible to tell docker build not to use caching (with --no-cache=true, as you can learn about in the link above), I would suggest that you review the way you create your Dockerfile to embrace and leverage the caching capabilities.
    – Bruno Reis
    Commented Jan 1, 2017 at 1:41
  • Cool read! Let me update my Dockerfile. From my quick reading it doesn't seem to solve my problem, no? Commented Jan 1, 2017 at 1:44
  • @Phillipe - the parameter I mentioned should solve the problem - it will disable the caching, so the "touch" will run on every single build. But there might be other ways in which you could achieve your goals without having to disable the cache...
    – Bruno Reis
    Commented Jan 1, 2017 at 1:47
  • I do not understand how the cache should affect the creation of a new file. I do understand that what you are pointing at is that docker may already have a cached image in which a file was created and as such doesn't re-run the command. However as it is, even after restarting from scratch using docker-compose down && docker-compose build --no-cache && docker-compose up --force-recreate;, I still do not have a file. Let me know what you think. Commented Jan 1, 2017 at 1:54

2 Answers 2

26

You are doing this during your build:

RUN touch /var/log/node.log && /
    node --help 2>&1 > /var/log/node.log

The file /var/log/node.log is created and fixed immutably into the resulting image.

Then you run the container with this volume mount:

volumes:
  - ./mongo/log/:/var/log/

Whatever is in ./mongo/log/ is mounted as /var/log in the container, which hides whatever was there before (from the image). This is the thing that's making it look like your touch didn't work (even though it probably worked fine).

You're thinking about this backward - your volume mount doesn't expose the container's version of /var/log externally - it replaces whatever was there.

Nothing you do in Dockerfile (build) will ever show up in an external mount.

2
  • 4
    Your answer is an ah-ha! moment. Great explanation. Clear, concise and to the point. Now I understand what @Bruno Reis meant in his comment under the OP. Ultimately since Socratees' answer provide a solution to my logging problem, I will take his as the accepted solution. However your answer is just as informative and valuable. Cheers! Commented Jan 1, 2017 at 2:36
  • Thanks a lot! That was also my problem... thus the volumes coming from the image I derive from... -.-
    – saitho
    Commented Mar 17, 2018 at 9:15
5

Instead of RUN node mongo-setup.js > /var/log/mongo-setup.log 2> /var/log/mongo-setup.error.log, within the container, what if you just say `RUN node mongo-setup.js'?

Docker recommends using docker logs. Like so:

docker logs container-name

To accomplish what you're after (see the mongo setup logs?), you can split the stdout & stderr of the container by piping the separate streams: and send them to files:

me@host~$ docker logs foo > stdout.log 2>stderr.log

me@host~$ cat stdout.log
me@host~$ cat stderr.log

Also, refer to the docker logs documentation

3
  • 1
    Interesting, I'll do a bit of research on how this can be achieved with docker-compose (and how docker logs behaves) and I'll post my results here. Commented Jan 1, 2017 at 2:07
  • After some research I found this nice article describing in an overview how logging takes place in docker: medium.com/@yoanis_gil/… Now I see docker's strategy in terms of docking. Great to know! Commented Jan 1, 2017 at 2:32
  • While I was setting up the syslog logging option, for some reason I started to have some prints from the faulty command (node mongo-setup.js). So now I have a syslog ready to go whenever I want to use such a logging system, and my command prints to the console otherwise. Thanks again for your answer :) Commented Jan 1, 2017 at 3:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.