93

I've recently tried running a cron job from within a linked docker container and run into an issue. My main docker container is linked to a postgres container and its port number is set as an environment variable by docker upon the containers creation. This environment variable is not set in ~/.profile or any other source file that I could load when running my cron job. How can I then access these environment variables from my cron job?

Thanks!

1

9 Answers 9

126

I ran into this same problem. I have a docker container that runs cron to execute some shell scripts periodically. I too had a hard time finding out why my scripts would run fine when I manually executed them inside the container. I tried all the tricks of creating a shell script that would run first to set the environment, but they never worked for me (most likely I did something wrong). But I continued looking and found this and it does work.

  1. Setup a start or entry point shell script for your cron container
  2. Make this the first line to execute printenv | grep -v "no_proxy" >> /etc/environment

The trick here is the /etc/environment file. When the container is built that file is empty, I think on purpose. I found a reference to this file in the man pages for cron(8). After looking at all the versions of cron they all elude to an /etc/? file that you can use to feed environment variables to child processes.

Also, note that I created my docker container to run cron in the foreground, cron -f. This helped me avoid other tricks with tail running to keep the container up.

Here is my entrypoint.sh file for reference and my container is a debian:jessie base image.

printenv | grep -v "no_proxy" >> /etc/environment

cron -f

Also, this trick worked even with environment variables that are set during, docker run commands.

14
  • 28
    printenv > /etc/environment was the only thing that worked for me after 4 days och experimenting with various options ! Thanks!
    – Deko
    Apr 13, 2017 at 19:07
  • 1
    Using "/etc/environment" worked for me, however please note that reading this file is not standard behavior for cron - it only works if you have the pam cron module installed, please see askubuntu.com/a/700126
    – llmora
    Nov 14, 2017 at 15:58
  • 4
    Watch out for the limitation of using /etc/environment file as there is no possibility to escape the comment sign #, so you can not use it in the values of the variables there...
    – AuHau
    Jan 18, 2018 at 19:35
  • 6
    It might be worth noting that the ... | grep -v " no_proxy part is specific to certain environments and it's just a red herring for most. As @Deko says, for most situations the magic is simply printenv > /etc/environment. (Also, using ">>" would append to any existing /etc/environment file but I've never seen one that's not empty to start. Does anyone have a different experience??)
    – Tom Wilson
    May 6, 2020 at 17:00
  • 2
    Please be aware that >> will cause add cron line as many times as docker container restart, better just >
    – Javier
    Feb 19, 2021 at 7:25
52

I would recommend using declare to export your environment and avoid escaping issues. Can be used in CMD or ENTRYPOINT or directly in a wrapper script which might be called by one of them:

declare -p | grep -Ev 'BASHOPTS|BASH_VERSINFO|EUID|PPID|SHELLOPTS|UID' > /container.env

Grep -v takes care of filtering out read-only variables.

You can later easily load this environment like this:

SHELL=/bin/bash
BASH_ENV=/container.env
* * * * * root /test-cron.sh
6
  • 1
    Thanks a lot, this works much better than the first answer with printenv that doesn't escape anything Aug 31, 2018 at 9:28
  • 2
    I think this is the right answer too, but I haven't used the "-v" of the "grep", I've preferred put the specific variables that I will use, like this: declare -p | grep -E 'VAR1|VAR2|VAR3|ETC' > /container.env Jun 24, 2019 at 18:22
  • 2
    Alex Fedulov's regex is too greedy. It will filter out more than the intended variable declarations (it would f.ex. filter out a variable named FLUID). I suggest using the expression declare -p | grep -Ev '^declare -[[:alpha:]]*r' which should filter out all the readonly declarations. Oct 15, 2019 at 10:34
  • 2
    @TomášPospíšek or maybe amend the pattern to respect word boundaries?: grep -Ev '\b(BASHOPTS|BASH_VERSINFO|EUID|PPID|SHELLOPTS|UID)\b', or even '\b(BASHOPTS|BASH_VERSINFO|EUID|PPID|SHELLOPTS|UID)='
    – laur
    Sep 30, 2020 at 13:22
  • 1
    Most elegant and bulletproof solution I found 👍 Of course, apart of fixing this in the container runtime. BTW - anybody knows how the envs are passed between container runtime and linux OS? The env is not part of e.g. /etc/profile and I'm scratching my head how it is working.
    – Jarek
    Dec 12, 2022 at 10:54
6

One can append the system environment variables to the top of a crontab file by using wrapper shell script to run the cron daemon. The following example is from CentOs 7,

In the Dockerfile

COPY my_cron /tmp/my_cron
COPY bin/run-crond.sh run-crond.sh
RUN chmod -v +x /run-crond.sh
CMD ["/run-crond.sh"]

run_cron.sh:

#!/bin/bash

# prepend application environment variables to crontab
env | egrep '^MY_VAR' | cat - /tmp/my_cron > /etc/cron.d/my_cron

# Run cron deamon
# -m off : sending mail is off 
# tail makes the output to cron.log viewable with the $(docker logs container_id) command
/usr/sbin/crond -m off  && tail -f /var/log/cron.log

This is based on a great blog post somewhere, but I lost the link.

6

inspired by @Kannan Kumarasamy answer:

  for variable_value in $(cat /proc/1/environ | sed 's/\x00/\n/g'); do
    export $variable_value
  done

I can't state for sure, what process with pid1 is and that its stable during lifetime of OS. but as it is the first process to be run, inside a container i guess we can take it for granted it is a process with desired env vars set. take all of this with pich of salt unless some linux/docker docs proves this is completely ok.

2
  • Read permission only for root. And one minor note, shellcheck warns "Useless cat" and suggests using $(sed 's/\x00/\n\g' < /proc/1/environ) instead. Jan 27, 2022 at 18:38
  • actually, you trim down further: sed 's/\x00/\n/g' /proc/1/environ sed takes a file as input and spits out the changes (unless specified otherwise, with -i inplace for example) May 24 at 6:25
4

You can run the following command

. <(xargs -0 bash -c 'printf "export %q\n" "$@"' -- < /proc/1/environ)

This is exceptionally works when you have the environment variables with special characters like ' " =

2
  • well. have a +1 from me. but could you point us to some docs that proves pid 1 is always the process that has the original environment variables? Jan 15, 2020 at 9:30
  • this one alos work with any special character in your envrionment variable. eval $(printenv | awk -F= '{print "export " "\""$1"\"""=""\""$2"\"" }' >> /etc/profile)
    – Amul
    Dec 2, 2020 at 15:35
3

The environment is set, but not available to the cron job. To fix that, you can do these two simple things

1) Save the env to a file in your ENTRYPOINT or CMD

CMD env > /tmp/.MyApp.env && /bin/MyApp

2) Then read that env into your cron command like this:

0 5 * * * . /tmp/.MyApp.env; /bin/MyApp
3
  • 3
    This won't work when env values contain characters such as whitespaces or pipes Feb 16, 2016 at 20:47
  • You'll need to properly escape env vars with whitespaces or pipes.
    – GDorn
    Mar 13, 2016 at 21:22
  • does .env file has have executable ?
    – Chang Zhao
    Oct 3, 2021 at 10:05
3

You should export your environment variable before you run cronjobs.

Other solutions are fine but they will fail when there are any special characters in your environment variable.

I have found the solution:

eval $(printenv | awk -F= '{print "export " "\""$1"\"""=""\""$2"\"" }' >> /etc/profile)

0

To escape any weird characters that could break your script, and according to reasoning from Mark's answer, add this line to your entrypoint.sh:

env | sed -r "s/'/\\\'/gm" | sed -r "s/^([^=]+=)(.*)\$/\1'\2'/gm" \ > /etc/environment

This way, if you have any variable like affinity:container==My container's friend, it will be converted to affinity:container='=My container\'s friend, and so on.

1
0

Building on this, I made a fully functional MVP here. I am sharing this as, being a nood, I struggled to build it and I guess I am not alone in that situation. It also has the "benefit" of output the echoes and prints to the container log.

TLDR:

Dockerfile

FROM python:3.11
ENV MY_ENV_VAR="default_value"

RUN apt-get update && apt-get -y install cron -y -qq

COPY cron /tmp/cron
COPY run_cron.sh run_cron.sh
RUN chmod +x run_cron.sh

COPY main.py /app/main.py

CMD ["/run_cron.sh"]

cron

*/1 * * * * root python /app/main.py > /proc/1/fd/1 2>/proc/1/fd/2

run_cron.sh

#!/bin/bash

#VARIOUS IMPORTANT FILES
CRON_FILE=/tmp/cron
ENV_FILE=/container.env
CRONTAB_BAK_FILE=/etc/crontab.bak
CRONTAB_FILE=/etc/crontab

CRON_STR=$(cat $CRON_FILE)

echo "=========="
echo "Welcome to run_crond.sh, a wrapper to $CRON_STR that loads the container environment variables into the cron shell"
echo "=========="
echo ""

echo "Exporting variables to $ENV_FILE"
echo ""
declare -p | grep -Ev 'BASHOPTS|BASH_VERSINFO|EUID|PPID|SHELLOPTS|UID' > "$ENV_FILE"

if test -f "$CRONTAB_BAK_FILE"; then
  echo "Retrieving $CRONTAB_FILE from $CRONTAB_BAK_FILE"
  cp "$CRONTAB_BAK_FILE" "$CRONTAB_FILE"
else
  echo "Copying $CRONTAB_FILE to $CRONTAB_BAK_FILE"
  cp "$CRONTAB_FILE" "$CRONTAB_BAK_FILE"
fi
echo ""

echo "SHELL=/bin/bash"    >> "$CRONTAB_FILE" && \
echo "BASH_ENV=$ENV_FILE" >> "$CRONTAB_FILE" && \
echo "$CRON_STR"          >> "$CRONTAB_FILE"

echo "$CRONTAB_FILE ready:"
cat "$CRONTAB_FILE"
echo ""

## Run cron deamon
echo "starting cron deamon in the foreground"
cron -f
echo ""

main.py

import os
my_string = "Hello, World! " + os.getenv('MY_ENV_VAR')
print(my_string)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.