14

Moved to using supervisod as a process control system.

I have a LONG and repeating ENVIRONMENT configuration in my supervisord.conf setting a lot of environment variables for a lot of processes. I need to define it one place and reuse it, to keep the configuration DRY and maintainable. is that possible with supervisor and how?

EDIT: Example of a non dry configuration

[program:node-app1]
command=node /home/ubuntu/server/node-app1/app.js
directory=/home/ubuntu/server/node-app1
autostart=true
autorestart=true
stderr_logfile=/home/ubuntu/supervisor/node_app1/err.log
stdout_logfile=/home/ubuntu/supervisor/node_app1/out.log
user=ubuntu
priority=998
startretries=20
ENVIRONMENT=BROKER_URL="amqp://user:password@path.to.rabbit:5672",
            NODE_ENV=envName,
            MONGO_URL="mongodb://path.to.mongo:27017",
            BASE_PUBLIC_API="http:path.to.api",
            REDIS_URL="redis://path.to.redis:6379",
            BACKEND_URL="https://path.to.backend",
            CHARTS_URL="https://path.to.charts"

[program:node-app2]
command=node /home/ubuntu/server/node-app2/app.js
directory=/home/ubuntu/server/node-app2
autostart=true
autorestart=true
stderr_logfile=/home/ubuntu/supervisor/node_app2/err.log
stdout_logfile=/home/ubuntu/supervisor/node_app2/out.log
user=ubuntu
priority=20
startretries=20
ENVIRONMENT=BROKER_URL="amqp://user:password@path.to.rabbit:5672",
            NODE_ENV=envName,
            MONGO_URL="mongodb://path.to.mongo:27017",
            BASE_PUBLIC_API="http:path.to.api",
            REDIS_URL="redis://path.to.redis:6379",
            BACKEND_URL="https://path.to.backend",
            CHARTS_URL="https://path.to.charts"

What could be shared: ENVIRONMENT, base directory for logs (only the end would change for each app), common variables like startsecs. etc

4
  • Not sure about supervisor, but did you try PM2 ? Dec 6, 2015 at 7:00
  • I know pm2. but we switched to supervisor because after having strange and un acceptable failures in production with another node.js "role your own" process control system, we moved so battle proven supervisod. Hence the question
    – alonisser
    Dec 6, 2015 at 7:57
  • Is there a problem with setting reused variables in the global environment for all processes under supervisord, and can you provide an example of your non-DRY configuration for 2 sub-processes?
    – lossleader
    Dec 9, 2015 at 19:07
  • @lossleader added a non DRY example
    – alonisser
    Dec 12, 2015 at 16:49

1 Answer 1

8
+100

Options

Via Inheritance from supervisord:

As long as you are using version 3.0a10 or above, you can set environment variables in [supervisord] environment and they will be in the environment of all processes under supervisord's management.

[supervisord]
...
environment=FAVORITE_VENTURE="RUSTY",FAVORITE_COLOR="RED"

Working with Variables supervisord inherited from a Shell

Supervisord also has a %(ENV_VARNAME)s expansion format for interpreting environment variables which would allow moving around variable names for different processes. But the configuration's environment section does not add to the environment variables available by the %(ENV_)s mechanism, so it would be necessary to call supervisord with variables already set by your shell.

As an example, if you use init scripts to start supervisord and are on a debian based system (i.e. Ubuntu) then you might start with the following in /etc/default/supervisor:

export SUPERVISOR_INCLUDES="main.conf test.conf"
export MAIN_RETRY_COUNT=2
export TEST_RETY_COUNT=1
MONGO_BASE="MONGO_URL=mongodb://path.to.mongo"
MAIN_MONGO_URL="${MONGO_BASE}:27017"
TEST_MONGO_URL="${MONGO_BASE}:27018"
export MAIN_ENV="${MAIN_MONGO_URL},OTHER_THING=\"Another thing with escaped quoting\""
export TEST_ENV=..

Then use them in configurations:

; supvervisord.conf
[includes]
files= %(here)s/subdir/other.conf %(ENV_SUPERVISOR_INCLUDES)s 

; main.conf
[group:main]
...
[program:mainbackend]
startretries=%(ENV_MAIN_RETRY_COUNT)s
environment=%(ENV_MAIN_ENV)s

If users can sudo and call supervisord directly, this method doesn't work very well since sudo both strips the users environment and doesn't run traditional shell init scripts. But you can source /etc/default/supervisor in root's .bashrc and use sudo bash -c "supervisord .." or source it before calling supervisord.

5
  • This is in the correct direction (using supervisord section) but not quite. 1. I have some common "groups", each has a little different syntax for environment variables. So if I could set "environment" in the "group" and not in the [supervisod] section that could work. Is that possible? 2. This does not solve other common variables, which are not "environment" like a common base dir for logs, or a common variable for startsecs, restart attemtps etc. Any idea about that one?
    – alonisser
    Dec 12, 2015 at 16:38
  • It sounds like you would want to use the %(ENV_)s method then, which would mean making sure the variables already exist when supervisord starts. In a debian/ubuntu like system you could create+export them in /etc/default/supervisor. For any system, you should be able to add them to the normal rc files, then they can be referred to all over the config as long as you use 3.2+.
    – lossleader
    Dec 12, 2015 at 21:26
  • Can you add an example how would using an rc file and the ENV would look like in the code? Yes I'm using 3.2
    – alonisser
    Dec 12, 2015 at 21:41
  • I'll accept the answer, but this isn't really what I was looking for
    – alonisser
    Dec 13, 2015 at 11:00
  • Thanks, I think that if you want reuse variables within the config your cleanest option is generating repetitive configs to include via a glob in [includes]->files from DRY templates as part of whichever automated build+deploy system you use. But it is interesting to see how supervisord options for dynamic configuration have advanced.
    – lossleader
    Dec 13, 2015 at 14:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.