6

I have been working on setting up my celery task as a daemon in order to process data on a schedule.

I have been following the docs in order to set up my daemon, but have been running into a log file permission error that has me stumped.

Below is the configuration i have set up on an ubuntu box on Digital Ocean

/etc/default/celeryd

# here we have a single node
CELERYD_NODES="w1"

CELERY_BIN = "/mix_daemon/venv/bin/celery"
CELERYD_CHDIR="/mix_daemon/"

CELERYD_OPTS="-A tasks worker --loglevel=info --beat"

# %n will be replaced with the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"
CELERY_CREATE_RUNDIR=0
CELERY_CREATE_LOGDIR=0

# Workers should run as an unprivileged user.
CELERYD_USER="celery"
CELERYD_GROUP="celery"

however, when i run

sh -x /etc/init.d/celeryd start

i get the following error message:

celery multi v3.1.7 (Cipater)
> Starting nodes...
> celery@mix: OK
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/john/mix_daemon/venv/lib/python2.7/site-packages/celery/bin/celeryd_detach.py", line 168, in <module> main()
File "/home/john/mix_daemon/venv/lib/python2.7/site-packages/celery/bin/celeryd_detach.py", line 165, in main
detached_celeryd(app).execute_from_commandline()
File "/home/john/mix_daemon/venv/lib/python2.7/site-packages/celery/bin/celeryd_detach.py", line 160, in execute_from_commandline
**vars(options)
File "/home/john/mix_daemon/venv/lib/python2.7/site-packages/celery/bin/celeryd_detach.py", line 42, in detach
with detached(logfile, pidfile, uid, gid, umask, working_directory, fake):
File "/home/john/mix_daemon/venv/local/lib/python2.7/site-packages/celery/platforms.py", line 314, in open
self.after_chdir()
File "/home/john/mix_daemon/venv/local/lib/python2.7/site-packages/celery/platforms.py", line 384, in after_chdir_do
logfile and open(logfile, 'a').close()
IOError: [Errno 13] Permission denied: '/var/log/celery/celery.log'
+ sleep 5
+ exit 0

I looked this error message up and saw this answer from a redmine project. So i tried the following to try to allow for the celery worker to write to the log file:

$ sudo mkdir -p -m 2755 /var/log/celery
$ sudo chown celery:celery /var/log/celery

but the same error remains when i try to start the daemon.

I am a celery noob, and any help on this would be greatly appreciated!

1

3 Answers 3

4

It helps me:

sudo chown -R myuser:root /var/run/celery/
sudo chown -R myuser:root /var/log/celery/
1
  • This was necessary for me when using systemd, where myuser is whatever the User is set to in the celery.service file. Commented Aug 8, 2018 at 14:16
3

According to the documentation:

The init scripts can only be used by root, and the shell configuration file must also be owned by root"

So, you need to run:

sudo chown root:root /etc/default/celeryd

and

sudo /etc/init.d/celeryd start

By the way, you have invalid syntax in your celeryd config file:

CELERY_BIN = "/mix_daemon/venv/bin/celery"

spaces is not allowed with "=" operator, the correct is:

CELERY_BIN="/mix_daemon/venv/bin/celery"

Also check, that you have your bin in /mix_daemon not /home//mix_daemon

0

Tl;dr

If you're using systemd for daemonization, your file /etc/tmpfiles.d/celery.conf should look like this:

d /run/celery 0755 myuser myusergroup -
d /var/log/celery 0755 myuser myusergroup -

A little more detail

For those using systemd, I found that I'd been using the wrong user when creating working directories for celery's logs and once I'd updated /etc/tmpfiles.d/celery.conf with the correct user and user group, my issues were solved.

The documentation mentions:

You can also use systemd-tmpfiles in order to create working directories (for logs and pid).

file: /etc/tmpfiles.d/celery.conf

And gives the example:

d /run/celery 0755 celery celery -
d /var/log/celery 0755 celery celery -

Your file should look like this:

d /run/celery 0755 myuser myusergroup -
d /var/log/celery 0755 myuser myusergroup -

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.