47

I'm trying to find some instructions on how to enable PyCharm debugging within my celery processes on a remote machine. The remote machine is running Ubuntu 14.04.

I am running PyCharm 4.x.

I've seen some other information that alludes others have it working, but haven't been able to locate any proper instructions.

11 Answers 11

46

You can have a Run Configuration to run your celery workers which then allows you to debug simply by clicking the debug button. Here is how I set that up in PyCharm 5:

pycharm celery

You need to set up a remote python interpreter and then set other configs like the image above. Note that the Working directory is pointing to the bin folder of the remote interpreter with celery installed.

2
  • this solution worked for me in pycharm 2020 pro edit: i suppose workdir was what's missing
    – safhac
    Mar 1, 2021 at 8:00
  • I had to put the full path to the celery executable in IntelliJ 2021.1. Apr 30, 2021 at 18:51
24

For Windows, tested in with Celery >= 4.0

Add a new (Run/Debug) configuration of type "Python":

Then, Under configuration tab: switch the first option (Target to run) from "Script path" to "Module name" and fill in the value:

celery.bin.celery

like the snapshot below:

enter image description here

3
  • 1
    Thanks, was looking for this!
    – Pnia
    Sep 21, 2020 at 22:30
  • 5
    Thanks, I found -P solo must be added
    – Dequn
    Jan 14, 2021 at 13:21
  • 5
    I had to use module name celery, not celery.bin.celery Feb 7, 2022 at 17:02
14

Just add the following config:

from celery import current_app
current_app.conf.CELERY_ALWAYS_EAGER = True
current_app.conf.CELERY_EAGER_PROPAGATES_EXCEPTIONS = True

Doing so makes celery execute in the same thread as the currently executing thread.

9

Unfortunately, most solutions don't work on Windows. (There was a separate question specifically about that, but unfortunately it got closed as a duplicate of this one. So I'll answer that question here now.)

The problem is that on Windows, the standalone celery command is a batch file, so PyCharm cannot attach the Python debugger to it.

Up until Celery 3.x, you can create a manage.py run configuration and call the celery worker command on it.

Screenshot of PyCharm run configuration for running manage.py celery worker

Note that you don't need to set --app here, as the application is defined by the management command via DJANGO_SETTINGS_MODULE.

Unfortunately, the celery management command was a feature of the django-celery library, which isn't supported by Celery 4.x. As of yet, I haven't found a solution for Celery 4.x.

2

If you are using Gevent for Celery do not forget to check Preferences > Build, Execution, Deployment > Python Debugger > Gevent Compatible checkbox.

1
  • Life-saving comment
    – vicusbass
    Dec 4, 2020 at 13:17
2

Adding to @Assam answer:

on linux\ubuntu simply choosing the "Script Path -> Module name" = celery did the trick for me

[1]: https://stackoverflow.com/users/149556/assem

2

I have PyCharm 2021.3(Professional Edition) and my library:

  • Python==3.8.9
  • celery==5.2.7

I use macOS M1. I followed the below steps for this problem:

  1. There are Edit Configurations.. in the option Run.
  2. Click '+' button which figured showed blue arrow and add new configuration.
  3. Add Python configuration.
  4. Select configuration Module name instead of Script Path as showed red arrow in Figure.
  5. Add Parameters like --workdir=/path/to/your_proj_direction -A your_proj.dir.celery_app worker -l info. It need to --workdir parameter for celery handle path. Except this, you write any script to start celery worker without celery word at the beginning.
  6. Add Python interpreter as your proj interpreter.
  7. Add Working directory to your virtual env path which installed celery library.
  8. Click Apply.
  9. You can use normal Debug PyCharm this configuration. İf you run configuration as Debug, PyCharm will start celery worker and you can debug.

Figure: Run/Debug Configurations

1

I'm using PyCharm 2017 and had to do something very similar to the answers above, but I specifically had to put the full/absolute path name to celery in the "Script" field

Also, I'm not sure if PyCharm 4 has this feature, but newer versions allow you to attach directly to a running python process by going to Run > Attach to Local Process...

This allows you to run celery however you were before (in the terminal, perhaps) then allow Pycharm to take over

1

in Windows , Add the following params in your debugging conf in Pycharm

-A YouAppName worker  --loglevel=debug -P solo --without-gossip --without-mingle --without-heartbeat
1

Example of configuration settings for celery on mac when using venv.

Where parametrapp is name of Celery object from your python code

app = Celery('project')

Configuration settings celery for mac

0

My working configuration:

  • Script: /home/app/env/bin/celery
  • Script parameters: worker -B -n qrc -Q qrc -l info --app=backend.celery

    • Where -B is for celerybeet, -n is node name, -Q is queue name, -l is log level, and --app is the app name, a django app with celery.py next to settings.py in my case.
  • Working directory: /home/app/server/ ie my django root folder

1
  • How do you start the web server and the celery worker at the same time? Dec 3, 2022 at 17:10

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.