55

I have a Django website, and one page has a button (or link) that when clicked will launch a somewhat long running task. Obviously I want to launch this task as a background task and immediately return a result to the user. I want to implement this using a simple approach that will not require me to install and learn a whole new messaging architecture like Celery for example. I do not want to use Celery! I just want to use a simple approach that I can set up and get running over the next half hour or so. Isn't there a simple way to do this in Django without having to add (yet another) 3rd party package?

4
  • You could just have the server return a Request Received type of response. Then if polling or websockets aren't an option, just have the server update a percent complete value maybe every 10% or 25%. And have a designated area for the user to check any running processes, and display the percent complete. This way the browser is only hitting the server when the user goes to or refreshes the running process page. Then if a process has 100% completion have a link to the results. Just some ideas.
    – Ryan G
    Feb 21, 2014 at 21:28
  • Note: The server updating the percent complete value is only on the server [eg a DB value]. This can be queried if the user goes to the given page. Also this will depend on how your infrastructure can handle concurrent connections. The running process may block, so you may need to look into something like tornado.
    – Ryan G
    Feb 21, 2014 at 21:30
  • 4
    I doubt very much that anything you write yourself is going to be simpler than just installing Celery and having done with it. Feb 21, 2014 at 21:36
  • 3
    "I doubt very much that anything you write yourself is going to be simpler than just installing Celery and having done with it." - let's see - I can think of a lot of things simpler than Celery. It's a great bit of software but it's a large glob of code that can handle large scale, high-volume distributed tasks. It requires separate installs of a key-value store, a broker and god knows what else. If you just want to run a background task then I would look for something else.
    – Andy Baker
    Dec 12, 2014 at 18:15

3 Answers 3

67

Just use a thread.

import threading

t = threading.Thread(target=long_process,
                            args=args,
                            kwargs=kwargs)
t.setDaemon(True)
t.start()
return HttpResponse()

See this question for more details: Can Django do multi-thread works?

2
  • 9
    I think this needs more visibility. I recently built a project around Celery and RabbitMQ when I really should have just been using threading. I built a minimal example of this here: github.com/nbwoodward/django-async-threading
    – nbwoodward
    Nov 15, 2018 at 18:39
  • It works great, just for confirmation ...does it have any drawbacks? Jun 4 at 10:55
18

Have a look at django-background-tasks - it does exactly what you need and doesn't need any additional services to be running like RabbitMQ or Redis. It manages a task queue in the database and has a Django management command which you can run once or as a cron job.

10
  • 4
    While this link may answer the question, it is better to include the essential parts of the answer here and provide the link for reference. Link-only answers can become invalid if the linked page changes. - From Review
    – Rao
    Mar 4, 2016 at 4:04
  • 3
    @Rao The link points to a module which solves the problem, not a description of how to solve the problem. I wouldn't know how else to rewrite the post, maybe you have some tips?
    – user226114
    Mar 4, 2016 at 9:31
  • 3
    This module currently only works with Django 1.4 through 1.8.
    – aboutaaron
    Aug 11, 2016 at 21:46
  • 3
    We pushed a more stable and robust version (1.1.6) of django-background-tasks to the Python Package Index today. This has support for the latest greatest Django version as well. Release notes: github.com/arteria/django-background-tasks/releases/tag/v1.1.6
    – phi
    Mar 29, 2017 at 12:53
  • 1
    Does anybody know how to fire a background job immediately in a view, (but not wait for it to complete) using django-background-tasks?
    – konstunn
    May 15, 2019 at 10:44
8

If you're willing to install a 3rd party library, but you want something a whole lot simpler than Celery, check out Redis Queue. It does require Redis, which is pretty easy in itself, but that can provide a lot of other benefits as well.

RQ itself has almost zero configuration. It's startlingly simple.

References:

2
  • 3
    This question is years old at this point, but I have now been using RQ for many such tasks. It really is startlingly simple and works like a charm. Highly recommended.
    – Marc
    Jan 31, 2019 at 0:43
  • 1
    Not sure how this "accepted answer" answers the question "Isn't there a simple way to do this in Django without having to add (yet another) 3rd party package?", but the project looks neat indeed Aug 26, 2021 at 18:03

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.