Questions tagged [celery]

Celery is a distributed task queue framework for Python, used for asynchronous and parallel execution.

Filter by
Sorted by
Tagged with
0 votes
0 answers
2 views

Celery task executed always but only returning if condition is met?

I'm working with Python 3 and Celery (v5.3.6). I have a workflow that looks something like... part1_task part2_task Both Part1 and Part2 hit different external APIs, and the time they take to finish ...
Julián Pérez's user avatar
0 votes
0 answers
28 views

Celery Beat schedules multiple tasks but only runs the first one

I'm facing an issue with Celery where I have multiple tasks scheduled in Celery Beat, but only the one task(notify_after_sobes) is being executed. In the Celery Beat logs, I can see that all the tasks ...
solovey1's user avatar
0 votes
0 answers
10 views

Celery Python - Run A Chord Callback Only After All Tasks In Body Have Finished All Retries Or Succeeded

I'm currently using celery with rabbitMQ to handle decentralization of message queues. I'm facing a bit of an issue, where sometimes the network im on may fall for an hour or so. I want to be able to ...
Gambizon's user avatar
0 votes
0 answers
7 views

Delayed execution of task causing empty task getting redelivered in queue

I have celery verison 5.1.2 (sun-harmonics) and using rabbitmq for broker. Celery task are getting redelivered to queue if there is delay in processing celery task. Celery queues are getting build up ...
Ajay Prajapati's user avatar
1 vote
0 answers
33 views

Connecting to Amazon SQS key (SSE-SQS) Encrypted SQS Queue via Django Celery [sqs]

I am unable to get my Celery working running while connecting to an SQS Encrypted Queue This is my settings.py for my django project SQS_AWS_ACCESS_KEY_ID = 'xxxx' SQS_AWS_SECRET_ACCESS_KEY = 'xxxx' ...
stackoverflow1234's user avatar
0 votes
0 answers
13 views

django celery tasks are executed before establishing a websocket connection

The user uploads the file to the form, after which the form with the file is sent to the server, and upon successful validation, I perform the following actions: def form_valid(self, form): ...
Mikhailo's user avatar
1 vote
1 answer
59 views

db.session.commit inside celery task: "Outside of request context"

I am trying to use celery to modify an SQLalchemy table. The task correctly prints into the console the target record, but upon committing I get the below error. Any ideas what I am missing? Many ...
Benjamin's user avatar
0 votes
0 answers
27 views

Celery returning error that task is not registered

I have been trying to solve this issue for two days now. Everything seems to be working when I run the task manually, and also celery beat and worker are working, but the issue is every time the ...
Nas's user avatar
  • 63
0 votes
0 answers
37 views

Celery instead of Async views with asyncio in django

I am new to django! (So basically i am seeking suggestions for this approach). In my project i had so many views and each view had at least 1 database i/o and as all my views are synchronous i find ...
varun_k_21's user avatar
0 votes
0 answers
13 views

Django Progress bar and results backend not updating even though the celery worker shows updates?

I'm new to using Celery with Django and using RabbitMQ as a message broker, and I'm having trouble being able to get a progress bar of sorts working. I have a function in the views.py, called ...
Jaimee-lee Lincoln's user avatar
0 votes
0 answers
22 views

how to deal with resources for celery docker

I am building python web app using dash and for long running tasks, I am using celery.The application is dockerized with following services: docker-compose.yaml db: # postgres image redis: # redis ...
winter's user avatar
  • 637
0 votes
0 answers
16 views

Celery SQS Consumer Issue: Visibility Timeout and Message Handling

I've implemented a Celery SQS consumer application with a visibility timeout of 3600 seconds configured in the broker_transport_options. The consumer processes messages from an SQS queue produced by ...
Dinesh Kumar's user avatar
1 vote
1 answer
52 views

Can't start django project in docker

At startup Redis does not work, therefore caching does not work, as I realized there is no connection to the database. I can't make loaddata fixtures and Celery can't connect to Redis either. In ...
Alexander Emelyanov's user avatar
-1 votes
0 answers
15 views

Problem reading State in DASH @long_callback

I am struggling to read State values in the @long_callback. Getting errors missing 2 required positional arguments @app.long_callback( output=[ Output('O_A', 'figure') ], inputs=[...
wasif's user avatar
  • 83
0 votes
0 answers
12 views

Can't delete Celery Task

I'm having a very weird Issue, I mounted a Django app with Docker, and added a Redis DB to implement Celery tasks, now, yesterday everything was working perfectly, but today I created a new task to ...
Efraín's user avatar
  • 453
0 votes
0 answers
36 views

Celery progress bar isn`t being updated

i am developing some Django app, which generates reports. I am using celery to enable progress bar after hitting 'submit' button. The problem is that I can't figure out what i am missing and why ...
Danylo Polishchuk's user avatar
0 votes
0 answers
32 views

LocalExecutor or CeleryExecutor for Airflow on a single machine?

My question may sound dumb, but we have a ETL process running in a remote machine (EC2) and as we only use it internally in the company and there are only 5 of us, we set up Airflow with Docker ...
WLD's user avatar
  • 17
0 votes
1 answer
23 views

How can I solve the error "ImportError: cannot import name ‘_request_ctx_stack’ from ‘flask’"?

When I am running a flask app with celery and redis, an importerror in appearing on the command prompt screen. ImportError: cannot import name ‘_request_ctx_stack’ from ‘flask I updated the versions ...
Raya_2002's user avatar
0 votes
1 answer
27 views

AttributeError: 'Logger' object has no attribute 'warn'

I'm trying to connect celery to my django project via docker. But when starting the worker container, I get the following error - File "/usr/local/lib/python3.13/site-packages/kombu/transport/...
Mikhailo's user avatar
0 votes
0 answers
39 views

How to keep scheduled jobs of celery worker in docker container? (django, celery, redis, docker)

I deploy my django app with docker. When I update my deployment, I stop current running container and start new container with new image. I start my celery worker in the created container. (exec -> ...
SSS's user avatar
  • 35
0 votes
0 answers
30 views

Wait until subtask is ready (celery + unittest)

I have a task which called by parent task. @shared_task def task1(): if condition: task2.apply_async() @shared_task def task2(): pass I want to test all flow starting by calling ...
hungry7's user avatar
  • 49
0 votes
0 answers
21 views

the .delay() method fails as celery is unable to connect to mongodb(aws document db)in kubernates

Python Version: 3.7 Celery Version: 5.2 below is the code being used # app.py (Flask application) from flask import Flask, jsonify from celery import Celery import db_connectors import os from ...
Just another object's user avatar
0 votes
0 answers
12 views

What is difference between Celery Beats, Celery worker, Threads and Multiprocessors?

What is the difference between:- Celery beats and Celery Worker, and if Celery worker works just like threads or multiprocessors, why do we use celery worker especially? Thank you. I tried working ...
user avatar
0 votes
0 answers
9 views

Unable to Terminate Django 2.1 Project Running on Ubuntu 20.04.2 LTS with NGINX, UWSGI, and Celery

I have a test project (a 'zombie' project) on the same server as the production version of my app. Both projects use Celery for background tasks, but they share the same RabbitMQ port. I noticed that ...
samuelSGDev's user avatar
0 votes
0 answers
33 views

How to retry failed tasks in celery chunks or celery starmap

I would like to know how to configure these tasks correctly so that the failed tasks can be retried. One thing I have noted is if one of the tasks in the chunk fails, the whole chunk (I assume ...
Eric O.'s user avatar
  • 503
0 votes
0 answers
87 views

Celery Flower UNAUTHENTICATED_API not working

I am working on a project using celery workers and Flower to monitor those workers. Both the worker and Flower are running as images inside one container. I want to change attributes of the worker ...
Swift's user avatar
  • 81
0 votes
0 answers
21 views

Celery Emails and Accessing Mailbox

I'm in the process of moving my emails over to Celery tasks to run asychronously. Porting over my existing tests I'm struggling with an AssertionError when I attempt to access the mail.outbox. I ...
user1387363's user avatar
0 votes
0 answers
19 views

Celery and Flask - RuntimeError('Working outside of application context)

In my flask application, I have app/celery_utils.py: from celery import Celery celery = Celery(__name__, broker='redis://localhost:6379/0', include=['backend....
8-Bit Borges's user avatar
  • 9,744
0 votes
0 answers
26 views

how to use redis in cpanel for python (django) module celery for sending email asynchrnously

I have used celery for sending email asynchronous task I use Redis in localhost for change password but when I want use it for send email asynchronous task in server Cpanel it gives error like :- ...
kartik prajapati's user avatar
0 votes
0 answers
16 views

Making Downloaded Files a Parameter for Subsequent Functions in Django, Celery, and Digital Ocean Spaces Integration

How can I utilize Django with Celery to handle a scenario where a function downloads a file from Digital Ocean Spaces, and once the download is complete, I want to pass the downloaded file as a ...
Jc John's user avatar
  • 1,809
0 votes
0 answers
28 views

Celery Task ECS Termination Issue - Need Help Updating Decorator for Handling ProtectionEnabled State Changes

Explanation: I have a Django application where I am running multiple Celery tasks on AWS Elastic Container Service (ECS), using SQS as the broker. I am encountering an issue where the Celery tasks are ...
Adrian's user avatar
  • 745
2 votes
0 answers
31 views

Celery task starts running before the flask app starts

The celery task is starting even before the flask app is running. I want to start a celery task when I make API call to a certain endpoint (eg: "/report"). So I started using celery for this....
Adithya sakaray's user avatar
0 votes
0 answers
28 views

Celery task result persistent strategy

I have a web server, a rabbitMQ server and two task processing servers in the system. Using Celery to manage the tasks. The web server needs to query the status of the tasks. Based on Celery's doc, I ...
Sam's user avatar
  • 1
0 votes
0 answers
36 views

Celery Tasks not Running on Mac M2 (Process 'ForkPoolWorker' exited with 'signal 11 (SIGSEGV)')

I am encountering an issue while trying to run Celery tasks on my Mac M1 machine. The error message I'm getting is as follows: The process has forked and you cannot use this CoreFoundation ...
Farhad's user avatar
  • 115
0 votes
1 answer
46 views

Celery with postgresql giving error "can't adapt type 'AsyncResult'"

I am using postgresql as my backend for Celery (v5.3.5). Celery is returning an SQL error when I call ready() on the task ASyncResult: sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) can'...
Giles's user avatar
  • 1,637
1 vote
1 answer
24 views

Do celery have backfill and catch up feature built in if not than how can we implement these features with celery?

Do celery have backfill and catch up feature built in if not than how can we implement these features with celery? Can we execute our job in past date using celery or can we catch up using celery? ...
Akash Jain's user avatar
0 votes
1 answer
38 views

How to configure celery with SQS as backend?

I'm trying to setup a SQS broker with a celery app, configured in a django project. Here's my setup: celery.py: import os from celery import Celery os.environ.setdefault("DJANGO_SETTINGS_MODULE&...
koleror's user avatar
  • 338
0 votes
0 answers
18 views

How to prevent overlapping tasks, that executing to schedule, in django celery beat?

I'm constructing some weather reminder API, where users can register, and take JWT token, for using api endpoints, such create subscriptions, with period of notifications, and adding cities to that ...
Vitaly Milestone's user avatar
0 votes
0 answers
23 views

Celery redis reduce freqeuncy of BRPOP messages

I'm running my worker with the following arguments --without-gossip --without-mingle --without-heartbeat with Redis as my celery broker I can see my redis instance is frequently receiving messages (...
Jack Evans's user avatar
  • 1,707
0 votes
0 answers
19 views

How to restart celery beat container or resolve KeyError(b'entries')?

My ecs fargate celery beat container is running well during some hours after deployment. But some times later, error raised like below Traceback (most recent call last): File "/usr/local/lib/...
user16613127's user avatar
0 votes
1 answer
62 views

Load and unload model on celery worker

I currently have a system that initiates tasks on whisper AI model using Celery. However, the existing setup involves loading the model inside each task, which is suboptimal due to the repeated ...
UnPapeur's user avatar
0 votes
0 answers
49 views

How to run celery workers with sequential task execution with "per-child" bounds

The context is that I'm using kubernetes for parallelism and workload balancing, similar to this. Given this, it is desirable to have no concurrency within the celery workers -- we get that by k8s ...
Dave's user avatar
  • 7,685
0 votes
0 answers
15 views

FileNotFoundError: [Errno 2] No such file or directory in celery container while exists in django application container

I am developing a django application in which I have to manipulate excel files. I have two containers: one for my main django application and another for celery where I process my files. These two ...
leauradmin's user avatar
0 votes
1 answer
85 views

Adding Celery settings - what does it do?

We upgrade Celery and started to get a warning: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine whether broker connection retries are made ...
D M's user avatar
  • 23
1 vote
2 answers
47 views

Django Celery Results Table Missing periodictaskname and taskname Fields

I have set up Django with Celery, using django_celery_results and django_celery_beat to schedule tasks and store the results. The tasks are running successfully, but I've encountered an issue with the ...
Ujwal Bhargav's user avatar
0 votes
1 answer
26 views

Celery concurrency configuration for long tasks

I'm looking for the right configuration of celery to handle long tasks executed in django. These are the keys of the case: the tasks take several minutes to complete, but the processing is done in a ...
klautern's user avatar
  • 139
0 votes
0 answers
16 views

Apache Superset 3.0.1 Sending Alert/Report in GMT TimeZone

In Apache Superset 3.0.1, I want to scheduled Alert/Report and get the email in basis of GMT timezone. Currently, it is working on UTC format only. Package Version Apache Superset 3.0.1 Celery 5.3.5 ...
Demonstration Nest's user avatar
0 votes
0 answers
12 views

Celery "unregistered task" error only on Heroku

My Flask app with celery works well locally on Mac. But on Heroku, the celery worker raises the error Received unregistered task of type 'app.task_func'. I am sure that the same commit of Git has been ...
Light Yagmi's user avatar
  • 5,145
0 votes
0 answers
40 views

Celery inspect remove 10 items limit

I need to get the worker queue size using python and Celery. The code works fine, but result is limited to 10 items. Is there any way to get entire list of queued tasks? I get 100+ items, but only 10 ...
Artem Kopytko's user avatar
-1 votes
1 answer
33 views

Problem communicating from celery container to redis

I'm Dockerizing my old project, but I came across this error: > Traceback (most recent call last):File > "/venv/lib/python3.10/site-packages/redis/connection.py", line 624, in > ...
Matheus Loeblein's user avatar

1
2 3 4 5
181