2

I am try to configure celery with django and using 3-tier project layout. I've already run the syncdb and migration commands. But I get some errors when I start the worker. Can someone please help me? Thanks alot.

Here is my Project Layout

- root/
    - src/
        - project/
            - __init__.py <<< celery imports added
            - urls.py
            - celery.py
        - core/
            tasks.py
        - manage.py

Commands I run

    $ cd root
    $ celery -A project --workdir=src/ worker -l info

Edit ->>> celery.py. Here is my INSTALLED_APP settings.

os.environ.setdefault('DJANGO_SETTINGS_MODULE',
                      'project.settings.local')

app = Celery('project', broker='django://')
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

CELERYBEAT_SCHEDULE = {
    'every-hour': {
        'task': 'core.tasks.sync_currency_rates',
        'schedule': crontab(minute='*/1')
    },
}

CELERY_TIMEZONE = 'UTC'
CELERY_ALWAYS_EAGER = True

Then I am getting following error.

-------------- [email protected] v3.1.16 (Cipater)
---- **** ----- 
--- * ***  * -- Darwin-13.4.0-x86_64-i386-64bit
-- * - **** --- 
- ** ---------- [config]
- ** ---------- .> app:         actionsuite:0x10f929890
- ** ---------- .> transport:   django://localhost//
- ** ---------- .> results:     disabled
- *** --- * --- .> concurrency: 4 (prefork)
-- ******* ---- 
--- ***** ----- [queues]
 -------------- .> celery           exchange=celery(direct) key=celery


[tasks]
  . core.tasks.sync_currency_rates

[2014-10-07 18:52:17,662: INFO/MainProcess] Connected to django://localhost//
[2014-10-07 18:52:17,678: ERROR/MainProcess] Unrecoverable error: OperationalError('no such table: djkombu_queue',)
Traceback (most recent call last):
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/celery/worker/__init__.py", line 206, in start
    self.blueprint.start(self)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/celery/bootsteps.py", line 123, in start
    step.start(parent)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/celery/bootsteps.py", line 374, in start
    return self.obj.start()
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/celery/worker/consumer.py", line 278, in start
    blueprint.start(self)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/celery/bootsteps.py", line 123, in start
    step.start(parent)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/celery/worker/consumer.py", line 608, in start
    c.connection, on_decode_error=c.on_decode_error,
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/celery/app/amqp.py", line 375, in __init__
    **kw
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/kombu/messaging.py", line 359, in __init__
    self.revive(self.channel)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/kombu/messaging.py", line 371, in revive
    self.declare()
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/kombu/messaging.py", line 381, in declare
    queue.declare()
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/kombu/entity.py", line 505, in declare
    self.queue_declare(nowait, passive=False)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/kombu/entity.py", line 531, in queue_declare
    nowait=nowait)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/kombu/transport/virtual/__init__.py", line 446, in queue_declare
    self._new_queue(queue, **kwargs)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/kombu/transport/django/__init__.py", line 36, in _new_queue
    Queue.objects.get_or_create(name=queue)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/models/manager.py", line 92, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/models/query.py", line 422, in get_or_create
    return self.get(**lookup), False
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/models/query.py", line 351, in get
    num = len(clone)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/models/query.py", line 122, in __len__
    self._fetch_all()
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/models/query.py", line 966, in _fetch_all
    self._result_cache = list(self.iterator())
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/models/query.py", line 265, in iterator
    for row in compiler.results_iter():
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 700, in results_iter
    for rows in self.execute_sql(MULTI):
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
    cursor.execute(sql, params)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
    return super(CursorDebugWrapper, self).execute(sql, params)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/utils.py", line 94, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/Users/sandhu/Envs/allout/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 485, in execute
    return Database.Cursor.execute(self, query, params)
OperationalError: no such table: djkombu_queue

EDIT 2 ->

I notice one more thing. When I run the celery -A ... command then It creates db.sqlite in src(This is wrong). When I execute syncdb command that is creating db.sqlite file in the root. Which is fine

11
  • could you share your INSTALLED_APPS setting? it could be because kombu.transport.django is not added to it
    – srj
    Commented Oct 7, 2014 at 19:18
  • @srj kombu.transport.django is added in kombu.transport.django.
    – Shah
    Commented Oct 7, 2014 at 19:21
  • please share the INSTALLED_APPS. How do you create the sql tables? via syncdb?
    – srj
    Commented Oct 7, 2014 at 19:22
  • @srj Yes sql tables are created with syncdb. I notice one more thing. When I run the celery -A ... command then It creates db.sqlite in src(This is wrong). When I execute syncdb command that is creating db.sqlite file in the root. Which is fine.
    – Shah
    Commented Oct 7, 2014 at 19:26
  • The app isn't in the settings.local[INSTALLED_APPS].
    – xbello
    Commented Oct 7, 2014 at 19:26

1 Answer 1

1

The celery executable is using the SQLite database at root, while Django uses another SQLite at project/settings/. Fix the path to the SQLite DB in settings/local.py with absolute paths:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(os.path.dirname(__file__), "my_database.sq3"),
    }
}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.