73

I'm trying to find a way to implement both a custom QuerySet and a custom Manager without breaking DRY. This is what I have so far:

class MyInquiryManager(models.Manager):
    def for_user(self, user):
        return self.get_query_set().filter(
                    Q(assigned_to_user=user) |
                    Q(assigned_to_group__in=user.groups.all())
                )

class Inquiry(models.Model):   
    ts = models.DateTimeField(auto_now_add=True)
    status = models.ForeignKey(InquiryStatus)
    assigned_to_user = models.ForeignKey(User, blank=True, null=True)
    assigned_to_group = models.ForeignKey(Group, blank=True, null=True)
    objects = MyInquiryManager()

This works fine, until I do something like this:

inquiries = Inquiry.objects.filter(status=some_status)
my_inquiry_count = inquiries.for_user(request.user).count()

This promptly breaks everything because the QuerySet doesn't have the same methods as the Manager. I've tried creating a custom QuerySet class, and implementing it in MyInquiryManager, but I end up replicating all of my method definitions.

I also found this snippet which works, but I need to pass in the extra argument to for_user so it breaks down because it relies heavily on redefining get_query_set.

Is there a way to do this without redefining all of my methods in both the QuerySet and the Manager subclasses?

3
  • 2
    Warning: The selected answer by T.Stone results in a severe performance penalty (from millisecond response times to multi-second responses) when .defer or .only methods are used. For example, in Django 1.3 a query such as: MyModel.objects.only('some_field').get(id=1) => returns in 3.7ms but, add the CustomManager as described above, and I get: MyModel.objects.only('some_field').get(id=1) => returns in ~ 357ms May 9, 2011 at 4:33
  • Has anybody else reproduced this? What about with Django 1.4?
    – fletom
    Jul 23, 2012 at 18:32
  • Okay. But why and how does this happen? Are the queries different, or did you profile that operation, without actually hitting the database? Sep 6, 2012 at 14:45

8 Answers 8

63

The Django 1.7 released a new and simple way to create combined queryset and model manager:

class InquiryQuerySet(models.QuerySet):
    def for_user(self, user):
        return self.filter(
            Q(assigned_to_user=user) |
            Q(assigned_to_group__in=user.groups.all())
        )

class Inquiry(models.Model):
    objects = InqueryQuerySet.as_manager()

See Creating Manager with QuerySet methods for more details.

2
  • 5
    This is the best way to do it, but it'd be got to exemplify how the for_user method should take a user and return self.[...] to chain together multiple operations. Aug 19, 2015 at 19:19
  • Funny how this is not he first answer here. Professionally this is the cleanest solution. Mar 12, 2021 at 12:39
57

Django has changed! Before using the code in this answer, which was written in 2009, be sure to check out the rest of the answers and the Django documentation to see if there is a more appropriate solution.


The way I've implemented this is by adding the actual get_active_for_account as a method of a custom QuerySet. Then, to make it work off the manager, you can simply trap the __getattr__ and return it accordingly

To make this pattern re-usable, I've extracted out the Manager bits to a separate model manager:

custom_queryset/models.py

from django.db import models
from django.db.models.query import QuerySet

class CustomQuerySetManager(models.Manager):
    """A re-usable Manager to access a custom QuerySet"""
    def __getattr__(self, attr, *args):
        try:
            return getattr(self.__class__, attr, *args)
        except AttributeError:
            # don't delegate internal methods to the queryset
            if attr.startswith('__') and attr.endswith('__'):
                raise
            return getattr(self.get_query_set(), attr, *args)

    def get_query_set(self):
        return self.model.QuerySet(self.model, using=self._db)

Once you've got that, on your models all you need to do is define a QuerySet as a custom inner class and set the manager to your custom manager:

your_app/models.py

from custom_queryset.models import CustomQuerySetManager
from django.db.models.query import QuerySet

class Inquiry(models.Model):
    objects = CustomQuerySetManager()

    class QuerySet(QuerySet):
        def active_for_account(self, account, *args, **kwargs):
            return self.filter(account=account, deleted=False, *args, **kwargs)

With this pattern, any of these will work:

>>> Inquiry.objects.active_for_account(user)
>>> Inquiry.objects.all().active_for_account(user)
>>> Inquiry.objects.filter(first_name='John').active_for_account(user)

UPD if you are using it with custom user(AbstractUser), you need to change
from

class CustomQuerySetManager(models.Manager):

to

from django.contrib.auth.models import UserManager

class CustomQuerySetManager(UserManager):
    ***
7
  • can you decide what should be done with stackoverflow.com/edit-suggestions/1216 Feb 1, 2011 at 4:19
  • Stone you are going to have to edit it in yourself, there is no way to take an edit after it was declined Feb 7, 2011 at 23:14
  • 4
    WARNING: I tried this method and discovered that it severely slows down .defer and .only calls. May 9, 2011 at 4:20
  • I've updated this response to a community wiki. Those with performance optimizations can adjust the code as necessary.
    – T. Stone
    Jan 6, 2012 at 17:17
  • 2
    probably similar to PassThroughManager from pypi.python.org/pypi/django-model-utils
    – Jelko
    Aug 2, 2013 at 7:41
14

You can provide the methods on the manager and queryset using a mixin.

This also avoids the use of a __getattr__() approach.

from django.db.models.query import QuerySet

class PostMixin(object):
    def by_author(self, user):
        return self.filter(user=user)

    def published(self):
        return self.filter(published__lte=datetime.now())

class PostQuerySet(QuerySet, PostMixin):
    pass

class PostManager(models.Manager, PostMixin):
    def get_query_set(self):
        return PostQuerySet(self.model, using=self._db)
11

You can now use the from_queryset() method on you manager to change its base Queryset.

This allows you to define your Queryset methods and your manager methods only once

from the docs

For advanced usage you might want both a custom Manager and a custom QuerySet. You can do that by calling Manager.from_queryset() which returns a subclass of your base Manager with a copy of the custom QuerySet methods:

class InqueryQueryset(models.Queryset):
    def custom_method(self):
        """ available on all default querysets"""

class BaseMyInquiryManager(models.Manager):
    def for_user(self, user):
        return self.get_query_set().filter(
                    Q(assigned_to_user=user) |
                    Q(assigned_to_group__in=user.groups.all())
                )

MyInquiryManager = BaseInquiryManager.from_queryset(InquiryQueryset)

class Inquiry(models.Model):   
    ts = models.DateTimeField(auto_now_add=True)
    status = models.ForeignKey(InquiryStatus)
    assigned_to_user = models.ForeignKey(User, blank=True, null=True)
    assigned_to_group = models.ForeignKey(Group, blank=True, null=True)
    objects = MyInquiryManager()
1
  • I like your solution ! as all methods defined in custom-queryset are available to manager. :-)
    – shahjapan
    Sep 15, 2021 at 17:22
2

A slightly improved version of T. Stone’s approach:

def objects_extra(mixin_class):
    class MixinManager(models.Manager, mixin_class):
        class MixinQuerySet(QuerySet, mixin_class):
            pass

        def get_query_set(self):
            return self.MixinQuerySet(self.model, using=self._db)

    return MixinManager()

Class decorators make usage as simple as:

class SomeModel(models.Model):
    ...
    @objects_extra
    class objects:
        def filter_by_something_complex(self, whatever parameters):
            return self.extra(...)
        ...

Update: support for nonstandard Manager and QuerySet base classes, e. g. @objects_extra(django.contrib.gis.db.models.GeoManager, django.contrib.gis.db.models.query.GeoQuerySet):

def objects_extra(Manager=django.db.models.Manager, QuerySet=django.db.models.query.QuerySet):
    def oe_inner(Mixin, Manager=django.db.models.Manager, QuerySet=django.db.models.query.QuerySet):
        class MixinManager(Manager, Mixin):
            class MixinQuerySet(QuerySet, Mixin):
                pass

            def get_query_set(self):
                return self.MixinQuerySet(self.model, using=self._db)

        return MixinManager()

    if issubclass(Manager, django.db.models.Manager):
        return lambda Mixin: oe_inner(Mixin, Manager, QuerySet)
    else:
        return oe_inner(Mixin=Manager)
2
  • That's pretty awesome, they have such decorator in the django itself Jan 13, 2016 at 15:03
  • 1
    My Django wants get_queryset to be overriden, not get_query_set. Jan 13, 2016 at 15:17
2

based on django 3.1.3 source code, i found a simple solution

from django.db.models.manager import BaseManager

class MyQuerySet(models.query.QuerySet):
      def my_custom_query(self):
          return self.filter(...)

class MyManager(BaseManager.from_queryset(MyQuerySet)):
     ...

class MyModel(models.Model):
     objects = MyManager()
0

There are use-cases where we need to call custom QuerySet methods from the manager instead of using the get_manager method of a QuerySet.

A mixin would suffice based on the solution posted in one of the accepted solution comments.

class CustomQuerySetManagerMixin:
    """
    Allow Manager which uses custom queryset to access queryset methods directly.
    """
    def __getattr__(self, name):
        # don't delegate internal methods to queryset
        # NOTE: without this, Manager._copy_to_model will end up calling
        # __getstate__ on the *queryset* which causes the qs (as `all()`)
        #  to evaluate itself as if it was being pickled (`len(self)`)
        if name.startswith('__'):
            raise AttributeError
        return getattr(self.get_queryset(), name)

For example,

class BookQuerySet(models.QuerySet):
    def published(self):
        return self.filter(published=True)

    def fiction(self):
        return self.filter(genre="fiction")

    def non_fiction(self):
        return self.filter(genre="non-fiction")

class BookManager(CustomQuerySetManagerMixin, models.Manager):
    def get_queryset(self):
        return BookQuerySet(self.model, using=self._db).published()

class Book(models.Model):
    title = models.CharField(max_length=200)
    genre = models.CharField(choices=[('fiction', _('Fiction')), ('non-fiction', _('Non-Fiction'))])
    published = models.BooleanField(default=False)
    author = models.ForeignKey(Author, on_delete=models.CASCADE, related_name="books")

    objects = BookManager()

class Author(models.Model):
    name = models.CharField(max_length=200)

With the above, we can access related objects (Book) like below without defining new methods in the manager for each queryset method.

fiction_books = author.books.fiction()
-1

The following works for me.

def get_active_for_account(self,account,*args,**kwargs):
    """Returns a queryset that is 
    Not deleted
    For the specified account
    """
    return self.filter(account = account,deleted=False,*args,**kwargs)

This is on the default manager; so I used to do something like:

Model.objects.get_active_for_account(account).filter()

But there is no reason it should not work for a secondary manager.

1
  • 3
    Try doing a filter, then using get_active_for_account. It works in your example, but not once you've already used a filter, and are then working with a QuerySet, which was my example.
    – Jack M.
    Jan 29, 2010 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.