51

I have a QuerySet, let's call it qs, which is ordered by some attribute which is irrelevant to this problem. Then I have an object, let's call it obj. Now I'd like to know at what index obj has in qs, as efficiently as possible. I know that I could use .index() from Python or possibly loop through qs comparing each object to obj, but what is the best way to go about doing this? I'm looking for high performance and that's my only criteria.

Using Python 2.6.2 with Django 1.0.2 on Windows.

2
  • Just for an update, first of all, query set is unordered. So, the index may vary for different iteration. You need to do order_by over any field and then following Vinay's answer will help in case if you just need the index.
    – Babu
    Sep 14, 2012 at 5:55
  • If we have 10 million records it is hard operation for database.
    – alexche8
    Jul 25, 2014 at 16:23

6 Answers 6

67

If you're already iterating over the queryset and just want to know the index of the element you're currently on, the compact and probably the most efficient solution is:

for index, item in enumerate(your_queryset):
    ...

However, don't use this if you have a queryset and an object obtained by some unrelated means, and want to learn the position of this object in the queryset (if it's even there).

2
  • 1
    this is not efficient and there is a memory leaking issue since you are looping through every single object in your table from your python code. the most efficient answer is to use a single DB query to get the index which is Richard's answer
    – Amin_mmz
    Dec 5, 2020 at 18:38
  • @Amin_mmz This depends on what the poster had wanted. If they're iterating over the queryset, have an object and wonder about its position - enumerate is all they need. That was my understanding of the question, and this is why there is no loop body. However, I see that it can be interpreted differently. If they have a queryset and an object they believe to be contained in there - you are correct that iterating over the queryset is an awful idea, and Richard's answer is the correct one. An even better one could be a query that would use SQL ROW_NUMBER() window function.
    – drdaeman
    Jan 5, 2021 at 3:30
54

If you just want to know where you object sits amongst all others (e.g. when determining rank), you can do it quickly by counting the objects before you:

    index = MyModel.objects.filter(sortField__lt = myObject.sortField).count()
4
  • 4
    In this case you would also want the sortField to be unique, because otherwise you would get the position of the first element with sortField equal to myObject.sortField.
    – naktinis
    Mar 22, 2013 at 12:58
  • 2
    Should it not be .count() - 1? For example, if there is 1 object in the queryset, the index is 0. Jan 29, 2018 at 19:21
  • 1
    @DavidD. in that case, the expression will give 0 as the count since the filter is using "less than" not "less than or equal".
    – C S
    Apr 25, 2018 at 17:33
  • This would be perfect if only sortfield is guaranteed to be in incremental/decremental order (in which case, an expected sequence). Doing this over something like an id can yeild unexpected results.
    – twohot
    Dec 23, 2022 at 11:05
25

Assuming for the purpose of illustration that your models are standard with a primary key id, then evaluating

list(qs.values_list('id', flat=True)).index(obj.id)

will find the index of obj in qs. While the use of list evaluates the queryset, it evaluates not the original queryset but a derived queryset. This evaluation runs a SQL query to get the id fields only, not wasting time fetching other fields.

2
  • 1
    This may be more efficient or it may be less efficient than just evaluating the original queryset, depending on whether you were going to do that anyway later on (since querysets cache their results).
    – Carl Meyer
    Jun 25, 2009 at 16:47
  • 1
    There is a problem with this method. qs.values_list('id', flat=True) return different values in each step. You obtain different index with the same query.
    – Virako
    Jan 20, 2017 at 7:31
24

QuerySets in Django are actually generators, not lists (for further details, see Django documentation on QuerySets).
As such, there is no shortcut to get the index of an element, and I think a plain iteration is the best way to do it.

For starter, I would implement your requirement in the simplest way possible (like iterating); if you really have performance issues, then I would use some different approach, like building a queryset with a smaller amount of fields, or whatever.
In any case, the idea is to leave such tricks as late as possible, when you definitely knows you need them.
Update: You may want to use directly some SQL statement to get the rownumber (something lie . However, Django's ORM does not support this natively and you have to use a raw SQL query (see documentation). I think this could be the best option, but again - only if you really see a real performance issue.

2
  • I see. It has been a while since I last touched SQL, but I figured maybe it would be possible in plain SQL and therefore possible using Django's QuerySet API. Jun 25, 2009 at 7:48
  • Yes, that could be an option. I have added it to the possible solutions.
    – rob
    Jun 25, 2009 at 8:10
8

It's possible for a simple pythonic way to query the index of an element in a queryset:

(*qs,).index(instance)

This answer will unpack the queryset into a list, then use the inbuilt Python index function to determine it's position.

1
5

You can do this using queryset.extra(…) and some raw SQL like so:

queryset = queryset.order_by("id")
record500 = queryset[500]

numbered_qs = queryset.extra(select={
    'queryset_row_number': 'ROW_NUMBER() OVER (ORDER BY "id")'
})

from django.db import connection
cursor = connection.cursor()
cursor.execute(
    "WITH OrderedQueryset AS (" + str(numbered_qs.query) + ") "
    "SELECT queryset_row_number FROM OrderedQueryset WHERE id = %s",
    [record500.id]
    )
index = cursor.fetchall()[0][0]

index == 501 # because row_number() is 1 indexed not 0 indexed
1
  • @alexche8 it performs well enough for me on result sets in the 10's of thousands (which I paginate afterward)
    – Jiaaro
    Jul 25, 2014 at 16:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.