81

I try to get a list of distinct foreign keys and I wrote this:

my_ids = Entity.objects.values('foreign_key').distinct()

But I get just a list of UNDISTINCT foreign keys... What am I missing?

Thanks!

4 Answers 4

70

Passing an argument to distinct doesn't work for MySQL-databases (AFAIK)

This one works and returns just one object:

Entity.objects.order_by('foreign_key').values('foreign_key').distinct()

1
  • 7
    I have the same issue on a Postgres9.1 database, Django==1.5.5. Adding order_by helped - stupid...
    – kev
    Jan 2, 2014 at 4:28
69

Perhaps you might want to go with this:

Entity.objects.order_by().values_list('foreign_key', flat=True).distinct()
1
  • 19
    WRONG. Wont't work when used without order_by('foregin_key'). E.g on M2M (postgres, django 1.8)
    – andilabs
    Jul 23, 2015 at 6:23
51
Entity.objects.values_list('foreign_key', flat=True).distinct().order_by()

distinct does not work without order_by, as explained in Django documentation:

Any fields used in an order_by() call are included in the SQL SELECT columns. This can sometimes lead to unexpected results when used in conjunction with distinct(). If you order by fields from a related model, those fields will be added to the selected columns and they may make otherwise duplicate rows appear to be distinct. Since the extra columns don’t appear in the returned results (they are only there to support ordering), it sometimes looks like non-distinct results are being returned.

Similarly, if you use a values() query to restrict the columns selected, the columns used in any order_by() (or default model ordering) will still be involved and may affect uniqueness of the results.

The moral here is that if you are using distinct() be careful about ordering by related models. Similarly, when using distinct() and values() together, be careful when ordering by fields not in the values() call.

3
  • 3
    O.M.G.! Bonus: that order_by() does not order really but solves the distinct problem. This is just insane.
    – Csaba Toth
    Mar 21, 2018 at 20:43
  • Does this work? It seems like distinct and order by would require a two column solution with the use of MIN or MAX to get the sorting right. I'm guessing order by just sorts by the values_list field, which is probably not the default sort order of the model. Proper solution would be to write a custom query: select value, min(sequence) as sequence from some_table group by value order by sequence; Dec 6, 2018 at 23:09
  • 1
    .order_by() without any args clears any default or existing ordering on the queryset. Oct 2, 2020 at 14:11
9
Entity.objects.order_by('foreign_key').distinct('foreign_key')

If you already have them as a list, then convert it to a set() to get the distinct values.

2
  • is not this converting to set is a costly idea with respect to memory as with set() everything will be filled in memory where as queryset its not loaded till its evaluated.. and this converting to list thing will evaluate the queryset as well ?? Mar 1, 2014 at 4:25
  • 1
    Sets are unsorted so you'll cancel the order_by, and in some cases that matters. Dec 6, 2018 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.