1

I have an issue auto-dereferencing sub-objects using aggregate and keep list order doing this. First: my data model. Let's break it down to two collections (and I use simplified ids to make it easier to understand):

meeting:

{
    'id': '1',
    'some': 'abc',
    'data': 'def',
    'agendaItem': [
        ObjectId('2'),
        ObjectId('3')
    ]
}

agenda_item:

{
    'id': ObjectId('2'),
    'some': 'ghi',
    'data': 'jkl'
},
{
    'id': ObjectId('3'),
    'some': 'mno',
    'data': 'pqr'
}

The result of my single(!) query should be like this:

{
    'id': ObjectId('1'),
    'some': 'abc',
    'data': 'def',
    'agenda_items': [
        {
            'id': ObjectId('2'),
            'some': 'ghi',
            'data': 'jkl'
        },
        {
            'id': ObjectId('3'),
            'some': 'mno',
            'data': 'pqr'
        }
    ]
}

The important part is that the list of agenda_items should stay in the same order, id 2 has to be on the top.

At the moment, I use a aggregate with unwind, lookup and group to do such a query:

[
    {
        '$unwind': {
            'preserveNullAndEmptyArrays': True,
            'path': '$agendaItem'
        }
    },
    {
        '$lookup': {
            'from': 'agenda_item',
            'as': 'agendaItem',
            'foreignField': '_id',
            'localField': 'agendaItem'
        }
    },
    {
        '$unwind': {
            'preserveNullAndEmptyArrays': True,
            'path': '$agendaItem'
        }
    },
    {
        '$group': {
            'agendaItem': {
            '$addToSet': '$agendaItem'
            }
        }
    }
]

This gives me the result I want - with one exception: the list is completely mixed up, which means in this example ObjectId('3') is before ObjectId('2').

Using addFields like in this blog post by MongoDb: http://www.kamsky.org/stupid-tricks-with-mongodb/using-34-aggregation-to-return-documents-in-same-order-as-in-expression is somehow the solution, but I have no real idea how to put it into the $unwind + $lookup chain. Can you help me? Or is my strategy completely wrong?

Background in German btw: https://oparl.org/spezifikation/online-ansicht/ and https://mirror.oparl.org/body/5a3d010888dbe04745b24e0a/meeting

6
  • Why are you using addToSet ? $addToSet is not guaranteed to maintain the order. Use $push and add a $sort stage before the $group stage for stable ordering. So last two stages should be {$sort:{"agendaItem.id":1}}, { '$group': {'_id':'$id', 'agendaItem': { '$push': '$agendaItem' } } }
    – s7vr
    Jan 19, 2018 at 15:27
  • I suppose because I still have my issues to understand the aggregation framework, so I just overlooked the issue with addToSet. But this works great for existing values. So the follow up issue: I don't want to have it ordered by id, instead exacly in order how it is saved in MongoDB list. I suppose the solution is $addField, but I have no clue where to add it. Can you give me a hint there? Jan 19, 2018 at 15:55
  • 1
    I'm guessing you are on 3.4 because you mentioned $addFields.Just don't $unwind and $group ( Its not needed in 3.4 ). Try just this and no other stages db.col.aggregate([ { '$lookup': { 'from': 'agenda_item', 'as': 'agendaItem', 'foreignField': '_id', 'localField': 'agendaItem' } } ])
    – s7vr
    Jan 19, 2018 at 15:57
  • $addField was something I googled searching for a solution. It's a new project, so I can choose any MongoDB version I want. But I created the code snippet when MongoDB 3.2 was the current version for another project. And: your hint works great. So this got "much" easier the last versions, and I just did not check if simplification would do the job now. Thanks a lot! Jan 19, 2018 at 16:12
  • The solution suggested by @user2683814 does not work, unfortunately. The $lookup stage does lookup all items of the array and outputs an array if localField is an array. However, it's not guaranteed the resulting field will be output in the same order as the input array.
    – sboisse
    Dec 17, 2019 at 21:15

1 Answer 1

0

I had the exact same problem to resolve.

The solution can be found here: Aggregate $lookup does not return elements original array order

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.