16

I am mongodb beginner and am working on a homework problem, the dataset looks like this

{ "_id" : { "$oid" : "50906d7fa3c412bb040eb577" }, "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb578" }, "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb579" }, "student_id" : 0, "type" : "homework", "score" : 14.8504576811645 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb57a" }, "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb57b" }, "student_id" : 1, "type" : "exam", "score" : 74.20010837299897 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb57c" }, "student_id" : 1, "type" : "quiz", "score" : 96.76851542258362 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb57d" }, "student_id" : 1, "type" : "homework", "score" : 21.33260810416115 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb57e" }, "student_id" : 1, "type" : "homework", "score" : 44.31667452616328 }

As part of the problem I have to delete for each student, the 'homework' document with lowest score. Here is my strategy

In aggregate pipeline
1: First filter all the documents with type:homeworks
2: Sort by student_id, score
3: do a group on student_id, find the first element

This will give me all the documents with lowest score,

however how I do delete these elements from the original dataset? Any guidance or hint?

4
  • .remove({}) , where {} contains condtions of sorting.
    – MegaMind
    Jun 3, 2015 at 9:34
  • possible duplicate of How remove min value in mongodb for group?
    – Neo-coder
    Jun 3, 2015 at 9:51
  • Dude, this homework seems familiar. If you can't figure out how to do it, you should rather go back to the lesson videos than to ask here. Without that basic knowledge, you'll be stuck in the weeks to come. Jun 4, 2015 at 8:28
  • @MarkusWMahlberg Thanks for the advise.
    – Max
    Jun 4, 2015 at 10:31

6 Answers 6

33

Use the cursor result from the aggregation to loop through the documents with the cursor's forEach() method and then remove each document from the collection using the _id as the query in the remove() method. Something like this:

var cursor = db.grades.aggregate(pipeline);
cursor.forEach(function (doc){
    db.grades.remove({"_id": doc._id});
});

Another approach is to create an array of the document's _ids using the map() method and remove the documents like:

var cursor = db.grades.aggregate(pipeline),
    ids = cursor.map(function (doc) { return doc._id; });
db.grades.remove({"_id": { "$in": ids }});

-- UPDATE --

For large deletion operations, it may be more efficient to copy the documents that you want to keep to a new collection and then use drop() on the original collection. To copy the essential documents your aggregation pipeline needs to return the documents without the lowest homework doc and copy them to another collection using the $out operator as the final pipeline stage. Consider the following aggregation pipeline:

db.grades.aggregate([    
    {
        '$group':{
            '_id': {
                "student_id": "$student_id",
                "type": "$type"
            },
            'lowest_score': { "$min": '$score'},
            'data': {
                '$push': '$$ROOT'
            }
         }
    },    
    {
        "$unwind": "$data"
    },
    {
        "$project": {
            "_id": "$data._id",
            "student_id" : "$data.student_id",
            "type" : "$data.type",
            "score" : "$data.score",
            'lowest_score': 1,            
            "isHomeworkLowest": {
                "$cond": [
                    { 
                        "$and": [
                            { "$eq": [ "$_id.type", "homework" ] },
                            { "$eq": [ "$data.score", "$lowest_score" ] }
                        ] 
                    },
                    true,
                    false
                ]
            }
        }
    },
    {
        "$match": {"isHomeworkLowest" : false}
    },
    {
        "$project": {           
            "student_id": 1,
            "type": 1,
            "score": 1
        }
    },
    {
        "$out": "new_grades"
    }
])

in which you can then drop the old collection by db.grades.drop() and then query on db.new_grades.find()

2
  • Is this the most efficient way to do this? I am asking because I just started learning. I thought that there would be a way where find/deletion can be done in one query?
    – Max
    Jun 3, 2015 at 9:35
  • @Dude For large deletion operations, it may be more efficient to copy the documents that you want to keep to a new collection and then use drop() on the original collection.
    – chridam
    Jun 3, 2015 at 9:58
5

I think this is the a database part of homework of MongoDB for Java Developers provided by MongoDB University. Where the requirement is to delete the lowest score from each individual student. anyway I solved this way. I hope It will be helpful for you. You can also clone my code from my github link(Provided below)

public class Homework2Week2 {

public static void main(String[] args) {
    // TODO Auto-generated method stub
    // Here the the documentation is used for mongo-jva-driver-3.2.2.jar
    /*If you want to use different versionof  mongo-jva-driver 
      then you have look for that version specificatios.*/
    MongoClient mongoClient = new MongoClient();
    // get handle to "students" database
    MongoDatabase database = mongoClient.getDatabase("students");
    // get a handle to the "grades" collection
    MongoCollection<Document> collection = database.getCollection("grades");
    /*
     * Write a program in the language of your choice that will remove the grade of type "homework" with the lowest score for each student from the dataset in the handout. 
     * Since each document is one grade, it should remove one document per student. 
     * This will use the same data set as the last problem, but if you don't have it, you can download and re-import.
     * The dataset contains 4 scores each for 200 students.
     * First, letâs confirm your data is intact; the number of documents should be 800.

     *Hint/spoiler: If you select homework grade-documents, sort by student
      and then by score, you can iterate through and find the lowest score
      for each student by noticing a change in student id. As you notice
      that change of student_id, remove the document.
     */
    MongoCursor<Document> cursor = collection.find(eq("type", "homework")).sort(new Document("student_id", 1).append("score", 1)).iterator();
    int curStudentId = -1;
    try
    {
    while (cursor.hasNext()) {
        Document doc = cursor.next();
        int studentId=(int) doc.get("student_id");
        if (studentId != curStudentId) {
            collection.deleteMany(doc);
            curStudentId = studentId;
        }
    }
    }finally {
        //Close cursor
        cursor.close();
    }   
    //Close mongoClient
    mongoClient.close();
}

}

In my Github account I have the complete project code. If anyone want's you can try from this link.

1
  • You can follow this int studentId= Integer.valueOf(doc.get("student_id").toString()); instead of this int studentId=(int) doc.get("student_id"); Jun 17, 2016 at 16:21
4
db.grades.aggregate( [ 
                            { 
                                $match:{type:'homework'}
                            }, 
                            { $group: 
                                 { _id: {student_id:"$student_id",type:'$type'},                                   
                                   score: { $max: "$score" } 
                                 } 
                            } 
                            ]).forEach(function(doc){
db.grades.remove({'student_id':doc._id.student_id,'score':doc.score})

})
2

Starting Mongo 4.4, the $group stage has a new aggregation operator $accumulator allowing custom accumulations of documents as they get grouped.

Coupled with an $out stage used in this case to replace the original collection with the result of the aggregation pipeline (that's been stripped out of the lowest score per student):

// > db.collection.find()
//     { "student_id" : 0, "type" : "exam",     "score" : 54.6535436362647  }
//     { "student_id" : 0, "type" : "homework", "score" : 14.8504576811645  }
//     { "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }
//     { "student_id" : 1, "type" : "homework", "score" : 21.33260810416115 }
//     { "student_id" : 1, "type" : "homework", "score" : 44.31667452616328 }
db.collection.aggregate(
  { $group: {
      _id: "$student_id",
      docs: { $accumulator: {
        accumulateArgs: ["$$ROOT"],
        init: function() { return []; },
        accumulate: function(docs, doc) { return docs.concat(doc); },
        merge: function(docs1, docs2) { return docs1.concat(docs2); },
        finalize: function(docs) {
          var min = Math.min(...docs.map(x => x.score));
          var i = docs.findIndex((doc) => doc.score == min);
          docs.splice(i, 1);
          return docs;
        },
        lang: "js"
      }}
  }},
  { $unwind: "$docs" },
  { $replaceWith: "$docs" },
  { $out: "collection" }
)
// > db.collection.find()
//     { "student_id" : 0, "type" : "exam",     "score" : 54.6535436362647  }
//     { "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }
//     { "student_id" : 1, "type" : "homework", "score" : 44.31667452616328 }

This:

  • $groups documents by student_id and accumulates them as arrays stripped down from the document with the lowest score:

    • accumulateArgs is the combination of fields (or in our case the whole document $$ROOT) used by the accumulation function.

    • each original accumulated array is initialised as an empty array.

    • documents are simply concatanated (accumulate and merge)

    • and finally, once all documents have been grouped, the finalize step allows finding the grouped document whose score is the lowest in order to remove it.

    • at the end of this stage, pipelined documents look like:

      {
        "_id" : 0,
        "docs" : [
          { "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 },
          { "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 },
          { "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }
        ]
      }
      ...
      
  • $unwinds the accumulated field of grouped documents to flatten the arrays of grouped documents, and get back to something like:

    { "_id" : 0, "docs" : { "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 } }
    { "_id" : 0, "docs" : { "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 } }
    { "_id" : 0, "docs" : { "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 } }
    ...
    
  • $replaceWith all existing fields in each document with the content of the accumulated field, in order to find the original format back. At the end of this stage, we have something like:

    { "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
    { "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }
    { "student_id" : 0, "type" : "homework", "score" : 63.98402553675503 }
    ...
    
  • $out inserts the result of the aggregation pipeline in the same collection. Note that $out conveniently replaces the content of the specified collection, making this solution possible.

0

int studentId=(int) doc.get("student_id");

gives an convert type error.Can you check again?

As I know we can make as below.

int studentId= Integer.valueOf(doc.get("student_id").toString());

1
  • For me , int studentId=(int) doc.get("student_id"); was working. Your one is also correct you can use.Could you write me the error then I will understand what's happening actually? Jun 16, 2016 at 0:25
0

This question is part of M101P: MongoDB for Developers Course by MongoDB University. The requirement here is to:-

Remove the grade of type "homework" with the lowest score for each student from the dataset. Since each document is one grade, it should remove one document per student.

So it means from every student_id there exists 4 'type' out of which two 'type' are 'homework' . We have to remove the lowest score out of two 'type':'homework' document.

The up and running code in pymongo is below:-

import pymongo
import sys

//establish a connection to database

connection=pymongo.MongoClient('localhost',27017)
//Get a handle to students database

db=connection.students
mycol=db.grades

def remove_documents():
        pipe=[
        {'$match':{'type':'homework'}},
        {'$group':{'_id':'$student_id','minscore':  {'$min':'$score'}}}
        ,{'$sort':{'_id':1}}
             ]
        result_cursor=mycol.aggregate(pipeline=pipe)
        counter=0

        for i in result_cursor:
            query={'student_id':i['_id'],'score':i['minscore']}
            del_record=mycol.delete_one(query)
            if (int(del_record.deleted_count) > 0):
                    counter+=1
            else:
                    continue
        print(counter)
remove_documents()

Terminal Output:- $python remove_grade.py

200

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.