Find Duplicate Records in MongoDB

You can use the aggregate framework to find duplicate records in MongoDB. To understand the concept, let us create a collection with the document. The query to create a collection with a document is as follows −

> db.findDuplicateRecordsDemo.insertOne({"StudentFirstName":"John"});
{
   "acknowledged" : true,
   "insertedId" : ObjectId("5c8a330293b406bd3df60e01")
}
> db.findDuplicateRecordsDemo.insertOne({"StudentFirstName":"John"});
{
   "acknowledged" : true,
   "insertedId" : ObjectId("5c8a330493b406bd3df60e02")
}
> db.findDuplicateRecordsDemo.insertOne({"StudentFirstName":"Carol"});
{
   "acknowledged" : true,
   "insertedId" : ObjectId("5c8a330c93b406bd3df60e03")
}
> db.findDuplicateRecordsDemo.insertOne({"StudentFirstName":"Sam"});
{
   "acknowledged" : true,
   "insertedId" : ObjectId("5c8a331093b406bd3df60e04")
}
> db.findDuplicateRecordsDemo.insertOne({"StudentFirstName":"Carol"});
{
   "acknowledged" : true,
   "insertedId" : ObjectId("5c8a331593b406bd3df60e05")
}
> db.findDuplicateRecordsDemo.insertOne({"StudentFirstName":"Mike"});
{
   "acknowledged" : true,
   "insertedId" : ObjectId("5c8a331e93b406bd3df60e06")
}

Display all documents from a collection with the help of find() method. The query is as follows −

> db.findDuplicateRecordsDemo.find();

The following is the output −

{ "_id" : ObjectId("5c8a330293b406bd3df60e01"), "StudentFirstName" : "John" }
{ "_id" : ObjectId("5c8a330493b406bd3df60e02"), "StudentFirstName" : "John" }
{ "_id" : ObjectId("5c8a330c93b406bd3df60e03"), "StudentFirstName" : "Carol" }
{ "_id" : ObjectId("5c8a331093b406bd3df60e04"), "StudentFirstName" : "Sam" }
{ "_id" : ObjectId("5c8a331593b406bd3df60e05"), "StudentFirstName" : "Carol" }
{ "_id" : ObjectId("5c8a331e93b406bd3df60e06"), "StudentFirstName" : "Mike" }

Here is the query to find duplicate records in MongoDB −

> db.findDuplicateRecordsDemo.aggregate(
   ... {"$group" : { "_id": "$StudentFirstName", "count": { "$sum": 1 } } },
   ... {"$match": {"_id" :{ "$ne" : null } , "count" : {"$gt": 1} } },
   ... {"$project": {"StudentFirstName" : "$_id", "_id" : 0} }
... );

The following is the output displaying only the duplicate records −

{ "StudentFirstName" : "Carol" }
{ "StudentFirstName" : "John" }