[SERVER-43313] bsonobj size is invalid because size is greater than 16mb Created: 13/Sep/19  Updated: 27/Oct/23  Resolved: 13/Sep/19

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: viney sharma Assignee: Carl Champain (Inactive)
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File Screenshot 2019-09-13 at 5.23.50 PM.png    
Operating System: ALL
Participants:

 Description   

I have a collection called Students which contain 515410 records of documents. i created a query for search and with limited dataset its working fine but with large dataset its failing and throw error (bsonobj size: is invalid. size must be between 0 and 16793600(16mb)).
Please help me to understand and let me know how can i handle this error.

Document screenshot attached please have a look and below is my mongo query for search.

 
student
      .aggregate([
        

{          $match: matchByFilterSet         }

,
        {
          $facet: {
            task1: [
              {
                $group: 

{                   _id: null,                   emailIds:\{$push: "$Email"}

,
                  count: 

{                     $sum: 1                   }

,
                  overallScholarshipGrantedCount:{
                    $sum: 

{                       $size: "$scholarshipHistory.ReceivedScholarships"                     }

                  },                  
                  scholarshipsGrantCount: {
                    $sum: {
                      $cond: [
                        {
                          $gte: [
                            

{                               $size: "$scholarshipHistory.ReceivedScholarships"                             }

,
                            1
                          ]
                        },
                        //{ $in: [ "$scholarshipHistory.ReceivedScholarships", [] ] }
                        1,
                        0
                      ]
                    }
                  },
                  profileCompletedCount: {
                    $sum: 

{                       $cond: [\{ $eq: ["$isProfileCompleted", true] }

, 1, 0]
                    }
                  },
                  statusApprovedCount: {
                    $sum: 

{                       $cond: [\{ $eq: ["$status", "Approved"] }

, 1, 0]
                    }
                  },
                  statusPendingCount: {
                    $sum: 

{                       $cond: [\{ $eq: ["$status", "Pending"] }

, 1, 0]
                    }
                  },
                  statusRejectCount: {
                    $sum: 

{                       $cond: [\{ $eq: ["$status", "Reject"] }

, 1, 0]
                    }
                  }
                }
              }
            ],
            task2: [
              {
                $sort: 

{                   updatedOn: -1                 }

              },
              

{                 $skip: offset               }

,
              

{                 $limit: limit               }

            ]
          }
        },
        

{           $unwind: "$task1"         }

,
        {
          $project: 

{             total_students: "$task1.count",             scholarshipsGrantCount: "$task1.scholarshipsGrantCount",             overallScholarshipGrantedCount:"$task1.overallScholarshipGrantedCount",             profileCompletedCount: "$task1.profileCompletedCount",             emailIds:"$task1.emailIds",             dataList: "$task2",             statusApprovedCount: "$task1.statusApprovedCount",             statusPendingCount: "$task1.statusPendingCount",             statusRejectCount: "$task1.statusRejectCount"           }

        }
      ])
      .exec();



 Comments   
Comment by viney sharma [ 14/Sep/19 ]

Thanks carl It's work for me.

Comment by Carl Champain (Inactive) [ 13/Sep/19 ]

Hi vineysharma1102@gmail.com,

Thanks for the report!

The $push operation in "task1" is causing the error since a document size must be less than 16MB. Please take a look at the documentation here. There is no error using a small dataset because you are pushing less emails into the array, therefore the size of the document becomes less than 16MB. You need to reorganize the aggregation pipeline to avoid building an oversized document.

For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag.

Kind regards,
Carl
  

Generated at Thu Feb 08 05:02:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.