[SERVER-68139] Resharding command fails if the projection sort is bigger than 100MB Created: 19/Jul/22  Updated: 29/Oct/23  Resolved: 20/Sep/22

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 5.0.14, 6.0.3, 6.1.0-rc3, 6.2.0-rc0

Type: Bug Priority: Major - P3
Reporter: Rachita Dhawan Assignee: Nandini Bhartiya
Resolution: Fixed Votes: 1
Labels: sharding-nyc-subteam1
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File initial_split_policy.h     File reshard_coll1.js    
Issue Links:
Backports
Depends
Related
related to SERVER-70012 Getting 'Sort exceeded memory limit' ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v6.1, v6.0, v5.0
Sprint: Sharding 2022-09-05, Sharding 2022-09-19, Sharding 2022-10-03
Participants:
Case:
Story Points: 3

 Description   

Problem: The resharding command fails if the newShardKeyProjection size * samplePerChunk * intialNumOfChunks is bigger than 100MB.

newShardKeyProjection: size of newShardKey document

Default value of initialNumOfChunks is the existing number of chunks before the resharding.

Default value of samplePerChunk: 10

Description: Resharding creates a pipeline query command and the query is then sorted on the new Shard Key as part of the projection. If the projection of over 100MB is passed onto sort, the following error is thrown 

"errmsg" : "PlanExecutor error during aggregation :: caused by :: Sort exceeded memory limit of 104857600 bytes, but did not opt in to external sorting. Aborting operation. Pass allowDiskUse:true to opt in.", 

The error is due to 
ReshardingSplitPolicy::_makePipelineDocumentSource creating ExpressionContext with allowDiskUse to false. 

auto expCtx = make_intrusive<ExpressionContext>(opCtx,                                                    boost::none, /* explain */                                                    false,       /* fromMongos */                                                    false,       /* needsMerge */                                                    false,       /* allowDiskUse */                                                    true,        /* bypassDocumentValidation */                                                    false,       /* isMapReduceCommand */                                                    ns,                                                    boost::none, /* runtimeConstants */                                                    nullptr,     /* collator */                                                    std::move(pi),                                                    std::move(resolvedNamespaces),                                                    boost::none); /* collUUID */ 

Steps to reproduce:
 Attached .js test and initial_split_policy.h to change the kDefaultSamplesPerChunk to 10000(from 10)
 

 

 



 Comments   
Comment by Githook User [ 21/Sep/22 ]

Author:

{'name': 'nandinibhartiyaMDB', 'email': 'nandini.bhartiya@mongodb.com', 'username': 'nandinibhartiyaMDB'}

Message: SERVER-68139: Test resharding success with limited memory

(cherry picked from commit 901ce6eed061d21df66d82ded539d294da6d3de4)
Branch: v6.0
https://github.com/mongodb/mongo/commit/e886b9117e348c4d0ca6dc280e859cea37b48a03

Comment by Githook User [ 21/Sep/22 ]

Author:

{'name': 'nandinibhartiyaMDB', 'email': 'nandini.bhartiya@mongodb.com', 'username': 'nandinibhartiyaMDB'}

Message: SERVER-68139: Test resharding success with limited memory

(cherry picked from commit 901ce6eed061d21df66d82ded539d294da6d3de4)
Branch: v6.1
https://github.com/mongodb/mongo/commit/6d0624e75a8a9ec1d253fc4cda7da4b42bfb07b8

Comment by Githook User [ 21/Sep/22 ]

Author:

{'name': 'nandinibhartiyaMDB', 'email': 'nandini.bhartiya@mongodb.com', 'username': 'nandinibhartiyaMDB'}

Message: SERVER-68139: Test resharding success with limited memory

(cherry picked from commit 901ce6eed061d21df66d82ded539d294da6d3de4)
Branch: v5.0
https://github.com/mongodb/mongo/commit/15575275d619d27970de98696951b884a49b4005

Comment by Githook User [ 19/Sep/22 ]

Author:

{'name': 'nandinibhartiyaMDB', 'email': 'nandini.bhartiya@mongodb.com', 'username': 'nandinibhartiyaMDB'}

Message: SERVER-68139: Test resharding success with limited memory
Branch: master
https://github.com/mongodb/mongo/commit/901ce6eed061d21df66d82ded539d294da6d3de4

Comment by Max Hirschhorn [ 04/Aug/22 ]

We should add C++ test cases to initial_split_policy_test.cpp which exercise $sort needing to spill to disk. The C++ test cases should reproduce the "Sort exceeded memory limit" error without needing to have >100MB worth of data. It looks like the DocumentSourceSort stage supports passing in maxMemoryUsageBytes. Either this mechanism can be used directly or perhaps it is already behind a general failpoint. Feel free to reach out to the Query team over Slack for additional guidance on the testing.

Generated at Thu Feb 08 06:10:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.