[SERVER-70012] Getting 'Sort exceeded memory limit' error when trying to reshard a collection Created: 27/Sep/22 Updated: 08/Nov/22 Resolved: 17/Oct/22 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | 5.0.12, 5.0.9 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Alexey Zarubin | Assignee: | Yuan Fang |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Steps To Reproduce: | Unfortunately, the only way to reproduce the problem is:
|
||||||||
| Participants: | |||||||||
| Description |
|
When trying to reshard a collection we are getting an error message telling that the sort limit is exceeded. The resarding command is being run as follows:
The collection contains approx. 2.4B documents split across 10 shards. Shard distribution looks like this:
The only reason I see for such behavior is that reshardCollection command does some preliminary queries/aggregation involving sorting for further work, like counting documents to copy etc., and during this phase something sort memory limit gets exceeded due to a large number of documents or chunks in the collection. Probably those sort operations need to be done with allowDiskUse option set to tru |
| Comments |
| Comment by Alexey Zarubin [ 08/Nov/22 ] |
|
Hi @Yuan Fang , The problem still occurs in 5.0.13. However, after studying the initial issue ( |
| Comment by Yuan Fang [ 17/Oct/22 ] |
|
We haven’t heard back from you for some time, so I’m going to close this ticket. If this is still an issue for you, please provide additional information and we will reopen the ticket. |
| Comment by Yuan Fang [ 30/Sep/22 ] |
|
alexey.zarubin@gmail.com, Sounds good. FYI that MongoDB 5.0.13 has been released! I'm going to leave this SERVER ticket open just in case the issue isn't solved by upgrading to it. |
| Comment by Alexey Zarubin [ 29/Sep/22 ] |
|
Greetings, Thanks for the information and sorry for the duplicate - my bad, but I could not find the referenced issue. Due to high complexity of testing, we will be expecting 5.0.13 release then, thanks! |
| Comment by Yuan Fang [ 28/Sep/22 ] |
|
Thank you for your report. The issue looks very similar to Regards, Yuan |