[SERVER-20057] Concurrent, sharded mapReduces can fail when temporary namespaces collide across mongos processes Created: 20/Aug/15 Updated: 06/Dec/22 Resolved: 09/Mar/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | MapReduce, Sharding |
| Affects Version/s: | 3.1.6 |
| Fix Version/s: | 4.4.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Kamran K. | Assignee: | [DO NOT USE] Backlog - Sharding Team |
| Resolution: | Done | Votes: | 1 |
| Labels: | 32qa | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Sharding
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Participants: | |||||||||
| Description |
|
It's possible for concurrent, sharded mapReduces to fail with DEAD plan executors when there's a collision in temporary namespaces across multiple mongos processes. This bug is intermittently triggered by the concurrency suite. This seems to be the sequence of events: 1 - A mongos process issues a drop command, on all shards, on a tmp.mrs namespace after finishing the mapReduce.shardedfinish command (in cluster_map_reduce_cmd.cpp). 2 - At the same time, another mongos process tries to initialize a ParallelSortClusteredCursor on the very same tmp.mrs namespace as part of another mapReduce.shardedfinish command. 3 - The drop invalidates cursors on the tmp.mrs namespace, which leads to a DEAD plan executor and a failed mapReduce command. Relevant log lines:
Test output:
|
| Comments |
| Comment by Charlie Swanson [ 09/Mar/20 ] |
|
This is no longer a problem after completing a recent project where we created a new implementation of mapReduce backed by the aggregation framework. |
| Comment by Githook User [ 21/Aug/15 ] |
|
Author: {u'username': u'kkmongo', u'name': u'Kamran Khan', u'email': u'kamran.khan@mongodb.com'}Message: The tests intermittently fail because temporary namespaces can collide |