[SERVER-36966] Sharded map reduce may fail to clean up temporary output collection Created: 31/Aug/18 Updated: 29/Oct/23 Resolved: 05/Nov/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | MapReduce, Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 4.0.5, 4.1.5 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Charlie Swanson | Assignee: | Charlie Swanson |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | todo_in_code | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||
| Backport Requested: |
v4.0, v3.6
|
||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Linked BF Score: | 14 | ||||||||||||||||||||
| Description |
|
This call is supposed to drop the temporary output collections if anything goes wrong in the command, but in some cases we don't reach there. In particular, I've reproduced a failure where this line can throw an exception due to a stale config. When that happens, the cleanup code doesn't run and the collection is left around. To fix this, we should use a ScopeGuard to make sure we clean up the temporary collections if we exit the function due to an exception. |
| Comments |
| Comment by Githook User [ 06/Nov/19 ] |
|
Author: {'username': 'cswanson310', 'email': 'charlie.swanson@mongodb.com', 'name': 'Charlie Swanson'}Message: |
| Comment by Githook User [ 19/Nov/18 ] |
|
Author: {'name': 'Charlie Swanson', 'email': 'charlie.swanson@mongodb.com', 'username': 'cswanson310'}Message: Also ensures that the temp collections are always up to date, and (cherry picked from commit 7dbcd710077bc4141e71730be9e12558880375e6) |
| Comment by Githook User [ 05/Nov/18 ] |
|
Author: {'name': 'Charlie Swanson', 'email': 'charlie.swanson@mongodb.com', 'username': 'cswanson310'}Message: Also ensures that the temp collections are always up to date, and |
| Comment by Charlie Swanson [ 05/Oct/18 ] |
|
blake.oler sure, I'd be happy to! greg.mckeon can you clarify why this was bumped into a sprint? I was treating this as a BF friday ticket since it is blocking a BF. The workflow for those tickets is no longer to drag them into the sprint.
|
| Comment by Blake Oler [ 02/Oct/18 ] |
|
I will be adding temporary blacklists for this test as part of |
| Comment by Charlie Swanson [ 24/Sep/18 ] |
|
david.storch yes, but I need to put up a further patch with the steps outlined above. |
| Comment by David Storch [ 24/Sep/18 ] |
|
charlie.swanson, is the code review for this ticket still active? |
| Comment by Charlie Swanson [ 21/Sep/18 ] |
|
Investigated more and talked to Esha - we think there are two bugs in map reduce:
We think we know how to fix this, but I didn't get enough time to implement the fix. We should |
| Comment by Ian Whalen (Inactive) [ 20/Sep/18 ] |
|
Next step is still for Charlie to talk to Esha. |