[SERVER-51107] Index build external sort files can collide with existing external sort files for resumed index builds Created: 23/Sep/20 Updated: 29/Oct/23 Resolved: 25/Sep/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 4.8.0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Gregory Noma | Assignee: | Gregory Noma |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Backwards Compatibility: | Fully Compatible |
| Operating System: | ALL |
| Sprint: | Execution Team 2020-10-05 |
| Participants: |
| Description |
|
Say a resumable index build with external sort file extsort-index.0 is interrupted for shutdown during the collection scan or bulk load phases and then resumed on startup. The resumed index build will continue to use the same external sort file. However, now consider a new index build started while the resumed index build is still in progress. This new index build will attempt to also use an external sort file named extsort-index.0 due to the filename counter starting back at 0. Another scenario is if the resume info failed to parse on startup, the external sort file extsort-index.0 will become orphaned until a subsequent restart without any index builds to resume (or an unclean shutdown). The external sort files can then collide in the same way as described above. |
| Comments |
| Comment by Githook User [ 25/Sep/20 ] |
|
Author: {'name': 'Gregory Noma', 'email': 'gregory.noma@gmail.com', 'username': 'gregorynoma'}Message: |
| Comment by Louis Williams [ 23/Sep/20 ] |
|
We might be able to do something similar to what we do for idents already, appending a random number generated at every startup, and also incrementing a counter. |