[SERVER-78947] Change QE cleanup to use in-memory priority queue for ESC anchors pending removal Created: 13/Jul/23  Updated: 29/Oct/23  Resolved: 20/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Erwin Pe Assignee: Erwin Pe
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Backwards Compatibility: Fully Compatible
Sprint: Security 2023-07-24
Participants:

 Description   

The original design/implementation of the QE cleanup algorithm uses a temporary collection to store the _id fields of anchors that will be deleted towards the end of the algorithm.  This persistence makes it so that the set of anchors that should be deleted can still be removed from the ESC even if the cleanup operation is interrupted before the deletions occur, and had to be resumed.   The security analysis of this implementation (WRITING-14802) found that since insertions into this temp collection occur as part of the individual transactions for each unique field/value pair, it leaks information on the number of compaction epochs for that field/value pair.  So, cleanup should be changed to instead use a in-memory priority queue.  As noted in the analysis, this change would cause some anchors to never be deleted in the case of a failure & resume, but this is an acceptable trade-off.

 

 



 Comments   
Comment by Githook User [ 20/Jul/23 ]

Author:

{'name': 'Erwin Pe', 'email': 'erwin.pe@mongodb.com', 'username': 'erwee'}

Message: SERVER-78947 Change QE cleanup to use in-memory priority queue for ESC anchors pending removal
Branch: master
https://github.com/mongodb/mongo/commit/946e3b13ae17e70685164b2cbe49bbc8fd2290be

Generated at Thu Feb 08 06:39:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.