Change QE cleanup to use in-memory priority queue for ESC anchors pending removal

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 7.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Fully Compatible
    • Security 2023-07-24
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      The original design/implementation of the QE cleanup algorithm uses a temporary collection to store the _id fields of anchors that will be deleted towards the end of the algorithm.  This persistence makes it so that the set of anchors that should be deleted can still be removed from the ESC even if the cleanup operation is interrupted before the deletions occur, and had to be resumed.   The security analysis of this implementation (WRITING-14802) found that since insertions into this temp collection occur as part of the individual transactions for each unique field/value pair, it leaks information on the number of compaction epochs for that field/value pair.  So, cleanup should be changed to instead use a in-memory priority queue.  As noted in the analysis, this change would cause some anchors to never be deleted in the case of a failure & resume, but this is an acceptable trade-off.

       

       

            Assignee:
            Erwin Pe
            Reporter:
            Erwin Pe
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: