Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-72513

Validation skips reporting inconsistencies on big collections

    • Type: Icon: Bug Bug
    • Resolution: Works as Designed
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • ALL

      We map keystrings to an array of size 65536 by hashing to efficiently validate index consistency during validation. In the array, for each location, we store the number of keys and the bytes it takes for them. We increment the key count when traversing through the collection, and decrement when traversing through the indexes, but increment the bytes for both. In the end, non-zero-count positions are considered inconsistent, and we will try to report them if the bytes don't exceed the memory limit for validation. 

      However, for big collections with lots of indexes, hash collisions can be common. And even though the problematic key is small enough to be reported, we might think it won't fit because we will see the total bytes used by all keys hashed to this position.

            Assignee:
            backlog-server-execution [DO NOT USE] Backlog - Storage Execution Team
            Reporter:
            yuhong.zhang@mongodb.com Yuhong Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: