Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-72513

Validation skips reporting inconsistencies on big collections

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Works as Designed
    • Icon: Major - P3 Major - P3
    • None
    • None
    • None
    • None
    • Storage Execution
    • ALL

    Description

      We map keystrings to an array of size 65536 by hashing to efficiently validate index consistency during validation. In the array, for each location, we store the number of keys and the bytes it takes for them. We increment the key count when traversing through the collection, and decrement when traversing through the indexes, but increment the bytes for both. In the end, non-zero-count positions are considered inconsistent, and we will try to report them if the bytes don't exceed the memory limit for validation. 

      However, for big collections with lots of indexes, hash collisions can be common. And even though the problematic key is small enough to be reported, we might think it won't fit because we will see the total bytes used by all keys hashed to this position.

      Attachments

        Activity

          People

            backlog-server-execution Backlog - Storage Execution Team
            yuhong.zhang@mongodb.com Yuhong Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: