Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-1140

Support 64 (or more) indexes

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 1.5.2
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None

      The current code supports up to 40 indexes per collection. I think that we can support 64 without too much extra work, and much much more if you're willing to use some of the reserved bytes in Namespace. AFAICT, the 40 index restriction is twofold:

      1. multiKeyIndexBits is a bitwise storage of the multi-key state for each index, and it uses an unsigned long long, limiting to 64 indexes

      2. the first 10 indexes are stored in the Namespace, the next 30 are stored in an extra block, currently limiting to 40 total indexes, however forethought was given to this, and there exists the capability to chain additional extra blocks.

      The attached patch implements the extra block chaining. It accounts for the possibility of 10 more "extra" blocks by naming them $extra, $extr0, $extr1, $extr2 ... $extr9. You could actually get another 300 using this, for a hard limit of 340. However, because of the multiKeyIndexBits limitation, we're really limited to just 64 indexes. I implemented the full chaining for future expansion.

      If you're interested in 128 indexes, just take 8 or more bytes from the reserved array, and create a multiKeyIndexMoreBits[] and carry the logic into the multiKey functions. I did not do this because (a) the extra 24 indexes should be enough for our needs, (b) those reserved bytes are precious few and once you use them, they're gone, (c) I know we're crazy to want 64 indexes, and I think whoever wants 128 is even crazier.

      This patch is not yet complete. It dies in db/pdfile.h +284 on an assertion "ofs >= headerSize()". But I've only spent 2.5 hours on this so far, and it's late, and I wanted to get my work in progress in front of others to review and comment.

        1. mongo-64-indexes.patch
          8 kB
        2. mongo-64-indexes.patch
          8 kB
        3. mongo-64-indexes.patch
          7 kB
        4. mongo-64-indexes-backport.patch
          13 kB

            Assignee:
            dwight@mongodb.com Dwight Merriman
            Reporter:
            douggreen Doug Green
            Votes:
            2 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: