Optimize index building during initial sync

XMLWordPrintableJSON

    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      I think it would be better if we fetched the indexes before the data and pulled the keys out of the objects while inserting (phase 1 of index building). Then once a collection is finished spawn a bg thread to sort and build all indexes on that collection using the prefetched keys. If we followed this pattern we would never need to read the whole objects off disk which should have a speedup for datasets larger than ram. Also we could make use of idle cycles while waiting for data to come over the network.

              Assignee:
              Unassigned
              Reporter:
              Mathias Stearn
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: