[SERVER-2731] Optimize index building during initial sync Created: 10/Mar/11 Updated: 11/Jul/11 Resolved: 11/Jul/11 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance, Performance, Replication |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Mathias Stearn | Assignee: | Unassigned |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Participants: | |||||||||
| Description |
|
I think it would be better if we fetched the indexes before the data and pulled the keys out of the objects while inserting (phase 1 of index building). Then once a collection is finished spawn a bg thread to sort and build all indexes on that collection using the prefetched keys. If we followed this pattern we would never need to read the whole objects off disk which should have a speedup for datasets larger than ram. Also we could make use of idle cycles while waiting for data to come over the network. |
| Comments |
| Comment by Eliot Horowitz (Inactive) [ 11/Jul/11 ] |
|
See |
| Comment by Matt McClure [ 09/Jul/11 ] |
|
Is this a duplicate of |