[SERVER-787] allow reIndex() to have option for background = True Created: 18/Mar/10 Updated: 06/Dec/22 Resolved: 22/May/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Kenny Gorman | Assignee: | Backlog - Query Team (Inactive) |
| Resolution: | Won't Fix | Votes: | 23 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||||||||||
| Issue Links: |
|
||||||||||||||||||||
| Assigned Teams: |
Query
|
||||||||||||||||||||
| Participants: | |||||||||||||||||||||
| Description |
|
In OLTP environments using traditional RDBMS's it's very common to periodically rebuild indexes for both performance and space savings. I am just guessing that mongoDB would need the same option. reIndex() works perfect, however, it's not enabled for background yet. I would contend it's most important for reIndex() because of the typical rollout of projects (create new indexes, run for a while, oh, we need to reindex but we are live..). |
| Comments |
| Comment by Eric Milkie [ 22/May/18 ] |
|
With the introduction of the WiredTiger storage engine, the reindex command has become less valuable. We are not planning on adding new features to it at this time. |
| Comment by Kenny Gorman [ 06/Oct/11 ] |
|
Another similar approach is to create a new slave entirely (which will then be compact, data too not just indexes), and then promote. This is what our strategy is to-date. -kg |
| Comment by Eliot Horowitz (Inactive) [ 22/Jun/11 ] |
|
@luke - In 2.0 there is a new compact command which should be many times faster |
| Comment by Luke Ehresman [ 22/Jun/11 ] |
|
Any more thoughts on this? Online reindexing would be incredibly helpful to us. We are heavy on deletes, deleting as much data as we insert. We have a rolling timeframe that we keep based on timestamp, so we prune old data as new data is inserted. Our indexes are getting enormous. Our solution for now is to periodically repair the entire database on slaves then switch them to primaries and repair the old primaries. This is getting slower and slower as our database grows with new customers. Currently it takes almost 24 hours to repair, which is nearing our oplog size. |
| Comment by Kenny Gorman [ 06/Oct/10 ] |
|
I agree with Matt's points. Perhaps just implement online collection reorg. This gives you a chance to not only compact the collection, but also it's indexes. There is no DML in MongoDB so online reorg is not needed for that. It could also (depending on implementation) useful in situations where you want to fill a collection and 'swap it out' with another for various reasons. For example if you want to 'push' new background data to a db, and have it all show up at once. Sometimes it's nice to use this method and not have to perform lots of updates with flags and the like. |
| Comment by Matt Keranen [ 06/Oct/10 ] |
|
> Instead of reIndex, I wonder if we could do an online index optimize. This makes sense, coming from the SQL Server (SE) approach where reindexing is an offline operation, but a reorg is an online, nonblocking op. A reindex can result in more compact structures, but a reorg is less resource intensive. It gives the user the option of choosing between impact and performance. |
| Comment by Eliot Horowitz (Inactive) [ 29/Apr/10 ] |
|
I think http://jira.mongodb.org/browse/SERVER-366 is the big problem actually. Instead of reIndex, I wonder if we could do an online index optimize. |
| Comment by Kenny Gorman [ 29/Apr/10 ] |
|
This file shows the index 'bloat' when data is deleted, and thus an online rebuild is required. |
| Comment by Eliot Horowitz (Inactive) [ 22/Apr/10 ] |
|
That's doable - but shouldn't be A dirty hack would be creating a new index on OLD_KEYS , BOGUS_KEY_NAME |
| Comment by Kenny Gorman [ 22/Apr/10 ] |
|
Yes, two at the same time and a quick lock to swap the new index in. Theoretically I think you are correct, but in practice my experience is background reindex() is a tool I want in my toolbox. A good example is a high delete environment. Would it work to create a background 2nd index on the same column and then drop the old one? |
| Comment by Eliot Horowitz (Inactive) [ 18/Mar/10 ] |
|
we try very hard to make sure you don't have to re-index, so i'd rather wait and see what comes up. |