[SERVER-2366] Speed up creation of (don't populate) index on unused field for sparse index Created: 17/Jan/11  Updated: 06/Dec/22  Resolved: 08/Dec/17

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor - P4
Reporter: David Lee Assignee: Backlog - Query Team (Inactive)
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-14772 Create a way to add an empty sparse i... Closed
Assigned Teams:
Query
Participants:
Case:

 Description   

This would help when schemas change and a user needs to start using another field within a collection. The user would create an index on the new field, and instead of uselessly having to scan through every document, index creation would happen immediately.

If collections kept track of which fields were in use, index creation for an unused could just be empty instead of having to check every document.

Even if collections don't keep track of exact fields, they can use a bloom filter to approximately keep track.

Finally, there should be an option to ensureIndex() that instantaneously creates an empty index instead of scanning through every document when the user is confident the collection does not contain documents that contain the field. The option can be

{empty:true}

or

{populate:false}

.



 Comments   
Comment by George Macon [ 02/Jul/12 ]

I am planning to use Ming to enforce a schema on my database. Because of this, I will have application-level knowledge that a certain field does not exist on any of the documents currently in the database and the database will not need to keep track of what fields exist. The option should be documented with a caveat to the effect that if you don't know that the field doesn't exist, don't use this option.

Comment by David Lee [ 17/Jan/11 ]

That's fine. That's the reason I proposed the explicit option to ensureIndex which allows users to create an initially empty index.

Comment by Eliot Horowitz (Inactive) [ 17/Jan/11 ]

These requires keeping meta data that I'm not sure we want to start keeping.

Comment by David Lee [ 17/Jan/11 ]

I forgot to mention that this would only work for sparse indexes without changing index behavior. I guess somebody could also want an empty non-sparse index, but that would behave differently from normal indexes and is out of the scope of this feature request.

Generated at Thu Feb 08 02:59:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.