[SERVER-3227] space used for index depends on fields fullnames legnth Created: 09/Jun/11  Updated: 29/Aug/11  Resolved: 09/Jun/11

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 1.8.0
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Yan Assignee: Unassigned
Resolution: Done Votes: 0
Labels: indexing
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Participants:

 Description   

Even if using non-compound indexes, indexes on 1 field = seems like every index entry stores full names of fields. In our test cases field names if very long and deep (many levels of supobjects).

I think index may have name, ONE fields desription, order, and many entries with ONLY values, or values + field position.

More space index occupy - less total objects count fits in RAM (even big RAM).
and more space index occupy - indexes searches is slower.

Compact indexes - happy customers.



 Comments   
Comment by Yan [ 09/Jun/11 ]

Thank you very much.
I checked again and index sizes match for all three test cases.
i was wrong, but it very cool.

>for (i = 0; i < 700000; i++) {db.small.save(

{"x" : i}

)};
>for (i = 0; i < 700000; i++) {db.big.save(

{"long_long_long_long_long_long_long_long_long_long_long_long_name_x" : i}

)};
>for (i = 0; i < 700000; i++) {db.deep.save({long : {long : {long : {long : {long : {long : {long : {long :

{long : i }

}}}}}}}})};

> db.small.ensureIndex(

{x : 1}

);
> db.big.ensureIndex(

{long_long_long_long_long_long_long_long_long_long_long_long_name_x : 1}

);
> db.deep.ensureIndex(

{"long.long.long.long.long.long.long.long.long" : 1}

);

> db.small.stats()
{
"ns" : "test.small",
"count" : 700000,
"size" : 25200024,
"avgObjSize" : 36.000034285714285,
"storageSize" : 47824896,
"numExtents" : 10,
"nindexes" : 2,
"lastExtentSize" : 14495232,
"paddingFactor" : 1,
"flags" : 1,
"totalIndexSize" : 52453376,
"indexSizes" :

{ "_id_" : 29122560, "x_1" : 23330816 }

,
"ok" : 1
}
> db.big.stats()
{
"ns" : "test.big",
"count" : 700000,
"size" : 70000020,
"avgObjSize" : 100.00002857142857,
"storageSize" : 99058176,
"numExtents" : 11,
"nindexes" : 2,
"lastExtentSize" : 25047552,
"paddingFactor" : 1,
"flags" : 1,
"totalIndexSize" : 52453376,
"indexSizes" :

{ "_id_" : 29122560, "long_long_long_long_long_long_long_long_long_long_long_long_name_x_1" : 23330816 }

,
"ok" : 1
}
> db.deep.stats()
{
"ns" : "test.deep",
"count" : 700000,
"size" : 86800000,
"avgObjSize" : 124,
"storageSize" : 100610048,
"numExtents" : 11,
"nindexes" : 2,
"lastExtentSize" : 30056960,
"paddingFactor" : 1,
"flags" : 1,
"totalIndexSize" : 52453376,
"indexSizes" :

{ "_id_" : 29122560, "long.long.long.long.long.long.long.long.long_1" : 23330816 }

,
"ok" : 1
}

Comment by Eliot Horowitz (Inactive) [ 09/Jun/11 ]

Field names are not stored in indexes for normal indexes.

If you have a doc like { "a" : 1 , "b" : 2 , "c" :

{ "x" : 3 }

}
These indexes do not have any field names:
(a)
(b)
(a,b)
(b,a)

An index on
(c)
Will not have "c" in it, but will have "x" in it.

Generated at Thu Feb 08 03:02:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.