[SERVER-64535] Add Clustered Index Information to collStats Output Created: 15/Mar/22  Updated: 29/Oct/23  Resolved: 28/Jun/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.3.0-rc3
Fix Version/s: 6.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Michael Gargiulo Assignee: Sulabh Mahajan
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File 53-clusteredIndex-collstatsoutput.json     File 53-normalcollection-collstatsoutput.json    
Issue Links:
Documented
is documented by DOCS-15457 Investigate changes in SERVER-64535: ... Backlog
Related
related to SERVER-67469 Add a test to validate the index-size... Backlog
is related to SERVER-67354 const getter returns garbage while no... Closed
Backwards Compatibility: Fully Compatible
Sprint: Execution Team 2022-03-21, Execution Team 2022-06-13, Execution Team 2022-06-27, Execution Team 2022-07-11
Participants:

 Description   

Clustered collections which were completed and enabled inĀ PM-2311 currently don't show index information in the collStats output (index number and size) which is needed for Atlas DE & Compass to properly display collection index informationĀ 



 Comments   
Comment by Githook User [ 28/Jun/22 ]

Author:

{'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com', 'username': 'sulabhM'}

Message: SERVER-64535 Add clustered index information to collStats output
Branch: master
https://github.com/mongodb/mongo/commit/e6af52584adbc558cc9b57b18167beb58ee8ce26

Comment by Anton Oyung [ 23/Jun/22 ]

Hi sulabh.mahajan@mongodb.com, yes this will be enough for our use case. Thank you for making the change!

Comment by Louis Williams [ 08/Jun/22 ]

I think we can report something in the indexDetails section to indicate that the _id is clustered, and also increment nindexes.

Comment by Louis Williams [ 16/Mar/22 ]

emily.pakulski, agreed that the inconsistency is a bit confusing. We decided to return a "fake" _id index after user feedback about it not being clear that the clustered index exists. We're definitely open to suggestions for how to improve the collStats output, but I'm not sure exactly what the desired output would be.

Comment by Emily Pakulski (Inactive) [ 16/Mar/22 ]

louis.williams thanks for explaining; I didn't gather that from my read of LABS-111. If the clustered _id index isn't a real index, then I don't have a well enough informed opinion as to whether we do or don't want to increment nindexes.

To mention it, I find it semantically slightly confusing then that we return it as an index with the listIndexes helper, but I understand now that it has to do with how we store the underlying clustered index data.

Enterprise atlas-13mzst-shard-0 [primary] myFirstDatabase> db.clusteredCollection.getIndexes()
[
  {
    v: 2,
    key: { _id: 1 },
    name: '_id_',
    unique: true,
    clustered: true
  }
]

Comment by Louis Williams [ 16/Mar/22 ]

emily.pakulski, a clustered _id index is not actually a real index, rather it is a property of the collection. The collection is ordered by _id. Because of that, we cannot report anything for _id in "indexSizes", "totalIndexSize", or "indexDetails" because there is nothing to report. The size of the clustered index is part of the size of the collection. This is a fundamental difference in behavior, and any values we include would be fictional.

We can, however, increment "nindexes", but I want to confirm that this difference in behavior is understood.

Comment by Emily Pakulski (Inactive) [ 15/Mar/22 ]

I am adding example output to demonstrate more clearly what the issue is. If you look at the clusteredIndex json, you'll notice that totalIndexSize, indexDetails, and indexSizes are empty, while in normalcollection json (which just has the default _id index), they are populated. nindexes is also not incremented when a clustered index is added.

Generated at Thu Feb 08 06:00:34 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.