[SERVER-69750] For certain index options, $listCatalog output could have inconsistent data type as $listIndexes Created: 15/Sep/22  Updated: 27/Oct/23  Resolved: 14/Dec/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Wenbin Zhu Assignee: Dan Larkin-York
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-73442 The type of the "bits" field can diff... Backlog
related to SERVER-69783 use integral type for TTL index test ... Closed
related to SERVER-72129 Document $listCatalog output format f... Closed
is related to SERVER-62006 Support majority read for _mdb_catalog Closed
is related to SERVER-64404 improve sharding support $listCatalog... Closed
Backport Requested:
v6.1, v6.0
Sprint: Execution Team 2022-12-26
Participants:
Linked BF Score: 34

 Description   

As an example, if user creates an index and specifies expireAfterSeconds option to be a floating point, like 101.23. The creation would succeed and $listIndexes outputs expireAfterSeconds as an integer 101, since this field is declared to be safeInt in IDL. However $listCatalog would still output expireAfterSeconds as floating point 101.23. This behavior causes issues and confusions to downstream consumers like C2C because the expectation is $listCatalog output is consistent with $listIndexes.



 Comments   
Comment by Wenbin Zhu [ 14/Dec/22 ]

Summary of my discussion with dan.larkin-york@mongodb.com:

  • The real problem is in $listIndex, it allows user to specify expireAfterSeconds as floating point, but outputs an integer. Unlike $listIndexes, $listCatalog outputs index specs exactly same as the on-disk catalog.
  • Ideally mongosync should create index on destination using source's on-disk index spec, because that guarantees source and destination can have the same catalog on disk, so that means the current $listCatalog implementation is preferred.
  • However while the current $listCatalog implementation is preferred, it causes confusion to mongosync developers because there is no documentation on $listCatalog output format, so we have to consult $listIndex documentation to know what types are expected in the output, which is exactly same as $listCatalog.
  • SERVER-72129 is created to add an internal documentation for $listCatalog to address the confusion for mongosync developers.
Comment by Connie Chen [ 07/Nov/22 ]

dan.larkin-york@mongodb.com to follow up with wenbin.zhu@mongodb.com on the use cases for mongosync to use $listCatalog and $listIndexes

Comment by Wenbin Zhu [ 04/Oct/22 ]

Hey connie.chen@mongodb.com, from mongosync's perspective, we're more concerned about the consistency between $listCatalog and $listIndexes output rather than precision. Any conversions done in mongosync and cannot guarantee always being consistent with server's $listIndexes output and is not future proof, which has risks for customers.

Comment by Connie Chen [ 04/Oct/22 ]

wenbin.zhu@mongodb.com is the crash happening in mongosync and would the replicator be able to workaround this issue?

We would prefer to keep $listCatalog as a floating point so that we can keep the precision.

Comment by Wenbin Zhu [ 20/Sep/22 ]

We haven't heard customers complaining on this other than our tests. So it's not too urgent, but we do hope this can be fixed soon in order to avoid customer frustrations since C2C would crash when experiencing this issue. Also according to offline discussion with Benety, it seems possible that other index options (in addition to expireAfterSeconds) might have similar issues. If this is true, fixing it sooner would result in less surprises. Thanks!

Generated at Thu Feb 08 06:14:17 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.