Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-67446

Ensure consistent wildcardProjection and columnstoreProjection specs in catalog

    • Query Execution
    • Minor Change
    • v6.1, v6.0, v5.3, v5.0

      In SERVER-25023 we started normalizing the wildcardProjection field of index specs that we persist in the catalog, in preparation for SERVER-47659 where we added wildcardProjection to the index signature. The normalization is intended to obviate any differences in how the user may specify the same projection, e.g. {"a.b": 1} and {a: {b: 1} } will both be transformed to the latter form. This allows us to easily compare the BSONObj specs when the user attempts to build a new index, to see whether an identical index already exists.

      However, it has recently been observed that building a wildcard index on a sharded cluster may permute the ordering of fields in the normalized spec across shards; for instance, we may end up with {name: 0, type: 0, _id: 1} on one shard and {type: 0, name: 0, _id: 1} on another. This appears to be because the projection tree holds its nodes as an unordered map and may therefore produce a field-order-nondeterministic normalization of the spec.

      While the Server itself will regard these specs as being equivalent and will behave correctly if the user submits multiple createIndex requests, this complicates the logic required by downstream products which read the set of index specs from the catalog, notably C2C.

      We should address this by doing one of the following (in likely order of desirability):

      • Persist the user's requested wildcardProjection verbatim and defer normalization until the point where we need to compare index specs.
      • Persist the user's requested wildcardProjection verbatim and add an operator== to the Projection tree to facilitate direct comparison
      • Store the projection nodes in a deterministic order so that the persisted normalized forms are also deterministic in their field ordering.

            kevin.cherkauer@mongodb.com Kevin Cherkauer
            bernard.gorman@mongodb.com Bernard Gorman
            0 Vote for this issue
            11 Start watching this issue