Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-76875

Exclude fields containing dots from indexes

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Minor - P4 Minor - P4
    • 7.1.0-rc0, 7.0.0-rc1, 6.0.7
    • Affects Version/s: None
    • Component/s: None
    • Labels:
      None
    • Query Execution
    • Fully Compatible
    • ALL
    • v7.0, v6.3, v6.0
    • QE 2023-05-15
    • 120

      When an index is defined on a path "obj.nested.field", it's supposed to index a document 

      { obj: {nested: {field: "value"}}}

      However, it also indexes 

      {obj: {"nested.field": "value"}}

      as it looks evident in this testcase 

       

      db.geo.createIndex({"obj.inner.geoLegacy": "2d"})
      db.geo.insert([ {_id: 17922, "obj": {_id: 17923, "str": "random string"} } ])
      db.geo.runCommand( {"update": "geo", "updates": [{"q": {}, "u": [{$set: {"obj": {$setField: {field: "inner.geoLegacy", input: "$obj", value: "$obj"}},}}, ], "upsert": false, "multi": false, }], "ordered": true, })
      db.geo.validate({full: true})
      

      where the created document

       

      {
        "_id": 17922,
        "obj": {
          "_id": 17923,
          "str": "random string",
          "inner.geoLegacy": {
            "_id": 17923,
            "str": "random string",
          }
        }
      }

      is using the content of the "inner.geoLegacy" field as a valid match for the indexed path "obj.inner.geoLegacy", and fails to treat it as a 2D legacy point.

       

      {
      ...
              "valid" : false,
              "repaired" : false,
      ...
              "errors" : [
                      "exception during collection validation: Location13026: geo values must be 'legacy coordinate pairs' for 2d indexes :: caused by :: { _id: 17922.0, obj: { _id: 17923.0, str: \"random string\", inner.geoLegacy: { _id: 17923.0, str: \"random string\" } } }"
              ],
      ...
              "advice" : "A corrupt namespace has been detected. See http://dochub.mongodb.org/core/data-recovery for recovery steps.",
              "ok" : 1
      }

       

       

      Given that we document that fields whose name is starting with "$" or containing "." are not included in indexes or encrypted, we should skip them rather than including them and yield wrong results.

       

      A maybe related issue is that the insertion of a dotted path matching the index key makes a document be excluded when the index is used, compared with when a collection scan is performed:

      db.x.insert([{a:1, b:{c:{x: 2, y:2}}}, {a:2, b:{c:{x:2, y:2}}}])
      db.x.update({a:2}, [{$replaceWith: {$setField: {field: "b.c", input: "$$ROOT", value: {x: 20, y:20}}}}])
      db.x.find()
      { "_id" : ObjectId("6454c87a0f747333893c632f"), "a" : 1, "b" : { "c" : { "x" : 2, "y" : 2 } } }
      { "_id" : ObjectId("6454c87a0f747333893c6330"), "a" : 2, "b" : { "c" : { "x" : 2, "y" : 2 } }, "b.c" : { "x" : 20, "y" : 20 } }
      
      db.x.find({"b.c": {$geoWithin: {$center: [ [2,2], 10]}}})
      { "_id" : ObjectId("6454c87a0f747333893c632f"), "a" : 1, "b" : { "c" : { "x" : 2, "y" : 2 } } }
      { "_id" : ObjectId("6454c87a0f747333893c6330"), "a" : 2, "b" : { "c" : { "x" : 2, "y" : 2 } }, "b.c" : { "x" : 20, "y" : 20 } }
      
      db.x.createIndex({"b.c":"2d"})
      
      db.x.find({"b.c": {$geoWithin: {$center: [ [2,2], 10]}}})
      { "_id" : ObjectId("6454c87a0f747333893c632f"), "a" : 1, "b" : { "c" : { "x" : 2, "y" : 2 } } }

       

       

            Assignee:
            alberto.massari@mongodb.com Alberto Massari
            Reporter:
            alberto.massari@mongodb.com Alberto Massari
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

              Created:
              Updated:
              Resolved: