Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-769

Validate top-level & index spec field names for the createIndexes command

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.0-rc0
    • Component/s: Indexing
    • Labels:
      None
    • Backwards Compatibility:
      Minor Change
    • Sprint:
      Integration 2016-10-10

      Description

      Add validation for createIndexes field names both at the top level and in the index specification. This is to prevent mis-spelled field names and correct field names being placed at the wrong level of the command specification (for example "background: true" being placed at the top level rather then within the index spec document.

        Issue Links

          Activity

          Hide
          wojcikstefan Stefan Wójcik added a comment -

          One more example of the same issue:

          > db.notification.getIndexes()[1]
          {
              "v" : 1,
              "key" : {
                  "_cls" : 1,
                  "membership" : 1
              },
              "ns" : "closeio.notification",
              "name" : "_cls_1_membership_1",
              "background" : false,
              "dropDups" : false
          }
          > db.notification.ensureIndex({ _cls: 1, membership: 1 })
          { "numIndexesBefore" : 14, "note" : "all indexes already exist", "ok" : 1 }
          > db.notification.ensureIndex({ _cls: 1, membership: 1 }, { background: true })
          { "numIndexesBefore" : 14, "note" : "all indexes already exist", "ok" : 1 }
          > db.notification.ensureIndex({ _cls: 1, membership: 1 }, { whatever: true })
          {
              "ok" : 0,
              "errmsg" : "Index with name: _cls_1_membership_1 already exists with different options",
              "code" : 85
          }
          > db.notification.ensureIndex({ _cls: 1, membership: 1, whatever:1 }, { whatever: true })
          {
              "createdCollectionAutomatically" : false,
              "numIndexesBefore" : 14,
              "numIndexesAfter" : 15,
              "ok" : 1
          }
          > db.notification.getIndexes()[14]
          {
              "v" : 1,
              "key" : {
                  "_cls" : 1,
                  "membership" : 1,
                  "whatever" : 1
              },
              "name" : "_cls_1_membership_1_whatever_1",
              "ns" : "closeio.notification",
              "whatever" : true
          }

          1. Calling ensureIndex with an unsupported option shouldn't consider the spec different from the existing one (i.e. the error shouldn't occur).
          2. Options that aren't supported shouldn't be stored on the index spec.

          Show
          wojcikstefan Stefan Wójcik added a comment - One more example of the same issue: > db.notification.getIndexes()[1] { "v" : 1, "key" : { "_cls" : 1, "membership" : 1 }, "ns" : "closeio.notification", "name" : "_cls_1_membership_1", "background" : false, "dropDups" : false } > db.notification.ensureIndex({ _cls: 1, membership: 1 }) { "numIndexesBefore" : 14, "note" : "all indexes already exist", "ok" : 1 } > db.notification.ensureIndex({ _cls: 1, membership: 1 }, { background: true }) { "numIndexesBefore" : 14, "note" : "all indexes already exist", "ok" : 1 } > db.notification.ensureIndex({ _cls: 1, membership: 1 }, { whatever: true }) { "ok" : 0, "errmsg" : "Index with name: _cls_1_membership_1 already exists with different options", "code" : 85 } > db.notification.ensureIndex({ _cls: 1, membership: 1, whatever:1 }, { whatever: true }) { "createdCollectionAutomatically" : false, "numIndexesBefore" : 14, "numIndexesAfter" : 15, "ok" : 1 } > db.notification.getIndexes()[14] { "v" : 1, "key" : { "_cls" : 1, "membership" : 1, "whatever" : 1 }, "name" : "_cls_1_membership_1_whatever_1", "ns" : "closeio.notification", "whatever" : true } 1. Calling ensureIndex with an unsupported option shouldn't consider the spec different from the existing one (i.e. the error shouldn't occur). 2. Options that aren't supported shouldn't be stored on the index spec.
          Hide
          kevin.pulo Kevin Pulo added a comment -

          Another case is creating a TTL index on a capped collection, which is can be created but doesn't actually achieve anything productive.

          > db.createCollection("test", {capped:true, size: 102400})
          { "ok" : 1 }
          > db.test.ensureIndex({foo:1}, {expireAfterSeconds: 10})
          {
                  "createdCollectionAutomatically" : false,
                  "numIndexesBefore" : 1,
                  "numIndexesAfter" : 2,
                  "ok" : 1
          }
          > db.test.insert({foo:new Date()})
          WriteResult({ "nInserted" : 1 })
          

          2016-01-19T18:17:11.210+1100 I STORAGE  [TTLMonitor] failing remove on a capped ns test.test
          2016-01-19T18:17:11.210+1100 E INDEX    [TTLMonitor] Error processing ttl index: { v: 1, key: { foo: 1.0 }, name: "foo_1", ns: "test.test", expireAfterSeconds: 10.0 } -- 10089 cannot remove from a capped collection
          

          If such indexes were disallowed in the first place, it could have avoided the effort on SERVER-11266, SERVER-16749 and SERVER-20920.

          In addition, the TTL docs state that creating such an index isn't possible, which isn't strictly true until this ticket is resolved.

          You cannot create a TTL index on a capped collection because MongoDB cannot remove documents from a capped collection.

          Show
          kevin.pulo Kevin Pulo added a comment - Another case is creating a TTL index on a capped collection, which is can be created but doesn't actually achieve anything productive. > db.createCollection("test", {capped:true, size: 102400}) { "ok" : 1 } > db.test.ensureIndex({foo:1}, {expireAfterSeconds: 10}) { "createdCollectionAutomatically" : false, "numIndexesBefore" : 1, "numIndexesAfter" : 2, "ok" : 1 } > db.test.insert({foo:new Date()}) WriteResult({ "nInserted" : 1 }) 2016-01-19T18:17:11.210+1100 I STORAGE [TTLMonitor] failing remove on a capped ns test.test 2016-01-19T18:17:11.210+1100 E INDEX [TTLMonitor] Error processing ttl index: { v: 1, key: { foo: 1.0 }, name: "foo_1", ns: "test.test", expireAfterSeconds: 10.0 } -- 10089 cannot remove from a capped collection If such indexes were disallowed in the first place, it could have avoided the effort on SERVER-11266 , SERVER-16749 and SERVER-20920 . In addition, the TTL docs state that creating such an index isn't possible, which isn't strictly true until this ticket is resolved. You cannot create a TTL index on a capped collection because MongoDB cannot remove documents from a capped collection.
          Hide
          charlie.swanson Charlie Swanson added a comment -

          Adding my two cents here: the test jstests/noPassthroughWithMongod/ttl1.js could be improved if we better validate the TTL arguments.

          Show
          charlie.swanson Charlie Swanson added a comment - Adding my two cents here: the test jstests/noPassthroughWithMongod/ttl1.js could be improved if we better validate the TTL arguments.
          Hide
          james.wahlin James Wahlin added a comment -

          Kevin Pulo - I created SERVER-26287 to address your TTL on capped collection concern. Please review that ticket and modify as needed.

          Show
          james.wahlin James Wahlin added a comment - Kevin Pulo - I created SERVER-26287 to address your TTL on capped collection concern. Please review that ticket and modify as needed.
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'jameswahlin', u'name': u'James Wahlin', u'email': u'james.wahlin@10gen.com'}

          Message: SERVER-769 Validate createIndexes field names
          Branch: master
          https://github.com/mongodb/mongo/commit/5563428f99af20c29cb334f97e84f0dcc1cb102a

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'jameswahlin', u'name': u'James Wahlin', u'email': u'james.wahlin@10gen.com'} Message: SERVER-769 Validate createIndexes field names Branch: master https://github.com/mongodb/mongo/commit/5563428f99af20c29cb334f97e84f0dcc1cb102a

            People

            • Votes:
              9 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                  Agile