Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-20529

WiredTiger allows capped collection objects to grow

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.0.2
    • Fix Version/s: 3.2.0-rc0
    • Component/s: Storage, WiredTiger
    • Labels:
      None
    • Backwards Compatibility:
      Minor Change
    • Operating System:
      ALL
    • Sprint:
      QuInt A (10/12/15)

      Description

      In MMAP items in a capped collection cannot grow - as per the documentation and an error is raised if they do.

      > db.createCollection("cap",{capped:true,size:128000})
      { "ok" : 1 }
      >  db.cap.update({_id:1},{$set:{test:"hello",vals:["one"]}},true,true)
      WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : 1 })
      >  db.cap.update({_id:1},{$set:{test:"hello"},$push:{vals:"two"}},true,true)
      WriteResult({
      	"nMatched" : 0,
      	"nUpserted" : 0,
      	"nModified" : 0,
      	"writeError" : {
      		"code" : 10003,
      		"errmsg" : "failing update: objects in a capped ns cannot grow"
      	}
      })
      

      In WiredTiger they are allowed to grow - is this deliberate?

      > db.createCollection("cap",{capped:true,size:128000})
      { "ok" : 1 }
      > db.cap.update({_id:1},{$set:{test:"hello",vals:["one"]}},true,true)
      WriteResult({ "nMatched" : 0, "nUpserted" : 1, "nModified" : 0, "_id" : 1 })
      > db.cap.findOne()
      { "_id" : 1, "test" : "hello", "vals" : [ "one" ] }
      > db.cap.update({_id:1},{$set:{test:"hello"},$push:{vals:"two"}},true,true)
      WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
      > db.cap.findOne()
      { "_id" : 1, "test" : "hello", "vals" : [ "one", "two" ] }
      

        Issue Links

          Activity

          Hide
          asya Asya Kamsky added a comment -

          Capped collection updates cannot be allowed regardless of storage engine since WT primary may be replicating to MMAPV1 secondary and the update would fail on the secondary.

          Show
          asya Asya Kamsky added a comment - Capped collection updates cannot be allowed regardless of storage engine since WT primary may be replicating to MMAPV1 secondary and the update would fail on the secondary.
          Hide
          david.hows David Hows added a comment - - edited

          Looking through the update code, as of 3.1.8 (SERVER-19551), WT will fail any update that grows a document in the oplog, but this check does not extend to capped collections as a whole.

          To me, Asya's suggestion around blocking all updates to capped collections seems the best direction to pursue as she is spot on about potential replication issues for any update to a capped collection and it would seem unwise to insist that each storage engine should perform this check at insert time.

          Does anyone know of cases where we currently need to update a capped collection internally? I couldn't think of any, can anyone else? Does MMS rely on doing updates (including non-growth updates) internally?

          The only suggested reason in SERVER-19551 for why we would allow non-growth updates to capped collections is to allow recovery in tricky situations where updating the oplog to turn an existing operation into a noop is required.

          Show
          david.hows David Hows added a comment - - edited Looking through the update code, as of 3.1.8 ( SERVER-19551 ), WT will fail any update that grows a document in the oplog, but this check does not extend to capped collections as a whole. To me, Asya's suggestion around blocking all updates to capped collections seems the best direction to pursue as she is spot on about potential replication issues for any update to a capped collection and it would seem unwise to insist that each storage engine should perform this check at insert time. Does anyone know of cases where we currently need to update a capped collection internally? I couldn't think of any, can anyone else? Does MMS rely on doing updates (including non-growth updates) internally? The only suggested reason in SERVER-19551 for why we would allow non-growth updates to capped collections is to allow recovery in tricky situations where updating the oplog to turn an existing operation into a noop is required.
          Hide
          geert.bosch Geert Bosch added a comment -

          By far the easiest and would be to uniformly disallow all updates that result in size changes. The reason we should probably still allow some updates that do not result in size changes is that these are sometimes used to fixup logOp entries, which is considered an important capability by our support team.

          Allowing records to shrink is not very good, as such an update cannot be undone (because that would result in growth), or we'd have to pad shrunk records, which is really a concept I'd like to avoid outside of MMAPv1.

          Show
          geert.bosch Geert Bosch added a comment - By far the easiest and would be to uniformly disallow all updates that result in size changes. The reason we should probably still allow some updates that do not result in size changes is that these are sometimes used to fixup logOp entries, which is considered an important capability by our support team. Allowing records to shrink is not very good, as such an update cannot be undone (because that would result in growth), or we'd have to pad shrunk records, which is really a concept I'd like to avoid outside of MMAPv1.
          Hide
          pasette Dan Pasette added a comment -

          For the reasons Geert gave, we need to allow updates which don't change the document size. However, I think we should restrict all storage engines to allow only updates which don't change the document size in capped collections. I don't think we want other storage engines to have to have the concept of a "Record" which wraps the document as we do in MMAPv1.

          This is a backward breaking change. Does anyone know of any legitimate use cases which this would break?

          Show
          pasette Dan Pasette added a comment - For the reasons Geert gave, we need to allow updates which don't change the document size. However, I think we should restrict all storage engines to allow only updates which don't change the document size in capped collections. I don't think we want other storage engines to have to have the concept of a "Record" which wraps the document as we do in MMAPv1. This is a backward breaking change. Does anyone know of any legitimate use cases which this would break?
          Hide
          xgen-internal-githook Githook User added a comment -

          Author:

          {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'}

          Message: SERVER-20529: Prevent size changes in capped updates
          Branch: master
          https://github.com/mongodb/mongo/commit/688e3d2650d1ca67ff9c9a6d09f02e6edbc02061

          Show
          xgen-internal-githook Githook User added a comment - Author: {u'username': u'GeertBosch', u'name': u'Geert Bosch', u'email': u'geert@mongodb.com'} Message: SERVER-20529 : Prevent size changes in capped updates Branch: master https://github.com/mongodb/mongo/commit/688e3d2650d1ca67ff9c9a6d09f02e6edbc02061

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                  Agile