Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-12266

Update no longer allows empty modifier objects

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Works as Designed
    • Affects Version/s: 2.5.4
    • Fix Version/s: None
    • Component/s: Write Ops
    • Labels:
      None
    • Operating System:
      ALL

      Description

      tl;dr:

      There is no consistent way to upsert an identifier-only document between 2.4.x and 2.5.x. Allowing modifiers in a newObj to be an empty BSON object would help.


      I noticed the following while testing Doctrine ODM against 2.5.x. In 2.4.x and earlier versions, I believe there was only a single way to upsert a document that only contained an identifier field:

      > db.foo.update({_id:1}, {$set: {}}, true); db.getLastErrorObj();
      {
      	"updatedExisting" : false,
      	"n" : 1,
      	"connectionId" : 1,
      	"err" : null,
      	"ok" : 1
      }
      > db.foo.update({_id:2}, {}, true); db.getLastErrorObj();
      {
      	"updatedExisting" : false,
      	"upserted" : ObjectId("52cb12553ad84dc22b80c72f"),
      	"n" : 1,
      	"connectionId" : 1,
      	"err" : null,
      	"ok" : 1
      }
      > db.foo.update({_id:3}, {$set: {_id:3}}, true); db.getLastErrorObj();
      {
      	"err" : "Mod on _id not allowed",
      	"code" : 10148,
      	"n" : 0,
      	"connectionId" : 1,
      	"ok" : 1
      }
      > db.foo.find()
      { "_id" : 1 }
      { "_id" : ObjectId("52cb12553ad84dc22b80c72f") }

      Using an empty object for the newObj argument results in the upsert ignoring the client-provided _id. $set cannot be used on _id, even if that would technically be OK for an upsert. $setOnInsert would make more sense, but it also doesn't work – it's also 2.4+ only, so I wouldn't rely on it for essential ODM logic.

      In 2.5.x, the one working method from 2.4.x no longer works. The two methods that didn't work in 2.4.x do work in 2.5.x:

      > db.foo.update({_id:1}, {$set: {}}, true); db.getLastErrorObj();
      {
      	"err" : "'$set' is empty. You must specify a field like so: {$mod: {<field>: ...}}",
      	"code" : 16840,
      	"n" : 0,
      	"connectionId" : 2,
      	"ok" : 1
      }
      > db.foo.update({_id:2}, {}, true); db.getLastErrorObj();
      {
      	"updatedExisting" : false,
      	"upserted" : 2,
      	"n" : 1,
      	"connectionId" : 2,
      	"syncMillis" : 0,
      	"writtenTo" : null,
      	"err" : null,
      	"ok" : 1
      }
      > db.foo.update({_id:3}, {$set: {_id:3}}, true); db.getLastErrorObj();
      {
      	"updatedExisting" : false,
      	"upserted" : 3,
      	"n" : 1,
      	"connectionId" : 2,
      	"syncMillis" : 0,
      	"writtenTo" : null,
      	"err" : null,
      	"ok" : 1
      }
      > db.foo.find()
      { "_id" : 2 }
      { "_id" : 3 }

      The strict validation that makes the 2.4.x solution no longer work looks to have been introduced in this commit for SERVER-7175.

        Issue Links

          Activity

          Hide
          behackett Bernie Hackett added a comment -

          Jacob Heller,

          Furthermore, with the default write concern the way it is (which I think is new), update exceptions are not explicitly reported back to the system.

          The default write concern of

          {w: 1}

          acknowledges all write operations and reports any errors. This has been the default ever since MongoClient was introduced in PyMongo quite a while back. Are you saying that the server is not reporting errors back to the client in certain cases?

          Show
          behackett Bernie Hackett added a comment - Jacob Heller , Furthermore, with the default write concern the way it is (which I think is new), update exceptions are not explicitly reported back to the system. The default write concern of {w: 1} acknowledges all write operations and reports any errors. This has been the default ever since MongoClient was introduced in PyMongo quite a while back. Are you saying that the server is not reporting errors back to the client in certain cases?
          Hide
          JacobGH111 Jacob Heller added a comment - - edited

          I'm using Pymogo (version 2.7.1). The issue arises when I'm using $set and $unset at the same time. If write (in Python):
          myDB.myTable.update(

          {'_id':12345}

          , {'$set':

          {'setThisField':123}

          , '$unset':{}})

          Then the function returns nothing. There is no indication that setting setThisField failed because the argument to $unset was emtpy. This is what I mean by data being mysteriously lost. Indeed, if I specify "w=1" as an argument to update, then it raises an exception indicating that the save failed, but any code that is designed for older versions of Mongo are potentially going to fall into this pitfall of failing silently.

          As Glenn Maynard suggested, this is a good place to create a warning, but you shouldn't lose data that you're trying to $set because $unset was empty.

          Show
          JacobGH111 Jacob Heller added a comment - - edited I'm using Pymogo (version 2.7.1). The issue arises when I'm using $set and $unset at the same time. If write (in Python): myDB.myTable.update( {'_id':12345} , {'$set': {'setThisField':123} , '$unset':{}}) Then the function returns nothing. There is no indication that setting setThisField failed because the argument to $unset was emtpy. This is what I mean by data being mysteriously lost. Indeed, if I specify "w=1" as an argument to update, then it raises an exception indicating that the save failed, but any code that is designed for older versions of Mongo are potentially going to fall into this pitfall of failing silently. As Glenn Maynard suggested, this is a good place to create a warning, but you shouldn't lose data that you're trying to $set because $unset was empty.
          Hide
          behackett Bernie Hackett added a comment -

          Please make sure you are using pymongo.MongoClient, not pymongo.Connection which has been deprecated since PyMongo 2.4 and is being removed in PyMongo 3.0. With MongoClient there is no need to pass w=1 in the update statement:

          >>> import pymongo
          >>> c = pymongo.MongoClient()
          >>> c.myDB.myTable.update({'_id': 12345}, {'$set': {'setThisField': 123}, '$unset': {}})
          Traceback (most recent call last):
            File "<stdin>", line 1, in <module>
            File "pymongo/collection.py", line 574, in update
              _check_write_command_response(results)
            File "pymongo/helpers.py", line 206, in _check_write_command_response
              raise OperationFailure(error.get("errmsg"), error.get("code"), error)
          pymongo.errors.OperationFailure: '$unset' is empty. You must specify a field like so: {$mod: {<field>: ...}}

          Also, make sure your application isn't globally setting w: 0 as its default write concern.

          Show
          behackett Bernie Hackett added a comment - Please make sure you are using pymongo.MongoClient, not pymongo.Connection which has been deprecated since PyMongo 2.4 and is being removed in PyMongo 3.0. With MongoClient there is no need to pass w=1 in the update statement: >>> import pymongo >>> c = pymongo.MongoClient() >>> c.myDB.myTable.update({'_id': 12345}, {'$set': {'setThisField': 123}, '$unset': {}}) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "pymongo/collection.py", line 574, in update _check_write_command_response(results) File "pymongo/helpers.py", line 206, in _check_write_command_response raise OperationFailure(error.get("errmsg"), error.get("code"), error) pymongo.errors.OperationFailure: '$unset' is empty. You must specify a field like so: {$mod: {<field>: ...}} Also, make sure your application isn't globally setting w: 0 as its default write concern.
          Hide
          JacobGH111 Jacob Heller added a comment -

          Ah I did not realize that pymongo.Connection had been replaced. I will replace that right away, but I still think $set and $unset should allow empty arguments. Thanks!

          Show
          JacobGH111 Jacob Heller added a comment - Ah I did not realize that pymongo.Connection had been replaced. I will replace that right away, but I still think $set and $unset should allow empty arguments. Thanks!
          Hide
          hrach Hrachya Mnatsakanyan added a comment -

          I think this restriction is not very important and it would be better to revert this change as I have to use version 2.4.12 instead of the latest to avoid this restriction as I would have to do more changes.

          Show
          hrach Hrachya Mnatsakanyan added a comment - I think this restriction is not very important and it would be better to revert this change as I have to use version 2.4.12 instead of the latest to avoid this restriction as I would have to do more changes.

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Days since reply:
                35 weeks, 3 days ago
                Date of 1st Reply: