[SERVER-17999] Regression between MongoDB 2.4 and 2.6 Created: 11/Apr/15  Updated: 22/May/15  Resolved: 21/May/15

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 2.6.7
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: NOVALUE Mitar Assignee: Ramon Fernandez Marina
Resolution: Duplicate Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-14662 Positional projection queries (and po... Closed
Related
related to DOCS-5491 getlasterror.n documentation is misle... Closed
Operating System: ALL
Steps To Reproduce:

First insert a test document:

db.test.insert({ _id: 'iJ7m2eW66eoGjR4tQ',
nested: 
 [ { required: 
      { _id: 'phitZKwpTqiH76BWS',
        username: 'person2',
        displayName: 'Person 2',
        field1: 'Field 2 - 1',
        field2: 'Field 2 - 2' } },
   { required: 
      { _id: 'phitZKwpTqiH76BWS',
        username: 'person2',
        displayName: 'Person 2' } },
   { required: { _id: 'Pua8HHJQzBbRtua9W' } },
   { required: { _id: 'Pua8HHJQzBbRtua9W' } },
   { required: 
      { _id: 'phitZKwpTqiH76BWS',
        username: 'person2',
        displayName: 'Person 2',
        field1: 'wrong' } },
   { required: { _id: 'Pua8HHJQzBbRtua9W' } }
 ]})

And then run the following update query:

db.test.update({ 'nested.required._id': 'phitZKwpTqiH76BWS',
  nested: 
   { '$elemMatch': 
      { '$or': 
         [ { 'required._id': 'phitZKwpTqiH76BWS',
             'required.username': { '$ne': 'person2' } },
           { 'required._id': 'phitZKwpTqiH76BWS',
             'required.displayName': { '$ne': 'Person 2' } },
           { 'required._id': 'phitZKwpTqiH76BWS',
             'required.field1': { '$ne': 'Field 2 - 1' } },
           { 'required._id': 'phitZKwpTqiH76BWS',
             'required.field2': { '$ne': 'Field 2 - 2' } } ] } } }, { '$set': 
   { 'nested.$.required.username': 'person2',
     'nested.$.required.displayName': 'Person 2',
     'nested.$.required.field1': 'Field 2 - 1',
     'nested.$.required.field2': 'Field 2 - 2' } })

If you test number of affected documents with:

db.runCommand({getLastError : 1})

Then on MongoDB 2.4 it will return number of affected documents "1" for first two runs of the update query, and the third one will be 0. As expected.

On MongoDB 2.6 update query will be returning:

WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 0 })

And "getLastError" will be returning "1" despite not really updating any document. Here are two issues:

Number of affected queries is 1 (node.js driver is reporting this as well) despite no document being really modified. This makes our code using this queries go into a loop because it is expecting that there are more things to still update (it is running the update query in a loop until there are no more changes done to the database).

The second issue is that no document is updated despite one being matched. It is unclear why the same update modifier does not work on 2.6 anymore.

Participants:

 Description   

When testing upgrading of our codebase to 2.6 from 2.4 some of our tests started failing. After analyzing tests we found a sample of failing update query which worked well on 2.4 but does not work as expected anymore on 2.6.



 Comments   
Comment by Ramon Fernandez Marina [ 22/May/15 ]

mitar, after asking around we've found an issue with the documentation and opened DOCS-5491 to fix it. Thanks for bringing this up.

Regards,
Ramón.

Comment by NOVALUE Mitar [ 22/May/15 ]

No, you are doing it right. But yea, documents were not updated also in your example. Why is the `n` set to 1? Shouldn't it be the same as `nModified`? So how many documents were really updated. Not how many documents were planned to be updated? I can get the planned by doing a count query.

Comment by Ramon Fernandez Marina [ 22/May/15 ]

Sorry, I forgot about the additional question. The documentation says that getLastError.n indicates the number of documents updated, and nModified is set to 0 because no documents were actually updated in the end. Here's a trivial repro:

> db.foo.insert({x:1, y:2})
WriteResult({ "nInserted" : 1 })
> db.foo.update({x:1}, {$set: {y:2}})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 0 })
> db.runCommand( { getLastError: 1, w: 1 } )
{
        "connectionId" : 22,
        "updatedExisting" : true,
        "n" : 1,
        "syncMillis" : 0,
        "writtenTo" : null,
        "err" : null,
        "ok" : 1
}

I'm not sure if this is a bug, or if this is expected and the documentation could be improved, or if I'm doing something wrong. I'll double-check, but by all means feel free to open a separate ticket to track this particular issue down.

Comment by NOVALUE Mitar [ 22/May/15 ]

Thanks for the explanation.

Also, the issue is with the return value of number of affected documents. Why it is returning non zero number when no document is really affected? I think this is a bug as well. Maybe I should open a new ticket for that separately?

Comment by Ramon Fernandez Marina [ 21/May/15 ]

mitar, upon further examination I can confirm that this is expected behavior and indeed a duplicate of SERVER-14662: starting with 2.6 the positional operator is unambiguously chosen.

Let's look at what 2.4 would do; consider your document and the update from the previous post (but updating a field named "field3" instead of "field2" for added clarity):

db.test.update({ 'nested.required._id': 'phitZKwpTqiH76BWS',
  nested: 
   { '$elemMatch': 
      { 'required._id': 'phitZKwpTqiH76BWS',
        'required.field2': { '$ne': 'Field 2 - 2' } } } }, { '$set': 
   { 'nested.$.required.field3': 'Field 2 - 2' } })

This would update the second field in nested.required to read:

                {
                        "required" : {
                                "_id" : "phitZKwpTqiH76BWS",
                                "username" : "person2",
                                "displayName" : "Person 2"
                                "field3" : "Field 2 - 2",
                        }
                },

Now let's reverse the order of the query predicate in the update command:

db.test.update({ nested: 
   { '$elemMatch': 
      { 'required._id': 'phitZKwpTqiH76BWS',
        'required.field2': { '$ne': 'Field 2 - 2' } } }, 'nested.required._id': 'phitZKwpTqiH76BWS'}, 
{ '$set': { 'nested.$.required.field3': 'Field 2 - 2' } })

This will change the first element in nested instead of the second, and it would look like this (this explains why I chose "field3"):

                        "required" : {
                                "_id" : "phitZKwpTqiH76BWS",
                                "displayName" : "Person 2",
                                "field1" : "Field 2 - 1",
                                "field2" : "Field 2 - 2",
                                "field3" : "Field 2 - 2",
                                "username" : "person2"
                        }

In other words, having the value of the positional operator depend on the order of the fields in the query predicate was the behavior in 2.4 and earlier versions and was considered ambiguous and error prone, while in 2.6 this choice was made unambiguous.

I see you've already commented on SERVER-14662 and are following that ticket, so you'll receive updates as they happen.

Regards,
Ramón.

Comment by NOVALUE Mitar [ 11/Apr/15 ]

I would not agree that it is a duplicate.

The update query is not ambiguous, it might be redundant, but it is not ambiguous. It is clear which array element this query is matching first. Only one first match like this exists. There is AND between conditions, so there is only one such first element in the array.

db.test.update({ 'nested.required._id': 'phitZKwpTqiH76BWS',
  nested: 
   { '$elemMatch': 
      { 'required._id': 'phitZKwpTqiH76BWS',
        'required.field2': { '$ne': 'Field 2 - 2' } } } }, { '$set': 
   { 'nested.$.required.field2': 'Field 2 - 2' } })

Also, the issue is with the return value of number of affected documents. Why it is returning non zero number when no document is really affected? I think this is a bug as well. Maybe I should open a new ticket for that separately?

Comment by Ramon Fernandez Marina [ 11/Apr/15 ]

mitar, I believe this is a duplicate of SERVER-14837, please see this comment for an explanation on the ambiguity of positional operators in 2.4.

You're also familiar with SERVER-11537, which points to SERVER-14662 for improvements in this area.

Regards,
Ramón.

Comment by NOVALUE Mitar [ 11/Apr/15 ]

But this one does:

db.test.update({
  nested: 
   { '$elemMatch': 
      { 'required._id': 'phitZKwpTqiH76BWS',
        'required.field2': { '$ne': 'Field 2 - 2' } } } }, { '$set': 
   { 'nested.$.required.field2': 'Field 2 - 2' } })

Comment by NOVALUE Mitar [ 11/Apr/15 ]

Even this one not:

db.test.update({ 'nested.required._id': 'phitZKwpTqiH76BWS',
  nested: 
   { '$elemMatch': 
      { 'required._id': 'phitZKwpTqiH76BWS',
        'required.field2': { '$ne': 'Field 2 - 2' } } } }, { '$set': 
   { 'nested.$.required.field2': 'Field 2 - 2' } })

Comment by NOVALUE Mitar [ 11/Apr/15 ]

Even this update query does not work:

db.test.update({ 'nested.required._id': 'phitZKwpTqiH76BWS',
  nested: 
   { '$elemMatch': 
      { 'required._id': 'phitZKwpTqiH76BWS',
        'required.field2': { '$ne': 'Field 2 - 2' } } } }, { '$set': 
   { 'nested.$.required.username': 'person2',
     'nested.$.required.displayName': 'Person 2',
     'nested.$.required.field1': 'Field 2 - 1',
     'nested.$.required.field2': 'Field 2 - 2' } })

Comment by NOVALUE Mitar [ 11/Apr/15 ]

The same issue appears also on 2.6.9 and 3.0.2.

Generated at Thu Feb 08 03:46:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.