[SERVER-36319] $ Positional Array Update Operator does not work as expected Created: 27/Jul/18  Updated: 27/Oct/23  Resolved: 31/Jul/18

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 3.6.6
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Marc Tinkler Assignee: Nick Brewer
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:

Insert this test document:

 

db.test.insert({
	"_id" : NumberLong("1046644416921"),
	"teachers" : [
		{ "lid" : 203745 },
		{ "lid" : 5551146 }
	],	
	"asn" : [
		{ "_id" : ObjectId("5b5a4a2cf3c2c0aad9e6237b") },
		{ "_id" : ObjectId("5b5a4abdf3c2c0aad9e62382") },
		{ "_id" : ObjectId("5b5a4c30f3c2c0aad9e62389") }
	]
})

 

Issue the following update:

 

db.test.update({ "asn._id" : new ObjectId("5b5a4a2cf3c2c0aad9e6237b"), "teachers.lid" : 5551146 , "_id" : 1046644416921 },
{ "$inc" : { "asn.$.mods" : 1 } })

Expected output of db.test.findOne()

 

 

{
	"_id" : NumberLong("1046644416921"),
	"teachers" : [
		{
			"lid" : 203745
		},
		{
			"lid" : 5551146
		}
	],
	"asn" : [
		{
			"_id" : ObjectId("5b5a4a2cf3c2c0aad9e6237b")
			"mods" : 1
		},
		{
			"_id" : ObjectId("5b5a4abdf3c2c0aad9e62382"),
		},
		{
			"_id" : ObjectId("5b5a4c30f3c2c0aad9e62389")
		}
	]
}

 

 

Actual output:

{
	"_id" : NumberLong("1046644416921"),
	"teachers" : [
		{
			"lid" : 203745
		},
		{
			"lid" : 5551146
		}
	],
	"asn" : [
		{
			"_id" : ObjectId("5b5a4a2cf3c2c0aad9e6237b")
		},
		{
			"_id" : ObjectId("5b5a4abdf3c2c0aad9e62382"),
			"mods" : 1
		},
		{
			"_id" : ObjectId("5b5a4c30f3c2c0aad9e62389")
		}
	]
}

Notice the wrong sub-document was updated!

Also, please note that if we do this query, we have the same problem.

db.test.update({ "asn" : {$elemMatch:{ _id:new ObjectId("5b5a4a2cf3c2c0aad9e6237b")}}, 
            "teachers.lid" : 5551146 , 
             "_id" : 1046644416921 },
{ "$inc" : { "asn.$.mods" : 1 } })

 

However, if you remove the "teachers.lid" from the query, it behaves as expected.  Also, if you do it with the "new-style" array filters, it works perfectly:

db.test.update({ "asn" : { $elemMatch:{_id:new ObjectId("5b5a4a2cf3c2c0aad9e6237b")}},
	"teachers.lid" : 5551146,
	"_id" : 1046644416921 }, 
	{ "$inc" : { "asn.$[element].mods" : 1 } },
{ arrayFilters:[{ 'element._id': new ObjectId("5b5a4a2cf3c2c0aad9e6237b")  }]})

 

 

Participants:

 Description   

If you include multiple array fields in a query, then the "$" positional array operator does not work, or worse, update the wrong (non-matching) sub-document.  



 Comments   
Comment by David Storch [ 29/Oct/18 ]

Hi tinkler@vocabulary.com,

It's not like the "$" operator is deprecated.

Correct. However, our intention is to pursue improvements to arrayFilters rather than to work on fixes to the positional operator, which could enable a potential future deprecation. Therefore, it's unlikely that we will schedule SERVER-18500 in the near term.

Also, currently the $ operator is the only way to update the first element of the array, so it's not like using arrayFilters is going to work for everyone in this situation.

Yes, great point. Adding a limit:1 capability to arrayFilters is on our list, but it looks like there is no SERVER ticket tracking this work. Would you like us to create one on your behalf?

Best,
Dave

Comment by Nick Brewer [ 31/Jul/18 ]

tinkler@vocabulary.com As you pointed out, this works as expected when the teachers.lid portion of the query is removed. The documentation does note that the $ operator should not be used with queries that traverse more than one array.

Sorry for not making that clearer in my first response.

Regards,
Nick

Comment by Marc Tinkler [ 31/Jul/18 ]

I understand that arrayFilters is the more "modern" way to do it, but there is no documentation that this type of behavior with the "$" operator even exists.  It's not like the "$" operator is deprecated.   Also, currently the $ operator is the only way to update the first element of the array, so it's not like using arrayFilters is going to work for everyone in this situation.

 

Furthermore, it should never update a document that doesn't even match the condition - that's just crazy.  Imagine adding this to the documentation:  "Be careful, if one of your query conditions is matching an array (even if it is an unrelated field), then the $ positional update operator may update a random sub-document that does not match your condition"

 

 

  

 

Comment by Nick Brewer [ 27/Jul/18 ]

tinkler@vocabulary.com Work to improve this behavior is being tracked in SERVER-18500 - you can follow along with that ticket for details.

That said, arrayFilters is the appropriate implementation for what you're trying to accomplish.

-Nick

Generated at Thu Feb 08 04:42:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.