[SERVER-50041] Preserve field order between $set clause and updated document Created: 31/Jul/20  Updated: 30/Nov/20  Resolved: 30/Nov/20

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: 俊仁 梁 Assignee: Bernard Gorman
Resolution: Won't Do Votes: 0
Labels: qexec-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

mongodb 4.0.4


Sprint: Query 2020-11-30, Query 2020-12-14
Participants:

 Description   

Using { $set:{ b:1, a:1 } } for updating results in { a:1, b:1 }.

If I use older version like mongodb 3.4.0, the result will be { b:1, a:1 }.



 Comments   
Comment by Bernard Gorman [ 30/Nov/20 ]

This change is a consequence of the update system modernization work undertaken in SERVER-28621, which was part of the broader "Array Updates" project in 3.6. As Eric outlined above, a $set statement is simply a series of directives about which fields to update with what values, but the order of fields in the $set does not guarantee the order of creation of the relevant fields if they do not already exist in the input document. The fact that this happened prior to 3.6 was simply an implementation detail of the previous update system. Under the new system, output documents will be generated consistently for all equivalent input $set statements. In view of this, I'm going to close this ticket as Won't Do.

Note, finally, that it is possible to define the order in which fields are added into a document, by using pipeline-style updates to encode the desired field ordering as part of the request:

replset:PRIMARY> db.testing.insert({_id: 0})
WriteResult({ "nInserted" : 1 })
replset:PRIMARY> db.testing.update({_id: 0}, [{$set: {c: 2}}, {$set: {a: 1}}])
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })
replset:PRIMARY> db.testing.find()
{ "_id" : 0, "c" : 2, "a" : 1 }

Comment by Eric Sedor [ 12/Aug/20 ]

To clarify, it is not currently true that the order of fields in a JSON or BSON document implies the order of creation of those fields.

If tracking the order of updates to documents in such a way is necessary for your use-case, we recommend encoding that information explicitly.

It's worth noting that the order of fields in queries as written can also be changed when queries defined in language-native data types (JSON in JavaScript, Dict in Python, etc.) are serialized to BSON for transmission to the server.

I'm going to pass this request on to an appropriate team for consideration.

Comment by 俊仁 梁 [ 06/Aug/20 ]

Like this:

  • Field order means the order of creation of that field, so field order just matters.
  • Updating a document  {a:1, c:1} with {$set:{a:2,b:1}} results in  {a:2, c:1, b:1}. Fine.
  • Updating a document  {a:1, c:1} with {$set:{c:2,a:1}} results in  {a:1, c:2}. Fine.

But:

  • Updating a document { } with {$set:{c:2,a:1}} results in {a:1, c:2}. Not Fine. {c:2,a:1} is wanted

 

My problem is order matters when creating new fields, which seems to be impossible now. 

Comment by Eric Sedor [ 05/Aug/20 ]

Can you clarify:

  • what is the document field order logically telling the application?
  • what field order does your use require when updating a document {a:1, c:1} with {$set:{a:2,b:1}}?
  • what about {a:1, c:1} receiving {$set:{c:2,a:1}}?

This information will help us understand the requirements and may help us suggest workarounds.

Comment by 俊仁 梁 [ 05/Aug/20 ]

Hello,

I need to add two more fields to a document with certain order in only one update op, which seems to be impossible without this improvement or bugfix.

Thanks.

 

Comment by Eric Sedor [ 04/Aug/20 ]

Hello,

Order of fields within documents is handled as described here.

But this does not apply to the order of fields in a $set clause at all. We'd consider this an improvement rather than a bug.

If you'd like to see MongoDB support such a guarantee can you describe your use-case and explain how you'd like to see $set order handled in situations where fields exist in the document in a different order than in the $set?

Thanks,
Eric

Generated at Thu Feb 08 05:21:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.