[SERVER-5143] Automatically create necessary subdoc structure from "dot" notation on updates when fields are null Created: 29/Feb/12  Updated: 06/Apr/23

Status: Backlog
Project: Core Server
Component/s: Write Ops
Affects Version/s: 2.0.3, 2.1.0
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Caleb Jones Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 18
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Assigned Teams:
Query Optimization
Participants:

 Description   

From discussion on user group:
http://groups.google.com/group/mongodb-user/browse_thread/thread/dfdbf0cc4a6a5891

When performing updates involving subdocuments, it would greatly simplify the edge cases which need to be handled client-side if the server would automatically create the necessary subdocument(s) indicated by the "dot" notation in the update when fields in chain are null (instead of the LEFT_SUBFIELD error that is currently returned). This would align the behavior more towards how it currently behaves when fields in the chain do not exist.

However, I would expect the LEFT_SUBFIELD error if a field is currently a non-null primitive since you wouldn't want to automatically overwrite that primitive data (which is current behavior).

One common use case where this would have a very positive impact is in data migrations. In migrations you often may need to replace a null field with a subdoc as schemas evolve but may not know (or want to worry about) whether that field is currently null (preventing sometimes expensive/complex document introspection prior to updating).

Current behavior when fields don't exist:

> db.testing.insert({"id":1})
> db.testing.update({"id":1}, {$set: {"data.id" : 1}})
> db.testing.find({"id":1})
{ "_id" : ..., "data" : { "id" : 1 }, "id" : 1 }
> db.testing.update({"id":1}, {$set: {"data.sub.sub.sub" : 1}})
> db.testing.find({"id":1})
> { "_id" : .., "data" : { "id" : 1, "sub" : { "sub" : { "sub" : 1 } } }, "id" : 1 }

Current behavior when field is null:

> db.testing.insert({"id":2, "data":null})
> db.testing.update({"id":2}, {$set: {"data.id" : 1}})
LEFT_SUBFIELD only supports Object: data not: 10 

Desired behavior when field is null (same as when fields don't exist):

> db.testing.insert({"id":2, "data":null})
> db.testing.update({"id":2}, {$set: {"data.id" : 1}})
> db.testing.find({"id":1})
{ "_id" : ..., "data" : { "id" : 1 }, "id" : 2 }

Maintain current behavior when field is non-null primitive:

> db.testing.insert({"id":3, "data":"non-null primitive"})
> db.testing.update({"id":3}, {$set: {"data.id" : 1}})
LEFT_SUBFIELD only supports Object: data not: 2



 Comments   
Comment by Marcel Bennett [ 14/Aug/14 ]

What are the best work arounds for this? Are any available other than to convert all null values to {} in our database?

I was quite surprised to find it worked the way it does, doesn't really make much sense to me blocking the database from handling dynamic updates, but I can see that this is how the browsers also handle such a set operation. Browsers fail on anything other than an object being present so in keeping with the way JS works in the browser perhaps it is better to be consistent. But it would be nice to override the default behaviour at least.

Comment by Scott Lowe [ 24/Oct/13 ]

Yeah, +1 this is a bit of a pain point at the moment.

Comment by Jon Murphy [ 21/Jan/13 ]

Adding my vote to this as we have data whereby one record maybe created with an array name from one data source but we make update that record from data from another source needing a new array name which throws this off as its updating the same record but under a different array.

Comment by Caleb Jones [ 01/Mar/12 ]

As an aside, this could evolve into general support for auto-expanding "dot" notation and could apply to things like inserts or upserts. It would essentially be a kind of short-hand for expressing documents (again only for null or non-existent keys).

Current behavior:

> db.testing.insert({"id":1, "data.sub1.sub2":1})
Thu Mar  1 09:30:11 uncaught exception: can't have . in field names [data.sub1.sub2]

Desired Behavior:

> db.testing.insert({"id":1, "data.sub1.sub2":1})
> db.testing.find({"id":1})
{"_id" : ..., "id" : 1, "data" : { "sub1" : { "sub2" : 1 } } } 

Comment by Caleb Jones [ 29/Feb/12 ]

Correction in find command:

Desired behavior when field is null (same as when fields don't exist):

> db.testing.insert({"id":2, "data":null})
> db.testing.update({"id":2}, {$set: {"data.id" : 1}})
> db.testing.find({"id":2})
{ "_id" : ..., "data" : { "id" : 1 }, "id" : 2 }

Generated at Thu Feb 08 03:08:01 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.