[SERVER-40070] ReplaceOne: The dollar ($) prefixed field is not valid for storage Created: 11/Mar/19  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: 4.0.6
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Jason R. Coombs Assignee: Backlog - Query Optimization
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Duplicate
is duplicated by SERVER-56462 Field starting with dollar in an embe... Closed
Assigned Teams:
Query Optimization
Backport Requested:
v4.0, v3.6
Sprint: Query 2019-04-22
Participants:

 Description   

With MongoDB 3.6, dollar-prefixed fields were allowed. Invoking ReplaceOne on such a document, however, results in an error:

rs0:PRIMARY> db.coll.insertOne({"a": {"$foo": 1}})
{
        "acknowledged" : true,
        "insertedId" : ObjectId("5c86835900b48b0b86ee197d")
}
rs0:PRIMARY> doc = db.coll.findOne()
{ "_id" : ObjectId("5c86835900b48b0b86ee197d"), "a" : { "$foo" : 1 } }
rs0:PRIMARY> doc
{ "_id" : ObjectId("5c86835900b48b0b86ee197d"), "a" : { "$foo" : 1 } }
rs0:PRIMARY> doc['b'] = 2
2
rs0:PRIMARY> db.coll.replaceOne({}, doc)
2019-03-11T11:49:14.749-0400 E QUERY    [js] WriteError: The dollar ($) prefixed field '$foo' in 'a.$foo' is not valid for storage. :
WriteError({
        "index" : 0,
        "code" : 52,
        "errmsg" : "The dollar ($) prefixed field '$foo' in 'a.$foo' is not valid for storage.",
        "op" : {
                "q" : {                },
                "u" : {
                        "_id" : ObjectId("5c86835900b48b0b86ee197d"),
                        "a" : {
                                "$foo" : 1
                        },
                        "b" : 2
                },
                "multi" : false,
                "upsert" : false
        }
})
WriteError@src/mongo/shell/bulk_api.js:461:48
Bulk/mergeBatchResults@src/mongo/shell/bulk_api.js:841:49
Bulk/executeBatch@src/mongo/shell/bulk_api.js:906:13
Bulk/this.execute@src/mongo/shell/bulk_api.js:1150:21
DBCollection.prototype.replaceOne@src/mongo/shell/crud_api.js:489:17
@(shell):1:1

This happened on MongoDB 4.0.5, but probably happens on all versions. This smells of a validation that should have been removed in 3.6 when these field names were allowed.

I can't think of any reason that documents valid for insertOne wouldn't be valid for replaceOne.



 Comments   
Comment by Jeffrey Yemin [ 15/Jun/20 ]

I tested with the latest patches all the way back to 3.6, and this is no longer the case:

MongoDB Enterprise > db.coll.insertOne({"a": {"$foo": 1}})
{
	"acknowledged" : true,
	"insertedId" : ObjectId("5ee796f51d819368b1c6392f")
}
MongoDB Enterprise > doc = db.coll.findOne()
{ "_id" : ObjectId("5ee796f51d819368b1c6392f"), "a" : { "$foo" : 1 } }
 
MongoDB Enterprise > db.coll.replaceOne({}, doc)
{ "acknowledged" : true, "matchedCount" : 1, "modifiedCount" : 0 }

Comment by Justin Seyster [ 22/Apr/19 ]

Thanks for bringing this to our attention. We have a broader project planned for making rules about $-prefixed fields more consistent, and we'll definitely be including this work in that project. There are initial code changes in review for document verification in the server, but drivers will also need updating to make sure they all follow the new document storage rules. Before that is all in place, rules for $-prefixed field names will still have some inconsistencies, and some operations won't be possible for paths with $-prefixed fields. The changes in 3.6 didn't exhaustively address $-prefixed field uses cases, because our motivation then was primarily to support internal requirements.

Comment by Jason R. Coombs [ 11/Mar/19 ]

Same issue exists for $set.

rs0:PRIMARY> db.coll.update({}, {"$set": {"a.$foo": 5}})
WriteResult({
        "nMatched" : 0,
        "nUpserted" : 0,
        "nModified" : 0,
        "writeError" : {
                "code" : 52,
                "errmsg" : "The dollar ($) prefixed field '$foo' in 'a.$foo' is not valid for storage."
        }
})

Generated at Thu Feb 08 04:53:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.