[SERVER-5674] Updates with $ sign in key name fail Created: 21/Apr/12 Updated: 09/Jul/16 Resolved: 27/Jul/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Usability, Write Ops |
| Affects Version/s: | 2.0.1, 2.0.4, 2.1.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Peter Uhrig | Assignee: | Scott Hernandez (Inactive) |
| Resolution: | Duplicate | Votes: | 2 |
| Labels: | update, validation | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Tested on 2.0.1 on 64bit Linux and on 2.0.4 on 32bit Windows XP |
||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Participants: | |||||||||
| Description |
|
db.test.insert( {"_id": "someid", "attribute": "somevalue"}); ,{$set: {"pos_list": {"PRP$": "PRP$ - Possessive pronoun"}}}); The second line returns the following error: Interestingly, inserting the same attribute-value pair works: ); |
| Comments |
| Comment by Jullian Chavez [ 26/Aug/12 ] | |||||||||||||||||||||
|
Ok thanks for the help. It makes sense that it's the driver causing my problem. And I agree, this should be a server-level enforcement. Oh well, guess it's time to write something to recursively rename the youtube field names as there are a TON of them. Thanks again. | |||||||||||||||||||||
| Comment by Glenn Maynard [ 26/Aug/12 ] | |||||||||||||||||||||
|
You should never be able to insert keys that start with a $. It seems like this is enforced at the driver level, which is a bit strange; it should be enforced at the server level. Doing it in drivers just duplicates work and creates inconsistent behavior. This bug is about not being able to perform updates when keys contain a non-leading $, which is valid. For example, while {$PRP: 1} is illegal, {PRP$: 1}is perfectly legal. You may be hitting a similar bug in the other direction, with your driver allowing you to insert "$t" when it shouldn't. (This is all fallout from the original problem, IMO: this should all be enforced uniformly at the server, not in each driver.) Also note that there are other cases of JSON that you can't store directly in Mongo. In particular, Mongo keys can't contain ".", which is valid in JSON. If you want to store external data that comes from JSON, you'll need to scan through and escape $ and . in keys. Otherwise, even if your driver accidentally allows it, you're liable to get bitten later on if Mongo adds (say) a "$t" operator. | |||||||||||||||||||||
| Comment by Jullian Chavez [ 26/Aug/12 ] | |||||||||||||||||||||
|
Let me clarify. I'm using the Codeigniter framework (PHP) to build my apps. One of the apps connects to Youtube's API to collect information about a user's videos. The response from Youtube looks something along the lines of: {"title":{"$t":"My Video Title"}} (obviously the whole response has a lot more data, but this bit is all that matters). I then turn this json-encoded string into a PHP array and add the video's id: Array( I then store the entire thing in mongodb using a library (PHP class) written for codeigniter: The equivalent of this in the mongo shell would be: Interestingly enough, using the PHP class to insert this data works perfectly fine, while running the shell command above gives the error: field names cannot start with $ [$t]. Not sure how the PHP class is getting by this, but mongo seems to have no issue adding fields containing a "$". Now, on to why this is relevant to this thread: When I use the same PHP class to UPDATE the document i just made, i get the error "not okForStorage" by using this command: The equivalent of which in the mongo shell would be: ,{$set:{title:{$t:"My New Video Title"}}}); In the end, my entire question is why would I be able to insert fields whose names contain a "$", but not update them? The only reason i am trying to do this is because i'm trying to store the ENTIRE Youtube API response, many field of which contain a "$" for whatever reason. Again, for some reason i can insert this data, but not update it. | |||||||||||||||||||||
| Comment by Glenn Maynard [ 26/Aug/12 ] | |||||||||||||||||||||
|
Jullian: Are you storing "{'title':{'$t':'My Youtube Video'}}" as a JSON-encoded string, or are you decoding it and attempting to store it as a Mongo structure directly? It looks like you're decoding it, because JSON uses double-quotes for strings, not single-quotes. You can't use keys starting with $, eg. "$t"; they need to be escaped manually. That's not a bug, but I'm not quite sure what you're doing, since Mongo won't let you insert it, either. (That would be related to this bug if you were using the literal string "{'title':{'$t':'My Youtube Video'}}" as a long document key, which is possible but seems unlikely...) In any case, remember that you can always use escaping to avoid inserting '$' in documents in any place they're causing issues. | |||||||||||||||||||||
| Comment by Jullian Chavez [ 25/Aug/12 ] | |||||||||||||||||||||
|
This is also causing a HUGE project-stopping issue. Youtube's API returns a JSON string with many keys containing a "$" in them. Storing this data into Mongo is not a problem the first time you insert the data, but when you try to update it, it returns "not okForStorage". | |||||||||||||||||||||
| Comment by Antonio Lobato [ 22/Aug/12 ] | |||||||||||||||||||||
|
I've just run into this bug as well and is a major show stopper. Using the Ruby driver, the following works: @data.update( {'_id' => id}, data, {:upsert => true, :safe => true})While this returns not okForStorage: data.each do |name, mdata| @data.update({'_id' => id} , {'$set' => {name => mdata}}, {:upsert => true, :safe => true}) | |||||||||||||||||||||
| Comment by Peter Uhrig [ 21/Apr/12 ] | |||||||||||||||||||||
|
It also causes no problems within a list: ,{$set: {"pos_list": [ {"PRP": "PRP - Personal pronoun"}, {"PRP$": "PRP$ - Possessive pronoun"}, {"RB": "RB - Adverb"}]}}); | |||||||||||||||||||||
| Comment by Scott Hernandez (Inactive) [ 21/Apr/12 ] | |||||||||||||||||||||
|
full trace
|