[JAVA-2810] Allow dots and $ in field names Created: 14/Mar/18 Updated: 27/Oct/19 Resolved: 12/Mar/19 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | Command Operations |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Alex Vayda | Assignee: | Scott L'Hommedieu (Inactive) |
| Resolution: | Won't Fix | Votes: | 7 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||
| Description |
|
MongoDB 3.6 allows using dots and dollar signs in the field names. Please remove this restriction from CollectibleDocumentFieldNameValidator |
| Comments |
| Comment by Oz Madar [ 13/Mar/19 ] |
|
Thank you scott.lhommedieu for the workaround |
| Comment by Scott L'Hommedieu (Inactive) [ 12/Mar/19 ] |
|
We'll deal with this in |
| Comment by Scott L'Hommedieu (Inactive) [ 12/Mar/19 ] |
|
Created |
| Comment by Scott L'Hommedieu (Inactive) [ 12/Mar/19 ] |
|
Hi Oz, There is a way to way to bypass these limitations on the write by using run command syntax from drivers. Here for Java. Until the whole of the MongoDB platform is geared toward supporting this we do not want to encourage this in Drivers. |
| Comment by Oz Madar [ 17/Jan/19 ] |
|
Thanks Scott, yet I don't understand why not allowing "power-users" that choose to skip this validation? Our data-model contains dots in fields names, the result of protecting us is that I have to modify the data model. By the way, I cloned driver-core git repo, deleted the validation, compiled it and it works fine for me. I would be very happy if I could cancel that validation with some simple configuration (env, java_opts etc.) and it will not have any effect on other parts (I not asking for queries capabilities, just not preventing an allowed insert behavior) |
| Comment by Scott L'Hommedieu (Inactive) [ 17/Jan/19 ] |
|
My apologies, I should have been more clear and accurate when I stated "usable way". The server has not adequately defined behavior regarding reading and writing . and $ in field names such that drivers can make a clear and lasting decision on an API to expose that behavior. If we make these decisions in drivers now the drivers would provide an interface that would likely be up for deprecation as soon as the server does implement this behavior and even worse would be susceptible to breaking changes from server changes. Also, there are currently options available to write and read values with keys that contain . and $. These options aren't as simple to use as we would like (i.e. no helper methods). But, this is also because the server support for these characters in field names is not complete. See: SERVER-30575 and Rather than add driver functionality that may be invalidated by the results of those server changes we opt to continue to disallow those in drivers, wait for the changes in server and advocate that users try the options detailed in the server tickets. |
| Comment by Alex Vayda [ 09/Jan/19 ] |
|
Not sure I'm following. What is your definition of a usable way? Lacking ability to query on fields with dots doesn't mean storing those fields isn't usable. If I get it right the main concern here is to be able just to store and get. Isn't is a valid case in your opinion? To me it looks like a unilateral decision for forbidding this functionality in the driver has already been made in the first place. |
| Comment by Oz Madar [ 09/Jan/19 ] |
|
@Scott L'Hommedieu, The server supports storing such documents, the problem is only in querying. Preventing insert from on the driver side is changing the declared behaviour of MongoDB. What about making it configurable, e.g by adding some JVM_OPTS that will allow the user disabling this validation? |
| Comment by Scott L'Hommedieu (Inactive) [ 09/Jan/19 ] |
|
We don't want to make a unilateral decision in any drivers for behavior of all query functionality. Server does not support this in a usable way. This is blocked on SERVER-30575 |
| Comment by Bastien Arata [ 31/Aug/18 ] |
|
@Michael Everett During my debugging, I found the flag in the Node.js driver, but it'is not documented. Flag is checkKeys, you can see an exemple on this comment. Maybe this hidden flag exists in the Java driver too ? |
| Comment by Michael Everett [ 23/Aug/18 ] |
|
@BHackett This seems like an overly restrictive approach to solving this problem. The reality is, there are many very valid reasons to want to store fields that contains these characters without ever wanting to query against them (storing JSON that you haven't generated, storing JsonSchema are two that come to mind). I'd imagine that these are the most common use cases for wanting fields containing these characters. If the server is supporting these characters, at the very least it seems the driver should include a flag to disable this validation.
|
| Comment by Terence Bodola [ 07/Aug/18 ] |
|
I'm sticking data into Mongo using Java in one environment and then pulling whole docs from collections by an indexed root property (rocket fast) and managing grouping/aggregation in JavaScript code in Node.JS in another environment, using bracket instead of dot notation (also rocket fast ...didn't do in Mongo because a bit complex of an operation for agg pipeline and dead simple in custom code). I get what you are saying, but there is definitely a use case for just being able to get the data in using Java at least.
|
| Comment by Bernie Hackett [ 07/Aug/18 ] |
|
Though dots and dollar signs are now allowed in keys by the server, there is still no way to unambiguously query on such fields. You can put the data in, but you may have a very hard time getting it back out. Until that problem is solved in the server we'll still be rejecting these keys in drivers. |
| Comment by Terence Bodola [ 07/Aug/18 ] |