[SERVER-13785] Can't canonicalize query: BadValue bad sort specification Created: 29/Apr/14 Updated: 10/Dec/14 Resolved: 01/May/14 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Querying |
| Affects Version/s: | 2.6.0 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Roman Tkachenko | Assignee: | Unassigned |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Backwards Compatibility: | Major Change | |
| Operating System: | ALL | |
| Steps To Reproduce: | Run in the Mongo shell:
Expected: Returned updated document. Actual: Returned an error. |
|
| Participants: |
| Description |
|
After upgrading from 2.4 to 2.6.0 our find_and_modify commands which use sorting started failing with the above error. PyMongo generates the following command for find_and_modify with sorting:
I tested it with 2.4 and it works fine, but on 2.6.0 it fails with the error (with the latest PyMongo 2.7):
Looks like MongoDB has stopped respecting "sort: [ [ "_lock", 1 ] ]" syntax. Is this a bug or expected change? This effectively breaks PyMongo's find_and_modify with sort functionality. |
| Comments |
| Comment by J Rassi [ 01/May/14 ] | |||||||||||||||||||||||||||||||||||||
|
r0mant: it seems that you're confusing PyMongo syntax with the server's query syntax. The ability to use an array as a sort spec is a convenience offered by some user-facing methods in PyMongo, which translate it into query syntax before sending it over to the server. It has never been valid to send these directly to the server (in 2.4 and earlier the server ignores invalid sort specs, but in 2.6 the server returns an error: see the sort() Specification Values page). To illustrate that the server ignores invalid sort specs, see the following shell session from 2.4.10. The document targeted by findAndModify is not the document with the lowest value of _lock.
Your application should be using the PyMongo find_and_modify() method (which performs sort spec translation), not the low-level command() method (which doesn't know what a sort spec is). Resolving as works-as-designed. | |||||||||||||||||||||||||||||||||||||
| Comment by Bernie Hackett [ 30/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
Sorry for the confusion. There was an assumption here that you were reporting a PyMongo bug, not a server bug. I apologize. It seems you are correct. In MongoDB 2.4 your syntax is allowed:
In MongoDB 2.6 it is not:
I have to let the kernel team investigate further. I'm not sure if this behavior change was intentional or not. | |||||||||||||||||||||||||||||||||||||
| Comment by Roman Tkachenko [ 30/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
Bernie, you're right - my point was that it used to work in 2.4.x. | |||||||||||||||||||||||||||||||||||||
| Comment by Roman Tkachenko [ 30/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
Bernie, This is the code snippet to reproduce the issue. It does not use "find_and_modify" method directly:
It gives me this:
So I guess this is not the issue with PyMongo per-se, but rather with Mongo core that does not accept sort specification like [("_lock", pymongo.ASCENDING)] anymore. If this is the intended behavior, feel free to close the issue. | |||||||||||||||||||||||||||||||||||||
| Comment by Bernie Hackett [ 29/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
I just realized that the example you posted with the error is from the mongodb shell, not PyMongo:
PyMongo takes a list of lists/tuples as it's sort argument because python dicts are not order preserving. It converts the list of lists/tuples to an ordered dict behind the scenes so the user doesn't have to deal with bson.son.SON or collections.OrderedDict. The mongo shell, on the other hand, expects you to pass a document for the sort parameter. | |||||||||||||||||||||||||||||||||||||
| Comment by Bernie Hackett [ 29/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
I can't reproduce this issue, with or without an index on _lock:
Note I'm not using datetime here since your example is a syntax error in python 2.x and 3.x:
Can you post the full traceback you are getting from find_and_modify? | |||||||||||||||||||||||||||||||||||||
| Comment by Roman Tkachenko [ 29/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
Hi Bernie, Originally I faced this behavior with PyMongo 2.5 but then (just today) installed the latest PyMongo 2.7 and got the same result. | |||||||||||||||||||||||||||||||||||||
| Comment by Bernie Hackett [ 29/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
r0mant, can you tell me what version of PyMongo you are using? | |||||||||||||||||||||||||||||||||||||
| Comment by Roman Tkachenko [ 29/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
Actually, correction: here I meant that PyMongo's "find" method generates sort object as a dict {"_lock": 1} which is understood by MongoDB. | |||||||||||||||||||||||||||||||||||||
| Comment by J Rassi [ 29/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
| Comment by Roman Tkachenko [ 29/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
Hi Jason, Appreciate you formatting my comment/description properly. So here's the call:
If I specify sort like this
it works. However, I haven't found any mention of this syntax deprecation. I have read through "2.6 Compatibility" guide and the only breaking change mentioned there is that use of "true"/"false" is not permitted anymore. Also, plain "find" handles that syntax just fine. Thanks, | |||||||||||||||||||||||||||||||||||||
| Comment by J Rassi [ 29/Apr/14 ] | |||||||||||||||||||||||||||||||||||||
|
Could you paste the full call you made to the pymongo find_and_modify() method to generate this? |