[SERVER-13122] sort stage must ensure validity of key even when sorting parallel arrays Created: 10/Mar/14  Updated: 11/Jul/16  Resolved: 11/Mar/14

Status: Closed
Project: Core Server
Component/s: Querying
Affects Version/s: 2.5.5
Fix Version/s: 2.6.0-rc2

Type: Bug Priority: Major - P3
Reporter: Steve Duan Assignee: hari.khalsa@10gen.com
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ubuntu


Operating System: Linux
Participants:

 Description   

I got this assert when doing query and sort. Its random but always show up once a while.

My code was doing the following in a loop for testing, I have around 300 documents in my collection:

ret1_cur = collection1.find({}).sort( [('B', -1), ('C', 1), ('A', 1)])
ret1 = [i for i in ret1_cur]

And I got the following assert once a while:
Traceback (most recent call last):

    ret1 = [i for i in ret1_cur]
  File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 904, in next
    if len(self.__data) or self._refresh():
  File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 848, in _refresh
    self.__uuid_subtype))
  File "/usr/local/lib/python2.7/dist-packages/pymongo/cursor.py", line 800, in __send_message
    self.__uuid_subtype)
  File "/usr/local/lib/python2.7/dist-packages/pymongo/helpers.py", line 100, in _unpack_response
    error_object["$err"])
OperationFailure: database error: assertion src/mongo/bson/bsonobjiterator.h:80



 Comments   
Comment by Githook User [ 11/Mar/14 ]

Author:

{u'username': u'hkhalsa', u'name': u'Hari Khalsa', u'email': u'hkhalsa@10gen.com'}

Message: SERVER-13122 if we must use the raw obj to sort with ensure it's well formatted
Branch: master
https://github.com/mongodb/mongo/commit/097f38faf2856f7776c75a48d985b6b3f1641860

Comment by hari.khalsa@10gen.com [ 11/Mar/14 ]

sduan Thanks for the bug report! A fix and a test are forthcoming.

Comment by Steve Duan [ 11/Mar/14 ]

Thanks for the reply. The collections I used are just some randomly generated documents for testing. Yes there are more than one of the A, B and C fields were arrays.

Comment by Mathias Stearn [ 10/Mar/14 ]

It looks like it is an issue with this function: https://github.com/mongodb/mongo/blob/r2.6.0-rc1/src/mongo/db/exec/sort.cpp#L149-196. In some cases it tries to sort by the whole object rather than errorring out. This can lead to the failure you observed when we try to compare these objects since they are larger than the sort key since the comparison function assumes that the objects to be compared have the same keys as the sort key: https://github.com/mongodb/mongo/blob/r2.6.0-rc1/src/mongo/db/jsobj.cpp#L553.

This ticket has now been scheduled for rc2, however, it is likely that the outcome will be that this will still error, but with a more accurate error message.

Would it be possible to get a copy of a collection that you are seeing this with, or at least a few objects? In particular, it would be nice to know if more than one of the A, B and C fields will ever be an array.

Comment by Steve Duan [ 10/Mar/14 ]

Thanks, here are some log from mongodb.log:

2014-03-10T16:47:36.429-0400 [clientcursormon] mem (MB) res:109 virt:6691
2014-03-10T16:47:36.429-0400 [clientcursormon]  mapped (incl journal view):6400
2014-03-10T16:47:36.429-0400 [clientcursormon]  connections:1
2014-03-10T16:47:46.896-0400 [initandlisten] connection accepted from 127.0.0.1:53208 #157 (2 connections now open)
2014-03-10T16:47:46.928-0400 [conn157] build index on: test.correctness0.657293304596 properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "test.correctness0.657293304596" }
2014-03-10T16:47:46.928-0400 [conn157]   added index to empty collection
2014-03-10T16:49:06.217-0400 [conn157] test.correctness0.657293304596 Assertion failure _pos <= _theend src/mongo/bson/bsonobjiterator.h 80
2014-03-10T16:49:06.223-0400 [conn157] test.correctness0.657293304596 0x114d711 0x10f0ab9 0x10d4b8e 0xb7004e 0xa93229 0xa9b977 0xa93e04 0xa97950 0xd1b6e8 0xd035d2 0xb592ce 0xb5e672 0x75e627 0x11048fb 0x7fdb573dbefc 0x7fdb5676cf8d 
 /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x21) [0x114d711]
 /usr/bin/mongod(_ZN5mongo10logContextEPKc+0x159) [0x10f0ab9]
 /usr/bin/mongod(_ZN5mongo12verifyFailedEPKcS1_j+0x17e) [0x10d4b8e]
 /usr/bin/mongod(_ZNK5mongo7BSONObj9woCompareERKS0_S2_b+0x6de) [0xb7004e]
 /usr/bin/mongod(_ZNK5mongo9SortStage20WorkingSetComparatorclERKNS0_16SortableDataItemES4_+0x29) [0xa93229]
 /usr/bin/mongod(_ZSt16__introsort_loopIN9__gnu_cxx17__normal_iteratorIPN5mongo9SortStage16SortableDataItemESt6vectorIS4_SaIS4_EEEElNS3_20WorkingSetComparatorEEvT_SB_T0_T1_+0x1b7) [0xa9b977]
 /usr/bin/mongod(_ZN5mongo9SortStage10sortBufferEv+0xa4) [0xa93e04]
 /usr/bin/mongod(_ZN5mongo9SortStage4workEPm+0x90) [0xa97950]
 /usr/bin/mongod(_ZN5mongo12PlanExecutor7getNextEPNS_7BSONObjEPNS_7DiskLocE+0x68) [0xd1b6e8]
 /usr/bin/mongod(_ZN5mongo11newRunQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x8e2) [0xd035d2]
 /usr/bin/mongod() [0xb592ce]
 /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x382) [0xb5e672]
 /usr/bin/mongod(_ZN5mongo16MyMessageHandler7processERNS_7MessageEPNS_21AbstractMessagingPortEPNS_9LastErrorE+0xa7) [0x75e627]
 /usr/bin/mongod(_ZN5mongo17PortMessageServer17handleIncomingMsgEPv+0x4fb) [0x11048fb]
 /lib/x86_64-linux-gnu/libpthread.so.0(+0x7efc) [0x7fdb573dbefc]
 /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fdb5676cf8d]
2014-03-10T16:49:06.223-0400 [conn157] assertion 0 assertion src/mongo/bson/bsonobjiterator.h:80 ns:test.correctness0.657293304596 query:{ $query: {}, $orderby: { B: -1, C: 1, A: 1 } }
2014-03-10T16:49:06.317-0400 [conn157] end connection 127.0.0.1:53208 (1 connection now open)
2014-03-10T16:52:37.039-0400 [clientcursormon] mem (MB) res:109 virt:6691

Comment by Andy Schwerin [ 10/Mar/14 ]

sduan, when this error occurs, you should also get entries in the server log referencing bsonobjiterator.h. If you can attach logs containing those messages, it will help us diagnose the failure.

Also, you might consider running the newest release candidate of MongoDB 2.6, 2.6.0-rc1, available for download in the development release section of http://www.mongodb.org/downloads. There have been numerous stability enhancements, though I don't particularly know of one that involved bsonobjiterator.

Comment by Steve Duan [ 10/Mar/14 ]

I'm using the 2.5.5 version of mongo server
mongod --version
db version v2.5.5
git version: 5f2ad3f6411cb1c727e4b836798b8ef06de25f2d

Thank you very much!

Comment by Bernie Hackett [ 10/Mar/14 ]

Hi,

That error message is from the server itself. What version of MongoDB are you using?

Generated at Thu Feb 08 03:30:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.