Originally posted as a question to StackOverflow (http://stackoverflow.com/questions/10559638/mongodb-assertion-error-starting-from-self-retrieved-pymongo-driver).

We're using a sharded replicaset, running pymongo 2.2 against mongo (version: 2.1.1-pre-). We're getting a traceback when a query returns more than one result document.

Traceback (most recent call last):
File "/usr/lib64/python2.6/threading.py", line 532, in __bootstrap_inner
self.run()
File "/opt/DCM/mods/plugin.py", line 25, in run
self._mod.collect_metrics_dcm()
File "/opt/DCM/plugins/res.py", line 115, in collect_metrics_dcm
ms.updateSpecificMetric(metricName, value, timestamp)
File "/opt/DCM/mods/mongoSaver.py", line 155, in updateSpecificMetric
latestDoc = self.getLatestDoc(metricName)
File "/opt/DCM/mods/mongoSaver.py", line 70, in getLatestDoc
for d in dlist:
File "/usr/lib64/python2.6/site-packages/pymongo/cursor.py", line 747, in next
if len(self.__data) or self._refresh():
File "/usr/lib64/python2.6/site-packages/pymongo/cursor.py", line 698, in _refresh
self.__uuid_subtype))
File "/usr/lib64/python2.6/site-packages/pymongo/cursor.py", line 668, in __send_message
assert response["starting_from"] == self.__retrieved
AssertionError

The code that give what dlist is is a simple find(). I've tried reIndex(), no joy. I've tried stopping and starting the mongo server, no joy.

Ok, so traced this down a bit, and I have a patch for pymongo for this assertion error.

There is a BUG in Mongo. When querying a sharded replicaset, Mongo returns an incorrect value for 'starting_from'. Instead of returning 0 on the first query, it's returning the number of records received instead of the offset value. I have a patch for pymongo to protect against this bad info:

File is site-packages/pymongo/cursor.py.

[user@hostname]$ diff cursor.py.orig cursor.py
668c668,671
< assert response["starting_from"] == self.__retrieved
—
> if ((self._retrieved != 0) and (response["starting_from"] != self._retrieved)):
> from pprint import pformat
> msg = "Server response of 'starting_from' is '%s', but self_retrieved (which is only set to nonzero below here) is '%s'." % (pformat(response), pformat(self._retrieved))
> assert False, msg

The 'starting_from' comes from helpers.py decoding the response from Mongo:

result["starting_from"] = struct.unpack("<i", response[12:16])[0]

So, MONGODB IS WRONG - it's the 12th thru the 15th byte of Mongo's response.