-
Type:
Bug
-
Resolution: Works as Designed
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Hi,
I had some code that worked for python2, but when moving to python3 it worked intermittently. Python 3.5.2. MongoDB:
> db.version()
3.4.1
Python snippet:
client = pymongo.MongoClient(MONGO_URI) db = client[database] potential_task = db.tasks.find_one({'processing':False}) if None == potential_task: print "Found no potentials tasks in database" return None update = copy.copy(potential_task) update['processing'] = True update['jobid'] = os.environ['SLURM_JOBID'] update['start'] = datetime.datetime.utcnow() result = db.tasks.replace_one(potential_task, update) if 1 == result.matched_count and 1 == result.modified_count: found_task = True print('Found task', update) return update client.close()
In practice, db.tasks.replace_one() often could not find a match for potential_task even though I just read it from the database. Here's an example, after I added some debug output:
Starting main loop
Connecting...
{'_id': ObjectId('5949aa5f729d186983e69e4c'), 'processing': False, 'payload': {'output_file': '/fh/scratch/delete30/peters_u/krcurtis/cidr_epic_work/chr11/epic_chr11_chunk_262.intermediate.nc', 'input_file': '/fh/scratch/delete30/peters_u/krcurtis/cidr_epic_work/chr11/epic_chr11_chunk_262.nc', 'platform': 'epic_omniexpressexome'}}
{'_id': ObjectId('5949aa5f729d186983e69e4c'), 'processing': False, 'payload': {'output_file': '/fh/scratch/delete30/peters_u/krcurtis/cidr_epic_work/chr11/epic_chr11_chunk_262.intermediate.nc', 'input_file': '/fh/scratch/delete30/peters_u/krcurtis/cidr_epic_work/chr11/epic_chr11_chunk_262.nc', 'platform': 'epic_omniexpressexome'}}
replace collision, matched=0 modified=0
I discovered that I could I just use the '_id' field instead, and that worked.
match_task = { '_id' : potential_task['_id'] } result = db.tasks.replace_one(match_task, update)
I heard from pycon 2017 that python dictionaries in 3.5 had some kind of randomized implementation, so thought trying to match on just one field, the _id field, might work better. Maybe this all goes away in python 3.6 when dictionaries were changed yet again.
Hope this helps.