[SERVER-31554] Make MapReduce batch BSONObjs before invoking scripting engine Created: 13/Oct/17  Updated: 27/Oct/23  Resolved: 21/Aug/20

Status: Closed
Project: Core Server
Component/s: JavaScript
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Spencer Jackson Assignee: Charlie Swanson
Resolution: Gone away Votes: 0
Labels: qopt-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Sprint: Query 2020-09-07
Participants:
Linked BF Score: 0

 Description   

It's difficult to assess how long a Javascript program will take to run. In map reduce, we should not take locks before vectoring into a script, which may result them being held for a long time.



 Comments   
Comment by Charlie Swanson [ 21/Aug/20 ]

Indeed! Closing this as "Gone away". Now that mapReduce uses aggregation's execution infrastructure there aren't any locks held when we're executing the JS pieces.

Comment by David Storch [ 14/Aug/20 ]

Sending this to the QO team's triage queue. This sounds like something that perhaps we can close given the work in 4.4 to re-implement the mapReduce command in terms of the aggregation subsystem?

Comment by Siyuan Zhou [ 15/Aug/18 ]

Thanks spencer.jackson and max.hirschhorn for the explaination. I'm leaving the backport question to Spencer.

Comment by Spencer Jackson [ 15/Aug/18 ]

siyuan.zhou, BF-6259 represents a deadlock caused by acquiring users while long running locks are held. That scenario can arise a few different ways. This ticket describes one way to prevent long running locks from being held. Namely, we shouldn't hold a lock while invoking user provided JavaScript. We're currently working onĀ  SERVER-31552 which should help mitigate the issue from a different direction by minimizing the need to acquire users. I suspect the recently merged SERVER-35890 may also reduce the occurrence of the BF, as we're now maintaining an LRU cache of recently used users.

Comment by Siyuan Zhou [ 15/Aug/18 ]

spencer.jackson, BF-6259 fails a lot and depends on this ticket, but this ticket is in Backlog. It's not obvious to me why the BF ticket depends on this one. Is this an optimization and not blocking BF-6259? Or shall we prioritize this ticket?

Generated at Thu Feb 08 04:27:26 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.