[SERVER-46740] establishCursors() must always drain the AsyncRequestsSender::_baton Created: 09/Mar/20  Updated: 29/Oct/23  Resolved: 09/Feb/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.9.0, 4.0.23, 4.4.4, 4.2.13

Type: Bug Priority: Major - P3
Reporter: Benjamin Caimano (Inactive) Assignee: Benjamin Caimano (Inactive)
Resolution: Fixed Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-50305 ARS::next() does not respect Operatio... Closed
Problem/Incident
Related
is related to SERVER-46255 establishCursors/AsyncRequestsSender ... Closed
is related to SERVER-54406 NetworkInterfaceMock should allow sim... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4, v4.2, v4.0
Sprint: Service Arch 2020-04-06, Service arch 2020-04-20, Service arch 2020-05-04, Service arch 2020-05-18, Service arch 2020-12-28, Service Arch 2021-01-25, Security 2021-01-25, Security 2021-02-08, Security 2021-02-22
Participants:
Case:
Linked BF Score: 46

 Description   

establishCursors() has a best effort try-catch-throw block here. However, our underlying pattern for the ARS is prone to throwing after interruption. We should introduce a separate draining pattern for the ARS and use it from establishCursors().

This is usually a seen as an invariant in this block in mongo/transport/baton_asio_linux.h:

Invariant failure _scheduled.empty() src/mongo/transport/baton_asio_linux.h 116



 Comments   
Comment by Billy Donahue [ 19/Feb/21 ]

There are several BFs linked to this commit.
Any idea what is causing those?

(Update: I see they're wfbf and linked to SERVER-54406.
I'll link it here directly.)

Comment by Githook User [ 09/Feb/21 ]

Author:

{'name': 'Ben Caimano', 'email': 'ben.caimano@10gen.com'}

Message: SERVER-46740 Add missing reference character
Branch: v4.2
https://github.com/mongodb/mongo/commit/622ba6fa36050f5a3f9d4decc4aca61eca840147

Comment by Githook User [ 09/Feb/21 ]

Author:

{'name': 'Ben Caimano', 'email': 'ben.caimano@10gen.com'}

Message: SERVER-46740 Add missing reference character
Branch: v4.0
https://github.com/mongodb/mongo/commit/07c6611b38d2aacbdb1846b688db70b3273170fb

Comment by Githook User [ 03/Feb/21 ]

Author:

{'name': 'Ben Caimano', 'email': 'ben.caimano@10gen.com'}

Message: SERVER-46740 Add missing reference character
Branch: v4.4
https://github.com/mongodb/mongo/commit/0a39553394d3975454e56fa1878b64c2c70cf6ef

Comment by Githook User [ 03/Feb/21 ]

Author:

{'name': 'Ben Caimano', 'email': 'ben.caimano@10gen.com'}

Message: SERVER-46740 Add missing reference character
Branch: master
https://github.com/mongodb/mongo/commit/72cc8e9c996f84205a0a1523942c58ab75ae5b31

Comment by Githook User [ 02/Feb/21 ]

Author:

{'name': 'Ben Caimano', 'email': 'ben.caimano@10gen.com'}

Message: SERVER-46740 establishCursors() must always drain the AsyncRequestsSender::_baton
Branch: v4.2
https://github.com/mongodb/mongo/commit/2f38d5622410a49d2e7cf2a5a7d9d0b18893c34c

Comment by Githook User [ 02/Feb/21 ]

Author:

{'name': 'Ben Caimano', 'email': 'ben.caimano@10gen.com'}

Message: SERVER-46740 establishCursors() must always drain the AsyncRequestsSender::_baton
Branch: v4.4
https://github.com/mongodb/mongo/commit/53ee01d154106c70f526651858a7667df93a6aa2

Comment by Githook User [ 01/Feb/21 ]

Author:

{'name': 'Ben Caimano', 'email': 'ben.caimano@10gen.com'}

Message: SERVER-46740 establishCursors() must always drain the AsyncRequestsSender::_baton
Branch: master
https://github.com/mongodb/mongo/commit/2421a667e1edcb119c0209a5f5af418ac038ab23

Comment by Githook User [ 01/Feb/21 ]

Author:

{'name': 'Ben Caimano', 'email': 'ben.caimano@10gen.com'}

Message: SERVER-46740 establishCursors() must always drain the AsyncRequestsSender::_baton
Branch: v4.0
https://github.com/mongodb/mongo/commit/fdf53c68a6eceb8d0dffeb8a48479cfd4d01861f

Comment by Benjamin Caimano (Inactive) [ 29/Jan/21 ]

spencer.brown, I believe our reviews have reached a conclusion. I'll attempt to land my changes on Monday.

Comment by Spencer Brown [ 29/Jan/21 ]

Can anything be done to complete the code reviews and get these fixes into the next point releases of 4.4/4.2/4.0? As you can see from the linked support cases there are a number of customers impacted by this. We have provided the workaround to those we know of, but more customers are going to hit this until it's fixed.

Comment by Benjamin Caimano (Inactive) [ 18/Nov/20 ]

This issue can usually be mitigated on v4.0 by setting these parameters:

setParameter.AsyncRequestsSenderUseBaton: false
setParameter.taskExecutorPoolSize: 0

See these docs for more details.

Generated at Thu Feb 08 05:12:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.