[SERVER-32935] improve exception handling in SyncTail::oplogApplication() Created: 26/Jan/18  Updated: 29/Oct/23  Resolved: 05/Jun/18

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 3.4.10, 3.6.2, 3.7.1
Fix Version/s: 3.4.16, 3.6.6, 4.0.0, 4.1.1

Type: Bug Priority: Major - P3
Reporter: Benety Goh Assignee: Benety Goh
Resolution: Fixed Votes: 0
Labels: replApplier
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
is related to SERVER-25240 set_terminate() should be done for ev... Closed
is related to SERVER-32913 Parallelize application of applyOps o... Closed
is related to SERVER-25071 Ensure replication batch finishes bef... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.0
Sprint: Repl 2018-05-21, Repl 2018-06-04, Repl 2018-06-18
Participants:
Linked BF Score: 50

 Description   

unexpected exceptions in SyncTail;:oplogApplication may sometimes cause OpQueueBatcher to terminate early and trigger the invariant on the _isDead field. We should consider more graceful shutdown semantics for the OpQueueBatcher or handle exceptions in oplogApplication() explicitly. This issue affects 3.4 through 3.6



 Comments   
Comment by Githook User [ 26/Feb/20 ]

Author:

{'name': 'Siyuan Zhou', 'username': 'visualzhou', 'email': 'siyuan.zhou@mongodb.com'}

Message: SERVER-42219 Remove logging death test of signalDrainComplete.

Reverted a5cbd93aea SERVER-32935 add characterization test for exception handling in oplog application loop

(cherry picked from commit 89a6d7bc3a0126cf8bfd177ad65b233181641175)
Branch: v4.0
https://github.com/mongodb/mongo/commit/79c200fc57255e31f4239983830cbdcb30ed357f

Comment by Githook User [ 05/Sep/19 ]

Author:

{'username': 'visualzhou', 'email': 'siyuan.zhou@mongodb.com', 'name': 'Siyuan Zhou'}

Message: SERVER-42219 Remove logging death test of signalDrainComplete.

Reverted a5cbd93aea SERVER-32935 add characterization test for exception handling in oplog application loop

(cherry picked from commit 89a6d7bc3a0126cf8bfd177ad65b233181641175)
Branch: v4.2
https://github.com/mongodb/mongo/commit/2a3748453d5d00d7f8de91ea75c985f5ad1ce14b

Comment by Githook User [ 23/Aug/19 ]

Author:

{'username': 'visualzhou', 'email': 'siyuan.zhou@mongodb.com', 'name': 'Siyuan Zhou'}

Message: SERVER-42219 Remove logging death test of signalDrainComplete.

Reverted a5cbd93aea SERVER-32935 add characterization test for exception handling in oplog application loop
Branch: master
https://github.com/mongodb/mongo/commit/89a6d7bc3a0126cf8bfd177ad65b233181641175

Comment by Githook User [ 08/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-32935 log exception during oplog application

This change uses the terminate handler to log unexpected exceptions thrown
inside SyncTail::oplogApplication().

(cherry picked from commit 09ea5600193cee30260c4d0a114f6464b870ab84)
Branch: v3.4
https://github.com/mongodb/mongo/commit/4ef027f98d5c00a0f4e507cbe39a22cab4c7a44c

Comment by Githook User [ 08/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-32935 add characterization test for exception handling in oplog application loop

(cherry picked from commit a5cbd93aeaa7de9857b82396888a91ef488c9c7c)
Branch: v3.4
https://github.com/mongodb/mongo/commit/5090ba6052d1cb13587c22356fcc247590e147a8

Comment by Githook User [ 07/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-32935 log exception during oplog application

This change uses the terminate handler to log unexpected exceptions thrown
inside SyncTail::oplogApplication().

(cherry picked from commit 09ea5600193cee30260c4d0a114f6464b870ab84)
Branch: v3.6
https://github.com/mongodb/mongo/commit/91d09244898d046880fdf7b82741795b56233883

Comment by Githook User [ 07/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-32935 add characterization test for exception handling in oplog application loop

(cherry picked from commit a5cbd93aeaa7de9857b82396888a91ef488c9c7c)
Branch: v3.6
https://github.com/mongodb/mongo/commit/76e15cc648c3dc8818de59cfc12279f78a969222

Comment by Githook User [ 07/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-32935 log exception during oplog application

This change uses the terminate handler to log unexpected exceptions thrown
inside SyncTail::oplogApplication().

(cherry picked from commit 09ea5600193cee30260c4d0a114f6464b870ab84)
Branch: v4.0
https://github.com/mongodb/mongo/commit/dc357342405345ff53f4f60358f61fd3e2f718dd

Comment by Githook User [ 07/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-32935 add characterization test for exception handling in oplog application loop

(cherry picked from commit a5cbd93aeaa7de9857b82396888a91ef488c9c7c)
Branch: v4.0
https://github.com/mongodb/mongo/commit/d814cdd418edb681b922e5d8ebc453d3d774f1ca

Comment by Githook User [ 05/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-32935 log exception during oplog application

This change uses the terminate handler to log unexpected exceptions thrown
inside SyncTail::oplogApplication().
Branch: master
https://github.com/mongodb/mongo/commit/09ea5600193cee30260c4d0a114f6464b870ab84

Comment by Githook User [ 05/Jun/18 ]

Author:

{'username': 'benety', 'name': 'Benety Goh', 'email': 'benety@mongodb.com'}

Message: SERVER-32935 add characterization test for exception handling in oplog application loop
Branch: master
https://github.com/mongodb/mongo/commit/a5cbd93aeaa7de9857b82396888a91ef488c9c7c

Comment by Spencer Brody (Inactive) [ 03/May/18 ]

william.schultz, check in with benety.goh about what the best thing to do here is.

Comment by Spencer Brody (Inactive) [ 08/Feb/18 ]

This is mainly about better logging around exception context to aid debugging.

Generated at Thu Feb 08 04:31:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.