[SERVER-29948] Implement stalled I/O testing for storage node watchdog Created: 30/Jun/17  Updated: 30/Oct/23  Resolved: 28/Jul/17

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 3.5.11

Type: Task Priority: Major - P3
Reporter: Mark Benvenuto Assignee: Mark Benvenuto
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Backwards Compatibility: Fully Compatible
Sprint: Platforms 2017-07-10, Platforms 2017-07-31
Participants:

 Description   

Using charybdefs, and FUSE on Linux, we can simulate slow/hung I/O in Linux.

We can specifically trigger a single file or the whole filesystem with calls like this via their Thrift API:

client.set_fault(['write'], False, 0, 100000, ".*watchdog_probe.txt", False, 50000000, False)

Recommended Branch with fixes for Direct I/O and a port conflict with Cockpit:
https://github.com/markbenvenuto/charybdefs/commit/313e0609d4ae917c225151d2c4da554cc9dbaf7c



 Comments   
Comment by Githook User [ 27/Jul/17 ]

Author:

{'email': 'geert@mongodb.com', 'username': 'GeertBosch', 'name': 'Geert Bosch'}

Message: SERVER-29948 Fix lint
Branch: moose
https://github.com/10gen/mongo-enterprise-modules/commit/72c6ffda8319a0b9096334339f56411a21aaee22

Comment by Githook User [ 27/Jul/17 ]

Author:

{'email': 'mark.benvenuto@mongodb.com', 'username': 'markbenvenuto', 'name': 'Mark Benvenuto'}

Message: SERVER-29948 Add tests for all watchdog monitored paths
Branch: moose
https://github.com/10gen/mongo-enterprise-modules/commit/bed57c6f779d5bb09d3ba2fb8fb763b115617f2e

Comment by Githook User [ 27/Jul/17 ]

Author:

{'email': 'geert@mongodb.com', 'username': 'GeertBosch', 'name': 'Geert Bosch'}

Message: SERVER-29948 Fix lint
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/cf727cb5f07d633c3280707f72a139d81934e337

Comment by Githook User [ 27/Jul/17 ]

Author:

{'email': 'mark.benvenuto@mongodb.com', 'username': 'markbenvenuto', 'name': 'Mark Benvenuto'}

Message: SERVER-29948 Add tests for all watchdog monitored paths
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/47013666d65b49da5a5f98fc7ccf4bf7fb679c5e

Comment by Githook User [ 19/Jul/17 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-29948 Implement stalled I/O testing for storage node watchdog
Branch: master
https://github.com/mongodb/mongo/commit/0cf88b47a6c4c396c83278c316a6637438a8b878

Comment by Githook User [ 19/Jul/17 ]

Author:

{u'username': u'markbenvenuto', u'name': u'Mark Benvenuto', u'email': u'mark.benvenuto@mongodb.com'}

Message: SERVER-29948 Implement stalled I/O testing for storage node watchdog
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/7360c8b972796810a029464cdb86855e645a482c

Generated at Thu Feb 08 04:22:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.