-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
None
-
Affects Version/s: 3.6.2
-
Component/s: None
-
Environment:VM details for primary and secondary: 8 vCPU, 64 GB RAM, 7.8TB SSD
OS: Ubuntu 16.04.6 LTS
MongoDB: 3.6.2
-
ALL
-
Cluster state
- Configuration
- 1 Primary
- 2 Secondaries (one secondary does not participate in voting and acts like a standby)
- 1 Arbiter
- Data size
- 1.9TB of data in one collection
- 30GB index size
Incident timeline (all times are in IST)
Incident date: 29th May 2022
- 12:12: One of the secondaries crashed due to a segmentation fault.
- 12:12: Increase in Pages queued for eviction along with errors in page eviction
- 12:12 - 14:20: Increase in pages queued for eviction as well as page eviction errors still ongoing with some occasional drops in the increase. These drops correspond to checkpoint activities.
- 14:30 - 14:52: Secondary comes back up and starts syncing from primary. Again seeing increase in page queued for eviction as well as page eviction errors.
- 14:52: All metrics normalized.
- 16:00 - now: Increase in page eviction queues and page eviction errors, not subsided yet. Happening as of now.
