[SERVER-34091] Oplog visibility rules can cause cappedTruncateAfter to erroneously skip record deletion in WiredTiger Created: 23/Mar/18  Updated: 29/Oct/23  Resolved: 23/Mar/18

Status: Closed
Project: Core Server
Component/s: Replication, Storage
Affects Version/s: None
Fix Version/s: 3.6.5, 3.7.4

Type: Bug Priority: Major - P3
Reporter: William Schultz (Inactive) Assignee: William Schultz (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
is related to SERVER-48934 Investigate replica set data mismatch... Closed
is related to SERVER-34701 manually set oplog visibility on roll... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.6
Participants:
Linked BF Score: 0

 Description   

The WiredTigerRecordStore::cappedTruncateAfter method truncates a capped collection after a given RecordId. For a non-inclusive truncation, it first establishes a forward scanning cursor on the collection and then iterates forward to the record immediately succeeding the given truncation point. If no next record is found it will return and not execute the capped truncation. This can be a problem when truncating the oplog collection, since the forward scanning record store cursor will be bound to oplog visibility rules, and if the oplog read timestamp has not been updated yet to the newest timestamp, this cursor may erroneously be exhausted before reaching the end of the oplog, making it seem like the oplog has less entries than it actually does. This can cause the oplog to not properly be truncated after a call to cappedTruncateAfter.



 Comments   
Comment by Githook User [ 27/Apr/18 ]

Author:

{'email': 'milkie@10gen.com', 'username': 'milkie', 'name': 'Eric Milkie'}

Message: SERVER-34091 Make sure we read all committed entries when truncating the oplog

This backports commit f5d4aae1587479ff917bd5aa81d2c0decc68a836.
Branch: v3.6
https://github.com/mongodb/mongo/commit/e5552ea011accdab31a4569dd340852580ca7e19

Comment by Eric Milkie [ 26/Apr/18 ]

Note: the above commit "Change boolean type to int" should have been marked as SERVER-33990, not SERVER-34091. It will not be backported as part of this ticket.

Comment by Githook User [ 26/Mar/18 ]

Author:

{'email': 'william.schultz@mongodb.com', 'name': 'William Schultz', 'username': 'will62794'}

Message: SERVER-34091 Change boolean type to int
Branch: master
https://github.com/mongodb/mongo/commit/68e4e5b84d700b26d0b3333f91ea72588c982c32

Comment by Githook User [ 23/Mar/18 ]

Author:

{'email': 'william.schultz@mongodb.com', 'name': 'William Schultz', 'username': 'will62794'}

Message: SERVER-34091 Make sure we read all committed entries when truncating the oplog
Branch: master
https://github.com/mongodb/mongo/commit/f5d4aae1587479ff917bd5aa81d2c0decc68a836

Comment by William Schultz (Inactive) [ 23/Mar/18 ]

milkie daniel.gottlieb I am considering the fix for this a temporary workaround until we can decide on a better all around solution.

Generated at Thu Feb 08 04:35:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.