[SERVER-45178] Rollback via refetch can cause rollback to happen successfully w/o updating the rollback id. Created: 16/Dec/19  Updated: 29/Oct/23  Resolved: 14/Feb/20

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 4.2.4, 3.6.18, 4.3.4, 4.0.17

Type: Bug Priority: Major - P3
Reporter: Suganthi Mani Assignee: Vesselina Ratcheva (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File repro.patch    
Issue Links:
Backports
Depends
is depended on by SERVER-42650 Remove stale comments mentioned in th... Closed
Related
is related to SERVER-27659 persist Rollback ID and read it at st... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.2, v4.0, v3.6
Sprint: Repl 2020-02-10, Repl 2020-02-24
Participants:

 Description   

Our current rollback via refetch currently increments the rollback id only after the completion of rollback (i.e. only after truncation of oplog). Consider a below scenario.

1) syncFixup() // rollback + Truncates the oplog.
                   ---> ***Unclean shutdown happens here***
2) incrementRollbackID() // Persists the updated Rollback ID.

On node restart, since we have truncated the oplog, the node doesn't transition to rollback state again. So, we don't increment the rollback id. This can lead to nodes silently completing the rollback without the other nodes knowing about it.

There are places in rollback and initial sync, we do check that sync source haven't rolled back during the process of rollback/initial sync using this rollback id. so, its necessary to update the rollback id on rollback success.

Note: It's not problem for rollback via RTT as we currently increment the rollback id before the rollback start/oplog truncation.

 



 Comments   
Comment by Githook User [ 26/Feb/20 ]

Author:

{'username': 'vessy-mongodb', 'name': 'Vesselina Ratcheva', 'email': 'vesselina.ratcheva@10gen.com'}

Message: SERVER-45178 Always update rollbackId before truncating oplog for rollback-via-refetch

create mode 100644 jstests/replsets/rollback_via_refetch_update_rollback_id_before_oplog_truncation.js

(cherry picked from commit 04a2c9acc7ca061fb86736b377b897b11f6c7c48)
Branch: v3.6
https://github.com/mongodb/mongo/commit/1939a3ca2a29d22e7b9976171bf2cc54bd9a1846

Comment by Githook User [ 26/Feb/20 ]

Author:

{'username': 'vessy-mongodb', 'name': 'Vesselina Ratcheva', 'email': 'vesselina.ratcheva@10gen.com'}

Message: SERVER-45178 Always update rollbackId before truncating oplog for rollback-via-refetch

create mode 100644 jstests/replsets/rollback_via_refetch_update_rollback_id_before_oplog_truncation.js

(cherry picked from commit 04a2c9acc7ca061fb86736b377b897b11f6c7c48)
Branch: v4.0
https://github.com/mongodb/mongo/commit/9e69da32cea83fc45724213215a8de40d2635f45

Comment by Githook User [ 26/Feb/20 ]

Author:

{'username': 'vessy-mongodb', 'name': 'Vesselina Ratcheva', 'email': 'vesselina.ratcheva@10gen.com'}

Message: SERVER-45178 Always update rollbackId before truncating oplog for rollback-via-refetch

create mode 100644 jstests/replsets/rollback_via_refetch_update_rollback_id_before_oplog_truncation.js

(cherry picked from commit 04a2c9acc7ca061fb86736b377b897b11f6c7c48)
Branch: v4.2
https://github.com/mongodb/mongo/commit/f74e24b16ec0cae0d1521c34379a0e0854cd4607

Comment by Githook User [ 14/Feb/20 ]

Author:

{'name': 'Vesselina Ratcheva', 'username': 'vessy-mongodb', 'email': 'vesselina.ratcheva@10gen.com'}

Message: SERVER-45178 Always update rollbackId before truncating oplog for rollback-via-refetch

create mode 100644 jstests/replsets/rollback_via_refetch_update_rollback_id_before_oplog_truncation.js
Branch: master
https://github.com/mongodb/mongo/commit/04a2c9acc7ca061fb86736b377b897b11f6c7c48

Generated at Thu Feb 08 05:08:07 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.