[SERVER-63387] StreamingCursor should return backup blocks in the order they were retrieved from the WiredTiger backup cursor Created: 07/Feb/22  Updated: 29/Oct/23  Resolved: 23/Mar/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.5
Fix Version/s: 5.0.8, 6.0.0-rc0

Type: Bug Priority: Major - P3
Reporter: Gregory Wlodarek Assignee: Yujin Kang Park
Resolution: Fixed Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File patch.diff     File repro.js    
Issue Links:
Backports
Depends
is depended on by SERVER-64739 backup_cursor_out_of_order.js should ... Closed
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.0
Sprint: Execution Team 2022-03-07, Execution Team 2022-03-21, Execution Team 2022-04-04
Participants:

 Description   

The server uses a streaming cursor to pre-fetch 1,000 backup blocks from WiredTiger instead of loading all of the backup blocks into memory at once.

Backup blocks returned by WiredTiger are pushed into a list, but when it's time to return these backup blocks to the user, we return them from the back of the list instead of the front. This causes us to return backup blocks out-of-order with what WiredTiger reported.

The Cloud back-end code expects that all the diffs for a file will be reported consecutively, without the diffs for any other files reported in between. The offsets are also reported in descending order instead of ascending order.



 Comments   
Comment by Githook User [ 19/Apr/22 ]

Author:

{'name': 'Yu Jin Kang Park', 'email': 'yujin.kang@mongodb.com', 'username': 'ykangpark'}

Message: SERVER-63387 Reverse return order of backup cursors in StreamingCursor
Branch: v5.0
https://github.com/mongodb/mongo/commit/250e543f214ee5c327a5b569f04f9f76278a3f82

Comment by Githook User [ 19/Apr/22 ]

Author:

{'name': 'Yu Jin Kang Park', 'email': 'yujin.kang@mongodb.com', 'username': 'ykangpark'}

Message: SERVER-63387 Reverse return order of backup cursors in StreamingCursor
Branch: v5.0
https://github.com/10gen/mongo-enterprise-modules/commit/f503d3f7119dcbd0cd2b8944c932a81c098a533b

Comment by Githook User [ 21/Mar/22 ]

Author:

{'name': 'Yu Jin Kang Park', 'email': 'yujin.kang@mongodb.com', 'username': 'ykangpark'}

Message: SERVER-63387 Change std::vector for std::deque to allow efficient pop front. Reverse return order of backup cursors in StreamingCursor
Branch: master
https://github.com/mongodb/mongo/commit/5f6240ddc9f36aafd211e96bc9313068918bc02a

Comment by Githook User [ 21/Mar/22 ]

Author:

{'name': 'Yu Jin Kang Park', 'email': 'yujin.kang@mongodb.com', 'username': 'ykangpark'}

Message: SERVER-63387 Change std::vector for std::deque to allow efficient pop front. Reverse return order of backup cursors in StreamingCursor
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/0ed8687779ff08dfca2c99032baf2853900a26a6

Generated at Thu Feb 08 05:57:40 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.