[SERVER-51530] mongomirror should periodically tail oplog between index builds Created: 14/Oct/20  Updated: 27/Oct/23  Resolved: 15/Oct/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Johnny Shields Assignee: Backlog - Triage Team
Resolution: Community Answered Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Server Triage
Participants:

 Description   

We recently tried to copy our DB to Atlas using mongomirror.

mongomirror has three phases:

  1. copy the collection snapshots
  2. build indexes
  3. tail the oplog for updates since step 1 completed

In our case, it took 48h for step 2 (index build) to complete, and our oplog only had 22h worth of data. When we reached step 3 (after waiting 2 days!) the tool could not proceed.

It would be better if the tool could inter-mix steps 2 and 3, i.e. tail the oplog each 1 hour between index builds to stay current. This would potentially save a lot of heartbreak for users in the future.



 Comments   
Comment by Dmitry Agranat [ 15/Oct/20 ]

Hi johnny.shields@gmail.com,

The SERVER project is for bugs and feature suggestions for the MongoDB server. To inform our Atlas team of issues you are having or feature requests you'd like to suggest, please reach out to the Atlas support team via Intercom. You can do this by clicking the round green button at the bottom right of the cluster view page in Atlas.

Regards,
Dima

Comment by Johnny Shields [ 14/Oct/20 ]

Just one correction, the sequence appears to be:

  1. copy the collection snapshots
  2. tail the oplog for updates since step 1 began
  3. build indexes
  4. tail the oplog for updates since step 3 began
Generated at Thu Feb 08 05:25:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.