Make aggregate command in ReshardingCollectionCloner specify maxStalenessSeconds

XMLWordPrintableJSON

    • Cluster Scalability
    • Fully Compatible
    • v8.1, v8.0
    • ClusterScalability Apr14-Apr28
    • 200
    • None
    • 3
    • None
    • None
    • None
    • None
    • None
    • None

      The aggregation command run by the ReshardingCollectionCloner against the each donor shard specifies readPreference "nearest" to avoid overloading the donor's primary. Currently, the command doesn't specify maxStalenessSeconds. As a result, the aggregate command could end up targeting a node that is so stale and later end up transitioning to the RECOVERING state because the oplog entries it needs have truncated etc. If the transition occurs while the read is waiting for the readConcern's afterClusterTime, by design the operation will be not interrupted. So the resharding operation would get stuck trying to start the cloning phase until the node is shutdown.

      According to the replication team, Atlas has the mechanism to detect that a node has transitioned to RECOVERING and perform intervention to turn the node back into a healthy state (e.g. via initial sync). So supposedly, this fix is only necessary for non-Atlas use cases.

            Assignee:
            Cheahuychou Mao
            Reporter:
            Cheahuychou Mao
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: