[SERVER-41074] Don't migrate prePostImageDoc session entries for CRUD operations outside of current migration range Created: 09/May/19  Updated: 29/Oct/23  Resolved: 17/May/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 4.1.12

Type: Bug Priority: Major - P3
Reporter: Blake Oler Assignee: Randolph Tan
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-39844 Create concurrency workload with migr... Closed
Related
related to SERVER-56127 Retryable update may execute more tha... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Sharding 2019-05-20, Sharding 2019-06-03
Participants:

 Description   

Problem Statement

When migrating retryable CRUD op session entries, we generate a prePostImage oplog entry if necessary.

Later in the process, we filter out these CRUD op entries, but not their corresponding prePostImageOplog entries. We skip the latter because the prePostImage oplog entries are created as no-ops.

We should also filter out corresponding prePostImage oplog entries.

Proposed Solution

We currently have two types of no-op entries that will come through the session catalog migration pipeline. These are either sentinel oplog entries or prePostImage oplog entries. We should:

  1. Automatically let sentinel oplog entries pass through, and
  2. Check the pre- or postImage in prePostImage oplog entries to filter based on a mismatched shard key.


 Comments   
Comment by Githook User [ 17/May/19 ]

Author:

{'email': 'randolph@10gen.com', 'name': 'Randolph Tan', 'username': 'renctan'}

Message: SERVER-41074 Make decision to send pre/post image oplog the same as the decision made for the originating oplog write.
Branch: master
https://github.com/mongodb/mongo/commit/32ade4aa6b6a73c3620486117388908ad9ad438c

Comment by Blake Oler [ 14/May/19 ]

Yes. That is the issue that we discussed today (ticket to follow). This issue exists independent of any changes to the document shard key. renctan

Comment by Randolph Tan [ 10/May/19 ]

With change shard key project, isn't it possible for the post-image to have different shard key and thus ending up on a different chunk but on the same shard? Or is findAndModify exempt from the in-place update optimization for change shard key?

Comment by Blake Oler [ 09/May/19 ]

renctan

 if (oplogEntryIsSentinel(oplog)) {
        return true;
    }
 
    // Skip appending CRUD operations that don't pertain to the ChunkRange being migrated.
    if (oplog.isCrudOpType() || oplog.getOpType() == repl::OpTypeEnum::kNoop) {
        auto shardKey = keyPattern.extractShardKeyFromDoc(oplog.getOperationToApply());
        if (!range.containsKey(shardKey)) {
            return false;
        }
    }
 
    return true;

Approach LGTY?

Generated at Thu Feb 08 04:56:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.