[SERVER-40350] Create a ReplOperation/Durable class delineation to indicate fields not persisted to disk Created: 26/Mar/19  Updated: 29/Oct/23  Resolved: 05/Apr/19

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 4.1.10

Type: Task Priority: Major - P3
Reporter: Blake Oler Assignee: Blake Oler
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-39940 Model a shard key update as a delete ... Closed
Backwards Compatibility: Fully Compatible
Sprint: Sharding 2019-04-08
Participants:

 Description   

Background

When adding operations to an ongoing transaction, we add the operations in the form of a ReplOperation struct. These ReplOperation s mirror the structure for future oplog entries. As a result, just like in an oplog entry, the o2 field in a ReplOperation will possess the document's document key (_id + shard key).

This document key is populated by extracting the correct fields from the update's postImage document. The document key that we retrieve is currently the same as the preImage document key – since you can't change the _id or the shard key, they are required to be the same. However, after this epic is complete, it will be possible for the pre- and post-Image document keys to be different.

Problem Statement

Currently, when we commit a transaction while a migration is running, the migration will observe ReplOperation s to be applied from the transaction. As shown in SERVER-39940, we will need both the pre- and post-Image document keys to verify if an update has happened. Since we don't currently have a notion of a preImageDocumentKey in a ReplOperation, we will need to add it. We have the constraint of not wanting to change the actual oplog entries, only the in-memory ReplOperation structs. If we add additional information that only persists in-memory, we avoid any upgrade/downgrade issues.

Proposed Solution

  1. Create a new field preImageDocumentKey in ReplOperation.
  2. Specify a custom serializer for preImageDocumentKey that will be a no-op.
  3. Specify a custom deserializer for preImageDocumentKey that will invariant that preImageDocumentKey was never set.

This change will allow us to consume the preImageDocumentKey in-memory for migrations, without ever changing the on-disk state of the oplog.



 Comments   
Comment by Githook User [ 05/Apr/19 ]

Author:

{'name': 'Blake Oler', 'username': 'BlakeIsBlake', 'email': 'blake.oler@mongodb.com'}

Message: SERVER-40350 Create a ReplOperation/Durable class delineation to indicate fields not persisted to disk
Branch: master
https://github.com/mongodb/mongo/commit/7a6a21915dcde1f232e7bbde5aa3738be69befd4

Comment by Siyuan Zhou [ 29/Mar/19 ]

ReplOperation is designed to be part of OplogEntry, either by its own or as an operation in an applyOps command. It's meant to represent the data written to disk. I'm wondering if it's possible to avoid adding the new field that only represent an in-memory state to the class.

Generated at Thu Feb 08 04:54:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.