Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-40350

Create a ReplOperation/Durable class delineation to indicate fields not persisted to disk

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.10
    • Component/s: Sharding
    • Labels:
      None

      Description

      Background

      When adding operations to an ongoing transaction, we add the operations in the form of a ReplOperation struct. These ReplOperation s mirror the structure for future oplog entries. As a result, just like in an oplog entry, the o2 field in a ReplOperation will possess the document's document key (_id + shard key).

      This document key is populated by extracting the correct fields from the update's postImage document. The document key that we retrieve is currently the same as the preImage document key – since you can't change the _id or the shard key, they are required to be the same. However, after this epic is complete, it will be possible for the pre- and post-Image document keys to be different.

      Problem Statement

      Currently, when we commit a transaction while a migration is running, the migration will observe ReplOperation s to be applied from the transaction. As shown in SERVER-39940, we will need both the pre- and post-Image document keys to verify if an update has happened. Since we don't currently have a notion of a preImageDocumentKey in a ReplOperation, we will need to add it. We have the constraint of not wanting to change the actual oplog entries, only the in-memory ReplOperation structs. If we add additional information that only persists in-memory, we avoid any upgrade/downgrade issues.

      Proposed Solution

      1. Create a new field preImageDocumentKey in ReplOperation.
      2. Specify a custom serializer for preImageDocumentKey that will be a no-op.
      3. Specify a custom deserializer for preImageDocumentKey that will invariant that preImageDocumentKey was never set.

      This change will allow us to consume the preImageDocumentKey in-memory for migrations, without ever changing the on-disk state of the oplog.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              blake.oler Blake Oler
              Reporter:
              blake.oler Blake Oler
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: