Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-40350

Create a ReplOperation/Durable class delineation to indicate fields not persisted to disk

    • Type: Icon: Task Task
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.1.10
    • Affects Version/s: None
    • Component/s: Sharding
    • Labels:
      None
    • Fully Compatible
    • Sharding 2019-04-08

      Background

      When adding operations to an ongoing transaction, we add the operations in the form of a ReplOperation struct. These ReplOperation s mirror the structure for future oplog entries. As a result, just like in an oplog entry, the o2 field in a ReplOperation will possess the document's document key (_id + shard key).

      This document key is populated by extracting the correct fields from the update's postImage document. The document key that we retrieve is currently the same as the preImage document key – since you can't change the _id or the shard key, they are required to be the same. However, after this epic is complete, it will be possible for the pre- and post-Image document keys to be different.

      Problem Statement

      Currently, when we commit a transaction while a migration is running, the migration will observe ReplOperation s to be applied from the transaction. As shown in SERVER-39940, we will need both the pre- and post-Image document keys to verify if an update has happened. Since we don't currently have a notion of a preImageDocumentKey in a ReplOperation, we will need to add it. We have the constraint of not wanting to change the actual oplog entries, only the in-memory ReplOperation structs. If we add additional information that only persists in-memory, we avoid any upgrade/downgrade issues.

      Proposed Solution

      1. Create a new field preImageDocumentKey in ReplOperation.
      2. Specify a custom serializer for preImageDocumentKey that will be a no-op.
      3. Specify a custom deserializer for preImageDocumentKey that will invariant that preImageDocumentKey was never set.

      This change will allow us to consume the preImageDocumentKey in-memory for migrations, without ever changing the on-disk state of the oplog.

            Assignee:
            blake.oler@mongodb.com Blake Oler
            Reporter:
            blake.oler@mongodb.com Blake Oler
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: