Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-32988

Oplog application, foreground index builds pin an unbounded amount of data in WiredTiger

    • ALL
    • Repl 2018-02-26, Repl 2018-03-12, Repl 2018-03-26

      Note, this only applies to the 3.7 development branch.

      SERVER-32188 started to timestamp writes on secondaries from command oplog entries. It uses a `TimestampBlock` to pass context on the OperationContext's RecoveryUnit which is applied when transactions commit.

      The index build uses a WriteUnitOfWork for each document resulting from the collection scan. This write gets the "commit timestamp" on the recovery unit. Foreground index builds use the recovery unit that is in context from the TimestampBlock. Background index builds use their own OperationContext. This causes background index builds to not timestamp their data writes.

      Moreover, foreground index builds block replication. When replication is not progressing, the oldest_timestamp does not advance. If the oldest_timestamp is not advancing, all of the data writes that are part of the index build stay pinned. This can unnecessarily activate lookaside.

            Assignee:
            vesselina.ratcheva@mongodb.com Vesselina Ratcheva (Inactive)
            Reporter:
            daniel.gottlieb@mongodb.com Daniel Gottlieb (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: