Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-6816

Improve journal data handling after multithreaded batch writing

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • 2.2.0-rc1
    • 2.2.1, 2.3.0
    • Replication, Storage
    • None

    Description

      On a secondary in a replica set, no journal commit is allowed during an oplog application after a replicated batch write. The batch write currently can write up to 256 megabytes of ops. If some amount of data close to 256 megs is written and then applied to the oplog, the journal tends to warn about this with warning messages such as

      [rsSync] warning: DR102 too much data written uncommitted 314.864MB
      [rsSync] warning: DR102 too much data written uncommitted 314.868MB
      [rsSync] warning: DR102 too much data written uncommitted 314.872MB
      [rsSync] warning: DR102 too much data written uncommitted 314.876MB
       
       warning assertion failure d.size() < 1024 src/mongo/util/concurrency/task.cpp 
       [journal] warning assertion failure a <= 256*1024*1024 src/mongo/util/alignedbuilder.cpp 90
       [rsSync] local.oplog.rs warning assertion failure _intents.size() < 2000000 src/mongo/db/dur_commitjob.h 101

      especially if a high volume of many small writes. The hardcoded trigger levels for these messages should be adjusted so that end users do not see them for routine use. Alternatively, the maximum batch size should be reevaluated (is it too large?).

      Attachments

        Issue Links

          Activity

            People

              milkie@mongodb.com Eric Milkie
              milkie@mongodb.com Eric Milkie
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: