Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-6816

Improve journal data handling after multithreaded batch writing

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.2.1, 2.3.0
    • Affects Version/s: 2.2.0-rc1
    • Component/s: Replication, Storage
    • None

      On a secondary in a replica set, no journal commit is allowed during an oplog application after a replicated batch write. The batch write currently can write up to 256 megabytes of ops. If some amount of data close to 256 megs is written and then applied to the oplog, the journal tends to warn about this with warning messages such as

      [rsSync] warning: DR102 too much data written uncommitted 314.864MB
      [rsSync] warning: DR102 too much data written uncommitted 314.868MB
      [rsSync] warning: DR102 too much data written uncommitted 314.872MB
      [rsSync] warning: DR102 too much data written uncommitted 314.876MB
      
       warning assertion failure d.size() < 1024 src/mongo/util/concurrency/task.cpp 
       [journal] warning assertion failure a <= 256*1024*1024 src/mongo/util/alignedbuilder.cpp 90
       [rsSync] local.oplog.rs warning assertion failure _intents.size() < 2000000 src/mongo/db/dur_commitjob.h 101
      

      especially if a high volume of many small writes. The hardcoded trigger levels for these messages should be adjusted so that end users do not see them for routine use. Alternatively, the maximum batch size should be reevaluated (is it too large?).

            Assignee:
            milkie@mongodb.com Eric Milkie
            Reporter:
            milkie@mongodb.com Eric Milkie
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: