[SERVER-9734] Single writes with journal=true are penalized by journalCommitInterval Created: 21/May/13  Updated: 06/Dec/22  Resolved: 06/Dec/16

Status: Closed
Project: Core Server
Component/s: Performance, Write Ops
Affects Version/s: 2.4.3
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Dmitry Naumov Assignee: Backlog - Storage Execution Team
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows 7 Professional SP1 x64


Assigned Teams:
Storage Execution
Participants:

 Description   

These lines are from durThread method in dur.cpp:

// commit sooner if one or more getLastError j:true is pending
sleepmillis(oneThird);
for( unsigned i = 1; i <= 2; i++ ) {
if( commitJob._notify.nWaiting() )
break;

As oneThird can't be less than 1 ms, every single write with j=true is penalized by that time. Should we move this check before sleepmillis?

if( !commitJob._notify.nWaiting() ) {
sleepmillis(oneThird);
for( unsigned i = 1; i <= 2; i++ ) {



 Comments   
Comment by Eric Milkie [ 06/Dec/16 ]

This ticket only applies to MMAPv1, and the logic for journal flushing has changed since this ticket was written.

Comment by Dmitry Naumov [ 10/Jun/13 ]

The following code states that in some cases we should commit faster that journalCommitInterval:

unsigned ms = cmdLine.journalCommitInterval;
if( ms == 0 ) { 
    // use default
    ms = samePartition ? 100 : 30;
}
 
unsigned oneThird = (ms / 3) + 1; // +1 so never zero
 
...
 
// commit sooner if one or more getLastError j:true is pending
sleepmillis(oneThird);
for( unsigned i = 1; i <= 2; i++ ) {
    if( commitJob._notify.nWaiting() )
        break;
    if( commitJob.bytes() > UncommittedBytesLimit / 2  )
        break;
    sleepmillis(oneThird);
}
                                        
durThreadGroupCommit();
 
...

Unfortunately despite some preference for those writers (

{getLastError=1, j=true}) there is still delay penalty of at least 1 ms for journalCommitInterval=2 (you can't use smaller value) and 100/3 + 1 = 34 ms for default journalCommitInterval = 100
I've already felt into this trap during my investigation of SERVER-9754 (http://dmitry-naumov.blogspot.ru/2013/05/mongodb-vs-ms-sql-journalcommitinterval.html)

I dare to suggest a bit different behavior - let's leave journalCommitInterval for those who doesn't wait for confirmation (getLastError=0), but for those who do we will flush pending commit ASAP. To do so we need to remove sleeps and replace them, for example, with conditional variable and timed wait. So any writer with {getLastError=1, j=true}

will notify possibly sleeping durThread before awaiting commit notification. It will help to avoid extra delays and increases throughput in such scenarios.

Generated at Thu Feb 08 03:21:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.