[SERVER-12402] POC for concurrent memory-mapped file flush on Windows Created: 17/Jan/14  Updated: 02/Aug/18  Resolved: 24/Mar/14

Status: Closed
Project: Core Server
Component/s: Concurrency, Storage
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Alexander Komyagin Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Windows on Azure
Persistent storage drive (no cache)


Issue Links:
Related
Participants:

 Description   

We need to verify that parallel flushes will allow us to overcome the disk latency.

One way to simply check this is to have multiple threads in mongoD flushing different data files in parallel.

In order for this test to make sense, the load on the database should be uniformly distributed.

Here is an example script used before in Azure testing:

db.getSisterDB("dat").test.ensureIndex({s : 1})
db.getSisterDB("dat").test.ensureIndex({x : 1, s : 1})
 
function makeid()
{
    var text = "";
    var possible = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
 
    for( var i=0; i < 20; i++ )
        text += possible.charAt(Math.floor(Math.random() * possible.length));
 
    return text;
}
 
while(true)
{
  sp = Math.floor((Math.random()*100)+1);
  for(i=0;i<sp;i++)
  {
    id = Math.floor((Math.random()*1000000000)+1);
    sx = makeid()
    db.getSisterDB("dat").test.save({_id : id, x: new ObjectId(), s : sx});
  }
  wait = 25000000;//Math.floor((Math.random()*1000000)+1);
  for(i=1000;i<wait;i++) {}
}

The expectation is that 2 threads should allow us to get ~200+ IOPS (writes/sec in windows perf mon)



 Comments   
Comment by Alexander Komyagin [ 21/Mar/14 ]

POC succeeded. In our testing we were able to get 500 disk ops/s with 8 threads and without Azure cache.

Generated at Thu Feb 08 03:28:27 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.