random-rw: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8 ... random-rw: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=8 fio-2.1.3 Starting 32 processes random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: Laying out IO file(s) (1 file(s) / 2048MB) random-rw: (groupid=0, jobs=1): err= 0: pid=64413: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394461B/s, iops=96, runt=2723757msec slat (usec): min=199, max=619576, avg=10289.19, stdev=38048.16 clat (usec): min=6, max=825021, avg=36264.61, stdev=70156.57 lat (usec): min=304, max=829828, avg=46556.02, stdev=78083.48 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6304], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[432128], | 99.99th=[618496] bw (KB /s): min= 25, max= 1396, per=3.14%, avg=385.84, stdev=297.96 write: io=1023.4MB, bw=393965B/s, iops=96, runt=2723757msec slat (usec): min=4, max=641040, avg=77.02, stdev=4221.79 clat (usec): min=65, max=828301, avg=36471.54, stdev=70060.47 lat (usec): min=71, max=846227, avg=36550.12, stdev=70250.33 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6496], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[415744], | 99.99th=[618496] bw (KB /s): min= 6, max= 1226, per=3.14%, avg=385.24, stdev=303.91 lat (usec) : 10=0.01%, 100=0.55%, 250=0.24%, 500=0.09%, 750=0.89% lat (usec) : 1000=1.99% lat (msec) : 2=12.34%, 4=40.39%, 10=17.58%, 20=6.91%, 50=1.35% lat (msec) : 100=0.01%, 250=17.48%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269295, majf=0, minf=9 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64414: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394903B/s, iops=96, runt=2720712msec slat (usec): min=184, max=638545, avg=10296.45, stdev=38182.42 clat (usec): min=5, max=825312, avg=36223.65, stdev=70105.69 lat (usec): min=326, max=828861, avg=46522.15, stdev=78056.63 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1608], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[423936], | 99.99th=[626688] bw (KB /s): min= 6, max= 1436, per=3.14%, avg=386.21, stdev=298.64 write: io=1023.4MB, bw=394406B/s, iops=96, runt=2720712msec slat (usec): min=4, max=633250, avg=58.19, stdev=3609.22 clat (usec): min=65, max=828840, avg=36431.34, stdev=70002.26 lat (usec): min=70, max=828848, avg=36491.18, stdev=70129.24 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6496], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 19, max= 1252, per=3.14%, avg=385.80, stdev=305.39 lat (usec) : 10=0.01%, 100=0.56%, 250=0.23%, 500=0.08%, 750=0.90% lat (usec) : 1000=1.99% lat (msec) : 2=12.43%, 4=40.36%, 10=17.46%, 20=6.95%, 50=1.40% lat (msec) : 250=17.45%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269523, majf=0, minf=9 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64415: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=395580B/s, iops=96, runt=2716050msec slat (usec): min=154, max=813189, avg=10264.40, stdev=38049.44 clat (usec): min=4, max=831539, avg=36087.86, stdev=70093.06 lat (usec): min=315, max=835928, avg=46354.25, stdev=78062.41 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1176], 10.00th=[ 1624], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3536], 60.00th=[ 4192], | 70.00th=[ 6176], 80.00th=[17536], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[407552], 99.95th=[432128], | 99.99th=[618496] bw (KB /s): min= 19, max= 1361, per=3.14%, avg=386.86, stdev=299.11 write: io=1023.4MB, bw=395083B/s, iops=96, runt=2716050msec slat (usec): min=4, max=462362, avg=73.29, stdev=4001.69 clat (usec): min=64, max=829113, avg=36442.34, stdev=70094.26 lat (usec): min=69, max=829118, avg=36517.14, stdev=70268.71 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3632], 60.00th=[ 4320], | 70.00th=[ 6304], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 19, max= 1236, per=3.14%, avg=386.36, stdev=306.83 lat (usec) : 10=0.01%, 100=0.58%, 250=0.21%, 500=0.08%, 750=0.87% lat (usec) : 1000=1.97% lat (msec) : 2=12.52%, 4=40.48%, 10=17.44%, 20=6.89%, 50=1.35% lat (msec) : 250=17.41%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.09%, sys=0.36%, ctx=269512, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64416: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393811B/s, iops=96, runt=2728256msec slat (usec): min=169, max=812764, avg=10318.57, stdev=38200.91 clat (usec): min=4, max=825207, avg=36238.28, stdev=70188.97 lat (usec): min=221, max=1020.4K, avg=46559.00, stdev=78176.14 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1176], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6304], 80.00th=[17536], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[432128], | 99.99th=[618496] bw (KB /s): min= 30, max= 1504, per=3.13%, avg=384.79, stdev=297.68 write: io=1023.4MB, bw=393315B/s, iops=96, runt=2728256msec slat (usec): min=4, max=455823, avg=64.74, stdev=3730.69 clat (usec): min=65, max=1016.5K, avg=36618.21, stdev=70216.99 lat (usec): min=70, max=1016.6K, avg=36684.58, stdev=70361.51 clat percentiles (usec): | 1.00th=[ 572], 5.00th=[ 1192], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6624], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 12, max= 1602, per=3.13%, avg=384.30, stdev=304.13 lat (usec) : 10=0.01%, 100=0.58%, 250=0.21%, 500=0.10%, 750=0.89% lat (usec) : 1000=2.05% lat (msec) : 2=12.33%, 4=40.13%, 10=17.67%, 20=6.98%, 50=1.38% lat (msec) : 100=0.01%, 250=17.49%, 500=0.15%, 750=0.04%, 1000=0.01% lat (msec) : 2000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269694, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64417: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393754B/s, iops=96, runt=2728650msec slat (usec): min=174, max=814307, avg=10322.78, stdev=38210.15 clat (usec): min=4, max=829812, avg=36286.84, stdev=70117.35 lat (usec): min=274, max=1019.9K, avg=46611.67, stdev=78096.39 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1176], 10.00th=[ 1608], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[419840], | 99.99th=[618496] bw (KB /s): min= 37, max= 1967, per=3.13%, avg=384.86, stdev=299.11 write: io=1023.4MB, bw=393258B/s, iops=96, runt=2728650msec slat (usec): min=4, max=451753, avg=62.25, stdev=3545.22 clat (usec): min=65, max=1019.9K, avg=36579.97, stdev=70201.19 lat (usec): min=71, max=1019.9K, avg=36643.75, stdev=70343.91 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1208], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 12, max= 1950, per=3.13%, avg=384.36, stdev=305.72 lat (usec) : 10=0.01%, 100=0.54%, 250=0.25%, 500=0.10%, 750=0.90% lat (usec) : 1000=2.04% lat (msec) : 2=12.39%, 4=40.07%, 10=17.72%, 20=6.93%, 50=1.38% lat (msec) : 100=0.01%, 250=17.51%, 500=0.15%, 750=0.04%, 1000=0.01% lat (msec) : 2000=0.01% cpu : usr=0.09%, sys=0.36%, ctx=270183, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64418: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393988B/s, iops=96, runt=2727027msec slat (usec): min=194, max=618936, avg=10306.87, stdev=38125.77 clat (usec): min=5, max=832412, avg=36262.70, stdev=70238.31 lat (usec): min=316, max=835800, avg=46571.79, stdev=78184.80 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6304], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[436224], | 99.99th=[626688] bw (KB /s): min= 31, max= 1532, per=3.13%, avg=384.99, stdev=298.00 write: io=1023.4MB, bw=393492B/s, iops=96, runt=2727027msec slat (usec): min=4, max=633209, avg=72.19, stdev=4066.03 clat (usec): min=65, max=830752, avg=36560.70, stdev=70115.08 lat (usec): min=71, max=830776, avg=36634.38, stdev=70288.43 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2256], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 6, max= 1368, per=3.13%, avg=384.36, stdev=304.91 lat (usec) : 10=0.01%, 100=0.55%, 250=0.24%, 500=0.08%, 750=0.89% lat (usec) : 1000=1.98% lat (msec) : 2=12.33%, 4=40.30%, 10=17.66%, 20=6.91%, 50=1.38% lat (msec) : 100=0.01%, 250=17.49%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269400, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64419: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=396114B/s, iops=96, runt=2712395msec slat (usec): min=159, max=814491, avg=10252.35, stdev=38022.03 clat (usec): min=4, max=825780, avg=35971.06, stdev=69875.44 lat (usec): min=330, max=829913, avg=46225.62, stdev=77858.47 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1192], 10.00th=[ 1608], 20.00th=[ 2160], | 30.00th=[ 2608], 40.00th=[ 3056], 50.00th=[ 3536], 60.00th=[ 4192], | 70.00th=[ 6048], 80.00th=[17536], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[423936], | 99.99th=[626688] bw (KB /s): min= 25, max= 1339, per=3.15%, avg=387.13, stdev=300.13 write: io=1023.4MB, bw=395615B/s, iops=96, runt=2712395msec slat (usec): min=4, max=458154, avg=71.21, stdev=3998.70 clat (usec): min=64, max=829138, avg=36461.63, stdev=70137.58 lat (usec): min=70, max=829144, avg=36534.24, stdev=70289.74 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1240], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3632], 60.00th=[ 4320], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 12, max= 1200, per=3.15%, avg=386.62, stdev=306.37 lat (usec) : 10=0.01%, 100=0.59%, 250=0.21%, 500=0.09%, 750=0.85% lat (usec) : 1000=2.00% lat (msec) : 2=12.50%, 4=40.72%, 10=17.24%, 20=6.86%, 50=1.37% lat (msec) : 250=17.41%, 500=0.14%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=268846, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64420: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394076B/s, iops=96, runt=2726416msec slat (usec): min=201, max=814759, avg=10304.34, stdev=38129.50 clat (usec): min=15, max=831435, avg=36266.21, stdev=70122.96 lat (usec): min=339, max=1018.6K, avg=46572.70, stdev=78082.15 clat percentiles (usec): | 1.00th=[ 604], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[423936], | 99.99th=[618496] bw (KB /s): min= 31, max= 1471, per=3.13%, avg=385.20, stdev=297.95 write: io=1023.4MB, bw=393581B/s, iops=96, runt=2726416msec slat (usec): min=4, max=462106, avg=72.36, stdev=3966.06 clat (usec): min=64, max=1018.6K, avg=36540.84, stdev=70120.89 lat (usec): min=70, max=1018.6K, avg=36614.65, stdev=70288.07 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[415744], | 99.99th=[618496] bw (KB /s): min= 12, max= 1261, per=3.13%, avg=384.66, stdev=304.13 lat (usec) : 20=0.01%, 100=0.57%, 250=0.22%, 500=0.07%, 750=0.88% lat (usec) : 1000=2.00% lat (msec) : 2=12.27%, 4=40.45%, 10=17.52%, 20=6.97%, 50=1.37% lat (msec) : 100=0.01%, 250=17.50%, 500=0.14%, 750=0.04%, 1000=0.01% lat (msec) : 2000=0.01% cpu : usr=0.08%, sys=0.36%, ctx=269677, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64421: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393744B/s, iops=96, runt=2728716msec slat (usec): min=160, max=813014, avg=10314.40, stdev=38156.84 clat (usec): min=5, max=1017.8K, avg=36283.63, stdev=70078.30 lat (usec): min=297, max=1024.7K, avg=46600.21, stdev=78064.21 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1176], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[423936], | 99.99th=[618496] bw (KB /s): min= 19, max= 2298, per=3.13%, avg=384.63, stdev=298.50 write: io=1023.4MB, bw=393249B/s, iops=96, runt=2728716msec slat (usec): min=4, max=441716, avg=70.60, stdev=3919.67 clat (usec): min=67, max=1017.8K, avg=36584.97, stdev=70332.47 lat (usec): min=73, max=1017.8K, avg=36657.09, stdev=70468.39 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1208], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 25, max= 2298, per=3.13%, avg=384.08, stdev=304.92 lat (usec) : 10=0.01%, 100=0.53%, 250=0.26%, 500=0.11%, 750=0.87% lat (usec) : 1000=2.05% lat (msec) : 2=12.35%, 4=40.07%, 10=17.75%, 20=6.94%, 50=1.38% lat (msec) : 100=0.01%, 250=17.50%, 500=0.15%, 750=0.04%, 1000=0.01% lat (msec) : 2000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269681, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64422: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393749B/s, iops=96, runt=2728685msec slat (usec): min=151, max=812273, avg=10322.52, stdev=38234.84 clat (usec): min=5, max=828136, avg=36428.44, stdev=70309.58 lat (usec): min=337, max=833561, avg=46753.19, stdev=78245.72 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1176], 10.00th=[ 1608], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[432128], | 99.99th=[626688] bw (KB /s): min= 31, max= 2272, per=3.13%, avg=384.76, stdev=298.79 write: io=1023.4MB, bw=393253B/s, iops=96, runt=2728685msec slat (usec): min=4, max=457410, avg=62.53, stdev=3683.36 clat (usec): min=65, max=824155, avg=36439.04, stdev=70025.83 lat (usec): min=72, max=824172, avg=36503.10, stdev=70145.26 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1208], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 25, max= 2318, per=3.13%, avg=384.23, stdev=305.86 lat (usec) : 10=0.01%, 100=0.54%, 250=0.25%, 500=0.10%, 750=0.86% lat (usec) : 1000=2.07% lat (msec) : 2=12.46%, 4=40.08%, 10=17.66%, 20=6.93%, 50=1.36% lat (msec) : 100=0.01%, 250=17.51%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.09%, sys=0.37%, ctx=269909, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64423: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394432B/s, iops=96, runt=2723959msec slat (usec): min=177, max=636340, avg=10306.72, stdev=38218.19 clat (usec): min=7, max=1022.4K, avg=36254.50, stdev=70051.11 lat (usec): min=331, max=1029.8K, avg=46563.40, stdev=78041.16 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[17536], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[419840], | 99.99th=[618496] bw (KB /s): min= 37, max= 1370, per=3.14%, avg=385.89, stdev=298.73 write: io=1023.4MB, bw=393936B/s, iops=96, runt=2723959msec slat (usec): min=4, max=633574, avg=60.34, stdev=3662.46 clat (usec): min=64, max=833446, avg=36486.94, stdev=70181.33 lat (usec): min=69, max=853093, avg=36548.88, stdev=70328.04 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2256], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[411648], | 99.99th=[626688] bw (KB /s): min= 6, max= 1167, per=3.14%, avg=385.31, stdev=304.47 lat (usec) : 10=0.01%, 100=0.57%, 250=0.22%, 500=0.08%, 750=0.87% lat (usec) : 1000=1.98% lat (msec) : 2=12.39%, 4=40.42%, 10=17.46%, 20=6.99%, 50=1.34% lat (msec) : 100=0.01%, 250=17.47%, 500=0.15%, 750=0.03%, 1000=0.01% lat (msec) : 2000=0.01% cpu : usr=0.09%, sys=0.36%, ctx=269787, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64424: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393751B/s, iops=96, runt=2728668msec slat (usec): min=155, max=814104, avg=10321.68, stdev=38237.34 clat (usec): min=4, max=831544, avg=36327.42, stdev=70130.87 lat (usec): min=341, max=1017.7K, avg=46651.40, stdev=78121.61 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1176], 10.00th=[ 1608], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[419840], | 99.99th=[618496] bw (KB /s): min= 24, max= 2528, per=3.13%, avg=384.81, stdev=298.83 write: io=1023.4MB, bw=393256B/s, iops=96, runt=2728668msec slat (usec): min=4, max=450123, avg=63.10, stdev=3709.80 clat (usec): min=65, max=834050, avg=36539.85, stdev=70164.82 lat (usec): min=71, max=834057, avg=36604.54, stdev=70298.65 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1192], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6496], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[411648], | 99.99th=[626688] bw (KB /s): min= 12, max= 2627, per=3.13%, avg=384.28, stdev=306.17 lat (usec) : 10=0.01%, 100=0.56%, 250=0.23%, 500=0.09%, 750=0.90% lat (usec) : 1000=2.06% lat (msec) : 2=12.36%, 4=40.08%, 10=17.71%, 20=6.96%, 50=1.36% lat (msec) : 100=0.01%, 250=17.51%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.09%, sys=0.37%, ctx=269581, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64425: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394104B/s, iops=96, runt=2726223msec slat (usec): min=199, max=638418, avg=10307.82, stdev=38175.58 clat (usec): min=5, max=832409, avg=36327.28, stdev=70128.41 lat (usec): min=305, max=847497, avg=46637.32, stdev=78112.96 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6304], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[419840], | 99.99th=[618496] bw (KB /s): min= 25, max= 1597, per=3.13%, avg=385.30, stdev=298.26 write: io=1023.4MB, bw=393609B/s, iops=96, runt=2726223msec slat (usec): min=4, max=633472, avg=67.83, stdev=4015.64 clat (usec): min=64, max=844272, avg=36474.58, stdev=70200.16 lat (usec): min=69, max=844279, avg=36544.01, stdev=70361.76 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2256], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[618496] bw (KB /s): min= 6, max= 1425, per=3.13%, avg=384.69, stdev=304.01 lat (usec) : 10=0.01%, 100=0.56%, 250=0.23%, 500=0.08%, 750=0.87% lat (usec) : 1000=1.98% lat (msec) : 2=12.33%, 4=40.45%, 10=17.49%, 20=6.95%, 50=1.38% lat (msec) : 100=0.01%, 250=17.49%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269339, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64426: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393957B/s, iops=96, runt=2727240msec slat (usec): min=138, max=813409, avg=10322.56, stdev=38276.88 clat (usec): min=5, max=822226, avg=36287.73, stdev=70018.21 lat (usec): min=325, max=1017.6K, avg=46612.42, stdev=78038.77 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[618496] bw (KB /s): min= 19, max= 1412, per=3.13%, avg=385.37, stdev=297.94 write: io=1023.4MB, bw=393462B/s, iops=96, runt=2727240msec slat (usec): min=4, max=446421, avg=57.44, stdev=3409.80 clat (usec): min=65, max=832789, avg=36541.40, stdev=70315.11 lat (usec): min=71, max=832804, avg=36600.31, stdev=70432.95 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2256], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[419840], | 99.99th=[626688] bw (KB /s): min= 25, max= 1425, per=3.13%, avg=384.86, stdev=305.90 lat (usec) : 10=0.01%, 100=0.55%, 250=0.24%, 500=0.09%, 750=0.94% lat (usec) : 1000=1.94% lat (msec) : 2=12.33%, 4=40.39%, 10=17.60%, 20=6.84%, 50=1.40% lat (msec) : 100=0.01%, 250=17.50%, 500=0.14%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.36%, ctx=269685, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64427: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394047B/s, iops=96, runt=2726619msec slat (usec): min=156, max=813760, avg=10314.40, stdev=38208.73 clat (usec): min=4, max=831085, avg=36169.35, stdev=69879.25 lat (usec): min=301, max=834572, avg=46485.86, stdev=77878.26 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[419840], | 99.99th=[618496] bw (KB /s): min= 31, max= 1478, per=3.13%, avg=385.27, stdev=297.83 write: io=1023.4MB, bw=393551B/s, iops=96, runt=2726619msec slat (usec): min=4, max=461219, avg=63.07, stdev=3776.19 clat (usec): min=63, max=828388, avg=36643.32, stdev=70365.95 lat (usec): min=70, max=828394, avg=36707.94, stdev=70507.61 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2256], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 12, max= 1260, per=3.13%, avg=384.74, stdev=304.41 lat (usec) : 10=0.01%, 100=0.58%, 250=0.22%, 500=0.08%, 750=0.89% lat (usec) : 1000=2.00% lat (msec) : 2=12.29%, 4=40.33%, 10=17.58%, 20=6.95%, 50=1.39% lat (msec) : 100=0.01%, 250=17.49%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.36%, ctx=269463, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64428: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393957B/s, iops=96, runt=2727242msec slat (usec): min=177, max=814169, avg=10308.07, stdev=38174.95 clat (usec): min=11, max=1019.7K, avg=36175.10, stdev=70110.15 lat (usec): min=336, max=1024.7K, avg=46485.33, stdev=78093.91 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1176], 10.00th=[ 1624], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[419840], | 99.99th=[626688] bw (KB /s): min= 31, max= 1423, per=3.13%, avg=385.21, stdev=298.03 write: io=1023.4MB, bw=393461B/s, iops=96, runt=2727242msec slat (usec): min=4, max=451683, avg=71.25, stdev=3835.23 clat (usec): min=66, max=1019.7K, avg=36654.34, stdev=70210.76 lat (usec): min=72, max=1019.7K, avg=36727.14, stdev=70355.32 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1208], 10.00th=[ 1640], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6496], 80.00th=[18304], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[411648], | 99.99th=[626688] bw (KB /s): min= 25, max= 1307, per=3.13%, avg=384.68, stdev=305.56 lat (usec) : 20=0.01%, 100=0.55%, 250=0.25%, 500=0.09%, 750=0.89% lat (usec) : 1000=2.00% lat (msec) : 2=12.42%, 4=40.32%, 10=17.43%, 20=6.98%, 50=1.39% lat (msec) : 250=17.50%, 500=0.15%, 750=0.04%, 1000=0.01%, 2000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269701, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64429: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393689B/s, iops=96, runt=2729102msec slat (usec): min=159, max=815295, avg=10316.26, stdev=38195.88 clat (usec): min=6, max=835951, avg=36338.41, stdev=70161.99 lat (usec): min=281, max=839772, avg=46656.95, stdev=78117.86 clat percentiles (usec): | 1.00th=[ 572], 5.00th=[ 1144], 10.00th=[ 1592], 20.00th=[ 2160], | 30.00th=[ 2608], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[419840], | 99.99th=[634880] bw (KB /s): min= 24, max= 5232, per=3.13%, avg=384.47, stdev=305.81 write: io=1023.4MB, bw=393193B/s, iops=95, runt=2729102msec slat (usec): min=4, max=462235, avg=70.01, stdev=3947.71 clat (usec): min=64, max=827443, avg=36540.54, stdev=70230.65 lat (usec): min=70, max=827452, avg=36612.17, stdev=70380.88 clat percentiles (usec): | 1.00th=[ 556], 5.00th=[ 1160], 10.00th=[ 1624], 20.00th=[ 2224], | 30.00th=[ 2672], 40.00th=[ 3152], 50.00th=[ 3632], 60.00th=[ 4320], | 70.00th=[ 6368], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 18, max= 5016, per=3.12%, avg=383.85, stdev=313.19 lat (usec) : 10=0.01%, 100=0.54%, 250=0.26%, 500=0.11%, 750=0.92% lat (usec) : 1000=2.09% lat (msec) : 2=12.58%, 4=39.94%, 10=17.56%, 20=6.90%, 50=1.40% lat (msec) : 100=0.01%, 250=17.51%, 500=0.14%, 750=0.04%, 1000=0.01% cpu : usr=0.09%, sys=0.36%, ctx=270271, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64430: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393956B/s, iops=96, runt=2727247msec slat (usec): min=121, max=630608, avg=10305.79, stdev=38137.51 clat (usec): min=4, max=1028.9K, avg=36052.83, stdev=69989.79 lat (usec): min=296, max=1039.9K, avg=46360.79, stdev=77946.09 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6176], 80.00th=[17536], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[428032], | 99.99th=[626688] bw (KB /s): min= 6, max= 1478, per=3.13%, avg=385.03, stdev=297.71 write: io=1023.4MB, bw=393461B/s, iops=96, runt=2727247msec slat (usec): min=4, max=633362, avg=73.71, stdev=4204.52 clat (usec): min=65, max=836512, avg=36776.88, stdev=70219.48 lat (usec): min=71, max=863829, avg=36852.11, stdev=70395.64 clat percentiles (usec): | 1.00th=[ 572], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2256], | 30.00th=[ 2704], 40.00th=[ 3184], 50.00th=[ 3664], 60.00th=[ 4384], | 70.00th=[ 6688], 80.00th=[18304], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 18, max= 1359, per=3.13%, avg=384.57, stdev=304.11 lat (usec) : 10=0.01%, 100=0.55%, 250=0.24%, 500=0.10%, 750=0.87% lat (usec) : 1000=2.04% lat (msec) : 2=12.28%, 4=40.20%, 10=17.65%, 20=7.01%, 50=1.37% lat (msec) : 100=0.01%, 250=17.51%, 500=0.15%, 750=0.04%, 1000=0.01% lat (msec) : 2000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269510, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64431: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393991B/s, iops=96, runt=2727005msec slat (usec): min=204, max=614431, avg=10295.30, stdev=38038.04 clat (usec): min=4, max=833488, avg=36371.63, stdev=70233.58 lat (usec): min=332, max=837123, avg=46669.09, stdev=78144.33 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[428032], | 99.99th=[618496] bw (KB /s): min= 12, max= 1408, per=3.13%, avg=385.10, stdev=297.87 write: io=1023.4MB, bw=393496B/s, iops=96, runt=2727005msec slat (usec): min=4, max=629303, avg=83.41, stdev=4550.11 clat (usec): min=65, max=822367, avg=36451.20, stdev=69991.00 lat (usec): min=71, max=833499, avg=36536.15, stdev=70192.02 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1240], 10.00th=[ 1672], 20.00th=[ 2256], | 30.00th=[ 2736], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6432], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[415744], | 99.99th=[618496] bw (KB /s): min= 18, max= 1236, per=3.13%, avg=384.67, stdev=305.20 lat (usec) : 10=0.01%, 100=0.54%, 250=0.24%, 500=0.08%, 750=0.85% lat (usec) : 1000=2.02% lat (msec) : 2=12.27%, 4=40.46%, 10=17.54%, 20=6.89%, 50=1.42% lat (msec) : 100=0.01%, 250=17.50%, 500=0.14%, 750=0.04%, 1000=0.01% cpu : usr=0.09%, sys=0.36%, ctx=269559, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64432: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393696B/s, iops=96, runt=2729051msec slat (usec): min=152, max=814459, avg=10320.65, stdev=38209.01 clat (usec): min=5, max=819024, avg=36255.49, stdev=69893.76 lat (usec): min=293, max=827261, avg=46578.39, stdev=77884.99 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1160], 10.00th=[ 1608], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[618496] bw (KB /s): min= 30, max= 5096, per=3.13%, avg=384.71, stdev=305.86 write: io=1023.4MB, bw=393201B/s, iops=95, runt=2729051msec slat (usec): min=4, max=463675, avg=65.31, stdev=3766.69 clat (usec): min=65, max=827245, avg=36622.15, stdev=70278.71 lat (usec): min=71, max=827251, avg=36689.11, stdev=70411.40 clat percentiles (usec): | 1.00th=[ 564], 5.00th=[ 1192], 10.00th=[ 1656], 20.00th=[ 2256], | 30.00th=[ 2704], 40.00th=[ 3184], 50.00th=[ 3664], 60.00th=[ 4384], | 70.00th=[ 6688], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 12, max= 4952, per=3.13%, avg=384.17, stdev=310.24 lat (usec) : 10=0.01%, 100=0.54%, 250=0.25%, 500=0.11%, 750=0.89% lat (usec) : 1000=2.04% lat (msec) : 2=12.41%, 4=39.91%, 10=17.76%, 20=7.01%, 50=1.37% lat (msec) : 100=0.01%, 250=17.52%, 500=0.14%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.38%, ctx=270029, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64433: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394047B/s, iops=96, runt=2726620msec slat (usec): min=152, max=813048, avg=10313.35, stdev=38213.10 clat (usec): min=4, max=837253, avg=36267.48, stdev=70084.86 lat (usec): min=330, max=1016.8K, avg=46582.96, stdev=78071.15 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6304], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[423936], | 99.99th=[626688] bw (KB /s): min= 30, max= 1613, per=3.13%, avg=385.23, stdev=298.37 write: io=1023.4MB, bw=393551B/s, iops=96, runt=2726620msec slat (usec): min=4, max=459441, avg=63.91, stdev=3838.63 clat (usec): min=66, max=837483, avg=36545.45, stdev=70227.19 lat (usec): min=73, max=837492, avg=36610.87, stdev=70360.09 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6496], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 12, max= 1441, per=3.13%, avg=384.71, stdev=305.07 lat (usec) : 10=0.01%, 100=0.55%, 250=0.25%, 500=0.08%, 750=0.86% lat (usec) : 1000=2.02% lat (msec) : 2=12.44%, 4=40.12%, 10=17.70%, 20=6.97%, 50=1.32% lat (msec) : 100=0.01%, 250=17.49%, 500=0.16%, 750=0.03%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269410, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64434: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393753B/s, iops=96, runt=2728656msec slat (usec): min=153, max=813074, avg=10308.75, stdev=38112.88 clat (usec): min=12, max=826216, avg=36298.35, stdev=70080.81 lat (usec): min=318, max=830370, avg=46609.35, stdev=78023.20 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1176], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[423936], | 99.99th=[618496] bw (KB /s): min= 31, max= 2003, per=3.13%, avg=384.90, stdev=298.42 write: io=1023.4MB, bw=393258B/s, iops=96, runt=2728656msec slat (usec): min=4, max=456566, avg=75.66, stdev=4304.47 clat (usec): min=66, max=822655, avg=36568.77, stdev=70117.07 lat (usec): min=72, max=822661, avg=36646.04, stdev=70282.28 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1208], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[411648], | 99.99th=[626688] bw (KB /s): min= 18, max= 2147, per=3.13%, avg=384.36, stdev=305.30 lat (usec) : 20=0.01%, 100=0.55%, 250=0.23%, 500=0.10%, 750=0.89% lat (usec) : 1000=2.01% lat (msec) : 2=12.38%, 4=40.05%, 10=17.69%, 20=7.01%, 50=1.40% lat (msec) : 100=0.01%, 250=17.51%, 500=0.15%, 750=0.03%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=270233, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64435: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393962B/s, iops=96, runt=2727208msec slat (usec): min=198, max=813045, avg=10317.59, stdev=38244.35 clat (usec): min=4, max=822116, avg=36165.71, stdev=70005.93 lat (usec): min=348, max=831564, avg=46485.58, stdev=78007.97 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1176], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6304], 80.00th=[17536], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[403456], 99.95th=[423936], | 99.99th=[618496] bw (KB /s): min= 32, max= 1666, per=3.13%, avg=385.52, stdev=298.04 write: io=1023.4MB, bw=393466B/s, iops=96, runt=2727208msec slat (usec): min=4, max=444774, avg=61.84, stdev=3508.39 clat (usec): min=65, max=831556, avg=36662.94, stdev=70263.41 lat (usec): min=71, max=831563, avg=36726.26, stdev=70390.31 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6624], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 19, max= 1480, per=3.13%, avg=385.01, stdev=303.64 lat (usec) : 10=0.01%, 100=0.55%, 250=0.24%, 500=0.09%, 750=0.89% lat (usec) : 1000=2.00% lat (msec) : 2=12.42%, 4=40.22%, 10=17.58%, 20=6.94%, 50=1.39% lat (msec) : 250=17.50%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269696, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64436: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393871B/s, iops=96, runt=2727835msec slat (usec): min=169, max=638590, avg=10309.92, stdev=38150.60 clat (usec): min=4, max=833343, avg=36250.44, stdev=70122.36 lat (usec): min=323, max=837326, avg=46562.66, stdev=78093.97 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1176], 10.00th=[ 1624], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6304], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[428032], | 99.99th=[618496] bw (KB /s): min= 31, max= 1668, per=3.13%, avg=384.99, stdev=298.58 write: io=1023.4MB, bw=393376B/s, iops=96, runt=2727835msec slat (usec): min=4, max=634718, avg=71.72, stdev=4149.87 clat (usec): min=65, max=834068, avg=36594.86, stdev=70143.55 lat (usec): min=71, max=834076, avg=36668.25, stdev=70299.07 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1208], 10.00th=[ 1640], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3632], 60.00th=[ 4320], | 70.00th=[ 6496], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[395264], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 6, max= 1526, per=3.13%, avg=384.37, stdev=306.21 lat (usec) : 10=0.01%, 100=0.56%, 250=0.23%, 500=0.10%, 750=0.88% lat (usec) : 1000=2.07% lat (msec) : 2=12.41%, 4=40.23%, 10=17.44%, 20=6.99%, 50=1.40% lat (msec) : 100=0.01%, 250=17.51%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.09%, sys=0.36%, ctx=269375, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64437: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393756B/s, iops=96, runt=2728636msec slat (usec): min=154, max=815425, avg=10314.33, stdev=38165.80 clat (usec): min=13, max=829340, avg=36389.13, stdev=70166.78 lat (usec): min=317, max=834125, avg=46705.79, stdev=78087.65 clat percentiles (usec): | 1.00th=[ 572], 5.00th=[ 1176], 10.00th=[ 1608], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[423936], | 99.99th=[618496] bw (KB /s): min= 36, max= 2620, per=3.13%, avg=384.89, stdev=299.50 write: io=1023.4MB, bw=393260B/s, iops=96, runt=2728636msec slat (usec): min=4, max=455141, avg=70.22, stdev=4030.80 clat (usec): min=65, max=825199, avg=36477.16, stdev=70002.76 lat (usec): min=70, max=825222, avg=36548.88, stdev=70155.67 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[411648], | 99.99th=[618496] bw (KB /s): min= 12, max= 2679, per=3.13%, avg=384.35, stdev=305.84 lat (usec) : 20=0.01%, 100=0.56%, 250=0.23%, 500=0.11%, 750=0.87% lat (usec) : 1000=2.00% lat (msec) : 2=12.42%, 4=40.14%, 10=17.57%, 20=7.00%, 50=1.39% lat (msec) : 100=0.01%, 250=17.52%, 500=0.15%, 750=0.03%, 1000=0.01% cpu : usr=0.09%, sys=0.36%, ctx=270036, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64438: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393874B/s, iops=96, runt=2727816msec slat (usec): min=186, max=813818, avg=10311.56, stdev=38177.65 clat (usec): min=4, max=834508, avg=36286.38, stdev=70157.85 lat (usec): min=332, max=841892, avg=46600.23, stdev=78129.42 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6304], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[423936], | 99.99th=[626688] bw (KB /s): min= 24, max= 1832, per=3.13%, avg=385.30, stdev=298.76 write: io=1023.4MB, bw=393379B/s, iops=96, runt=2727816msec slat (usec): min=4, max=446103, avg=70.02, stdev=3942.75 clat (usec): min=65, max=841883, avg=36558.29, stdev=70215.14 lat (usec): min=71, max=841890, avg=36629.87, stdev=70368.27 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1208], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[618496] bw (KB /s): min= 12, max= 1970, per=3.13%, avg=384.78, stdev=305.34 lat (usec) : 10=0.01%, 100=0.55%, 250=0.24%, 500=0.10%, 750=0.87% lat (usec) : 1000=1.97% lat (msec) : 2=12.38%, 4=40.27%, 10=17.60%, 20=6.94%, 50=1.38% lat (msec) : 250=17.49%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.09%, sys=0.37%, ctx=269728, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64439: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394254B/s, iops=96, runt=2725187msec slat (usec): min=184, max=637802, avg=10307.86, stdev=38220.72 clat (usec): min=4, max=839864, avg=36227.75, stdev=69995.50 lat (usec): min=321, max=1020.2K, avg=46537.77, stdev=77980.57 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[419840], | 99.99th=[626688] bw (KB /s): min= 6, max= 1795, per=3.14%, avg=385.80, stdev=299.29 write: io=1023.4MB, bw=393758B/s, iops=96, runt=2725187msec slat (usec): min=4, max=633865, avg=63.95, stdev=3770.60 clat (usec): min=65, max=842348, avg=36546.72, stdev=70325.13 lat (usec): min=71, max=844557, avg=36612.17, stdev=70469.39 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3632], 60.00th=[ 4320], | 70.00th=[ 6496], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 18, max= 1603, per=3.14%, avg=385.40, stdev=304.97 lat (usec) : 10=0.01%, 100=0.56%, 250=0.22%, 500=0.09%, 750=0.90% lat (usec) : 1000=1.99% lat (msec) : 2=12.39%, 4=40.36%, 10=17.51%, 20=6.91%, 50=1.40% lat (msec) : 100=0.01%, 250=17.48%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.07%, sys=0.37%, ctx=269347, majf=0, minf=8 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64440: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=395109B/s, iops=96, runt=2719289msec slat (usec): min=185, max=613296, avg=10281.61, stdev=38143.91 clat (usec): min=4, max=824895, avg=36067.13, stdev=69935.97 lat (usec): min=315, max=828459, avg=46350.81, stdev=77899.56 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2160], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6176], 80.00th=[17536], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[423936], | 99.99th=[618496] bw (KB /s): min= 6, max= 1610, per=3.14%, avg=386.42, stdev=298.73 write: io=1023.4MB, bw=394612B/s, iops=96, runt=2719289msec slat (usec): min=4, max=631680, avg=67.78, stdev=3982.70 clat (usec): min=64, max=828447, avg=36549.71, stdev=70241.56 lat (usec): min=70, max=828461, avg=36619.07, stdev=70393.77 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1224], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3632], 60.00th=[ 4320], | 70.00th=[ 6496], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 6, max= 1438, per=3.14%, avg=385.93, stdev=305.50 lat (usec) : 10=0.01%, 100=0.58%, 250=0.21%, 500=0.09%, 750=0.87% lat (usec) : 1000=1.98% lat (msec) : 2=12.41%, 4=40.48%, 10=17.46%, 20=6.89%, 50=1.40% lat (msec) : 100=0.01%, 250=17.43%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269472, majf=0, minf=9 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64441: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393752B/s, iops=96, runt=2728663msec slat (usec): min=143, max=812717, avg=10322.09, stdev=38245.47 clat (usec): min=5, max=832621, avg=36185.07, stdev=69926.93 lat (usec): min=189, max=1022.2K, avg=46509.44, stdev=77960.98 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1176], 10.00th=[ 1608], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[618496] bw (KB /s): min= 30, max= 2482, per=3.13%, avg=384.68, stdev=299.11 write: io=1023.4MB, bw=393257B/s, iops=96, runt=2728663msec slat (usec): min=4, max=436248, avg=62.52, stdev=3639.77 clat (usec): min=65, max=1017.5K, avg=36682.37, stdev=70467.13 lat (usec): min=71, max=1017.5K, avg=36746.36, stdev=70591.62 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1208], 10.00th=[ 1640], 20.00th=[ 2256], | 30.00th=[ 2704], 40.00th=[ 3184], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6624], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[403456], 99.95th=[436224], | 99.99th=[626688] bw (KB /s): min= 18, max= 2568, per=3.13%, avg=384.17, stdev=305.11 lat (usec) : 10=0.01%, 100=0.54%, 250=0.25%, 500=0.10%, 750=0.87% lat (usec) : 1000=2.04% lat (msec) : 2=12.39%, 4=40.03%, 10=17.74%, 20=6.95%, 50=1.40% lat (msec) : 250=17.50%, 500=0.15%, 750=0.04%, 1000=0.01%, 2000=0.01% cpu : usr=0.09%, sys=0.37%, ctx=270021, majf=0, minf=9 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64442: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=393837B/s, iops=96, runt=2728074msec slat (usec): min=157, max=638953, avg=10312.60, stdev=38200.01 clat (usec): min=4, max=837508, avg=36277.22, stdev=70113.28 lat (usec): min=324, max=841617, avg=46591.99, stdev=78087.85 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1176], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4256], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[423936], | 99.99th=[618496] bw (KB /s): min= 38, max= 1762, per=3.13%, avg=385.05, stdev=298.19 write: io=1023.4MB, bw=393341B/s, iops=96, runt=2728074msec slat (usec): min=4, max=624233, avg=69.97, stdev=3861.36 clat (usec): min=65, max=841601, avg=36574.36, stdev=70227.83 lat (usec): min=71, max=841612, avg=36645.90, stdev=70374.59 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1208], 10.00th=[ 1656], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[18048], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 12, max= 1664, per=3.13%, avg=384.46, stdev=304.57 lat (usec) : 10=0.01%, 100=0.57%, 250=0.22%, 500=0.09%, 750=0.90% lat (usec) : 1000=2.02% lat (msec) : 2=12.31%, 4=40.27%, 10=17.56%, 20=6.99%, 50=1.40% lat (msec) : 100=0.01%, 250=17.50%, 500=0.14%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269663, majf=0, minf=9 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64443: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=394167B/s, iops=96, runt=2725790msec slat (usec): min=203, max=812786, avg=10304.74, stdev=38189.51 clat (usec): min=4, max=1020.2K, avg=36325.32, stdev=70229.64 lat (usec): min=340, max=1025.2K, avg=46632.23, stdev=78182.24 clat percentiles (usec): | 1.00th=[ 596], 5.00th=[ 1192], 10.00th=[ 1624], 20.00th=[ 2192], | 30.00th=[ 2640], 40.00th=[ 3088], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6240], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[403456], 99.95th=[428032], | 99.99th=[626688] bw (KB /s): min= 38, max= 1735, per=3.13%, avg=385.59, stdev=298.45 write: io=1023.4MB, bw=393671B/s, iops=96, runt=2725790msec slat (usec): min=4, max=439493, avg=69.60, stdev=3877.85 clat (usec): min=64, max=1016.8K, avg=36465.15, stdev=70093.28 lat (usec): min=70, max=1016.8K, avg=36536.25, stdev=70246.88 clat percentiles (usec): | 1.00th=[ 580], 5.00th=[ 1208], 10.00th=[ 1640], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3152], 50.00th=[ 3664], 60.00th=[ 4320], | 70.00th=[ 6560], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 19, max= 1584, per=3.13%, avg=385.06, stdev=305.14 lat (usec) : 10=0.01%, 100=0.56%, 250=0.23%, 500=0.08%, 750=0.90% lat (usec) : 1000=2.01% lat (msec) : 2=12.34%, 4=40.38%, 10=17.50%, 20=6.94%, 50=1.38% lat (msec) : 100=0.01%, 250=17.49%, 500=0.14%, 750=0.04%, 1000=0.01% lat (msec) : 2000=0.01% cpu : usr=0.09%, sys=0.36%, ctx=269774, majf=0, minf=9 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 random-rw: (groupid=0, jobs=1): err= 0: pid=64444: Fri Sep 4 15:07:00 2015 read : io=1024.7MB, bw=395875B/s, iops=96, runt=2714032msec slat (usec): min=190, max=813664, avg=10265.64, stdev=38151.33 clat (usec): min=6, max=832209, avg=36024.38, stdev=69958.14 lat (usec): min=294, max=835563, avg=46292.19, stdev=77967.38 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1176], 10.00th=[ 1608], 20.00th=[ 2160], | 30.00th=[ 2608], 40.00th=[ 3056], 50.00th=[ 3568], 60.00th=[ 4192], | 70.00th=[ 6176], 80.00th=[17536], 90.00th=[179200], 95.00th=[189440], | 99.00th=[199680], 99.50th=[201728], 99.90th=[399360], 99.95th=[428032], | 99.99th=[618496] bw (KB /s): min= 38, max= 1785, per=3.15%, avg=386.99, stdev=299.74 write: io=1023.4MB, bw=395377B/s, iops=96, runt=2714032msec slat (usec): min=4, max=446264, avg=63.87, stdev=3775.34 clat (usec): min=65, max=833432, avg=36452.13, stdev=70286.39 lat (usec): min=70, max=833439, avg=36517.68, stdev=70408.88 clat percentiles (usec): | 1.00th=[ 588], 5.00th=[ 1208], 10.00th=[ 1640], 20.00th=[ 2224], | 30.00th=[ 2704], 40.00th=[ 3120], 50.00th=[ 3632], 60.00th=[ 4320], | 70.00th=[ 6368], 80.00th=[17792], 90.00th=[179200], 95.00th=[189440], | 99.00th=[197632], 99.50th=[201728], 99.90th=[399360], 99.95th=[415744], | 99.99th=[626688] bw (KB /s): min= 19, max= 1601, per=3.15%, avg=386.47, stdev=306.85 lat (usec) : 10=0.01%, 100=0.57%, 250=0.22%, 500=0.08%, 750=0.91% lat (usec) : 1000=2.00% lat (msec) : 2=12.55%, 4=40.42%, 10=17.46%, 20=6.84%, 50=1.37% lat (msec) : 100=0.01%, 250=17.39%, 500=0.15%, 750=0.04%, 1000=0.01% cpu : usr=0.08%, sys=0.37%, ctx=269315, majf=0, minf=9 IO depths : 1=0.1%, 2=0.1%, 4=0.1%, 8=100.0%, 16=0.0%, 32=0.0%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=100.0%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% issued : total=r=262309/w=261979/d=0, short=r=0/w=0/d=0 Run status group 0 (all jobs): READ: io=32789MB, aggrb=12302KB/s, minb=384KB/s, maxb=386KB/s, mint=2712395msec, maxt=2729102msec WRITE: io=32747MB, aggrb=12287KB/s, minb=383KB/s, maxb=386KB/s, mint=2712395msec, maxt=2729102msec Disk stats (read/write): sdb: ios=8393873/7973262, merge=0/1596, ticks=84672724/229886452, in_queue=314564564, util=100.00%