SuperMicro X9DRH-iF (CSE-219A-R920LPB, BPN-SAS-213A) 2x E5-2690 @ 2.90GHz (8 cores with 2 HTs) 128 GB 1600MHz ECC LSI-620J 1x ESM 2x WidePort Uplink 24x Seagate ST9146852SS via LSI SAS 9207-8e (PCIe 3.0 x8, IT P15) 2x WidePort Uplink Solaris 11.1 SRU 5.5, mxpio=enabled setenv JBOD1 /dev/chassis/JBOD1 + zpool create [-f] -o failmode=continue -O compression=on -O atime=off pool1 \ $JBOD1/SLOT_01_/disk $JBOD1/SLOT_02_/disk \ $JBOD1/SLOT_03_/disk $JBOD1/SLOT_04_/disk \ $JBOD1/SLOT_05_/disk $JBOD1/SLOT_06_/disk \ $JBOD1/SLOT_07_/disk $JBOD1/SLOT_08_/disk \ $JBOD1/SLOT_09_/disk $JBOD1/SLOT_10_/disk \ $JBOD1/SLOT_11_/disk $JBOD1/SLOT_12_/disk \ $JBOD1/SLOT_13_/disk $JBOD1/SLOT_14_/disk \ $JBOD1/SLOT_15_/disk $JBOD1/SLOT_16_/disk \ $JBOD1/SLOT_17_/disk $JBOD1/SLOT_18_/disk \ $JBOD1/SLOT_19_/disk $JBOD1/SLOT_20_/disk \ $JBOD1/SLOT_21_/disk $JBOD1/SLOT_22_/disk \ $JBOD1/SLOT_23_/disk $JBOD1/SLOT_24_/disk Simple TESTs to check upper bounds: + time sh -c 'mkfile 32g bla ; sync' ; \ + time sh -c 'mkfile 32g blabla ; sync' ; \ + time sh -c 'mkfile 32g blablabla ; sync' ; \ + time sh -c 'mkfile 32g blablablabla ; sync' + chmod 644 b* + time dd if=bla of=/dev/null bs=128k ; \ + time dd if=blabla of=/dev/null bs=128k ; \ + time dd if=blablabla of=/dev/null bs=128k ; \ + time dd if=blablablabla of=/dev/null bs=128k --------------------------------------------------------------------------- compression=on, atime=off, recordsize=128K --------------------------------------------------------------------------- write 2035.3 2141.7 2127.8 2087.1 MiB/s read 1655.0 1630.3 1622.2 1638.4 MiB/s filebench 1.4.8 (0.175.1.0.0.24.2) as is with 'set $dir=/pool1/test': ops ops/s r/w mb/s cpu/op latency [µs] [ms] bringover: 6002 3000.8 500/500 15.0 200 0.9 6002 6001.4 1001/1000 30.0 173 0.3 6002 6001.4 1001/1000 30.0 173 0.3 6002 6001.5 1001/1000 30.0 207 0.3 copyfiles: 6002 5995.4 1000/999 30.0 197 0.4 6002 6001.4 1001/1000 30.0 252 0.2 6002 5995.5 1000/999 30.0 177 0.2 6002 6001.4 1001/1000 30.0 203 0.2 createfiles: 149979 74983.2 0/24994 390.5 370 0.4 149975 74907.7 0/24968 390.1 343 0.4 149978 74909.2 0/24969 390.1 447 0.5 149973 74944.3 0/24980 390.3 348 0.4 deletefiles: 50000 24986.2 0/0 0.0 0 0.0 50000 12487.0 0/0 0.0 0 0.0 50000 16654.8 0/0 0.0 0 0.0 50000 24986.1 0/0 0.0 0 0.0 fileserver: 6525512 108329.5 9848/19696 2626.0 1041 1.4 5032068 83523.2 7593/15186 2025.4 1370 1.9 5257596 87349.4 7941/15882 2118.3 1308 1.8 5579590 92574.8 8416/16832 2243.3 1225 1.7 listdirs: 17212476 286621.2 0/0 59.5 0 0.0 17371666 289275.8 0/0 60.1 0 0.0 17401520 289750.4 0/0 60.2 0 0.0 17403659 289800.1 0/0 60.2 0 0.0 makedirs: 10000 9999.1 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9999.0 0/0 0.0 0 0.0 10000 9998.8 0/0 0.0 0 0.0 mongo: 7006 7005.4 1001/1001 23.0 126 0.4 7006 7005.4 1001/1001 23.0 131 0.3 7006 6998.3 1000/1000 23.0 121 0.3 7006 6998.1 1000/1000 23.0 137 0.3 multistreamread: 857915 14284.6 14285/0 14270.6 278 0.3 857482 14279.7 14280/0 14265.7 279 0.3 855288 14240.2 14240/0 14226.2 279 0.3 857187 14271.8 14272/0 14257.9 279 0.3 multistreamreaddirect: 859665 14313.1 14313/0 14299.1 278 0.3 857958 14286.0 14286/0 14272.0 279 0.3 855820 14249.9 14250/0 14236.0 279 0.3 857604 14278.8 14279/0 14264.8 279 0.3 multistreamwrite: 538142 8963.0 0/8963 8963.0 1314 0.4 539087 8979.3 0/8979 8979.3 1314 0.4 542888 9042.5 0/9043 9042.5 1313 0.4 542559 9036.7 0/9037 9036.7 1316 0.4 multistreamwritedirect: 542403 9036.2 0/9036 9036.1 1311 0.4 545801 9089.2 0/9089 9089.2 1311 0.4 546050 9095.8 0/9096 9095.7 1309 0.4 542833 9041.1 0/9041 9041.0 1311 0.4 networkfs: 36661 610.7 90/90 3.7 354 0.5 36661 610.5 90/90 3.7 379 0.5 36661 610.8 90/90 3.7 359 0.5 36661 610.9 90/90 3.9 378 0.5 oltp: 5813874 96822.4 48180/48146 191.8 84 0.0 5466197 91036.7 45304/45266 180.3 84 0.0 5608409 93401.3 46484/46439 185.0 86 0.0 5456367 90868.4 45222/45180 180.0 85 0.0 openfiles: 26266337 437371.6 0/0 0.0 0 0.0 25689134 427760.8 0/0 0.0 0 0.0 25894791 431158.6 0/0 0.0 0 0.0 25202798 419691.1 0/0 0.0 0 0.0 randomfileaccess: 134497 2239.6 498/249 5620.2 6594 6.7 126837 2111.9 469/235 5156.7 7005 7.1 131470 2189.1 486/243 5503.4 6705 6.8 134559 2240.5 498/249 5648.9 6578 6.7 randomread: 11886820 198088.6 198089/0 1547.6 8 0.0 11703317 194949.5 194950/0 1523.0 8 0.0 11784724 196312.8 196313/0 1533.7 8 0.0 11408875 190066.8 190067/0 1484.9 9 0.0 randomrw: 12286404 204565.2 130455/74110 1598.2 12 0.0 12204432 203200.3 130078/73122 1587.5 12 0.0 12372436 205997.4 130001/75997 1609.3 12 0.0 12265351 204214.6 129470/74745 1595.4 12 0.0 randomwrite: 7246915 120657.0 0/120657 942.6 15 0.0 7113209 118433.1 0/118433 925.2 15 0.0 7214720 120122.8 0/120123 938.4 15 0.0 7080535 117983.9 0/117984 921.7 15 0.0 ratelimcopyfiles: 6002 60.0 10/10 0.3 977 0.5 6002 60.6 10/10 0.3 953 0.4 6002 60.0 10/10 0.3 913 0.4 6002 60.0 10/10 0.3 907 0.4 removedirs: 10000 9999.2 0/0 0.0 0 0.0 10000 9989.2 0/0 0.0 0 0.0 10000 9999.1 0/0 0.0 0 0.0 10000 9999.3 0/0 0.0 0 0.0 singlestreamread: 185536 3091.5 3092/0 3090.9 329 0.3 189032 3147.3 3147/0 3146.7 320 0.3 180352 3002.8 3003/0 3002.2 337 0.3 185170 3083.0 3083/0 3082.4 329 0.3 singlestreamreaddirect: 185204 3083.6 3084/0 3083.0 328 0.3 189857 3161.2 3161/0 3160.5 319 0.3 185597 3090.1 3090/0 3089.5 328 0.3 188103 3132.0 3132/0 3131.4 324 0.3 singlestreamwrite: 127910 2130.9 0/2131 2130.9 1277 0.5 126692 2109.9 0/2110 2109.9 1308 0.5 128178 2134.5 0/2135 2134.5 1286 0.5 126644 2109.8 0/2110 2109.7 1296 0.5 singlestreamwritedirect: 125844 2095.4 0/2095 2095.4 1307 0.5 128123 2134.8 0/2135 2134.7 1280 0.5 126904 2113.4 0/2113 2113.4 1305 0.5 128089 2132.6 0/2133 2132.6 1274 0.5 tpcso: varmail: 817356 13620.5 2095/2096 43.6 238 3.8 814651 13576.1 2089/2089 43.9 246 3.8 801651 13357.0 2055/2055 41.9 242 3.8 814680 13575.9 2088/2089 43.4 244 3.8 videoserver: 23212 386.7 386/0 1028.2 3244 1.4 23200 386.4 386/0 1028.1 3298 1.4 23208 386.6 386/0 1028.3 3235 1.4 23200 386.4 386/0 1028.0 3268 1.4 webproxy: 8454970 140700.4 37026/7405 353.0 491 2.2 8914579 148374.0 39046/7809 371.1 466 2.1 8947061 148984.0 39206/7841 372.4 461 2.1 8763260 145828.8 38376/7675 364.5 461 2.1 webserver: 36037969 600303.7 193646/19366 3182.7 95 0.2 35550365 592304.1 191065/19108 3141.0 96 0.2 30202939 503005.1 162259/16227 2667.2 155 0.4 35501706 591410.8 190777/19079 3136.2 96 0.2 --------------------------------------------------------------------------- compression=off, atime=off, recordsize=128K --------------------------------------------------------------------------- write 1780.9 1771.2 1743.0 1743.0 MiB/s read 1743.0 1724.6 1724.6 1733.8 MiB/s filebench 1.4.8 (0.175.1.0.0.24.2) as is with 'set $dir=/pool1/test': ops ops/s r/w mb/s cpu/op latency [µs] [ms] bringover: 6002 3000.8 500/500 15.0 216 0.5 6002 2997.7 500/499 15.0 176 0.5 6002 3000.8 500/500 15.0 171 0.5 6002 2999.3 500/500 15.0 206 0.5 copyfiles: 6002 3000.7 500/500 15.0 206 0.6 6002 2997.8 500/499 15.0 182 0.6 6002 2000.5 334/333 10.0 206 1.1 6002 3000.8 500/500 15.0 206 0.5 createfiles: 149974 74945.2 0/24980 390.3 347 0.4 149978 74946.1 0/24981 390.3 427 0.5 149973 74945.3 0/24980 390.3 354 0.4 149982 74948.6 0/24984 390.3 414 0.4 deletefiles: 50000 8326.1 0/0 0.0 0 0.0 50000 24986.2 0/0 0.0 0 0.0 50000 16654.6 0/0 0.0 0 0.0 50000 8326.1 0/0 0.0 0 0.0 fileserver: 7160943 118991.1 10817/21635 2886.0 922 1.2 4285968 71048.4 6459/12918 1722.0 1593 2.2 7048476 117249.5 10659/21318 2843.2 946 1.3 6954824 115469.3 10497/20995 2798.7 959 1.3 listdirs: 17583441 292760.0 0/0 60.8 0 0.0 17372226 289268.1 0/0 60.1 0 0.0 17610267 293279.2 0/0 60.9 0 0.0 17365976 289153.7 0/0 60.1 0 0.0 makedirs: 10000 9989.5 0/0 0.0 0 0.0 10000 9989.2 0/0 0.0 0 0.0 10000 9989.2 0/0 0.0 0 0.0 10000 9999.0 0/0 0.0 0 0.0 mongo: 7006 7005.5 1001/1001 23.0 119 0.5 7006 7005.3 1001/1001 23.0 94 0.4 7006 7005.3 1001/1001 23.0 120 0.4 7006 7005.4 1001/1001 23.0 111 0.4 multistreamread: 770343 12837.2 12837/0 12824.6 310 0.3 768974 12805.3 12805/0 12792.8 311 0.3 767202 12786.2 12786/0 12773.7 312 0.3 754232 12569.9 12570/0 12557.7 311 0.3 multistreamreaddirect: 768709 12810.6 12811/0 12798.1 311 0.3 768623 12803.3 12803/0 12790.7 311 0.3 769855 12828.6 12829/0 12816.0 310 0.3 769735 12827.7 12828/0 12815.1 311 0.3 multistreamwrite: 230930 3845.3 0/3845 3845.3 1087 1.0 228567 3806.5 0/3807 3806.4 1075 1.0 224357 3736.8 0/3737 3736.8 1073 1.1 227919 3795.7 0/3796 3795.7 1066 1.0 multistreamwritedirect: 231602 3857.6 0/3858 3857.5 1087 1.0 229285 3817.8 0/3818 3817.7 1076 1.0 233292 3885.5 0/3885 3885.4 1055 1.0 232368 3869.6 0/3870 3869.5 1083 1.0 networkfs: 36661 610.9 90/90 3.7 444 1.3 36661 610.8 90/90 4.1 464 1.4 36123 602.0 89/89 3.7 454 1.5 36661 610.7 90/90 3.6 437 1.4 oltp: 5414189 90167.3 44874/44831 178.6 84 0.0 5574288 92828.7 46191/46162 183.9 83 0.0 5226198 87030.8 43301/43284 172.4 72 0.0 5481513 91276.3 45425/45383 180.8 85 0.0 openfiles: 24977978 415878.7 0/0 0.0 0 0.0 26172462 435843.7 0/0 0.0 0 0.0 25223279 419995.8 0/0 0.0 0 0.0 26123105 434966.7 0/0 0.0 0 0.0 randomfileaccess: 132851 2212.2 492/246 5612.2 6545 6.8 133353 2220.3 493/247 5656.3 6512 6.7 129870 2162.9 481/240 5298.0 6549 6.9 128584 2142.0 476/238 5386.9 6749 7.0 randomread: 11620841 193611.1 193611/0 1512.6 8 0.0 11401133 189909.5 189909/0 1483.7 9 0.0 11350249 189014.9 189015/0 1476.7 9 0.0 11384928 189731.9 189732/0 1482.3 9 0.0 randomrw: 12273826 204354.7 133997/70358 1596.5 12 0.0 12570432 209399.9 134142/75258 1635.9 12 0.0 12186582 203042.0 132680/70362 1586.3 12 0.0 12095511 201434.7 129675/71760 1573.7 13 0.0 randomwrite: 7169774 119467.3 0/119467 933.3 15 0.0 7113597 118546.5 0/118547 926.1 15 0.0 6987066 116355.7 0/116356 909.0 15 0.0 7165944 119310.3 0/119310 932.1 15 0.0 ratelimcopyfiles: 6002 60.0 10/10 0.3 1100 1.7 6002 60.0 10/10 0.3 1110 1.7 6002 60.0 10/10 0.3 1120 1.8 6002 60.0 10/10 0.3 1070 1.8 removedirs: 10000 9999.1 0/0 0.0 0 0.0 10000 9999.1 0/0 0.0 0 0.0 10000 9999.1 0/0 0.0 0 0.0 10000 9989.2 0/0 0.0 0 0.0 singlestreamread: 174364 2903.1 2903/0 2902.5 349 0.3 176883 2945.0 2945/0 2944.5 342 0.3 175639 2926.4 2926/0 2925.8 346 0.3 171759 2859.8 2860/0 2859.3 340 0.3 singlestreamreaddirect: 104712 1744.0 1744/0 1743.6 419 0.6 105648 1759.3 1759/0 1758.9 417 0.6 175326 2919.1 2919/0 2918.6 347 0.3 11932 198.7 199/0 198.7 1500 5.0 singlestreamwrite: 124091 2066.3 0/2066 2066.3 1091 0.5 123728 2061.5 0/2062 2061.5 1079 0.5 99686 1659.8 0/1660 1659.8 1117 0.6 125179 2084.9 0/2085 2084.9 1090 0.5 singlestreamwritedirect: 123899 2063.1 0/2063 2063.1 1088 0.5 124998 2081.3 0/2081 2081.3 1082 0.5 125204 2084.6 0/2085 2084.5 1083 0.5 125435 2089.4 0/2089 2089.4 1084 0.5 tpcso: varmail: 799376 13319.1 2049/2049 43.0 266 3.9 804515 13404.9 2062/2062 41.0 231 3.8 799639 13321.2 2049/2050 43.3 270 3.8 826113 13764.1 2117/2118 42.7 229 3.7 videoserver: 23198 386.4 386/0 1028.1 2966 3.2 23138 385.4 385/0 1027.7 2980 3.9 23192 386.4 386/0 1028.4 2981 3.4 23137 385.4 385/0 1027.8 2977 3.2 webproxy: 8507635 141578.2 37257/7451 354.4 493 2.2 8052362 134101.3 35290/7058 336.6 531 2.3 10207979 169758.9 44673/8935 424.4 391 1.8 9324785 155211.4 40845/8169 388.6 426 2.0 webserver: 35281625 587714.7 189585/18959 3116.1 95 0.2 35860149 597395.1 192708/19272 3167.9 95 0.2 35902662 598128.1 192944/19296 3171.8 95 0.2 35320672 588454.5 189824/18983 3120.1 96 0.2 --------------------------------------------------------------------------- compression=off, atime=off, sync=disabled, recordsize=128K --------------------------------------------------------------------------- write 1830.6 1743.0 1743.0 1743.0 MiB/s read 1583.0 1706.7 1697.8 1697.8 MiB/s filebench 1.4.8 (0.175.1.0.0.24.2) as is with 'set $dir=/pool1/test': ops ops/s r/w mb/s cpu/op latency [µs] [ms] bringover: 6002 5995.6 1000/999 30.0 142 0.4 6002 6001.6 1001/1000 30.0 94 0.1 6002 6001.4 1001/1000 30.0 167 0.5 6002 6001.5 1001/1000 30.0 201 0.5 copyfiles: 6002 5995.5 1000/999 30.0 151 0.4 6002 6001.5 1001/1000 30.0 137 0.4 6002 6001.6 1001/1000 30.0 137 0.4 6002 6001.4 1001/1000 30.0 112 0.3 createfiles: 149973 74943.9 0/24980 390.3 354 0.4 149972 74907.2 0/24967 390.1 344 0.4 149972 74907.1 0/24967 390.1 350 0.4 149973 74944.7 0/24980 390.3 440 0.5 deletefiles: 50000 12487.0 0/0 0.0 0 0.0 50000 9993.5 0/0 0.0 0 0.0 50000 12493.2 0/0 0.0 0 0.0 50000 9989.7 0/0 0.0 0 0.0 fileserver: 7073978 117389.1 10672/21344 2847.8 961 1.3 6481628 107814.0 9801/19603 2613.2 1040 1.4 6505792 108114.3 9829/19657 2621.9 1049 1.4 5406577 89781.9 8162/16324 2176.6 1271 1.7 listdirs: 17656117 294004.4 0/0 61.1 0 0.0 17402239 289780.4 0/0 60.2 0 0.0 17662329 294082.5 0/0 61.1 0 0.0 17177358 285999.5 0/0 59.4 0 0.0 makedirs: 10000 9999.4 0/0 0.0 0 0.0 10000 9999.1 0/0 0.0 0 0.0 10000 9999.3 0/0 0.0 0 0.0 10000 9999.4 0/0 0.0 0 0.0 mongo: 7006 3499.4 500/500 11.5 140 0.5 7006 7005.2 1001/1001 23.0 115 0.5 7006 7005.3 1001/1001 23.0 104 0.4 7006 6998.4 1000/1000 23.0 111 0.4 multistreamread: 769800 12816.8 12817/0 12804.2 311 0.3 753654 12556.2 12556/0 12543.9 311 0.3 769208 12809.6 12810/0 12797.1 310 0.3 768595 12797.5 12798/0 12785.0 311 0.3 multistreamreaddirect: 767939 12797.8 12798/0 12785.3 311 0.3 768242 12803.4 12803/0 12790.9 311 0.3 767581 12786.6 12787/0 12774.1 312 0.3 768622 12797.4 12797/0 12784.9 311 0.3 multistreamwrite: 238397 3971.1 0/3971 3971.1 1054 1.0 236098 3932.0 0/3932 3931.9 1054 1.0 233803 3893.9 0/3894 3893.8 1055 1.0 231569 3856.2 0/3856 3856.1 1053 1.0 multistreamwritedirect: 228259 3801.7 0/3802 3801.6 1059 1.0 240517 4005.6 0/4006 4005.6 1049 1.0 240087 3998.7 0/3999 3998.6 1050 1.0 238135 3966.0 0/3966 3965.9 1044 1.0 networkfs: 36661 610.9 90/90 3.7 399 0.9 36661 610.8 90/90 3.4 405 0.9 36661 611.0 90/90 3.6 425 1.4 36661 610.6 90/90 3.3 427 1.3 oltp: 9833006 163746.2 81461/81446 324.4 48 0.0 9662373 160890.1 80033/80032 318.8 50 0.0 9675822 161127.4 80154/80147 319.2 51 0.0 9626248 160284.9 79732/79731 317.5 51 0.0 openfiles: 25747686 428764.3 0/0 0.0 0 0.0 26181572 435919.0 0/0 0.0 0 0.0 26075459 434236.7 0/0 0.0 0 0.0 26028416 433361.3 0/0 0.0 0 0.0 randomfileaccess: 135324 2253.2 501/250 5650.7 6506 6.6 133553 2224.0 494/247 5651.4 6595 6.7 131185 2184.5 485/243 5437.9 6684 6.8 133211 2218.5 493/247 5585.4 6582 6.7 randomread: 11555079 192414.8 192415/0 1503.2 8 0.0 11365959 189310.3 189310/0 1479.0 9 0.0 11028668 183679.1 183679/0 1435.0 9 0.0 11300656 188297.3 188297/0 1471.1 9 0.0 randomrw: 12461906 207486.9 132932/74555 1621.0 12 0.0 12357605 205750.5 129849/75902 1607.4 12 0.0 12375381 206046.4 130882/75164 1609.7 12 0.0 12482562 207831.0 131077/76754 1623.7 12 0.0 randomwrite: 7682631 128016.3 0/128016 1000.1 14 0.0 7628737 127068.2 0/127068 992.7 14 0.0 7649918 127474.1 0/127474 995.9 14 0.0 7399032 123204.3 0/123204 962.5 14 0.0 ratelimcopyfiles: 6002 60.0 10/10 0.3 1160 1.8 6002 60.0 10/10 0.3 1055 1.7 6002 60.0 10/10 0.3 1070 1.9 6002 60.0 10/10 0.3 1020 1.8 removedirs: 10000 9999.0 0/0 0.0 0 0.0 10000 9998.9 0/0 0.0 0 0.0 10000 9999.0 0/0 0.0 0 0.0 10000 9999.3 0/0 0.0 0 0.0 singlestreamread: 174773 2909.9 2910/0 2909.3 350 0.3 173838 2894.3 2894/0 2893.7 352 0.3 174033 2897.6 2898/0 2897.0 351 0.3 175517 2922.5 2922/0 2921.9 349 0.3 singlestreamreaddirect: 175078 2915.0 2915/0 2914.4 346 0.3 176147 2934.7 2935/0 2934.1 344 0.3 174750 2909.5 2909/0 2908.9 347 0.3 168583 2806.8 2807/0 2806.3 349 0.4 singlestreamwrite: 125688 2093.7 0/2094 2093.7 1068 0.5 124650 2075.6 0/2076 2075.6 1077 0.5 124739 2077.6 0/2078 2077.6 1078 0.5 124732 2076.0 0/2076 2076.0 1081 0.5 singlestreamwritedirect: 125048 2082.8 0/2083 2082.7 1078 0.5 124529 2073.4 0/2073 2073.3 1079 0.5 124360 2071.2 0/2071 2071.2 1092 0.5 124744 2077.4 0/2077 2077.4 1089 0.5 tpcso: varmail: 15544299 258843.0 39822/39822 932.1 155 0.2 16073091 267661.5 41179/41179 962.6 149 0.2 16093688 267982.9 41228/41228 965.9 146 0.2 15245434 253849.2 39054/39054 913.8 159 0.2 videoserver: 23189 386.3 386/0 1028.2 2956 3.5 23191 386.5 386/0 1028.5 2951 4.3 23174 386.0 386/0 1028.1 2978 4.9 23200 386.5 386/0 1028.1 2927 5.8 webproxy: 8901175 148205.3 39001/7800 371.5 464 2.1 9974343 165951.7 43671/8734 415.0 398 1.9 8153671 135755.5 35725/7145 339.5 523 2.3 9315573 155044.8 40801/8160 387.7 438 2.0 webserver: 35627622 593597.5 191483/19149 3148.0 96 0.2 35620238 593426.7 191428/19144 3146.8 96 0.2 35495594 591324.7 190749/19076 3135.4 96 0.2 35861559 597427.3 192718/19273 3167.6 95 0.2 --------------------------------------------------------------------------- compression=on, atime=off, sync=disabled, recordsize=128K --------------------------------------------------------------------------- write 2010.3 2060.9 2087.1 2060.9 MiB/s read 1630.3 1638.4 1638.4 1638.4 MiB/s filebench 1.4.8 (0.175.1.0.0.24.2) as is with 'set $dir=/pool1/test': ops ops/s r/w mb/s cpu/op latency [µs] [ms] bringover: 6002 5995.4 1000/999 30.0 188 0.4 6002 6001.5 1001/1000 30.0 152 0.3 6002 6001.4 1001/1000 30.0 137 0.2 6002 6001.5 1001/1000 30.0 193 0.2 copyfiles: 6002 6001.5 1001/1000 30.0 158 0.2 6002 6001.5 1001/1000 30.0 143 0.2 6002 6001.5 1001/1000 30.0 152 0.2 6002 6001.5 1001/1000 30.0 157 0.2 createfiles: 149975 74945.8 0/24981 390.3 352 0.4 149971 74943.3 0/24979 390.3 453 0.5 149972 74944.0 0/24980 390.3 351 0.4 149972 74943.5 0/24979 390.3 349 0.4 deletefiles: 50000 12487.0 0/0 0.0 0 0.0 50000 49945.3 0/0 0.0 0 0.0 50000 12490.1 0/0 0.0 0 0.0 50000 16649.3 0/0 0.0 0 0.0 fileserver: 5968049 99280.0 9025/18051 2403.9 1143 1.5 4676569 77467.6 7042/14085 1879.2 1480 2.0 5147626 85399.7 7763/15528 2070.9 1339 1.8 7627643 126757.0 11523/23047 3072.7 883 1.1 listdirs: 17418270 290043.0 0/0 60.2 0 0.0 17398929 289692.2 0/0 60.2 0 0.0 17173391 285942.9 0/0 59.4 0 0.0 17418135 290054.7 0/0 60.2 0 0.0 makedirs: 10000 9989.3 0/0 0.0 0 0.0 10000 9999.3 0/0 0.0 0 0.0 10000 9999.4 0/0 0.0 0 0.0 10000 9999.4 0/0 0.0 0 0.0 mongo: 7006 7005.3 1001/1001 23.0 108 0.2 7006 7005.5 1001/1001 23.0 113 0.2 7006 7005.4 1001/1001 23.0 109 0.2 7006 6998.5 1000/1000 23.0 41 0.0 multistreamread: 856780 14267.3 14267/0 14253.3 279 0.3 855863 14252.7 14253/0 14238.7 279 0.3 856108 14253.9 14254/0 14239.9 279 0.3 855322 14252.1 14252/0 14238.2 279 0.3 multistreamreaddirect: 858711 14297.2 14297/0 14283.2 278 0.3 854709 14230.5 14231/0 14216.6 280 0.3 857158 14271.3 14271/0 14257.4 279 0.3 857111 14270.7 14271/0 14256.7 279 0.3 multistreamwrite: 543669 9055.4 0/9055 9055.3 1320 0.4 548379 9135.6 0/9136 9135.5 1314 0.4 545042 9079.9 0/9080 9079.8 1318 0.4 552014 9180.0 0/9180 9180.0 1308 0.4 multistreamwritedirect: 545134 9076.2 0/9076 9076.1 1320 0.4 549312 9151.3 0/9151 9151.2 1311 0.4 548550 9137.8 0/9138 9137.8 1311 0.4 552356 9202.8 0/9203 9202.7 1306 0.4 networkfs: 36661 610.9 90/90 3.5 397 0.6 36661 611.0 90/90 3.8 378 0.5 36661 610.9 90/90 3.7 383 0.5 36661 610.9 90/90 3.6 390 0.5 oltp: 9850070 164027.5 81609/81577 325.0 48 0.0 9803550 163265.3 81221/81207 323.5 49 0.0 9427177 156976.9 78095/78077 311.0 57 0.0 9618135 160158.5 79669/79668 317.3 50 0.0 openfiles: 25807116 429691.7 0/0 0.0 0 0.0 25309165 421475.6 0/0 0.0 0 0.0 25761700 428990.4 0/0 0.0 0 0.0 25417529 423301.3 0/0 0.0 0 0.0 randomfileaccess: 133003 2214.4 492/246 5584.2 6688 6.8 131246 2185.2 486/243 5397.3 6762 6.8 134362 2237.3 497/249 5560.6 6601 6.7 134442 2238.4 497/249 5576.7 6584 6.7 randomread: 11679678 194636.5 194636/0 1520.6 8 0.0 11355427 189123.8 189124/0 1477.5 9 0.0 11736334 195458.1 195458/0 1527.0 8 0.0 11581382 193002.5 193003/0 1507.8 8 0.0 randomrw: 12539768 208784.1 131509/77275 1631.1 12 0.0 12486791 207901.4 131630/76271 1624.2 12 0.0 12535002 208704.1 131919/76785 1630.5 12 0.0 12596707 209729.7 135198/74532 1638.5 12 0.0 randomwrite: 7590270 126473.2 0/126473 988.1 14 0.0 7481991 124591.3 0/124591 973.4 14 0.0 7633751 127126.0 0/127126 993.2 14 0.0 7548945 125733.1 0/125733 982.3 14 0.0 ratelimcopyfiles: 6002 60.0 10/10 0.3 926 0.3 6002 60.0 10/10 0.3 901 0.3 6002 60.0 10/10 0.3 916 0.4 6002 60.0 10/10 0.3 996 0.4 removedirs: 10000 9999.2 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 singlestreamread: 183864 3061.2 3061/0 3060.6 333 0.3 180949 3012.8 3013/0 3012.2 338 0.3 182149 3032.7 3033/0 3032.1 334 0.3 184489 3071.8 3072/0 3071.2 331 0.3 singlestreamreaddirect: 187756 3126.0 3126/0 3125.4 323 0.3 190050 3167.0 3167/0 3166.4 318 0.3 188033 3130.8 3131/0 3130.1 323 0.3 187150 3115.9 3116/0 3115.3 325 0.3 singlestreamwrite: 128122 2133.4 0/2133 2133.4 1291 0.5 127285 2119.2 0/2119 2119.2 1298 0.5 128323 2136.9 0/2137 2136.8 1280 0.5 128210 2135.3 0/2135 2135.3 1275 0.5 singlestreamwritedirect: 128399 2138.2 0/2138 2138.1 1301 0.5 128425 2138.4 0/2138 2138.4 1281 0.5 127810 2128.3 0/2128 2128.3 1280 0.5 127649 2125.7 0/2126 2125.7 1294 0.5 tpcso: varmail: 13583563 226181.9 34797/34797 812.1 192 0.2 11321076 188521.1 29003/29003 677.2 254 0.3 14217877 236743.7 36422/36422 850.6 185 0.2 14530032 241961.5 37225/37225 868.0 178 0.2 videoserver: 23200 386.5 386/0 1028.3 3313 1.4 23204 386.5 386/0 1028.1 3287 1.4 23200 386.5 386/0 1028.1 3263 1.4 23200 386.4 386/0 1027.9 3201 1.4 webproxy: 10612166 176713.6 46504/9301 441.5 444 1.8 8993602 149710.7 39397/7880 375.0 462 2.1 8250763 137398.9 36157/7232 344.4 474 2.3 8439399 140529.4 36981/7396 351.4 507 2.2 webserver: 35570150 592602.7 191162/19117 3142.5 95 0.2 35435671 590345.9 190434/19044 3130.2 96 0.2 35787388 596142.0 192303/19231 3162.0 95 0.2 35751806 595535.0 192108/19212 3157.8 95 0.2