SuperMicro X9DRH-iF (CSE-219A-R920LPB, BPN-SAS-213A) 2x E5-2690 @ 2.90GHz (8 cores with 2 HTs) 128 GB 1600MHz ECC LSI-620J 1x ESM 2x WidePort Uplink 24x Seagate ST9146852SS via LSI SAS 9207-8e (PCIe 3.0 x8, IT P15) 2x WidePort Uplink via LSI SAS 9207-8i (PCIe 3.0 x8, IT P15) Solaris 11.1 SRU 5.5, mxpio=enabled 12x 2-way mirrors ZFS + 1x ZeusRAM log setenv JBOD1 /dev/chassis/JBOD1 + zpool create [-f] -o failmode=continue -O compression=on -O atime=off pool1 \ mirror $JBOD1/SLOT_01_/disk $JBOD1/SLOT_02_/disk \ mirror $JBOD1/SLOT_03_/disk $JBOD1/SLOT_04_/disk \ mirror $JBOD1/SLOT_05_/disk $JBOD1/SLOT_06_/disk \ mirror $JBOD1/SLOT_07_/disk $JBOD1/SLOT_08_/disk \ mirror $JBOD1/SLOT_09_/disk $JBOD1/SLOT_10_/disk \ mirror $JBOD1/SLOT_11_/disk $JBOD1/SLOT_12_/disk \ mirror $JBOD1/SLOT_13_/disk $JBOD1/SLOT_14_/disk \ mirror $JBOD1/SLOT_15_/disk $JBOD1/SLOT_16_/disk \ mirror $JBOD1/SLOT_17_/disk $JBOD1/SLOT_18_/disk \ mirror $JBOD1/SLOT_19_/disk $JBOD1/SLOT_20_/disk \ mirror $JBOD1/SLOT_21_/disk $JBOD1/SLOT_22_/disk \ mirror $JBOD1/SLOT_23_/disk $JBOD1/SLOT_24_/disk \ spare /dev/chassis/SYS/HDD1/disk /dev/chassis/SYS/HDD9/disk \ log /dev/chassis/SYS/HDD19/disk Simple TESTs to check upper bounds: + time sh -c 'mkfile 32g bla ; sync' ; \ + time sh -c 'mkfile 32g blabla ; sync' ; \ + time sh -c 'mkfile 32g blablabla ; sync' ; \ + time sh -c 'mkfile 32g blablablabla ; sync' + chmod 644 b* + time dd if=bla of=/dev/null bs=128k ; \ + time dd if=blabla of=/dev/null bs=128k ; \ + time dd if=blablabla of=/dev/null bs=128k ; \ + time dd if=blablablabla of=/dev/null bs=128k --------------------------------------------------------------------------- compression=on, atime=off, recordsize=128K --------------------------------------------------------------------------- write 2048.0 2087.1 2087.1 2073.9 MiB/s read 1622.2 1622.2 1630.3 1638.4 MiB/s filebench 1.4.8 (0.175.1.0.0.24.2) as is with 'set $dir=/pool1/test': ops ops/s r/w mb/s cpu/op latency [µs] [ms] bringover: 6002 6001.5 1001/1000 30.0 220 0.1 6002 5995.7 1000/999 30.0 197 0.2 6002 6001.5 1001/1000 30.0 152 0.3 6002 6001.4 1001/1000 30.0 152 0.2 copyfiles: 6002 5995.6 1000/999 30.0 142 0.2 6002 5995.7 1000/999 30.0 142 0.2 6002 5995.6 1000/999 30.0 157 0.2 6002 5995.5 1000/999 30.0 166 0.3 createfiles: 149978 74984.0 0/24994 390.5 366 0.4 149977 74944.4 0/24980 390.3 349 0.4 149982 74949.1 0/24984 390.3 356 0.4 149974 74944.9 0/24980 390.3 339 0.4 deletefiles: 50000 49996.0 0/0 0.0 0 0.0 50000 16654.8 0/0 0.0 0 0.0 50000 16654.8 0/0 0.0 0 0.0 50000 24986.0 0/0 0.0 0 0.0 fileserver: 6762110 112270.5 10206/20413 2723.0 1005 1.3 5062845 83893.1 7627/15254 2033.2 1370 1.8 6337844 105074.8 9552/19105 2551.1 1078 1.5 4288271 71129.2 6466/12933 1723.7 1615 2.2 listdirs: 15209978 253284.1 0/0 52.6 0 0.0 15674777 260993.8 0/0 54.2 0 0.0 15872463 264280.8 0/0 54.9 0 0.0 15882406 264476.8 0/0 54.9 0 0.0 makedirs: 10000 9989.4 0/0 0.0 0 0.0 10000 9989.4 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9989.5 0/0 0.0 0 0.0 mongo: 7006 6998.4 1000/1000 23.0 108 0.2 7006 7005.5 1001/1001 23.0 93 0.2 7006 6998.2 1000/1000 23.0 124 0.2 7006 7005.5 1001/1001 23.0 108 0.2 multistreamread: 853674 14217.0 14217/0 14203.0 280 0.3 854766 14232.2 14232/0 14218.2 280 0.3 853678 14215.4 14215/0 14201.5 280 0.3 855982 14251.9 14252/0 14238.0 279 0.3 multistreamreaddirect: 851664 14192.4 14192/0 14178.5 281 0.3 854941 14236.0 14236/0 14222.1 279 0.3 855750 14248.7 14249/0 14234.8 279 0.3 853855 14218.1 14218/0 14204.2 280 0.3 multistreamwrite: 542661 9038.6 0/9039 9038.5 1318 0.4 536793 8942.0 0/8942 8941.9 1316 0.4 534888 8910.4 0/8910 8910.4 1319 0.4 541312 9017.6 0/9018 9017.5 1315 0.4 multistreamwritedirect: 543195 9049.2 0/9049 9049.2 1318 0.4 543654 9057.0 0/9057 9057.0 1317 0.4 543251 9049.4 0/9049 9049.4 1316 0.4 544230 9063.4 0/9063 9063.3 1318 0.4 networkfs: 36661 610.9 90/90 3.7 386 0.4 36661 610.7 90/90 4.1 384 0.4 36661 610.9 90/90 3.8 377 0.4 36661 610.9 90/90 3.8 371 0.4 oltp: 8215713 136820.6 68065/68054 271.1 79 0.0 8026267 133654.7 66493/66476 264.8 87 0.0 8267674 137681.9 68497/68479 272.8 79 0.0 8293039 138099.4 68709/68683 273.6 78 0.0 openfiles: 25658873 427230.4 0/0 0.0 0 0.0 26210457 436393.7 0/0 0.0 0 0.0 26036220 433512.9 0/0 0.0 0 0.0 26124863 434988.9 0/0 0.0 0 0.0 randomfileaccess: 133129 2216.7 493/246 5521.9 6686 6.7 137259 2285.6 508/254 5935.0 6494 6.5 134452 2238.6 497/249 5739.6 6618 6.7 128268 2135.8 475/237 5265.6 6924 7.0 randomread: 11555355 192531.7 192532/0 1504.2 8 0.0 11565389 192715.4 192715/0 1505.6 8 0.0 11697229 194915.0 194915/0 1522.8 8 0.0 11820973 196971.7 196972/0 1538.8 8 0.0 randomrw: 12135434 202051.3 128012/74040 1578.5 13 0.0 12476188 207725.0 130535/77190 1622.8 12 0.0 12220968 203475.2 130055/73420 1589.6 12 0.0 12453503 207345.0 131078/76267 1619.9 12 0.0 randomwrite: 7120513 118552.6 0/118553 926.2 15 0.0 7179272 119532.5 0/119533 933.8 15 0.0 7176314 119572.6 0/119573 934.1 15 0.0 7195192 119796.4 0/119796 935.9 15 0.0 ratelimcopyfiles: 6002 60.0 10/10 0.3 948 0.3 6002 60.0 10/10 0.3 988 0.4 6002 60.0 10/10 0.3 977 0.3 6002 60.0 10/10 0.3 937 0.3 removedirs: 10000 9999.2 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9989.1 0/0 0.0 0 0.0 singlestreamread: 180534 3006.7 3007/0 3006.1 339 0.3 181922 3029.0 3029/0 3028.4 336 0.3 181477 3022.5 3022/0 3021.9 337 0.3 184058 3064.5 3065/0 3063.9 332 0.3 singlestreamreaddirect: 187618 3123.8 3124/0 3123.2 324 0.3 188919 3145.4 3145/0 3144.8 320 0.3 187200 3116.8 3117/0 3116.2 324 0.3 186826 3110.6 3111/0 3109.9 326 0.3 singlestreamwrite: 127397 2121.3 0/2121 2121.2 1280 0.5 125825 2095.6 0/2096 2095.6 1295 0.5 126188 2102.0 0/2102 2102.0 1304 0.5 126585 2108.4 0/2108 2108.3 1292 0.5 singlestreamwritedirect: 127495 2123.6 0/2124 2123.6 1288 0.5 127301 2119.6 0/2120 2119.6 1282 0.5 126460 2105.5 0/2106 2105.5 1290 0.5 127506 2123.1 0/2123 2123.1 1284 0.5 tpcso: varmail: 3840591 63963.5 9840/9841 230.3 430 0.8 3931366 65476.3 10073/10073 235.8 405 0.8 3661725 60995.6 9384/9384 219.8 452 0.8 4284066 71365.7 10979/10979 256.6 373 0.7 videoserver: 23240 387.1 387/0 1028.1 3280 1.4 23200 386.5 386/0 1028.1 3290 1.4 23200 386.4 386/0 1028.1 3286 1.4 23200 386.5 386/0 1028.3 3288 1.4 webproxy: 6091629 101440.4 26695/5339 255.7 554 3.1 9346694 155639.0 40957/8192 389.3 419 2.0 9278274 154385.7 40628/8126 386.9 429 2.0 7135630 118811.4 31266/6253 298.1 522 2.6 webserver: 35574008 592672.6 191184/19119 3142.8 96 0.2 35354705 589025.0 190008/19002 3123.8 96 0.2 35650540 593980.6 191606/19162 3149.5 96 0.2 35779603 596088.8 192286/19230 3161.0 95 0.2 --------------------------------------------------------------------------- compression=off, atime=off, recordsize=128K --------------------------------------------------------------------------- write 1382.6 1359.7 1382.6 1382.6 MiB/s read 1326.6 1310.7 1300.3 1300.3 MiB/s filebench 1.4.8 (0.175.1.0.0.24.2) as is with 'set $dir=/pool1/test': ops ops/s r/w mb/s cpu/op latency [µs] [ms] bringover: 6002 5995.5 1000/999 30.0 146 0.4 6002 5995.6 1000/999 30.0 137 0.4 6002 6001.4 1001/1000 30.0 152 0.4 6002 6001.6 1001/1000 30.0 137 0.4 copyfiles: 6002 5995.6 1000/999 30.0 142 0.4 6002 6001.4 1001/1000 30.0 108 0.3 6002 5995.6 1000/999 30.0 157 0.4 6002 6001.5 1001/1000 30.0 142 0.3 createfiles: 149972 74981.5 0/24992 390.5 459 0.5 149976 74946.4 0/24982 390.3 449 0.5 149973 74907.3 0/24967 390.1 350 0.4 149975 74982.0 0/24992 390.5 356 0.4 deletefiles: 50000 16654.8 0/0 0.0 0 0.0 50000 12496.2 0/0 0.0 0 0.0 50000 24985.8 0/0 0.0 0 0.0 50000 16654.8 0/0 0.0 0 0.0 fileserver: 5420907 90090.6 8190/16380 2182.6 1262 1.7 4705643 78097.8 7100/14200 1892.0 1464 2.0 6023516 99880.4 9080/18160 2421.6 1133 1.5 5362709 89092.2 8099/16199 2158.6 1275 1.7 listdirs: 16002234 266429.6 0/0 55.3 0 0.0 16038906 267040.1 0/0 55.5 0 0.0 16086428 267840.4 0/0 55.6 0 0.0 16076167 267682.1 0/0 55.6 0 0.0 makedirs: 10000 9989.4 0/0 0.0 0 0.0 10000 9999.1 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9989.3 0/0 0.0 0 0.0 mongo: 7006 6998.6 1000/1000 23.0 120 0.4 7006 7005.1 1001/1001 23.0 90 0.0 7006 6998.4 1000/1000 23.0 48 0.0 7006 7005.6 1001/1001 23.0 43 0.0 multistreamread: 762327 12693.8 12694/0 12681.4 313 0.3 751238 12518.2 12518/0 12506.0 311 0.3 766752 12767.7 12768/0 12755.2 311 0.3 764863 12734.8 12735/0 12722.3 313 0.3 multistreamreaddirect: 758944 12638.5 12639/0 12626.1 312 0.3 764683 12736.0 12736/0 12723.5 311 0.3 764389 12726.9 12727/0 12714.4 312 0.3 762923 12711.9 12712/0 12699.5 312 0.3 multistreamwrite: 118506 1974.1 0/1974 1974.0 1244 2.0 116985 1948.3 0/1948 1948.2 1248 2.0 117760 1960.8 0/1961 1960.7 1238 2.0 117944 1964.1 0/1964 1964.0 1246 2.0 multistreamwritedirect: 121868 2029.9 0/2030 2029.8 1240 2.0 120676 2009.9 0/2010 2009.9 1241 2.0 117147 1951.1 0/1951 1951.0 1253 2.0 118087 1966.8 0/1967 1966.7 1283 2.0 networkfs: 36661 610.7 90/90 3.9 480 1.3 36661 610.7 90/90 3.5 480 1.2 36661 610.8 90/90 3.7 459 1.4 36661 610.9 90/90 3.7 477 1.3 oltp: 7875572 131141.6 65244/65225 259.8 91 0.0 8054182 134122.4 66728/66706 265.7 86 0.0 8240813 137230.3 68272/68255 271.9 80 0.0 7886497 131338.9 65336/65329 260.2 91 0.0 openfiles: 26088158 434371.2 0/0 0.0 0 0.0 26739851 445236.2 0/0 0.0 0 0.0 26124706 435000.1 0/0 0.0 0 0.0 26153297 435490.5 0/0 0.0 0 0.0 randomfileaccess: 134848 2245.6 499/250 5736.1 6477 6.7 135153 2250.3 500/250 5755.6 6563 6.6 134972 2247.6 499/250 5821.5 6603 6.7 132157 2200.4 489/245 5524.5 6686 6.8 randomread: 11493141 191401.5 191401/0 1495.3 8 0.0 11216679 186789.2 186789/0 1459.3 9 0.0 11597027 193260.4 193260/0 1509.8 8 0.0 11648258 194105.6 194106/0 1516.4 8 0.0 randomrw: 12705877 211552.0 139234/72318 1652.7 12 0.0 12170121 202627.3 129943/72684 1583.0 12 0.0 12098230 201430.0 127809/73621 1573.7 13 0.0 12441901 207288.7 135219/72070 1619.4 12 0.0 randomwrite: 7228892 120357.2 0/120357 940.3 15 0.0 7081324 117900.2 0/117900 921.1 15 0.0 7188097 119784.7 0/119785 935.8 15 0.0 7211965 120077.1 0/120077 938.1 15 0.0 ratelimcopyfiles: 6002 60.0 10/10 0.3 1171 1.4 6002 60.0 10/10 0.3 1185 1.6 6002 60.0 10/10 0.3 1155 1.9 6002 60.0 10/10 0.3 1145 2.0 removedirs: 10000 9989.0 0/0 0.0 0 0.0 10000 9999.1 0/0 0.0 0 0.0 10000 9989.2 0/0 0.0 0 0.0 10000 9989.1 0/0 0.0 0 0.0 singlestreamread: 167995 2797.2 2797/0 2796.7 346 0.4 24993 416.3 416/0 416.2 894 2.4 167948 2798.4 2798/0 2797.8 345 0.4 174676 2908.3 2908/0 2907.7 347 0.3 singlestreamreaddirect: 174956 2913.1 2913/0 2912.5 351 0.3 17211 286.8 287/0 286.7 1362 3.5 173367 2886.6 2887/0 2886.0 352 0.3 175757 2926.4 2926/0 2925.8 343 0.3 singlestreamwrite: 113817 1895.3 0/1895 1895.3 1148 0.5 55245 920.0 0/920 920.0 1331 1.1 114833 1912.5 0/1912 1912.5 1171 0.5 114130 1901.0 0/1901 1901.0 1178 0.5 singlestreamwritedirect: 117838 1962.9 0/1963 1962.9 1109 0.5 54704 911.4 0/911 911.4 1339 1.1 114464 1905.8 0/1906 1905.8 1180 0.5 113825 1895.6 0/1896 1895.6 1184 0.5 tpcso: varmail: 4372760 72830.1 11205/11205 262.8 348 0.7 4514944 75195.6 11568/11569 271.3 326 0.7 4404691 73353.3 11285/11285 264.6 346 0.7 4837397 80572.7 12396/12396 289.9 307 0.6 videoserver: 23195 386.3 386/0 1027.8 3251 9.3 23179 386.2 386/0 752.1 2929 6.2 23200 386.4 386/0 1028.1 3296 10.1 23198 386.4 386/0 752.0 3148 6.4 webproxy: 7271498 121047.3 31854/6371 304.2 471 2.6 9562763 159156.0 41883/8377 397.7 419 2.0 9140711 152219.6 40058/8012 380.6 435 2.1 9120231 151845.1 39959/7992 379.9 420 2.1 webserver: 35345640 588835.4 189947/18996 3122.2 96 0.2 35850821 597324.8 192685/19270 3166.9 95 0.2 35383847 589507.3 190163/19017 3125.3 97 0.2 35704890 594762.0 191858/19187 3153.8 95 0.2 --------------------------------------------------------------------------- compression=off, atime=off, sync=disabled, recordsize=128K --------------------------------------------------------------------------- write 1382.6 1412.4 1382.6 1388.5 MiB/s read 1290.1 1359.7 1359.7 1371.1 MiB/s filebench 1.4.8 (0.175.1.0.0.24.2) as is with 'set $dir=/pool1/test': ops ops/s r/w mb/s cpu/op latency [µs] [ms] bringover: 6002 6001.6 1001/1000 30.0 127 0.4 6002 5995.6 1000/999 30.0 79 0.1 6002 6001.6 1001/1000 30.0 98 0.1 6002 6001.5 1001/1000 30.0 152 0.4 copyfiles: 6002 6001.4 1001/1000 30.0 157 0.4 6002 6001.4 1001/1000 30.0 69 0.1 6002 5995.7 1000/999 30.0 75 0.1 6002 5995.7 1000/999 30.0 127 0.4 createfiles: 149974 74944.8 0/24980 390.3 345 0.4 149971 74944.0 0/24979 390.3 423 0.5 149974 74945.3 0/24981 390.3 360 0.4 149973 74945.1 0/24980 390.3 426 0.5 deletefiles: 50000 12487.0 0/0 0.0 0 0.0 50000 12487.0 0/0 0.0 0 0.0 50000 24986.0 0/0 0.0 0 0.0 50000 16654.7 0/0 0.0 0 0.0 fileserver: 5597162 92960.2 8451/16902 2252.4 1225 1.7 6188772 102903.3 9355/18710 2496.2 1102 1.5 6151381 102156.0 9287/18574 2474.0 1100 1.5 5130162 85157.0 7742/15483 2062.5 1340 1.8 listdirs: 15930011 265249.1 0/0 55.1 0 0.0 16034177 266995.7 0/0 55.5 0 0.0 16060465 267469.3 0/0 55.5 0 0.0 16089853 267966.6 0/0 55.7 0 0.0 makedirs: 10000 9989.2 0/0 0.0 0 0.0 10000 9999.1 0/0 0.0 0 0.0 10000 9999.4 0/0 0.0 0 0.0 10000 9999.3 0/0 0.0 0 0.0 mongo: 7006 7005.6 1001/1001 23.0 53 0.0 7006 6998.5 1000/1000 23.0 102 0.4 7006 7005.2 1001/1001 23.0 120 0.0 7006 7005.5 1001/1001 23.0 37 0.0 multistreamread: 767546 12784.1 12784/0 12771.6 312 0.3 765264 12744.2 12744/0 12731.7 312 0.3 765361 12745.4 12745/0 12732.9 312 0.3 766371 12771.7 12772/0 12759.2 312 0.3 multistreamreaddirect: 767182 12773.3 12773/0 12760.8 312 0.3 766231 12760.5 12761/0 12748.0 312 0.3 766962 12773.9 12774/0 12761.4 311 0.3 765862 12758.4 12758/0 12745.9 312 0.3 multistreamwrite: 120184 2001.4 0/2001 2001.3 1241 2.0 116700 1943.7 0/1944 1943.6 1232 2.1 115852 1929.7 0/1930 1929.6 1278 2.1 118147 1967.6 0/1968 1967.5 1251 2.0 multistreamwritedirect: 121190 2018.1 0/2018 2018.0 1235 2.0 121002 2015.3 0/2015 2015.2 1242 2.0 118989 1981.9 0/1982 1981.8 1243 2.0 119987 1998.3 0/1998 1998.3 1237 2.0 networkfs: 36661 610.8 90/90 3.6 456 1.3 36661 610.7 90/90 3.6 417 1.0 36661 610.8 90/90 4.0 448 1.3 36661 610.9 90/90 3.7 450 1.2 oltp: 9816935 163469.7 81314/81318 323.9 48 0.0 9341246 155558.7 77388/77374 308.2 58 0.0 9803107 163233.3 81200/81197 323.4 48 0.0 9815407 163441.1 81305/81298 323.8 49 0.0 openfiles: 26320469 438317.2 0/0 0.0 0 0.0 25660398 427330.9 0/0 0.0 0 0.0 26157560 435526.7 0/0 0.0 0 0.0 25159611 418971.2 0/0 0.0 0 0.0 randomfileaccess: 135415 2255.0 501/251 5817.6 6527 6.6 137036 2281.8 507/254 5838.1 6475 6.6 134712 2243.0 498/249 5619.7 6587 6.7 134075 2232.8 496/248 5584.9 6595 6.7 randomread: 11591464 193104.7 193105/0 1508.6 8 0.0 11531347 192100.2 192100/0 1500.8 8 0.0 11361847 189230.6 189231/0 1478.4 9 0.0 11401015 189905.0 189905/0 1483.6 9 0.0 randomrw: 12633696 210421.4 129222/81200 1643.9 12 0.0 12621476 210143.6 132543/77601 1641.7 12 0.0 12459209 207440.3 129692/77748 1620.6 12 0.0 12233866 203689.6 128327/75363 1591.3 12 0.0 randomwrite: 7699257 128235.7 0/128236 1001.8 14 0.0 7658040 127570.2 0/127570 996.6 14 0.0 7540052 125598.2 0/125598 981.2 14 0.0 7461902 124277.8 0/124278 970.9 14 0.0 ratelimcopyfiles: 6002 60.0 10/10 0.3 1161 1.6 6002 60.0 10/10 0.3 1230 1.8 6002 60.0 10/10 0.3 1180 1.8 6002 60.0 10/10 0.3 1165 1.7 removedirs: 10000 9999.0 0/0 0.0 0 0.0 10000 9999.1 0/0 0.0 0 0.0 10000 9999.3 0/0 0.0 0 0.0 10000 9999.0 0/0 0.0 0 0.0 singlestreamread: 176126 2932.5 2932/0 2931.9 345 0.3 172654 2876.8 2877/0 2876.2 350 0.3 174782 2910.1 2910/0 2909.5 349 0.3 174635 2907.6 2908/0 2907.0 350 0.3 singlestreamreaddirect: 174065 2898.1 2898/0 2897.5 348 0.3 174675 2908.4 2908/0 2907.8 348 0.3 173460 2888.0 2888/0 2887.5 352 0.3 172815 2877.3 2877/0 2876.7 354 0.3 singlestreamwrite: 115230 1919.1 0/1919 1919.1 1147 0.5 115039 1916.2 0/1916 1916.2 1136 0.5 114682 1910.0 0/1910 1910.0 1166 0.5 115362 1920.9 0/1921 1920.8 1160 0.5 singlestreamwritedirect: 114652 1909.9 0/1910 1909.8 1195 0.5 114267 1903.2 0/1903 1903.2 1167 0.5 114022 1898.6 0/1899 1898.6 1185 0.5 114491 1906.2 0/1906 1906.2 1180 0.5 tpcso: varmail: 15741542 262105.9 40324/40324 945.9 158 0.2 15139590 252095.2 38784/38784 910.2 167 0.2 15648562 260587.6 40090/40090 938.5 170 0.2 15563510 259197.2 39876/39877 934.0 154 0.2 videoserver: 23198 386.4 386/0 752.0 2615 10.6 23122 385.1 385/0 1027.6 3290 13.1 23182 386.1 386/0 1027.9 3298 11.3 23184 386.2 386/0 1028.0 3251 12.0 webproxy: 9470713 157563.1 41464/8293 393.3 429 2.0 8839223 147193.1 38735/7747 368.0 474 2.1 8855418 147459.5 38805/7761 368.9 441 2.1 9360571 155831.8 41008/8202 389.6 432 2.0 webserver: 35182046 586114.5 189069/18908 3107.7 97 0.2 34932702 581889.3 187706/18772 3085.9 119 0.3 35721927 595153.6 191985/19200 3155.6 96 0.2 35481918 591091.7 190674/19068 3134.6 96 0.2 --------------------------------------------------------------------------- compression=on, atime=off, sync=disabled, recordsize=128K --------------------------------------------------------------------------- write 2155.8 2141.7 2087.1 2073.9 MiB/s read 1655.0 1630.3 1646.6 1655.0 MiB/s filebench 1.4.8 (0.175.1.0.0.24.2) as is with 'set $dir=/pool1/test': ops ops/s r/w mb/s cpu/op latency [µs] [ms] bringover: 6002 6001.5 1001/1000 30.0 152 0.2 6002 6001.3 1001/1000 30.0 162 0.3 6002 5995.4 1000/999 30.0 110 0.1 6002 5995.6 1000/999 30.0 70 0.1 copyfiles: 6002 6001.5 1001/1000 30.0 152 0.3 6002 6001.5 1001/1000 30.0 156 0.2 6002 6001.3 1001/1000 30.0 99 0.1 6002 6001.5 1001/1000 30.0 123 0.2 createfiles: 149970 74905.9 0/24966 390.1 340 0.4 149973 74944.6 0/24980 390.3 443 0.5 149970 74942.6 0/24978 390.3 347 0.4 149974 74907.0 0/24968 390.1 350 0.4 deletefiles: 50000 16654.8 0/0 0.0 0 0.0 50000 16649.3 0/0 0.0 0 0.0 50000 24973.5 0/0 0.0 0 0.0 50000 12487.1 0/0 0.0 0 0.0 fileserver: 4888517 81103.6 7373/14746 1966.6 1414 1.9 5033650 83654.1 7605/15210 2027.6 1370 1.8 6370263 105852.8 9623/19246 2565.1 1073 1.4 6149733 102170.2 9288/18577 2477.0 1104 1.5 listdirs: 15475442 257687.9 0/0 53.5 0 0.0 15675470 261039.5 0/0 54.2 0 0.0 15731546 261948.3 0/0 54.4 0 0.0 15536656 258745.4 0/0 53.7 0 0.0 makedirs: 10000 9999.4 0/0 0.0 0 0.0 10000 9998.8 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9999.4 0/0 0.0 0 0.0 mongo: 7006 7005.5 1001/1001 23.0 38 0.0 7006 6998.2 1000/1000 23.0 81 0.1 7006 7005.4 1001/1001 23.0 114 0.2 7006 7005.4 1001/1001 23.0 120 0.2 multistreamread: 856663 14264.4 14264/0 14250.4 279 0.3 855366 14251.8 14252/0 14237.8 279 0.3 856222 14260.1 14260/0 14246.2 279 0.3 856607 14262.3 14262/0 14248.4 279 0.3 multistreamreaddirect: 857297 14273.5 14274/0 14259.6 279 0.3 857795 14294.6 14295/0 14280.6 278 0.3 853532 14210.9 14211/0 14197.0 280 0.3 854953 14234.6 14235/0 14220.6 280 0.3 multistreamwrite: 546629 9103.9 0/9104 9103.8 1318 0.4 544367 9065.9 0/9066 9065.9 1315 0.4 543944 9059.8 0/9060 9059.7 1325 0.4 541815 9023.8 0/9024 9023.7 1317 0.4 multistreamwritedirect: 546858 9108.2 0/9108 9108.1 1319 0.4 545653 9089.6 0/9090 9089.5 1320 0.4 544128 9062.0 0/9062 9061.9 1321 0.4 547033 9111.4 0/9111 9111.3 1318 0.4 networkfs: 36661 610.9 90/90 3.6 393 0.5 36661 610.7 90/90 3.8 399 0.5 36661 610.8 90/90 3.7 406 0.5 36661 610.8 90/90 3.7 407 0.5 oltp: 9692686 161393.1 80286/80280 319.7 51 0.0 9598393 159838.6 79513/79506 316.7 50 0.0 9818651 163507.3 81335/81334 323.9 49 0.0 9695976 161460.9 80317/80316 319.9 51 0.0 openfiles: 25894071 431186.9 0/0 0.0 0 0.0 25442880 423620.6 0/0 0.0 0 0.0 25718619 428258.4 0/0 0.0 0 0.0 24944272 415318.9 0/0 0.0 0 0.0 randomfileaccess: 136691 2275.9 506/253 5866.3 6529 6.6 132298 2202.9 489/245 5502.2 6732 6.8 135593 2257.8 502/251 5794.0 6574 6.6 133922 2229.7 495/248 5706.1 6660 6.7 randomread: 11779789 196188.4 196188/0 1532.7 8 0.0 11415329 190186.8 190187/0 1485.8 9 0.0 11646806 193972.0 193972/0 1515.4 8 0.0 11498933 191605.1 191605/0 1496.9 8 0.0 randomrw: 12751297 212305.4 131365/80940 1658.6 12 0.0 12583093 209503.0 133652/75851 1636.7 12 0.0 12458281 207426.9 130862/76565 1620.5 12 0.0 12540001 208800.7 132888/75913 1631.2 12 0.0 randomwrite: 7691663 128175.4 0/128175 1001.4 14 0.0 7577073 126185.9 0/126186 985.8 14 0.0 7554265 125819.1 0/125819 983.0 14 0.0 7666179 127657.6 0/127658 997.3 14 0.0 ratelimcopyfiles: 6002 60.0 10/10 0.3 1022 0.2 6002 60.0 10/10 0.3 1012 0.3 6002 60.0 10/10 0.3 1077 0.3 6002 60.0 10/10 0.3 992 0.3 removedirs: 10000 9999.1 0/0 0.0 0 0.0 10000 9999.2 0/0 0.0 0 0.0 10000 9999.0 0/0 0.0 0 0.0 10000 9999.0 0/0 0.0 0 0.0 singlestreamread: 183143 3050.9 3051/0 3050.3 332 0.3 181245 3017.6 3018/0 3017.0 336 0.3 183876 3061.4 3061/0 3060.8 332 0.3 181870 3029.7 3030/0 3029.1 336 0.3 singlestreamreaddirect: 191122 3182.2 3182/0 3181.5 316 0.3 190581 3173.1 3173/0 3172.5 317 0.3 188638 3140.7 3141/0 3140.1 320 0.3 191972 3196.3 3196/0 3195.6 314 0.3 singlestreamwrite: 127401 2121.8 0/2122 2121.8 1290 0.5 127055 2115.7 0/2116 2115.7 1307 0.5 128684 2143.4 0/2143 2143.4 1288 0.5 127377 2121.6 0/2122 2121.5 1291 0.5 singlestreamwritedirect: 128412 2138.3 0/2138 2138.3 1282 0.5 128737 2143.7 0/2144 2143.7 1287 0.5 127050 2115.6 0/2116 2115.6 1296 0.5 128753 2143.8 0/2144 2143.8 1277 0.5 tpcso: varmail: 14743304 245512.9 37771/37771 882.2 178 0.2 15382742 256156.8 39409/39409 921.0 159 0.2 13888187 231246.8 35576/35576 830.3 181 0.2 13925430 231912.6 35679/35679 832.7 179 0.2 videoserver: 23200 386.5 386/0 1028.2 3232 1.4 23200 386.4 386/0 1028.1 3338 1.4 23200 386.4 386/0 1028.1 3238 1.4 23200 386.4 386/0 1028.0 3318 1.4 webproxy: 8353253 139084.4 36601/7320 348.4 505 2.3 9090848 151324.1 39822/7964 377.9 449 2.1 9495518 158129.2 41613/8323 394.5 422 2.0 7333118 122123.6 32138/6428 306.6 594 2.6 webserver: 35574696 592628.6 191170/19118 3142.5 96 0.2 35335471 588705.0 189905/18992 3121.4 97 0.2 35649034 593928.9 191590/19160 3150.1 96 0.2 36014675 600025.0 193556/19357 3182.1 94 0.2