- CPU: 2 x Intel L5630 (Westmere microarchitecture, so 2x4x2 = 16 hardware threads and lots of caches)
- RAM: 2 x 6 x 8GB = 96GB DDR3 ECC+Reg Dual-Rank DIMMs
- Disks: 12 x Western Digital (WD) RE4 (model: WD2003FYYS – 2TB SATA 7200rpm)
- RAID controllers: Adaptec 51645 and LSI MegaRaid 9280-16i4e
O_DIRECTon 64 files, for a total of 100GB of data. Some observations:
- Formatting XFS with the optimal values for
swidthdoesn't lead to much better performance. The gain is about 2%, except for sequential writes where it actually makes things worse. Yes, there was no partition table, the whole array was formatted directly as one single big filesystem.
- Creating more allocation groups in XFS than physical threads doesn't lead to better performance.
- XFS has much better random write throughput at low concurrency levels, but quickly degrades to the same performance level as ext4 with more than 8 threads.
- ext4 has consistently better random read/write throughput and latency, even at high concurrency levels.
- Similarly, for random reads ext4 also has much better throughput and latency.
- By default XFS creates too few allocation groups, which artificially limits its performance at high concurrency levels. It's important to create as many AGs as hardware threads. ext4, on the other hand, doesn't really need any tuning as it performs well out of the box.