Hooray! I've finally updated my home file server. This is the first non-drive change made to the system since I built it back in 2010.
The original build allowed up to 20x SATA drives sitting on top of LSI 2068E cards and a motherboard with three 8x PCIe slots on an Intel X58 chipset. This design was to allow for enough bandwidth back to the CPU for full controller-speed (SATA II) for all 20 drives in the chassis.
The system has served me well to date, but as I expand the array I'd like to keep on top of disk density - and if the opportunity arises, I wouldn't mind something a bit quieter.
Also beyond a crazy growth rate of ~300+GiB/mo for the first year or so, my growth rate has calmed massively and I'm only sitting at around 11TiB of data all up spread over 8x 2TB disks. So maybe supporting up to 20 drives isn't necessary after all.
I had a bit long rant preparated about all the goals and issues, but honestly it boils down to:
I ended up settling for a design with a chassis supporting 12x 3.5" + 2x 2.5", with passive backplanes and different card on the LSI SAS2008 chipset. Whilst the dollar-per-bay isn't as great on the new case, it's a much higher build quality and has better airflow/cabling.
And it runs great - I've rebuilt the array onto 6x 4T drives still using linux software RAID but now using btrfs as the filesystem (I'll redo this as raw disk btrfs raid when the code for that is a bit more mature).
Now that the data migration and deduplication run have completed, I can cross this off my TODO list - and decide what to do with the spare gear I have left over, enough to build 2 file servers.