TL;DR

  • A B550 motherboard paired with a Ryzen 3 Pro 4350G is about as cost-effective as it gets for an all-new build with ECC memory. The CPU’s 7 nm process also brings low idle power consumption and decent single-core performance.
  • TrueNAS Core provides native ZFS, with snapshots and other lovely features, while avoiding the many pitfalls of Btrfs.
  • Striped mirrored vdevs balance reliability—including rebuild speed—usable capacity, performance, and expandability.
  • An X710-DA2 also let me put together a simple 10 GbE network along the way.

Requirements

My existing NAS was a rather underpowered J3455 build from late 2016. About six months later I moved it into an InWin MS04 case, but otherwise it has been serving me for four years. Both its configuration and the way I use it have been fairly improvised:

  • It has always run Linux directly, with basic file sharing set up manually using Samba and similar tools.
  • Two drives form a hardware RAID 1 array for important data; the other two are running without any protection.
  • I/O performance is weak, so I usually SSH in to work with larger files, mostly using vifm. Efficient enough, but it does make me feel a little uncivilized…

Over those four years, I considered upgrading many times, only to put it off for one reason or another. This time I finally made up my mind. My requirements are actually quite simple:

  • Other services, such as Plex Server and Bitwarden, already run on a cluster of NUCs. I only need a storage server. Storage should be decoupled from everything else: no all-in-one box.
  • 10 GbE support. The NAS bottleneck has dampened my enthusiasm for 10 GbE for years; this should finally fix it. New hardware over the past few years has also made a simple 10 GbE setup considerably cheaper and less of a hassle.
  • Use new components wherever possible, rather than second-hand enterprise castoffs. This is storage, after all: stability comes first.
    • For the same reason, ECC is a must.

Choosing an OS

I decided on FreeNAS/TrueNAS a long time ago and did not spend much time investigating OMV or other systems. ZFS has a lot of wonderful features:

  • Copy-on-write
  • Snapshots
  • RAID-Z

This remark about ZFS from Jeff Bonwick also left quite an impression on me:

  • fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans.

As for TrueNAS Core (BSD) versus TrueNAS Scale (Linux):

  • Scale is not quite stable enough yet.
  • I do not need the extensibility that Scale offers.
  • Bill Joy wrote BSD during his Berkeley years, and ZFS came out of his company, Sun. Now that is a proper lineage! Running it on Linux would obviously be heresy!

Hardware

First, the final parts list:

component model
Case Silverstone CS381
Motherboard ASUS TUF GAMINGB550M-PLUS
CPU AMD Ryzen Pro 4350G
RAM Micron 32G DDR4 ECC * 2
HBA LSI 9300-8i
Network Intel X710-DA2
Power Supply Silverstone ST45SF-G
USB Stick (OS) SanDisk CZ430 32GB * 3
SSD (SLOG) Intel Optane M1 16G * 2
SSD (L2ARC) SAMSUNG 980 EVO 1T
CPU Cooler NOCTUA NH-L9a-AM4
Fan NOCTUA NF-A12x25 FLX *2
Cables mini-SAS SFF-8643 0.5m * 2

CS381

I originally wanted another Silverstone case, the rackmount RM21-308: 2U and only 48 cm deep. But for now I cannot completely separate the NAS from my living space, and rackmount fan noise could be difficult to deal with, so I played it safe. I am not entirely happy with the CS381 either. First, it is much larger than I expected—almost like two MS04s side by side. Second, its stock case fans are fairly noisy, which forced me to buy two NF-A12s as replacements.

B550 + 4350G: Cost-Effective ECC

An ordinary B550 motherboard and a 4350G make a very cost-effective combination with ECC support. I initially considered a non-APU such as the 3100 or 3300X, but that would ideally call for a motherboard with IPMI/BMC, such as the X470D4U or X570D4U, to avoid potential problems booting without a GPU. I do not particularly need IPMI on this machine, so the extra cost was hard to justify. After some deliberation, I fell back to this configuration. One thing worth noting: among APUs, only the Ryzen Pro series supports ECC—for example, the 4350G and 4750G in the 4000 series—so the choices are relatively limited.

USB Drives for the OS

FreeNAS strongly favors giving each component a single job, so the boot drive cannot also be used for storage or cache. Mirroring the OS across two or three physically small USB drives is therefore a fairly economical choice. The CZ430 barely protrudes when plugged into a rear USB port and fits in nicely. Even 16 GB would certainly have been enough, but the 32 GB version cost only one yuan more, so that is what I bought…

SLOG/ZIL

As explained in ServeTheHome’s What is the ZFS ZIL SLOG and what makes a good one, SLOG/ZIL in ZFS addresses the performance of synchronous writes. With synchronous writes, the system waits for an acknowledgment that a write has completed before proceeding, protecting against data loss in the event of a power failure or system or network crash. The ZIL logs the intended writes before they happen. Once the ZIL write is complete, the write is considered committed to persistent storage, so it can logically be thought of as a write cache. Given this behavior, a device with extremely fast writes, such as Optane, is naturally the best choice. Exploring the Best ZFS ZIL SLOG SSD with Intel Optane and NAND shows Optane’s significantly better latency as a SLOG device. The 16 GB and 32 GB M10/M15 models are now very cheap, making a mirrored pair a good fit for SLOG.

L2ARC

As Brendan Gregg explains on his blog, L2ARC is the second level of the ARC. Unlike the ZIL, it is a true cache serving a different purpose. If a workload repeatedly reads hot data and the ARC in DRAM is too small, adding L2ARC can help. With 64 GB of RAM and no large hot-data workload, I do not really need it, but I had a spare 980 Evo, so I installed it anyway. Another point worth noting is that L2ARC behaves much like ARC: even though its medium is persistent storage, it is cleared on reboot. Keeping a long uptime therefore matters here.