unRAID is a slackware based system, designed boot from a USB drive and run in memory. More than just an operating system, though, unRAID includes a unique RAID system that offers all the benefits of RAID5/6 but with more flexibility. For example, with unRAID you can have a collection of drives of different sizes. Also, since each drive operates independently, they can spin down when not in use -- saving power and reducing wear and tear on the drive. This makes it ideal for a home environment.
The downsides? unRAID is proprietary software. And while it is free (at least, in terms of price) for a home NAS with 3 drives, a license for 4-7 drives will cost you USD$69. IMHO, the price is more than reasonable, but it is still an additional cost to consider.
From my perspective, a more frustrating downside is that unRAID is built upon slackware. While slackware is a very mature, and generally very stable Linux distribution, it has a painful, somewhat esoteric approach to package management. And being one of the less popular distributions, you'll find a lot of less documentation and support online than you'll find for say, Ubuntu or Fedora. Plus, since unRAID runs in memory, the standard approach to installing additional software packages won't work. You'll need to perform a little bit of jiggery pokery to get things working.
Fortunately, unRAID has a really strong community with lots of forum uses to help answer your questions. So definitely check it out.
2. NAS4FREE / FreeNAS
For years, FreeNAS was arguably the most popular choice for home NAS users. A few years ago, a series of events led to the fracture of FreeNAS into two distinct projects. iXsystems sponsored the development of a new, forked version of FreeNAS while a number of developers continued development of the existing code base under the name NAS4FREE.
I've only used FreeNAS briefly (mostly to test it) and I've never used NAS4FREE, so I can't offer much in the way of comments on either. They are both based on FreeBSD, which probably means they are pretty stable and very secure. But it also means that my Linux experience doesn't really apply. Anyway... check them out.
3. Open Media Vault
During the schism in the FreeNAS community, one of the lead developers decided to go in a different direction and establish a completely new, Debian-based NAS operating system. Open Media Vault (OMV) includes many of the features and benefits of the original FreeNAS, but runs over the top of a 100% compatible Debian base.
The beauty of this arrangement is that you can do all your basic day-to-day server administration using the simple, clean GUI. But if you need to get under the hood and tinker, you'll been working with a pure Debian system... meaning you can use Debian packages, and rely on Debian documentation and support from the Debian community. The OMV community is small, but growing slowly and generally fairly responsive.
The downsides? OMV installs to a hard drive and takes up that hard drive completely. The OS itself isn't very big, so it's a bit frustrating seeing a perfectly good hard drive at only 5% capacity. And that's a pain if your NAS doesn't haven't a lot of spare HDD capacity (ie, slots). Theoretically, you can install OMV to a USB thumbdrive. But it hasn't been optimised for this type of install, so the constant writing (eg, log files) will quickly wear out the drive... so it's generally not recommended.
The other downside is that OMV only supports standard RAID configuration. Which means it lacks some of the flexibility of unRAID. Of course, you could combine OMV with FlexRAID or SnapRAID and get the best of both worlds (more on that in a later post).
Overall, though, OMV is an excellent choice and as I said, it has been my operating system of choice for more than a year now. Definitely check it out.
So there are some pretty good options already out there. Check them out and decide whether any of them suit your needs. After playing around with them, decide if you're still game to build your NAS from scratch, or whether the out-of-the-box option is good enough.