The right raid array for NAS

[qasdfdsaq]

Hmm, maybe I just have really poor standards for "quiet". All my machines use at least 150w while idle so anything with less than six fans or 100w power consumption under load is automatically "quiet" in comparison to what I have :P

[Qtx]

Some are more sensitive than sound than others but looking at your setup, you are accustomed to much more sound anyway

It's funny how an office can sound pretty quiet until there is a power cut and then you realise how much noise the computer fans and air conditioning make all day every day!

[qasdfdsaq]

Heh. While rack servers do tend to be pretty noisy out of the box in that particular one I've built some custom fan control relays to keep the noise down, it's a lot quieter than you'd expect. Nothing like the server rooms they are normally used in, where providing hearing protection for employees is a mandatory health and safety requirement...

That said my desktop has one of the noisiest graphics cards on the market today and boy does that thing make a racket... I'm sure my portable vacuum is quieter (and probably has less airflow too!)

[General Maximus]

what do you use your array for and what is the cumulative capacity. I am genuinely curious because when I get my new NAS I'll have about 31TB of storage altogether which I thought was a bit OTT.

[Ignitionnet]

My N54 is very quiet. It is no louder than a PVR, no-one notices it. I don't work in a noisy environment so would notice a loud bit of kit and do when I fire Big Daddy Dell up.

No idea what would require that amount of storage. I use less than 3TB across all devices including 1TB devoted to lab/VMs. I guess video content, that's the usual suspect. Odd as it may sound given it is me nearly all of my video content is on its original Blu Rays or is streamed. I am not a big TV watcher.

[Qtx]

Interesting to hear different opinions on the fan noise of the N54. Need to listen to one myself maybe.

Some people rip their blu-rays to a NAS so they can do away with using the disks and easily browse their media from any location on their network, or even when out and about on mobile devices. Others download similar material but the premise is the same, a central media library. If you do a full bluray you are looking around 56GB per film although you can remux quality down slightly and keep only DTS-MA 5.1 for around 25GB per film. If you look at downloads they are around 7GB per film for a 720 with DTS sound at 1.5Kbps or around 11Gb for it at 1080. So times that by x amount of movies in your library.

Streaming movies from a service is good enough quality for most people and would save the need for the storage but some prefer a higher bitrate for the video on their large tv and better sound quality for their surround sound setup.

I tend to download TV shows even though I can watch them live or through a catch up service, simply because I can automate it completely with the NAS and use the same easy to use interface (xbmc) for watching everything, rather than flicking between inputs to different devices for different things and multiple inferior front ends. They get deleted after being watched but sometimes it takes a while to catch up on them, so the GB of them do stack up, especially with some of them being 1080p.

Add on top music, then all the windows and linux iso's, VM backups, home video backups, lossless music samples for creating music etc and space can easily get taken.

[qasdfdsaq]

Originally Posted by General Maximus

what do you use your array for and what is the cumulative capacity. I am genuinely curious because when I get my new NAS I'll have about 31TB of storage altogether which I thought was a bit OTT.

I use it for... well... Storage. The main array is a 76TB array in triple-parity mode giving 64TB usable. Backup storage is a 25TB array.

Aside from the usual suspects (movies, porn, other downloaded warez) it also stores backups of most of my machines, and my own photo/video collection. And several dozen virtual machines I can't be bothered going through before deleting.

I also used to do a lot of freelance data recovery work and imaging four or five 500GB-1TB drives at a time, plus extra space to store the data recovered off them...

---------- Post added at 12:14 ---------- Previous post was at 12:09 ----------

Originally Posted by Ignitionnet

My N54 is very quiet. It is no louder than a PVR, no-one notices it. I don't work in a noisy environment so would notice a loud bit of kit and do when I fire Big Daddy Dell up.

No idea what would require that amount of storage. I use less than 3TB across all devices including 1TB devoted to lab/VMs. I guess video content, that's the usual suspect. Odd as it may sound given it is me nearly all of my video content is on its original Blu Rays or is streamed. I am not a big TV watcher.

The storage I "need" these days is probably closer to about 10TB. 2-3TBs are my own photo/video collection, as a photographer I'm pretty lazy and shoot everything in maximum quality RAW+JPEG so a single image (which my camera takes 5 per second of) is 30MB. I sometimes shoot as many as 2000 a day and over a decade it adds up...

Also my camera's hacked to record HD video at 200Mbps which probably doesn't help.

Do I "require" any of it? Probably not, given the number of times I've had a drive/memory card fail and lose hundreds of gigabtyes of hugely important data, yet I'm still alive...

---------- Post added at 12:18 ---------- Previous post was at 12:14 ----------

Originally Posted by Qtx

Add on top music, then all the windows and linux iso's, VM backups, home video backups, lossless music samples for creating music etc and space can easily get taken.

Music is probably the one thing that really needs to be on storage, since I play my whole music collection on shuffle all day every day. Changing CDs every 3 minutes would be a PITA.

Movies/Blu-Rays etc. I tend to find I only ever watch once, if I ever watch it again it's several years later, it could probably be better off/cheaper in a WORM tape archive. Or just on disk, in which case I'd have to to stop being lazy and actually insert a disk once every few weeks...

Originally Posted by qasdfdsaq

I use it for... well... Storage. The main array is a 76TB array in triple-parity mode giving 64TB usable. Backup storage is a 25TB array.

Aside from the usual suspects (movies, porn, other downloaded warez) it also stores backups of most of my machines, and my own photo/video collection. And several dozen virtual machines I can't be bothered going through before deleting.

I also used to do a lot of freelance data recovery work and imaging four or five 500GB-1TB drives at a time, plus extra space to store the data recovered off them...

---------- Post added at 12:14 ---------- Previous post was at 12:09 ----------


The storage I "need" these days is probably closer to about 10TB. 2-3TBs are my own photo/video collection, as a photographer I'm pretty lazy and shoot everything in maximum quality RAW+JPEG so a single image (which my camera takes 5 per second of) is 30MB. I sometimes shoot as many as 2000 a day and over a decade it adds up...

Also my camera's hacked to record HD video at 200Mbps which probably doesn't help.

Do I "require" any of it? Probably not, given the number of times I've had a drive/memory card fail and lose hundreds of gigabtyes of hugely important data, yet I'm still alive...

This is a great example of how Technology has advanced in quality and price in the last ten or so years.

In 2001, I undertook a trip to the USA (on behalf of BT Cellnet, when I was the IT Programme Manager on the Data Warehousing programme) - we visited a number of sites (Pittsburg, Little Rock, St Louis, and Montreal) to look at what was (at the time) leading edge in DW, focusing on Telecoms and CRM companies. The site in Little Rock had one of the biggest (not including NSA) Data Warehouses for CRM, and it was 25TB useable space (they used RAID1 at the time).

How things have changed......

[qasdfdsaq]

Well it sure has advanced in capability though I'm not so sure about price, I did spend a whole 2-3 months' salary on it :P

Miniaturization has been incredible though. An 80GB MP3 player from 10 years ago:
http://img52.imageshack.us/img52/50/p1010855w.jpg
And a 128GB one today:
http://2.imimg.com/data2/KC/SM/MY-25...le-250x250.jpg

I think the 2000's were probably the heyday of hard drive technology though, since about 2010 or so we've seen far less progress

[Ignitionnet]

Originally Posted by qasdfdsaq

Well it sure has advanced in capability though I'm not so sure about price, I did spend a whole 2-3 months' salary on it :P

Miniaturization has been incredible though. An 80GB MP3 player from 10 years ago:
http://img52.imageshack.us/img52/50/p1010855w.jpg
And a 128GB one today:
http://2.imimg.com/data2/KC/SM/MY-25...le-250x250.jpg

I think the 2000's were probably the heyday of hard drive technology though, since about 2010 or so we've seen far less progress

Maxing out on bit density per platter. SSDs are where it's at now.

[Qtx]

Everything we have now is better quality. Cameras and video cameras (including mobile phones) have more megapixels which = more space and we love to have the best possible quality if we can. Movies are better quality after going from DVD to bluray. People want to store lossless music such as FLAC instead of the lower quality MP3. People are also becoming more aware of backups and raid.

Luckily hard drives have got bigger and in general cost less per GB as time goes by

I'm also guilty of not cleaning up old computers or backups, so have a number of messy home directories backup up from as long as 15 years ago, as well as some very old virtual machines I should actually delete. There may be a time I can't find something and want to fire them up to see if it's on them though :p

[qasdfdsaq]

Originally Posted by Ignitionnet

Maxing out on bit density per platter. SSDs are where it's at now.

Well there's still HAMR and shingled recording to come, the latter being a pretty desperate and backwards move IMO.

SSDs on the other hand, I've had too many fail disastrously to put any trust in them anymore, it seems they're yet to figure out how to design them with any sort of sensible failure mode.

Plus, aren't we already nearing the limits of process shrinking and write endurance on stupidly tiny 12nm or so NANDs as well?

[Ignitionnet]

3D NAND is your friend.

Prices on SSDs have dropped a ton. I purchased a 240GB SSD a couple of years back at 2 quid per GB, got a 240GB a month ago with better read and write performance for 33p per GB.

I always install monitoring on the SSD to keep an eye on the individual segments as they fail.

[qasdfdsaq]

Yeah prices have dropped like mad, though again the rate of decrease seems to have slowed. My 256GB SSD (mid-range, so not the cheapest) bought in late 2012 was £120, or 47p per gigabyte.

Monitoring is all good and all, if wear is the actual problem. Unfortunately often they show no signs of wear before one day the controller just catastrophically fails. Not only catastrophically failing itself, but then going on to explicitly overwrite known-good data while the device is inaccessible! Go figure.

Plus you can't always monitor SD cards properly, especially when used in phones or cameras.