RAID 5 vs 10 for Library Server and Archiving

[UnderTow]

Originally Posted by tpad

Also, I wouldn't recommended a workflow where you are "working" on the data in the NAS unit. You should always copy the data into your computer hard drive and work on it there (then put the revised version back on the NAS). Otherwise, you're going to be "thrashing" the NAS drives, which is good way to induce premature disk drive failure.

I suggest you read a bit more diligently. My "Errrmm no" was specifically in response to your comment about what people are assuming. It was in direct contradiction to what others and myself had clearly mentioned: The need for backups. The word backup was even in bold in my post. Hard to miss. I think I was the third or fourth person in this thread to mention backups.

And again I have to disagree with the content of your post:

First I never mentioned anything about a NAS. Just a RAID system. Not that it matters but still, you are responding to something I never wrote.

Secondly, a RAID system is meant to be used actively and that is what most people and companies do. There is no issue with trashing the drives if you use quality components. Certainly no more than the local disks in your system.

Thirdly we are talking about an archive and a sample library. That is completely different than an active working disk. The archive wouldn't be "thrashed". Projects are written once and read sporadically. Samples are just written once and loaded into the project (mostly RAM on PT10 and good VI samplers). (Not that it matters as per point 2).

Fourth, even as an active project drive it would not be thrashed with Pro Tools 10 on a system with enough RAM: The project is entirely cached in RAM. Audio files are read once when you open the project. New files are written once. There is ZERO disk thrashing.

So in short, errmmm no.

Alistair

[UnderTow]

Originally Posted by JohnOmix

I little OT, but since we have some experts present :
I'm thinking of using this set up as JBOD

OWC OWCMRP1UF8U3EP Rack Pro/Desktop 4-Bay RAID... in stock at OWC

Do you think accessing 2 or 3 drives at a time via eSATA will bottleneck? Not doing anything hugely drastic, some VIs reading samples from 2 drives, medium size Pro Tools sessions reading/writing on another drive...

BTW, thanks for the RAID info too.

I didn´t see a response to this post. So here goes:

In a JBOD system (Just A Bunch Of Disks) all the drives are bunched together to form one big (virtual) partition that looks like one big drive to your OS. The important thing to know about JBOD is that if any of the disks fail, the whole system goes down. The more disks in the JBOD system, the higher the risk of failure. Each additional disk added to the JBOD is an added point of potential failure. There is no redundancy or protection for your data.

JBOD is a way to get big disks cheaply. Not recommended for mission critical data!

Alistair

[ggegan]

Originally Posted by UnderTow

I didn´t see a response to this post. So here goes:

In a JBOD system (Just A Bunch Of Disks) all the drives are bunched together to form one big (virtual) partition that looks like one big drive to your OS. The important thing to know about JBOD is that if any of the disks fail, the whole system goes down. The more disks in the JBOD system, the higher the risk of failure. Each additional disk added to the JBOD is an added point of potential failure. There is no redundancy or protection for your data.

JBOD is a way to get big disks cheaply. Not recommended for mission critical data!

Alistair

I thought that was RAID 0, striping with no redundancy or parity, and that JBOD showed up as individual disks.

[UnderTow]

Originally Posted by ggegan

I thought that was RAID 0, striping with no redundancy or parity, and that JBOD showed up as individual disks.

There is no striping in JBOD. Just a bunch of separate disks that that are concatenated to show up as one large logical disk to your operating system. The data is written over all the disks (spanning) and a failure of any of the disks takes the whole system down. It is the least secure way to use disks yet doesn´t offer any speed benefits like RAID 0 does. Just using the individual disks as separate drives is safer than JBOD.

It used to be a way to combine smaller disks in to one large virtual disks. Either to be able to put very large files on it or just for convenience if you want to have one large library of data or something like that.

In my opinion it is best avoided.

PS: Edited for clarity and here is an article on the subject: http://macs.about.com/od/usingyourmac/ss/raidjbod.htm
PS2: That article says that files in a JBOD set never span over drive boundaries but beware as that is implementation dependent!

Alistair

[tsvisser]

RAID is not a good backup plan, it is only designed to maintain uptime. A drive goes bad, the bank can still run it's financial applications and not go offline, etc... Always backup to an external volume or media, do not rely on RAID alone as your backup plan.

[insomaniac]

Originally Posted by ggegan

I thought that was RAID 0, striping with no redundancy or parity, and that JBOD showed up as individual disks.

Yeah, it is, though you option for creating a "concatenated" or "spanned" data set when a JBOD is attached (or multiple drives, for that matter) that will do what Alistair described. It's risky as all hell, but doesn't have the more stringent requirements of RAID (all disks same size/model, etc.)

[tpad]

...First I never mentioned anything about a NAS. Just a RAID system. Not that it matters but still, you are responding to something I never wrote....

You seem to think that everything I'm writing is directed to you personally, and you're mistaken. You're not the only one in this discussion and you're not the only one making assumptions. If you look back, another poster (DaVogi) originally brought up the topic of NAS units, and I was merely pointing out -TO OTHERS- that focusing on RAID topologies so intently, ignored the fact that other things like power supplies can fail, and leave you dead in the water, no matter how good your RAID (or NAS or SAN) is.

As far as the thrashing goes, I'm well aware that RAID useage falls into many different working paradigms. My own preference is to not pound on the network storage equipment to prolong the life of the disks and reduce the probability of failure. Some choose to ignore good advice and live dangerously!

[UnderTow]

Originally Posted by tpad

You seem to think that everything I'm writing is directed to you personally, and you're mistaken.

That might be because you put my name at the top of your post.

Alistair

[tpad]

You were interjecting commentary. Anyway, next time I'll address them to whomever it may concern

[JohnOmix]

I made mistake, thought JBOD was the same as Independent Mode, which is what I really want. From the OWC site:

"When using the OWC Mercury Rack Pro in Independent mode*, all four drives appear on your system as individual volumes drives, so you can mix and match drives of any size and brand and hot swap** the drives too."

Here's the drag:
"**When using in Independent Mode and wanting to switch out one or more drives, ALL drives must be ejected (unmounted) from the Desktop to avoid data corruption. "

I'm trying to streamline my current setup of using several separate drives, plugging certain ones in, unplugging some, forgetting to turn off one, etc.

Anything against going this way? Is this a totally non-RAID way to go?

[JSt0rm]

Glyph gt key will do that

[JohnOmix]

Originally Posted by JSt0rm

Glyph gt key will do that

It's been discontinued... And I'd like one where just bare drives can be used. Wishful thinking I guess

[ggegan]

Originally Posted by JohnOmix

It's been discontinued... And I'd like one where just bare drives can be used. Wishful thinking I guess

Why not just get a few of these?

Newer Tech Voyager

[tpad]

You're JBOD probably employs some sort of common SATA drive control card in the enclosure, which is why the disks are not completely independent. If really you want independent drives, why not buy Firewire single drive enclosures and just daisy-chain them?

[George Necola]

Most of the stuff written in this thread is bllocks.

You can build a raid5 with minimal 4 disks (3 data+1hs)
The more disks you have in a raid, the faster the rw speed.
The rw speed is primary determined with the speed of each hd.
The more disks, the higher the speed.
If you want more safety within a raid enclosure get a bbu on the raid controller card and a good controller.
You dont need a "server". You need a drive enclosure with intelligence (check fujitsu eternus-dx60 or IBM ds3000..).
You can attach such a drive enclosure to a network/host via iscsi, sas or fibre channel. (And I am pretty sure the gs community will find more ways to connect )

RAID is NOT backup!

Gesendet von meinem GT-I9300 mit Tapatalk

[JohnOmix]

Originally Posted by ggegan

Why not just get a few of these?

Newer Tech Voyager

Ha, I DO have a couple of those. Looking for a more elegant, less cluttered way than having a bunch of drives on my desk. Also would like to run one eSATA cable for the drives; it's faster and I use Firewire for my audio interface. Running a MacBook Pro so I don't have PCIe to add more slots.

I'm going to check it out and if it's no good I'll return it.

[tsvisser]

Originally Posted by George Necola

RAID is NOT backup!

Really, this should be THE takeaway from this whole thread.

[tpad]

The OP (ggegan) said he wanted a "Library Server" for Archiving, set up so that "2 or 3 workstations [could] access the server simultaneously". Then he made a contextual jump into talking about RAIDs, without anyone noting that RAID they were so fervently discussing are not file Servers, which is what he said he wanted (maybe he doesn't really know what he wants!). An ethernet based NAS unit would be ideal as a LIBRARY SERVER. 4-disk NAS units from better vendors such as Buffalo-Tech, Iomega, Synology, etc come prestriped in RAID-5, and so they are ready to go right out of the box. All you need to do is plug them in and turn them on.

[ggegan]

Originally Posted by tpad

The OP (ggegan) said he wanted a "Library Server" for Archiving, set up so that "2 or 3 workstations [could] access the server simultaneously". Then he made a contextual jump into talking about RAIDs, without anyone noting that RAID they were so fervently discussing are not file Servers, which is what he said he wanted (maybe he doesn't really know what he wants!). An ethernet based NAS unit would be ideal as a LIBRARY SERVER. 4-disk NAS units from better vendors such as Buffalo-Tech, Iomega, Synology, etc come prestriped in RAID-5, and so they are ready to go right out of the box. All you need to do is plug them in and turn them on.

The reason I started the thread is because I am not very knowledgable about either RAID or servers, although I have learned quite a bit due to the discussion that has taken place, so thanks to everyone who contributed. What I needed was a large, fast storage system to host Library and Archive partitions that would primarily serve one system, but that could also be accessed by one or two other systems on occasion.

Since I started the thread I have subsequently set up a 12TB RAID 10 that is attached to my primary Mac Pro and tested it to see if a second and third system could also mount the RAID volumes using ethernet file sharing to access the Library and Archive partitions simultaneously. That is working great, no problems whatsoever, read and write speeds are more than acceptable. This seems to be a very convenient way to approach things for now, I just have to boot my primary Mac and everything works as it should, and that is what would generally happen anyway. If I needed to have the RAID accessable to other systems without booting my primary Mac, I could attach the RAID to a spare iMac running Lion Server, but that doesn't seem necessary right now.

[George Necola]

How do you backup 12 TB of data? Tapedrive?

Gesendet von meinem GT-I9300 mit Tapatalk

[ggegan]

Originally Posted by George Necola

How do you backup 12 TB of data? Tapedrive?

Gesendet von meinem GT-I9300 mit Tapatalk

The 12TB RAID 10 can only store 6TB of data. There is a 2TB partition for my library, which also exists on 2 other external hard drives. The other 4TB partition is for archiving. In addition to the system drive, an audio drive and a video drive, each of my Macs has a 4th 2TB "vault" drive where I also keep copies of archived files, plus I try to burn DVDs of all the material that gets transfered to the RAID archive, but I'm always behind on burning the DVDs because it's so time consuming. It's not a perfect system. I'm considering eventually getting a tape drive, but it isn't in the budget right now.

[tpad]

You're archiving audio - right? So, how fast is "fast"?

[ggegan]

Originally Posted by tpad

You're archiving audio - right? So, how fast is "fast"?

I ran the Blackmagic Disk Speed Test app on both an internal drive and the RAID. The internal drive was a 7200rpm 1TB Western Digital Caviar Black, the RAID drives are 7200rpm 3TB Hitachi Ultrastars. Speeds are approximate averages based on several different tests.

Internal speed test: both read and write approx. 115MB/sec
RAID speed test: read approx. 170MB/sec, write approx. 120MB/sec

[jeronimo]

Hey ggegan,
Could you let us know what speeds are you achieving thru Ethernet?
Also, what OS are you running on your Macpro AND what enclosure are you using?
Thanks a lot!

[George Necola]

Originally Posted by ggegan

I ran the Blackmagic Disk Speed Test app on both an internal drive and the RAID. The internal drive was a 7200rpm 1TB Western Digital Caviar Black, the RAID drives are 7200rpm 3TB Hitachi Ultrastars. Speeds are approximate averages based on several different tests.

Internal speed test: both read and write approx. 115MB/sec
RAID speed test: read approx. 170MB/sec, write approx. 120MB/sec

your experiments are dangerous. I just to want to note, that no one should go this route if he wants/needs data safety.

get something like a Fujitsu eternus DX-80 (or IBM DS3xxx/HP MSA storage with ethernet interfaces + something to backup all this data, which is some sort of problem if you don't run a tape library (talking about backup tapes.. )

the cheaper alternative are NAS-based storage devices available from synoglogy.

cheers
George