# 20-30 gig backup ideas



## kilowatt (Mar 6, 2005)

If you had to backup 20 to 30 gigs of unique data every week to a removable format, what would it be?

Has to last a long time (over 10 years, climate-controlled, of course), has to be compatible with Mac OS X (scsi, sata, or firewire connected), cart's can't cost more than $50

So, any ideas?

Thanks!


----------



## JPigford (Mar 6, 2005)

Does it have to be on seperated or could you just back it up all to one place each week? For instance, I do nightly backups to a 250GB external harddrive I have.

Other than that, your $50 price limit could be pushing it. And I'm assuming that's $50 per backup?


----------



## chornbe (Mar 6, 2005)

If you don't need to keep the backed up data for more than a week or two, external hard drives could be the answer. Buy 5 of them and rotate them during the week. 

Or invest some cash up-front and get a decent tape system like DLT or something.


----------



## kilowatt (Mar 6, 2005)

Well, the data has to be archived for years, which is why I didn't swing for a set of hard drives. And yeah, I can't delete previous backups.

I'm looking at DLT drives at the moment, but I'm wondering what alternatives exist.


----------



## mdnky (Mar 7, 2005)

Iomega makes the REV Drives.  35GB uncompressed (70GB compressed on a Mac, 90GB compressed on a PC) per cartridge.  Cartridge Shelf-life is >30 years.  Cartridges are $60/ea if bought singularly, $50/ea if bought in packs of 4 (from Iomega store); $55/ea if bought singularly, $47.50/ea if bought in packs of 4 (from CDW).


----------



## Randman (Mar 7, 2005)

$579 for the Iomega burner and 6 disks is pretty hefty. You could get a couple of good external HDs with more space for less than that total.

That said, under $50 doesn't sound very likely unless you're going to spend lots of time burning dvds of backed up stuff.


----------



## kilowatt (Mar 9, 2005)

$579 for the drive and some media isn't bad at all, compared with a new DLT system (which can easily cost over $1,000).

My only question would be, how reliable are these things - its basically the inside of a hard drive, minus the heads, right?


----------



## chornbe (Mar 9, 2005)

With the exception of the ZIP drive, I found every piece of IOMega hardware to be of questionable quality. I used to work in a service center and we saw an inodinant number of IOMega tape drives and especially Jazz drives (and that other drive with the little lever and hard plastic cartridge - forget the name now) come back, usually with really bad problems. It kind of soured me against using their hardware in any "real" system or situation.

$.02


----------



## Pengu (Mar 9, 2005)

um. spanned data onto DVD+/-Rs?
DLT (get's kinda expensive though, tapes are $100 each here in Aus.)
um. Zip/Stuffit/TarGzipped archives backed up onto network RAID storage, and then backed up daily onto rotating tape/dvd/hdd..

20-30Gb of data every week is quite a bit.. as much as it's "old" technology i'd be guessing DLT is the way to go. Just make sure your drive is kept in good repair. I work in an educational institute with 8 campuses (well.. 35ish now that we're all joined into TAFE SA and we've had easily half a dozen drives fail recently.. it seems this was mostly caused by bad tapes.. so if you get a quality unit, and quality tapes used once only (except for restores obviously) then you should be ok..


----------



## Go3iverson (Mar 9, 2005)

What data is going to be placed on this backup?  I'm assuming if its being backed up, that you might find it vital?  

Having gone through months of disaster recovery planning, depending on your usage and environment will dictate what they will find as acceptable for backup.  Remember there is only one reason for backup and that is restoration.  If you use a fairly non-mainstream media source now, what are your odds that you'll be able to find a replacement drive in 10 years to restore from, if need be, if you don't migrate that data off to a more mainstream media type.

For example, I looked at sait drives during my research, but decided to go with the more popular LTO2, not because the technology is better, or even more cost effective, but simply because more people are using it, which will make it easier to get support.  Go back to VHS vs Beta and apply that argument to this one. 

For the amount of data your trying to move, you can really use almost anything, which gives you tons of options.  The idea of DLT or SDLT isn't a bad idea.  I used SDLT for a long time and, again, its a widely used tape media.  You do, then, have the cost of getting that data to the tape itself, as OS X doesn't have any built in tape drivers.  For an environment of that size, I'd recommend BRU by Tolis Group.  It scales pretty well and has some very nice command line tools.  Licensing isn't very expensive and I would prefer it over a smaller, possibly cheaper alternative because of the support and stability of Tolis Group...you know they'll be around, in theory, longer than a unknown.  You'll find that many drives actually ship with a BRU license, so that'll save you some cash right there! 

If you want to read more about it, grab the documentation from my site.  I have to update the BRU section, though.  When I originally had to have the doc done, I wasn't able to get the new version of BRU to run with the new tape device I had just received.  When I revisited it, that all changed and I'm backing up about 2TB with it on average.   My site's in my sig, just click on the Projects tab...the mirror should be faster to get the PDF, if you're interested.


----------



## chornbe (Mar 9, 2005)

Excellent write-up, Go3iverson. Well done. Good, unbiased, objective.


----------



## Go3iverson (Mar 9, 2005)

Thanks Chornbe. 

My server's being pushed for the last three weeks since I released the doc and I've had some pretty interesting hits in the logs 

I tried to be fair and unbiased.  I also didn't want to bash any product.  I'm sure you can tell I was less than thrilled with my overall experience with TSM, but its not a bad product, it just didn't work out for me, probably due to the engineering group I worked with, who will remain nameless.  Each one of those products can address a different need, market and data size, with some nice overlap.  

It took forever and a day to get all that info together, so I'm glad folks are enjoying it!


----------



## chornbe (Mar 9, 2005)

Looking to hire a programmer?


----------



## Robn Kester (Mar 9, 2005)

This thread raises some interesting questions about what we back up and how safe it really is.

I have used pretty much every kind of storage there is out there over the years in one form or another... Floppies (the big and small ones), HD, CD, DVD, DVD-RAM (those cartridge suckers), Zip, Jazz, Syquest, Bernoullis (still got a few of the carts), tape backups of all kinds, etc. Anyone remember the days of using cassette tape drives? Any of those still around? lol.

but I never really put much thought into whether it was going to be around in say 5 or 10 years. In most cases for me, the media is still around but the drives for it arent.

At points during my computerlife, I have moved some of my old content worth saving to new media, including moving zip and jaz to CDs, moving CDs to DVDs, etc. That doesnt include the hundreds and hundreds of floppies I no longer can use in any mac. (I use mediafour's macdrive on my PC for this and it works nicely.) I dare to say that 1 out of every 4 of my floppies is bad in some spot.

What about all of those CDs we backed up 5-10 years ago that we wrote on with a magic marker. Or the tapes that have sat unused and have possibly become brittle.

Is there really a truly reliable source for safe long term (10-30 year) backup that is consumer friendly? Or can only those who are dead serious about backups lasting afford it?

Just some thoughts.


----------



## chornbe (Mar 9, 2005)

DVDs seem to be the shiney new favorite for long term. For me, I just make sure I've always got another system running in my house and run my backups to that. The office is another story, but at home, I run machine-to-machine rsyncs (or the like) and periodically burn a dvd (CDs in earlier days) and toss it in the car. It's not fool proof, but in the 5-6 years I've lost darn little data... even thru' two major system crashes (both brand-name Win PCs - go figger). 

So that's something.


----------



## pipermalibu (Mar 9, 2005)

I have another alternative altogether....one thing for sure is the hard drive is going to be arond for a while longer. I have started using a low cost raid5 backup solution. I have two 1TB systems that mirror to each other everynight. This gives me the redundancy of raid5 PLUS the redundancy of it mirroring to the other system off-site. Each system costs me about $1200. So the math says thats $1.20 per GB. That fits into the original poster's cost analysis. If you are interested the product is known as REBYTE.  (www.rebyte.com) and I have found them to be pretty cool.

My .02


----------



## Go3iverson (Mar 9, 2005)

Hmmm....a programmer?  I'm not that big of a company on my own, but send me an email, I don't mind referring out jobs that may come my way. 

There are multiple considerations here:

1.  Do you want your backup to be offsite, in case of disaster?
2.  What is your rotation of media going to be?  Is it going to be 10 years and down?
3.  Are you considering any data growth?

Media such as DVDs and CDs aren't going anywhere for quite sometime, so for the average user, it'll work.  If your considering 30GB/week for 10 years, that's a lot of DVD's to keep track of and mind you, if your just simply going to do this by hand, you don't really have a cataloging system in the background to keep track of your data for restoration purposes.

Again, I'll repeat:

The whole point of backup is restoration!


----------



## aicul (Mar 10, 2005)

Go3iverson said:
			
		

> The whole point of backup is restoration!



The backup process enables recovery, conservation, and archiving. Restoration is only really essential to recovery.


----------



## chornbe (Mar 10, 2005)

I think he's thinking along the lines, if you can't restore from a backup then your backup is useless. Picking a long-term backup strategy is not a game for the cavalier.  It's probably one of the least aggressive technology segments out there... in the big-money corporate world, for the sole purpose of backward compatibility (at least in read-back mode).

When people start using phrases like "yeah, just toss all that onto a tape for me" then you'll know you have standards and reliability in inter-system operation. Until then, when you think universal and inter-platform, people think CD and DVD.

That's why companies like IOMega couldn't ever make it the long-term corporate DR market segment. Companies... especially ones that have lived thru' a major system outage... know that what was backed up yesterday had *BETTER* be restorable today. Extrapolate that out over a 3-5 year time span.

Name one retail tape drive or low-end tape drive technology that was standardized, fast and which was left alone without 10000 revisions, updates and media changes, in the last 10 years. I can think of only one... QIC-40; which is now so horribly undersized as to be completely useless in today's market.

$.02


----------



## Go3iverson (Mar 10, 2005)

aicul said:
			
		

> The backup process enables recovery, conservation, and archiving. Restoration is only really essential to recovery.





Yes, but if you put all of your archives and backups (conservation) onto media that you cannot restore from, what good is it to you? 

The real bottom line is the ability to access the data over a specific given period of time.  If its your archive, you'll need to know you can easily access that data in X-months or X-years when you need the data again.  If its your backup and your system goes out, you need to know you can get back up and running ASAP in an emergency.  It really doesn't matter what the task is called, it really is a form of restoration.  Most people won't 'work' from their backups directly.  They'll want to keep the integrity of the backup and the state of the data of that given point in time, so they'll restore the data to another machine to access it.

The real keys that people overlook are all linked together.  Its the ability to properly restore the data.  You'd be shocked how many folks just assume the data is safe.  Secondly, is the use of a catalog.  That's where true backups are handy, in that they will keep track of versions and dates for you, of your data and also can help you save space, by allowing differential/incremental backups, instead of continual, in this case, 30GB backups, unless that's just the size of data growth.


----------



## kilowatt (Mar 10, 2005)

Well,

Thanks everyone for the information. I have to say, I've never had so many people with so many opinions and ideas on how to do this. 

DVD's and CDs are out of the question, for two reasons:
1) Backups have to span multiple units
2) I don't think consumer cd-r and dvd+/-r formats have really been proven for longevity. Sure, a glass master-produced cd lasts decently long, but a cdr? I'm not so sure.

At the moment, I'm leaning towards DLT/SDLT, for two reasons:
1) Backup fits on one drive.
2) Its a somewhat common format.

One thing I am also considering is HP's Ultra Density Optical (UDO). UDO 'disks' are $50 or so each, but the drive system is around $4,000. Also, I'm not clear on if Mac OS X is supported - HPs site says many other platforms are supported by various ISVs, so I have to look into that. I'd think though, if it were a standard scsi storage component, I could just address it as such.

The backups will be stored in an extremely nice climate-controlled room, with its own independent cooling. They'll also be stored off-site in someone's spare room or something. 

The data that I would be storing is multitrack PCM digital audio, from a Tascam MX-2424. My idea is to just store the entire drive image (compressed) onto the tape. The drives are formated in HFS+, so I could alternatively just drag'n'drop the files.

Any opinions on UDO?


----------



## chornbe (Mar 10, 2005)

I'd stick with the (s)DLT solution. If your drive took a dump or if drives changed a lot, you'll still find support out there for years and years to come. UDO is, as yet, largely unproven in as near a widespread market as the DLT options.


----------



## Go3iverson (Mar 10, 2005)

SDLT is a nice, widely used and supported technology.  I'd look around to see what drives Apple may sell on their store, as they may have a bundled BRU license.  If not, call Tolis Group (BRU).  I can give you a good contact there, who can really get the info you need for you.  This way, you'll be sure that the drive you settle on is fully supported by the software.  

For what your trying to do, go with BRU.  I'm not a big fan of Retrospect.  I feel the product cripples a great deal of the features of OS X in general and for your data store, backup requirements and price range, those are your two 'main' competitors.


----------



## aicul (Mar 11, 2005)

*kilowatt*, Have a look at http://www.synerway.fr/index.html.en. They address your concerns and more.



			
				Go3iverson said:
			
		

> The real bottom line is the ability to access the data over a specific given period of time.


 Once people will learn that there is a difference between recovery, conservation, and archiving they will learn how to use the backup process correctly.


----------



## Go3iverson (Mar 11, 2005)

aicul said:
			
		

> *kilowatt*, Have a look at http://www.synerway.fr/index.html.en. They address your concerns and more.
> 
> Once people will learn that there is a difference between recovery, conservation, and archiving they will learn how to use the backup process correctly.



I'm thinking in the way that you said that, you are implying that I do not understand the difference between these concepts.  The terms themselves without actual retention policies, data storage policies, etc don't really mean a whole lot.  

For example, archiving to what?  Are we talking online archiving?  Near line?  Offline?  How large can your archive grow in comparison to your overall data storage?  Then, depending on those factors, and many others, then decide your backup strategy around it.

My average, full network backup is about 4.5TB, for my servers.  Now, disk is relatively cheap these days, so I have a lower speed RAID where I have non-live projects archived, in case a user needs that data, which does come up now and again.  That data is not continually backed up, because that .75TB can make a difference in your window of time to get your data properly to tape, or whatever your trying to do with it, for that matter.  The archives require special permissions to write to, so this way, I can know if data has been added, updated, or a project is being moved to the archive, then I can add that bay into my backup again.  In the case of a project being moved to archive, I can now move the data off of my primary server to the lower speed server and not worry about daily backups of that data, since its not being used.

Some people don't have that luxury of additional storage space.  Their archives are offline, probably on tape, in a remote location (hopefully).  Again, the remote location comes into play for two reasons.  One, if disaster were to hit your primary location, you still have all of your data safely off site and secondly its a factor in your restoration times.  You then have to factor in, where is your remote location?  Who's in charge of the media?  Is it you, where you can just jump into your car and go get them at a moments notice, or is it a company where fees may come into play or scheduling?  Sure, its easy enough to keep an identical set of archive tapes on site, but its also not much fun to continually search those backups for the data that your users want!  The fun is always the consideration that disk is much faster than tape, but tape is more durable for shipping of site than disk.  

Of course, you can also look at larger products, such as TSM and Time Navigator, who have capabilities to archive your data for you.  Enterprise solutions like this can be set full of variables and storage thresholds, as well as retention times, which will then migrate data for you to different storage pools that you've previously defined, which could be lower speed disk, tape, etc.  I'm thinking, though, for 30GB, this particular poster isn't going to go quite that far, though I could be wrong!

Heck, and we haven't even touched on the content management piece that you can throw in on Tivoli!


----------



## aicul (Mar 12, 2005)

Didn't mean to offend you by what I said. Just pointing out a fact I could not see in the thread.


----------



## Go3iverson (Mar 12, 2005)

aicul said:
			
		

> Didn't mean to offend you by what I said. Just pointing out a fact I could not see in the thread.



Its all good.   It is a question that hasn't been answered by the poster.  

If we're simply talking 30GB of data, period, then topics like archiving and such don't really come into play much, as you'd hope your server is robust enough to support that load of data!

Now, if we're talking 30GB of growth, per week, over 10 years, then you have a whole new white board to fill up with diagrams and colored lines!


----------



## kilowatt (Mar 14, 2005)

its just 30GB of data, every week. So, week 1 is 30 GB of data. Week 2 is 30 GB of data, ammounting to 60GB of total archived data. Week 3 is 30 GB of data, ammounting to 90GB of total archived data.


----------



## Go3iverson (Mar 14, 2005)

Yeah, that'll be *a lot* of DVDs at the end of 10 years! 

I'd say tape.  Whatever fits your budget that's widely used, like SDLT.  You never know, you may be able to find a good deal on LTO even! 

Keep in mind (I think I said this earlier) that OS X has no built in ability to write to tape, so your compatability is really at the hands of the software you use.  I still say to try BRU out.  Low cost, which will let you spend more on your hardware, good company history, good support record and dedicated to your platform.  It'll let you also do diff/inc backups as well, which will help.

If your not worried about off site, Xserve RAIDs are nice for backup as well!


----------



## Go3iverson (Mar 14, 2005)

Oh, if you want, send me an email and I can give you a good contact in Apple's Enterprise division, if you'd like to learn a bit more about what's available.  I usually go to him for most of my orders/future planning, so I know he's good!


----------



## chornbe (Mar 14, 2005)

Go3iverson said:
			
		

> Hmmm....a programmer?  I'm not that big of a company on my own, but send me an email, I don't mind referring out jobs that may come my way.



Would it help if I talked about desperation? Need? And an overwhelming desire to run like hell from the insanity that has become my day job.


----------



## FLASH1296 (Mar 20, 2005)

Here is a very low-cost notion:

Get "BurnitAgainSam" shareware. This will permit you to repeatedly backup to a standard (virtually costless) CD-R  You can do this weekly. Then, on a monthly basis,you can store this CD in a metallic envelope (such as RAM is shipped in), date the CD,  and store it securely off-site, and go back to the regular weekly backup.  Then, yearly, you can create a disk image of ALL 12 monthly CD's and burn THAT image to a DVD and, naturally, store that as your annual backup. Repeat every decade creating a new Ten-Year ArchivalDVD.


----------



## pedz (May 22, 2006)

On a similar topic, what is the storage expectancy of data on a hard drive that is not in use?  I'm planning on archiving data to SATA drives.  Is that going to be o.k.?


----------



## fryke (May 22, 2006)

No. Hard drives are _not_ meant for archiving data. I mean: They _could_ hold as long as you wish them to, but from my experience, you just don't want to archive on harddrives. You want DLTs. Or DVD-Rs (BluRay, HD-DVDs later on) if you can live with duplicating them from time to time (5-10 years)...


----------

