back

by magnetic·8y ago·view on hn ↗
If you have a large amount of data to backup, cloud storage may be too expensive.

My data set is about 8 TB (my wife is a professional photographer), and it would be too expensive to keep in S3, so I have an "offsite backup system" that is hosted at my in-laws. It's just a RaspberryPi + an 8 TB drive encrypted with LUKS (in case it gets stolen or tampered with).

Every night, the RPi syncs the data from my house with rsnapshot (which is a TimeMachine like tool that uses hard links with rsync over ssh).

Because of how rsnapshot works, I can always go there and look for a file in one of the directories: it looks just like the original hierarchy, and I can just browse the filesystem as I would on the original system.

I also don't have to "trust a 3rd party" that the data is really there. I remember some posts on HN about people who used some backup services successfully... until restore time. I'm always cautious about the magical "set it and forget it" service that is a black box.

The first sync has to be done "on the premises" because of the sheer amount of data to transfer, but then the few daily Gigs of data can easily go over the net while everyone is sleeping.

2 comments
S3 is expensive not just because it's "cloud", but because it keeps multiplies copies in monitored drives, whereas your single drive is a ticking time bomb.

I have a couple of drives in RAID 1 (mirror), yet I still don't rely on it exclusively for really important data.

Well, all of my data exists in 3 copies. One in my house, one in the detached "granny unit", and one in the offsite location that I described.

My single offsite drive is indeed a ticking time bomb, but it's easily replaceable with no loss of data when it dies.

The problematic case is when all the drives hosting the 3 copies happen to die at the same time. Perhaps I don't have good protection about this case, but I think I've reached diminished returns in terms of backups.

He doesn't just have a single drive though, he has (at least) two (the drive in active use and the off-site backup drive).

Not the ideal 3-2-1 rule of thumb, but also not a ticking time bomb.

Glacier would cost about $30 per month to store 8 terabytes, according to the price calculator (calculator.s3.amazonaws.com). S3 infrequent access storage would cost $100 per month. These amounts seem pretty cheap for storing 8 terabytes online and offline respectively.
If you do the math, you see that an 8TB drive can be found for $125 (granted, it's a special deal) at https://slickdeals.net/f/10919999-8tb-seagate-backup-plus-hu... - but it's quite common to find them around $200. Then the RPi 3 is under $50.

Without even talking about the difficulty of dealing with Glacier files in an incremental backup scheme, doing the initial sync, and checking the backup data regularly, we're talking about something that will probably cost $250 for a couple of years of backup (assuming the drive only lasts 2 years) vs $720 (glacier) or $2,400 (S3) if I use your numbers.

It seems like a significant difference to me, especially because the assumptions on how often my drive will fail are quite pessimistic.