Transparent Compression

Richard K Miller richardkmiller at gmail.com
Mon Jun 12 15:20:29 MDT 2006


On Jun 12, 2006, at 2:49 PM, Ryan Nielsen wrote:

> Jonathan Duncan wrote:
>> On Mon, 12 Jun 2006, Ryan Nielsen wrote:
>>> I'm currently backing up data using rsync over ssh to remote hard  
>>> drives. Those drives have filled up quite quickly.  Has anyone  
>>> worked with transparent compression on file systems?  Does it  
>>> work well?  Are there any risks?  I'm not (at the moment) worried  
>>> about speed, just space.  Is there a better way to do this?
>> How about an extra hard drive?  I just picked up a 320gb for  
>> $120.  Do you need to be saving all this data cumulatively or can  
>> you use rsync to overwrite previously saved data?  Is this like a  
>> nightly backup or a additionaly data to large and growing  
>> project?  Usually, compression is more of a problem that it is  
>> worth, in my opinion.  Especially in a day when disk space really  
>> is cheap.
> My boss is pulling his save money card.  I would have just upgraded  
> to bigger drives, but I am trying to research it a bit at least.   
> Your comment on being inefficient is exactly what I wanted to  
> hear.  Any other opinions?
>

Here is the backup statement we use at work:

rsync -aruvz --delete --link-dest=/backup/`date -v -1d +'%d'` --files- 
from=/list_of_paths_to_backup / backupmachine.domain.com:/backup/ 
`date +'%d'`

Our entire server gets backed up to another machine each day and then  
rotates each month, so we have 31 incremental copies of the data.   
The secret to not wasting hard drive space is the --link-dest  
argument, which tell rsync to make a hardlink to the previous copy of  
the data if it's identical.  (`date +'%d` returns a numeric version  
of today's day of the month (01-31) while `date -v -1d +1'%d'`  
returns yesterday's day of the month.)    This means we get 31 full  
copies of the data, but it only takes up as much space as 1 copy of  
the data + incremental changes (which isn't very much in our case.)   
I've found this to be very useful.

You could change %d to %w to do 7 backups, one for each day of the week.

Richard







More information about the PLUG mailing list