Skip to content
Fragmented Development

Problems with rsync and link-dest

I use rsync to back up many servers and other machines. Because of my very different sources and targets, I have a collection of very complex bash scripts that manage these backups. When I moved one stage of these backups to a new server, it started behaving strangely - and it was difficult to discover why.

The backup that went haywire was the offsite backup step; when I take all of the daily backups that collect on the server, and once weekly, I copy these daily backups onto an external drive for long-term protection.

To prevent each snapshot from being a full backup, I specify the previous daily backup directory using the --link-dest argument. This tells rsync to take the previous daily backup, and only copy files that differ - all other files should be hard links of the previous files. For some reason, even though I was using the --link-dest argument, it was making a full backup and not using the hard-link technique to save space.

I had just moved my backups to a new server, and thought initially that I had accidentally changed permissions on the files (which would make them appear to be new/changed files). That wouldn't explain why every snapshot would be copied anew; it would have only affected the first newly copied snapshot directory.

It took me a rather long time, but I found the culprit thanks to a StackOverflow answer: If link-dest receives a relative path, it is relative to the destination directory. I had changed the line in my script where I create the temporary directory, and it had switched from an absolute path to a relative one. Once I switched it to an absolute path again, everything clicked and started behaving.

My rickety, fragile backup solution lives again!

Tags: server terminal

Add Your Comment