Problems with rsync and link-dest
Posted on 2021-04-12 (Updated on 2021-04-12)
I use rsync to back up many servers and other machines. Because of my very
different sources and targets, I have a collection of very complex bash scripts
that manage these backups. When I moved one stage of these backups to a new
server, it started behaving strangely - and it was difficult to discover why.
The backup that went haywire was the offsite backup step; when I take all of the daily backups that collect on the server, and once weekly, I copy these daily backups onto an external drive for long-term protection.
To prevent each snapshot from being a full backup, I specify the previous daily
backup directory using the --link-dest argument. This tells rsync to take the
previous daily backup, and only copy files that differ - all other files should
be hard links of the previous files. For some reason, even though I was
using the --link-dest argument, it was making a full backup and not using the
hard-link technique to save space.
I had just moved my backups to a new server, and thought initially that I had accidentally changed permissions on the files (which would make them appear to be new/changed files). That wouldn't explain why every snapshot would be copied anew; it would have only affected the first newly copied snapshot directory.
It took me a rather long time, but I found the culprit thanks to a StackOverflow
answer: If link-dest receives a relative path, it is relative to the
destination directory. I
had changed the line in my script where I create the temporary directory, and it
had switched from an absolute path to a relative one. Once I switched it to an
absolute path again, everything clicked and started behaving.
My rickety, fragile backup solution lives again!