Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I've been using this extensively recently. I was setting up remote virtual machines that boot a live ISO containing all the software for the machine. Sometimes I need to change a small config file, which would lead to generating a new 1.7GiB ISO, but 99.9% of that ISO is identical to the previous one. So I used rsync. Blew my mind when after a day of working on these images, uploading 1.7GiB ISO after 1.7GiB ISO, wireguard showed that I had only sent 600MiBs.

Fun surprise, rsync uses file size and modified time first to see if the files are identical. I build these ISOs with nix. Nix sets the time to Jan 1st 1970 for reproducible builds, and I suspect the ISOs are padded out to the next sector. So rsync was not noticing the new ISO images when I made small changes to config files until I added the --checksum flag.



> Fun surprise, rsync uses file size and modified time first to see if the files are identical. [...] time to Jan 1st 1970 for reproducible builds

I think it'd be a good idea for rsync to not trust timestamp 0.


In the past I downloaded daily diffs from iso which were only few MB. I then applied this diff to my iso from yesterday. Forgot the name of this tool though. I did this on my machine, if parent wants to update in a remote machine I'm not sure it works the same way.


If it was a 100GB image on the other hand—good luck! It'd be faster to copy it from scratch every time than to use rsync.


You'll find something like BorgBackup will be far more efficient than rsync.


But rsync is widely available, usually installed by default on linux or unix-like systems. You can just use it.


Borg is available for download as a standalone binary, easily dropped onto any Linux system even with very limited privs. And in the repos of every distro easily installed and kept up-to-date.

By avoiding that one step and using rsync instead, you're resigning yourself to "send 600MiBs" over the network for every tiny config change. Not a good trade-off.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: