Published: Sep 16, 2022

rdiff - patching files

Use Case

A dockerized application is build locally and saved as a .tar image file that will be uploaded to a remote server. I was looking for a way to reduce the amount of data that needs to be uploaded when updating the image file.

Using diff I noticed that the difference between images is usually quite small. While diff and patch have the capability to do what I want and come preinstalled with debian, they are unfortunately rather slow.

Requirements

Install rdiff using your package manager of choice.

apt install rdiff

Steps

Create a signature file from the current file (on the remote server):

rdiff signature docker-image.tar signature.bin

2. Create a delta file containing the differences between the current and new file: ```sh rdiff delta signature.bin docker-image-new.tar patch.bin ```
3. (Upload patch.bin to the remote server)
4. Patch the current file ```sh rdiff patch docker-image.tar patch.bin docker-image-patched.tar ```
Now docker-image-patched.tar is equal to docker-image-new.tar.

Conclusion

In my usecase this reduced the data to be uploaded from ~1GB per update to a few MB. Creating and applying the patch only takes a few seconds, even on a slow machine. Overall rdiff is a very useful tool that can save a lot of time and bandwidth. It’s also easy to integrate into existing scripts and applications.