I've recently had the luxury of being handed 6x1tb external HDD's filled with scientific data that needed to be "uploaded" to a storage array for analysis. At first I thought to myself I should just use rsync to upload the files. Obviously it was going to take a while to scan the files then upload the files, this would typically be the command that I would use.

$ rsync -arxvHP /SOURCE/ host:/DEST/

After a bit of thinking, I thought, may be using gnu tar with ssh might have been a better choice to do the initial transfer, then just use rsync to update the dataset if there are any changes. I used something similar to the following command.

$ tar czf - SOURCEFILES | ssh host 'tar xzf - -C DESTDIR/'

The above is just a note to myself as I keep forgetting I can just pipe stuff over ssh.