Notice the logarithmic scale (scp vs tar)
Transferring a few hundred small files over a high latency network using scp
became a bit too slow for my taste, and using ssh
+ tar
on the same network
was blazingly fast. But how fast.
Shameful scp performance… Let’s see it on the logarithmic scale to drive the point home.
Here’s the dataset, for the curious.
Time: | Seconds | Minutes |
---|---|---|
tar + ssh |
7.6 | 0.127 |
tar + ssh -C |
7 | 0.117 |
scp -r |
2216 | 36.93 |
scp -Cr |
2220 | 37.67 |
And the commands used:
#tar + ssh
ssh <HOST> tar -cf - <DIRECTORY> | tar -xf -
#tar + ssh -C (Enable ssh compression)
ssh -C <HOST> tar -cf - <DIRECTORY> | tar -xf -
#scp -r
scp -r <HOST>:<DIRECTORY> ./
#scp -rC (Enable ssh compression)
scp -Cr <HOST>:<DIRECTORY> ./