Child pages
  • Transfer a file incrementally over a flapping connection
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 5 Current »

Shipboard outages of network can happen when the ship turns.  This can be unpredictable.  I use this technique to "set and forget" transfer data over HiSeasNet, using a combination of rsync, SSH and the Bourne-Again SHell (BASH) on a UNIX-type system.  You'll need to set up password-less SSH keys to do this.

1. Create an ssh-key for your shipboard host, if you haven't already done so.  `man ssh-keygen` for more info on this.  This is a potential security risk, so you should understand the caveats and benefits.

chmod 0600 ~/.ssh/id*
cat ~/.ssh/ | ssh me@myShoreServer "cat - >> ~/.ssh/authorized_keys"

2. Login to a shore-based server, use the faster link to stage the data:

ssh me@myShoreServer
wget -N -nd http://someServer/some.file

3. On the local shipboard system, use rsync to grab the data in a loop that will execute forever until rsync succeeds.  The -P flag will move resume a partially transferred file, so once data is moved once, it will stay on board and be updated; the -c flag will ensure the shore-side and shipboard final file(s) checksum to be the same.

while ! rsync -Pcrtz myShoreServer:/dir/to/data/some.file /local/target/dir; do echo "`date -u` Restarting rsync..."; done

If you want the data to show up, but not hog all the bandwidth while you leave this alone for hours (days?), use the --bwlimit flag.  --bwlimit is calculated in kiloBytes per second (kBps), not kilobits per second (kbps), so a value of 2 kBps per second is 16 kbps.

while ! rsync --bwlimit=2 -Pcrtz myShoreServer:/dir/to/data/some.file /local/target/dir; do echo "`date -u` Restarting rsync..."; done
  • No labels