Back to SFB1102-Storage.
Expert Mode
This document is for advanced users, only. If you do not understand the concepts referred to here, this document is not for you.
Explanations will be kept to a necessary minimum, it is expected that you are able to fill in the gaps yourself. If you have suggestions or questions, please contact us via email to sfb-help@coli.uni-saarland.de
In this document we use a username of "sfb-1" as an example.
Access methods
There exist different access methods for the SFB1102 storage, please find a list and the corresponding server names here.
You may have a use case for WebDAV, FTPS or SMB/CIFS via VPN. This document will focus on SSH/SFTP and shortly mention rsync.
SSH/SFTP
The SFB1102 storage is acessible via SSH/SFTP which allows for many handy applications.
scp
Next to a graphical user client like Filezilla, the command line scp is the obvious application for SFTP access. Please note the path to your user/group files:
scp sfb-1@sftp.hidrive.strato.com:users/sfb-1/foo /local
and vice versa
scp /local/bar sfb-1@sftp.hidrive.strato.com:users/sfb-1
- You may want to use the -cblowfish option to speed up transfers.
sshfs
This may be the single most useful function for the expert user. Instead of manually copying your data to and from local storage, simply mount it into your file system hierarchy.
- create mount point
mkdir /local/sfb-1
- mount
sshfs -o reconnect -C -o workaround=all sfb-1@sftp.hidrive.strato.com: /local/sfb-1
- remove mount point
fusermount -u /local/sfb-1
A mount point is blocked, even if a mount is unsuccessful or if the connection terminates abnormally. In this case you may need to remove the moint point before you can try munting again.
By default your mount point is only available to the user who created it. On a multi user cluster node, however, you may wish to have other users access to the mount point, too, e.g. for accessing a group account simulataneously. To do so you will need to
- edit /etc/fuse.conf and add
user_allow_other
- mount with -o allow_other
sshfs -o reconnect -C -o workaround=all -o allow_other sfb-1@sftp.hidrive.strato.com: /local/sfb-1
Be aware that every user who can log in to the cluster node will be able to access this directory read/write! To mitigate this problem you may create an intermediate directory which is only accessible to a specific user group, like so:
mkdir -p /local/mygroupdir/sfb-1
chown :mygroup /local/mygroupdir
chmod ug+rwXs,o-rwx /local/mygroupdir
sshfs -o reconnect -C -o workaround=all -o allow_other sfb-1@sftp.hidrive.strato.com: /local/mygroupdir/sfb-1
Only users who beling to mygroup can access /local/mygroup and further access /local/mygroup/sfb-1.
Known Bugs in sshfs
The "dd" command will occasionally break the sshfs connection, which is quite odd. Simply do not use dd, or use a construct like dd | cat.
File synchronisation tools
rsync
Use like so:
rsync -rltDv -e rsh sfb-1@rsync.hidrive.strato.com:/users/sfb-1/t/ /local
- You may want to use the --delete option.
- You may want to use the --dry-run option.
- Trivia: A trailing slash at the end of the source path (e.g. "..sfb-1/t/", as in the example above) denotes all the files in the source directory. If you leave out the trailing slash (e.g. "..sfb-1/t") denotes the directory ("t") itself.
Find more documentation here.
unison (Linux)
Unison is a very handy tool to keep copies of data sets in sync accross different machines. Unison will take care of both way changes and uses an efficient transfer protocol. Unfortunately, unison is not (yet) directly supported by Strato. However, you may mount your SFB1102 storage account locally via sshfs, see above, and use unison "locally", synchronising from the mounted directory to a local one. unison should be available via the package management system of most Linux distributions.
FreeFileSync (Windows/Linux)
A free file synchronisation utility. There exists a Windows version which supports SFTP. You may also use the Linux version as a local sync via sshfs, see above, just like unison.
GoodSync (Windows/Linux/MacOS)
A commercial (29EUR) file synchronisation utility. All Platforms are supported as is SFTP.
Encryption
You may encrypt your files using Truecrypt, please see here.