User Tools

Site Tools


processing_stream_-_file_transfer

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

processing_stream_-_file_transfer [2010/05/26 18:28] (current)
jochen Copying remote data
Line 1: Line 1:
 +====== File transfer ======
 +
 +===== Motivation =====
 +You need the files (well, it is that simple, really...)
 +
 +===== Requirements =====
 +The files you wish to retrieve can be either on a fixed disk (USB key, external harddrive, CD, etc.) or, as more commonly occurring, on a machine accessible through the network.
 +
 +In the first case, you simply need the media itself, and for network access, you need a login (and for the sake of data security, I suppose and strongly suggest this to be a secure shell, aka SSH, account).
 +
 +On the remote network node, the SSH server (sshd) must be running, and the Linux/​Unix/​MacOSX tar command must be available. For Windows-based servers, I assume that comparable solutions can be found.
 +
 +**Remember: on the local host it is of course best to have a suitable [[Processing stream - project folder structure|project folder structure]] //before// you perform the actual data import!**
 +
 +===== Usage =====
 +If your machine also supports the tar command, the easiest way to copy data from a remote node is by
 +
 +  * opening a terminal window
 +  * changing into the folder on your local machine where the data should be retrieved into
 +  * potentially creating a new subfolder (for that particular subject/​session)
 +
 +Then you enter the command (please replace items in ALL CAPS with whatever you need in your setup, and leave out optional parts within square brackets--[ and ]!):
 +
 +<​code>​ssh [-p SSH_SERVER_PORT] REMOTEUSER@REMOTEHOST "cd /​FOLDER/​WHERE/​DATA/​IS/​STORED ; tar -cf - REMOTE_FOLDER_NAME [REMOTE_FOLDER_NAME2 ...]" | tar -xvf -</​code>​
 +
 +Here's one exemplary version:
 +
 +<​code>​cd /​Volumes/​CHIP_project/​Imaging/​Subjects
 +ssh neurodata@fmri.brainland.org "cd /​import/​projects_a_to_d/​CHIP ; tar -cf - CHIP23_WT1956 CHIP24_RM1988"​ | tar -xvf -</​code>​
 +
 +This will change into the local folder where the CHIP project is stored, then ask for the password of user neurodata on host fmri.brainland.org,​ and then transfer the TAR-archived stream of two subjects (CHIP23 and CHIP24) via secure channel to the local host where it is un-tarred on the fly.
  
processing_stream_-_file_transfer.txt ยท Last modified: 2010/05/26 18:28 by jochen