This shows you the differences between two versions of the page.
— |
processing_stream_-_file_transfer [2010/05/26 18:28] (current) jochen Copying remote data |
||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== File transfer ====== | ||
+ | |||
+ | ===== Motivation ===== | ||
+ | You need the files (well, it is that simple, really...) | ||
+ | |||
+ | ===== Requirements ===== | ||
+ | The files you wish to retrieve can be either on a fixed disk (USB key, external harddrive, CD, etc.) or, as more commonly occurring, on a machine accessible through the network. | ||
+ | |||
+ | In the first case, you simply need the media itself, and for network access, you need a login (and for the sake of data security, I suppose and strongly suggest this to be a secure shell, aka SSH, account). | ||
+ | |||
+ | On the remote network node, the SSH server (sshd) must be running, and the Linux/Unix/MacOSX tar command must be available. For Windows-based servers, I assume that comparable solutions can be found. | ||
+ | |||
+ | **Remember: on the local host it is of course best to have a suitable [[Processing stream - project folder structure|project folder structure]] //before// you perform the actual data import!** | ||
+ | |||
+ | ===== Usage ===== | ||
+ | If your machine also supports the tar command, the easiest way to copy data from a remote node is by | ||
+ | |||
+ | * opening a terminal window | ||
+ | * changing into the folder on your local machine where the data should be retrieved into | ||
+ | * potentially creating a new subfolder (for that particular subject/session) | ||
+ | |||
+ | Then you enter the command (please replace items in ALL CAPS with whatever you need in your setup, and leave out optional parts within square brackets--[ and ]!): | ||
+ | |||
+ | <code>ssh [-p SSH_SERVER_PORT] REMOTEUSER@REMOTEHOST "cd /FOLDER/WHERE/DATA/IS/STORED ; tar -cf - REMOTE_FOLDER_NAME [REMOTE_FOLDER_NAME2 ...]" | tar -xvf -</code> | ||
+ | |||
+ | Here's one exemplary version: | ||
+ | |||
+ | <code>cd /Volumes/CHIP_project/Imaging/Subjects | ||
+ | ssh neurodata@fmri.brainland.org "cd /import/projects_a_to_d/CHIP ; tar -cf - CHIP23_WT1956 CHIP24_RM1988" | tar -xvf -</code> | ||
+ | |||
+ | This will change into the local folder where the CHIP project is stored, then ask for the password of user neurodata on host fmri.brainland.org, and then transfer the TAR-archived stream of two subjects (CHIP23 and CHIP24) via secure channel to the local host where it is un-tarred on the fly. | ||