While for educational purposes it is quite useful to perform steps more manually, on a daily basis it is much more convenient (and less error prone) to have one or several scripts do the “work” and only take care of their configuration.

For this example, the following programs must be installed and information available:

• bash, find, mkdir, mv, rsync, ssh, tar (come with most OSes that have a terminal or shell)
• Matlab v7.0 or higher, SPM5/SPM8, and NeuroElf v0.9a or higher
• location of subject data at the scanning facility (server)
• local folder and knowledge about desired structure
• onsets in a tabular structure where one column codes the subject and negative differences between onset times code the run separation (for an example, see below)

As part of this scripted approach requires the shell, there are multiple scripts. Part of the data handling is performed by shell commands (data retrieval, archiving) whereas the processing is implemented via SPM's job manager. The following steps are covered:

• data import (from a remote SSH-accessible server)
• correct renaming and creation of folders (into existing study structure)
• archiving (possibly configured as a regularly executed job via crontab)
• fMRI quality assessment (per run)
• preprocessing (SPM5 or SPM8)
• conversion into BrainVoyager QX's VTC format
• semi-automatic logfile parsing (PRT generation)
• computing a RFX-GLM of all available subjects (upon request)

SCANUnit_import_subject.bash

#!/bin/bash
 
# settings ; PLEASE ADAPT
LOCAL_FOLDER=/Users/XXX/Desktop/CPU
REMOTE_FOLDER=/export/data/scanner/a_to_c/CPU
REMOTE_HOST=host.domain.columbia.edu
REMOTE_USER=xxx
 
# ensure there are arguments
if [[ "a$@" == "a" ]] ; then
    echo "No subjects requested."
    echo "USAGE (import one subject): $0 subject1"
    echo "USAGE (import several subjects): $0 subject1 subject2 subject3 ..."
    echo "USAGE (import several subjects, pattern based): $0 subj\*patt"
    echo "---"
    echo "This script uses ssh. You will be asked to enter"
    echo "the password of user $REMOTE_USER on host $REMOTE_HOST."
    echo "---"
    echo "Data will be stored in $LOCAL_FOLDER."
    exit 1;
  fi
 
# change into local folder
cd $LOCAL_FOLDER
 
# retrieve requested files
ssh -l $REMOTE_USER REMOTE_HOST "cd $REMOTE_FOLDER ; tar -cf - $@" | tar -xf -

SCANUnit_subject_foldernames.bash

#!/bin/bash
 
# settings ; PLEASE ADAPT
LOCAL_FOLDER=/Users/XXX/Desktop/CPU
SUBJ_PATTERN=CPU*
SUBFOLDERS="functional onsets structural"
RAWFOLDER=raw
 
# Note: the SUBFOLDERS variable can be easily adapted to create
# several levels of folders:
# SUBFOLDERS="functional functional/raw functional/prepro ..."
 
# for every folder found (matching the pattern)
find $LOCAL_FOLDER -type d -mindepth 1 -maxdepth 1 -name "$SUBJ_PATTERN" | while read subject_folder ; do
 
    # make sure all subfolders we need exist
    for sub in $SUBFOLDERS ; do
        mkdir $subject_folder/$sub 2>/dev/null
      done
 
    # rename original folder, if existent, to raw
    find $subject_folder -type d -mindepth 1 -maxdepth 1 -name "$SUBJ_PATTERN" -exec mv {} $RAWFOLDER \;
  done

For this, there is really no need for a script, simply let rsync do the work:

rsync_example.sh

rsync -aut /Users/XXX/Desktop/CPU /Volumes/CPU/ImagingData

This can be easily added as a line to crontab (callable via crontab -e):

rsync_crontab_snip.txt

30 2 * * * /usr/bin/rsync -aut /Users/XXX/Desktop/CPU /Volumes/CPU/ImagingData

Note: On MacOSX to ensure that the backuping volume is always available, add it to the user's startup items list in System Preferences, and store the password to that network volume in your keychain!

This part of the scripted process is already covered at its own, proper page. See image file conversion for more information.

Using the spm5_preprojobs function, we can easily setup a small script that preprocesses all (remaining) subjects:

SCANUnit_CPU_prepro.m

% folder location
subjfolder = '/Users/Desktop/XXX/CPU';
subjpattern = 'CPU*';
funcfolder = 'functional';
anatfolder = 'struct';
 
% settings (for spm5_preprojobs)
funcpattern = 'r*340';
anatpattern = '';
usestruct = true;
runjobs = true;
disdaqs = 5;
smoothmm = 6;
sliceorder = 'aio';
tr = 2;
 
% complete settings
if ~isempty(funcpattern)
    ffp = [funcfolder '/' funcpattern];
else
    ffp = funcfolder;
end
if ~isempty(anatpattern)
    anatpattern = [anatfolder '/' anatpattern];
else
    anatpattern = anatfolder;
end
ppopts = struct( ...
    'fun2str', usestruct, ...
    'jobrun', runjobs, ...
    'skip', disdaqs, ...
    'smk', smoothmm, ...
    'sto', sliceorder, ...
    'sttr', tr);
 
% find subjects
subjects = findfiles(subjfolder, subjpattern, 'depth=1', 'dirs=1');
 
% for each subject
for sc = 1:numel(subjects)
 
    % determine whether the subject needs processing
    rps = findfiles([subjects{sc} '/' funcfolder], 'rp*.txt');
    if ~isempty(rps)
        continue;
    end
 
    % get subject ID
    [basefld, subjid] = fileparts(subjects{sc});
    subjid(subjid == '_') = [];
 
    % run fMRI quality on all runs
    runs = findfiles([subjects{sc} '/' funcfolder], funcpattern, 'dirs=1', 'depth=1');
    for rc = 1:numel(runs)
        q = fmriquality(findfiles(runs{rc}, {'*.img', '*.nii'}));
        save(sprintf('%s/fmriquality.mat', runs{rc}), 'q');
        clear q;
    end
 
    % run preprocessing
    [j, jh, ppfiles] = spm5_preprojobs(subjects{sc}, ffp, anatpattern, ppopts);
 
    % for each run, create one VTC
    for rc = 1:numel(ppfiles)
        [basefld, runid] = fileparts(fileparts(ppfiles{rc}{1}));
        vtc = importvtcfromanalyze(ppfiles{rc});
        vtc.TR = round(1000 * tr);
        vtc.SaveAs(sprintf('%s/%s/%s_%s.vtc', subjects{sc}, funcfolder, subjid, runid);
        vtc.ClearObject;
    end
end