XNAT-App (Upload)

Slides from a presentation

Prerequisites

Note

To successfully run the application you need a linux or mac osx operating system, java 1.8 and the terminal application curl

To use quality control you need (enabled by Default)

To use the defacement of a head structure you need

To transform data to BIDS-Format

  • FSL

  • GDCM (linux: apt-get install libgdcm-tools | mac: brew install gdcm)

Download and install

xnat-app

Unzip and copy the app to a location you like. After this you can configure a variable in your ~/.bash_profile to call the java app.

export app=/Users/userbame/Desktop/xnat-app
alias xnatupload='cd $app; java -jar $app/xnat-upload.jar'
source ~/.bash_profile

Configure xnat-app

In the properties subfolder of the xnat-app folder you find different files to configure.

Note

config.properties
To define the ip / url of the xnat-server, your username and password

Danger

Mark this file as read only (chmod 400).
Otherwise it is possible for users on the system to read your password.

xnat.server.address=http://148.***.***.**
xnat.scp.receiver=dicom://xnat@148.***.***.**:8081/XNAT
xnat.username=admin
xnat.password=**********

To successfully run the xnat-app you need to name the sequences with the appropriate project name you defined in xnat: project-name_sequence (e.g. dwm_002 ...)

Note

studynames.conf
Insert all the project id names which you defined in your xnat projects.

Note

sequences.yml
Insert all the sequence names and their types which should be parsed (this is needed for recursive parsing of the folder structure). You can name the types as you like - the app dynamically define itself (with some exception cases explained at the end of this paragraph). Don´t insert the scan number (of the sequences e.g. sequence-name_10. Don´t insert _10). Exceptions for the free type naming: dti has to be type: dti. Also all fcmri and rest data have to contain _fcMRI or _func in their filenames (name: ...).

# Example sequences.yml
type: t1
name: anat.nii.gz # nifti-name (output)
sequence: t1_mprage_sag_p2_iso_08
---
type: t2
name: fla.nii.gz
sequence: t2_spc_dafl_sag_p2_iso
---
type: fmaps
name: fieldmaps.nii.gz
sequence: gre_field_mapping

If you like to transform you data into a valid BIDS-format you need to define bids.yml.

Note

bids.yml
This config-file is similar to the sequences.yml file. At the beginning of the line you can define the type (e.g. name: t1). In the next line you need to define the ending name of the output nifti/json file (e.g. name: _T1w). After this you have to name the folder where the bids data is created (e.g. folder: anat). At the last line you have to name the sequence-name of the type (e.g. sequence: t1_mprage_sag_p2_iso_0.8). All functional data has to move into a func folder. The other data you can define as you like.

# Example bids.yml
type: t1
name: _T1w
folder: anat
sequence: t1_mprage_sag_p2_iso_0.8
---
type: t2
name: _T2w
folder: anat
sequence: t2_spc_da-fl_sag_p2_iso
---
type: fcmri1
name: _task-rest_bold
folder: func
sequence: Pre_Baseline_resting_state_ep2d_bold_moco_p3 

Hint

If you have succesfully converted the dicom data into bids-format you can check with the bids-validator if the format is valid.

Danger

All dicom images must be in the folder which is named in the sequences.yml or in the bids.yml file. It is not possible to have subfolders in this folder. Be careful that there are no whitespaces at the end of each line (sequences.yml).

Note

sort.conf
To sort the data for cluster computation you can link the defined type to folders you like.

# Example sort.conf
t1=anat
t2=flair
fmaps=fieldmaps

Anonymization

There is a file named anon.das in the properties folder of you xnat-app. By editing this file you can define which dicom tags are removed. With the dicom-browser you can examine the tags.

// remove dicom tags 
- (0010,0030)       // patient birth date
- (0010,0040)       // patient sex
- (0010,1010)       // patient age
- (0010,1020)       // patient size

Run xnat-app

Change into you xnat-app folder (cd /path/xnat-app)

xnatupload -i input-folder -o output-folder
xnatupload -h # help

If you don´t like to configure a variable you can also run the program on this way:

java -jar xnat-upload.jar -i path_to_input_folder -o path_to_output_folder

Warning

The input folder needs the following structure: DWM_HC_ ... / subfolder / sequences. Project name (e.g. DWM_HC_m_013_20164_t2): The first letters mark the project name. Subfolder: Any name. Sequences: Field type(s): Sequence name(s) of your project - defined in sequences.yml.

Hint

If following error message appears: "Exception in thread "main" org.apache.commons.io.FileExistsException: Destination '/folder' already exists" you have to empty your output folder first.

Success

If the xnat-app throws an unexpected Error: One reason could be that you have compressed dicom images. GDCM helps you by decompressing the images: gdcmconv (linux: apt-get install libgdcm-tools | mac: brew install gdcm)

find . -name "*.dcm" -exec gdcmconv -V --raw {} {} \; # decompress images

xnatupload [parameters]

-b (--bids) -ba (--bids-anonymized) -c (--copy) -d (--decompress) -w (--delay)
Convert to BIDS Convert to BIDS (anonymized) Copy dicom images Decompress dicom images Upload delay (niftis)
-fm (--face-masking) -i (--input) -o (--output) -qc (--quality-contro) -s (--sort) -t1 (--t1iso)
Nifti defacement Input folder Output folder SNR / FoV quality control Sort niftis (Cluster) Change iso value
-u (--upload) -un (--upload-nifti)
Upload all Upload only niftis

with parameter -b

To transform the data into a bids valid structure. The will be no upload to the xnat server. Before you can convert the data all Dicom Images has to be copied into the input folder (without any folder structure).

# copy dicom images to input folder example
find SIMNFB_ACTIVE_001_m_t1 -name "*.dcm" -exec cp {} ../input/sub-simnfb001/ \;
xnatupload -i ../input -o ../output -b true

with parameter -ba

The same as with parameter -b before but with anonymized dicom meta data.

xnatupload -i ../input -o ../output -ba true

with parameter -c

To copy the dicom images into the output folder (enabled by default)

xnatupload -i ../input -o ../output -c false

with parameter -d

To decompress comressed dicom images (disabled by default)

xnatupload -i ../input -o ../output -d true

with parameter -w

For the delay in minutes until the niftis are uploaded (it´s needed because the xnat-server needs time to build the subject(s). 15 minutes are set by default)

xnatupload -i ../input -o ../output -w 5

with parameter -fm

For an defacement of the head structure (disabled by Default)

xnatupload -i ../input -o ../output -fm true

with parameter -i

For the input folder with dicom images (needed to run)

xnatupload -i ../input -o ../output 

with parameter -o

For the output folder (needed to run)

xnatupload -i ../input -o ../output

with parameter -qc

To create a qualitiy control (field of view and signal to noise ratio. Enabled by default)

xnatupload -i ../input -o ../output -qc true 
xnatupload -i ../input -o ../output # it´s the same, because qc is enabled by default 

with parameter -s

To sort the output for the processing in the cluster. The files in the output folder get sorted. Nothing else is done.

xnatupload -i ../input -o ../output -s true

with parameter -t1

To change the the value of the 11-iso (default: 0.8)

xnatupload -i ../input -o ../output -t1 1.0

with parameter -u

To upload the dicom images and the created niftis to the xnat-server (enabled by default)

xnatupload -i ../input -o ../output -u false

with parameter -un

To upload only the created niftis to the xnat-server (disabled by default)

xnatupload -i ../input -o ../output -un true

Upload script (nifti, eeg, ...)

If you need to upload already generated data you can use the upload script 'nifti-upload.sh' in the scripts folder of the xnat-app. For the defacement you need to install fsl, nibabel, nipype, pydeface.

./niftis_upload.sh [input folder] [xnat project id] [defacement] [resource type] 
[input folder] [nat project id] [defacement] [rsesource type]
Full Path to: Folders with the subject names and niftis Id of the xnat project true / false NIFTI, QC, EEG, SURVEY

Important

The structure of the input folder needs the exact subject name with all the niftis you like to upload: subject / [niftiname1.nii.gz, niftiname2.nii.gz]