XNAT-App (Upload)

Slides from a presentation



To successfully run the application you need a linux or mac osx operating system, java 1.8 and the terminal application curl

To use quality control you need (enabled by Default)

To use the defacement of a head structure you need

To transform data to BIDS-Format

  • FSL

  • GDCM (linux: apt-get install libgdcm-tools | mac: brew install gdcm)

Download and install


Unzip and copy the app to a location you like. After this you can configure a variable in your ~/.bash_profile to call the java app.

export app=/Users/userbame/Desktop/xnat-app
alias xnatupload='cd $app; java -jar $app/xnat-upload.jar'
source ~/.bash_profile

Configure xnat-app

In the properties subfolder of the xnat-app folder you find different files to configure.


To define the ip / url of the xnat-server, your username and password


Mark this file as read only (chmod 400).
Otherwise it is possible for users on the system to read your password.


To successfully run the xnat-app you need to name the sequences with the appropriate project name you defined in xnat: project-name_sequence (e.g. dwm_002 ...)


Insert all the project id names which you defined in your xnat projects.


Insert all the sequence names and their types which should be parsed (this is needed for recursive parsing of the folder structure). You can name the types as you like - the app dynamically define itself (with some exception cases explained at the end of this paragraph). Don´t insert the scan number (of the sequences e.g. sequence-name_10. Don´t insert _10). Exceptions for the free type naming: dti has to be type: dti. Also all fcmri and rest data have to contain _fcMRI or _func in their filenames (name: ...).

# Example sequences.yml
type: t1
name: anat.nii.gz # nifti-name (output)
sequence: t1_mprage_sag_p2_iso_08
type: t2
name: fla.nii.gz
sequence: t2_spc_dafl_sag_p2_iso
type: fmaps
name: fieldmaps.nii.gz
sequence: gre_field_mapping

If you like to transform you data into a valid BIDS-format you need to define bids.yml.


This config-file is similar to the sequences.yml file. At the beginning of the line you can define the type (e.g. name: t1). In the next line you need to define the ending name of the output nifti/json file (e.g. name: _T1w). After this you have to name the folder where the bids data is created (e.g. folder: anat). At the last line you have to name the sequence-name of the type (e.g. sequence: t1_mprage_sag_p2_iso_0.8). All functional data has to move into a func folder. The other data you can define as you like.

# Example bids.yml
type: t1
name: _T1w
folder: anat
sequence: t1_mprage_sag_p2_iso_0.8
type: t2
name: _T2w
folder: anat
sequence: t2_spc_da-fl_sag_p2_iso
type: fcmri1
name: _task-rest_bold
folder: func
sequence: Pre_Baseline_resting_state_ep2d_bold_moco_p3 


If you have succesfully converted the dicom data into bids-format you can check with the bids-validator if the format is valid.


All dicom images must be in the folder which is named in the sequences.yml or in the bids.yml file. It is not possible to have subfolders in this folder. Be careful that there are no whitespaces at the end of each line (sequences.yml).


To sort the data for cluster computation you can link the defined type to folders you like.

# Example sort.conf


There is a file named anon.das in the properties folder of you xnat-app. By editing this file you can define which dicom tags are removed. With the dicom-browser you can examine the tags.

// remove dicom tags 
- (0010,0030)       // patient birth date
- (0010,0040)       // patient sex
- (0010,1010)       // patient age
- (0010,1020)       // patient size

Run xnat-app

Change into you xnat-app folder (cd /path/xnat-app)

xnatupload -i input-folder -o output-folder
xnatupload -h # help

If you don´t like to configure a variable you can also run the program on this way:

java -jar xnat-upload.jar -i path_to_input_folder -o path_to_output_folder


The input folder needs the following structure: DWM_HC_ ... / subfolder / sequences. Project name (e.g. DWM_HC_m_013_20164_t2): The first letters mark the project name. Subfolder: Any name. Sequences: Field type(s): Sequence name(s) of your project - defined in sequences.yml.


If following error message appears: "Exception in thread "main" org.apache.commons.io.FileExistsException: Destination '/folder' already exists" you have to empty your output folder first.


If the xnat-app throws an unexpected Error: One reason could be that you have compressed dicom images. GDCM helps you by decompressing the images: gdcmconv (linux: apt-get install libgdcm-tools | mac: brew install gdcm)

find . -name "*.dcm" -exec gdcmconv -V --raw {} {} \; # decompress images

xnatupload [parameters]

-b (--bids) -ba (--bids-anonymized) -c (--copy) -d (--decompress) -w (--delay)
Convert to BIDS Convert to BIDS (anonymized) Copy dicom images Decompress dicom images Upload delay (niftis)
-fm (--face-masking) -i (--input) -o (--output) -qc (--quality-contro) -s (--sort) -t1 (--t1iso)
Nifti defacement Input folder Output folder SNR / FoV quality control Sort niftis (Cluster) Change iso value
-u (--upload) -un (--upload-nifti)
Upload all Upload only niftis

with parameter -b

To transform the data into a bids valid structure. The will be no upload to the xnat server. Before you can convert the data all Dicom Images has to be copied into the input folder (without any folder structure).

# copy dicom images to input folder example
find SIMNFB_ACTIVE_001_m_t1 -name "*.dcm" -exec cp {} ../input/sub-simnfb001/ \;
xnatupload -i ../input -o ../output -b true

with parameter -ba

The same as with parameter -b before but with anonymized dicom meta data.

xnatupload -i ../input -o ../output -ba true

with parameter -c

To copy the dicom images into the output folder (enabled by default)

xnatupload -i ../input -o ../output -c false

with parameter -d

To decompress comressed dicom images (disabled by default)

xnatupload -i ../input -o ../output -d true

with parameter -w

For the delay in minutes until the niftis are uploaded (it´s needed because the xnat-server needs time to build the subject(s). 15 minutes are set by default)

xnatupload -i ../input -o ../output -w 5

with parameter -fm

For an defacement of the head structure (disabled by Default)

xnatupload -i ../input -o ../output -fm true

with parameter -i

For the input folder with dicom images (needed to run)

xnatupload -i ../input -o ../output 

with parameter -o

For the output folder (needed to run)

xnatupload -i ../input -o ../output

with parameter -qc

To create a qualitiy control (field of view and signal to noise ratio. Enabled by default)

xnatupload -i ../input -o ../output -qc true 
xnatupload -i ../input -o ../output # it´s the same, because qc is enabled by default 

with parameter -s

To sort the output for the processing in the cluster. The files in the output folder get sorted. Nothing else is done.

xnatupload -i ../input -o ../output -s true

with parameter -t1

To change the the value of the 11-iso (default: 0.8)

xnatupload -i ../input -o ../output -t1 1.0

with parameter -u

To upload the dicom images and the created niftis to the xnat-server (enabled by default)

xnatupload -i ../input -o ../output -u false

with parameter -un

To upload only the created niftis to the xnat-server (disabled by default)

xnatupload -i ../input -o ../output -un true

Upload script (nifti, eeg, ...)

If you need to upload already generated data you can use the upload script 'nifti-upload.sh' in the scripts folder of the xnat-app. For the defacement you need to install fsl, nibabel, nipype, pydeface.

./niftis_upload.sh [input folder] [xnat project id] [defacement] [resource type] 
[input folder] [nat project id] [defacement] [rsesource type]
Full Path to: Folders with the subject names and niftis Id of the xnat project true / false NIFTI, QC, EEG, SURVEY


The structure of the input folder needs the exact subject name with all the niftis you like to upload: subject / [niftiname1.nii.gz, niftiname2.nii.gz]