Pre-processing TIGER K01 data

From Wikipedia, the free encyclopedia

This article is about the writing implement. For other uses, see Pencil (disambiguation).

Overview

General subject folder structure: ~/project-tiffany/projects/TIGER-K01/raw/sub-XXX

‘Raw’ Subject Data Sub-Folders

• EJA_SVS_SLASER_0005: Contains DICOM files for a spectroscopy scan, possibly using the Siemens SVS/SLASER sequence.
• EP2D_BOLD_1_6MM_ISO_P2_SMS5_HCP_PA_CMRR_0014: Contains DICOM files for a BOLD (blood oxygen level dependent) fMRI scan using the Siemens HCP (Human Connectome Project) sequence.
• EP2D_BOLD_1_6MM_ISO_P2_SMS5_HCP_PA_CMRR_SBREF_0013: Contains DICOM files for a BOLD fMRI scan using the Siemens HCP sequence, for a single-shot gradient-echo echo-planar imaging (GE-EPI) acquisition.
• EP2D_BOLD_1_6MM_ISO_P2_SMS5_HCP_PA_CMRR_SBREF_0015: Contains DICOM files for a BOLD fMRI scan using the Siemens HCP sequence, for a single-shot GE-EPI acquisition, for the reference scan.
• LOCALIZER_0007: Contains DICOM files for a localizer scan, which is typically a low-resolution image used for positioning subsequent scans.
• SPINECHOFIELDMAP_AP_CMRR_0010: Contains DICOM files for a field map scan in the anterior-posterior (AP) direction, possibly for use in distortion correction of subsequent scans.
• SPINECHOFIELDMAP_AP_CMRR_SBREF_0009: Contains DICOM files for a field map reference scan in the anterior-posterior (AP) direction.
• SPINECHOFIELDMAP_PA_CMRR_0012: Contains DICOM files for a field map scan in the posterior-anterior (PA) direction, possibly for use in distortion correction of subsequent scans.
• SPINECHOFIELDMAP_PA_CMRR_SBREF_0011: Contains DICOM files for a field map reference scan in the posterior-anterior (PA) direction.
• SUB_S4-S3_1_0102: Contains DICOM files for a T1-weighted anatomical scan.
• SVS_SLASER_DKD_RACC_WS_64AV_0006: Contains DICOM files for a spectroscopy scan, possibly using the Siemens SVS/SLASER sequence.
• T1_MP2RAGE_SAG_P3_1MM_INV2_0003: Contains DICOM files for a T1-weighted anatomical scan, possibly using the Siemens MP2RAGE sequence.
• _MPR_RANGE[1]__0101: Contains DICOM files for an anatomical scan in the axial plane, possibly using the Siemens MPRAGE sequence.
• _MPR_RANGE__0100: Contains DICOM files for an anatomical scan in the axial plane, possibly using the Siemens MPRAGE sequence.

Batch DICOM to NIFTI Conversion

Navigate to the directory:

cd ~/project-tiffany/projects/TIGER_K01/

Then, run the following command:

python batch_dcm2niix.py

That’s it!

Python Script: batch_dcm2niix.py

import os
import subprocess

# Set the root directory for the study
root_dir = os.getcwd()

# Find all subject directories in the raw data folder
subj_dirs = [d for d in os.listdir(os.path.join(root_dir, 'raw')) if d.startswith('sub-')]

# Loop through each subject directory
for subj_dir in subj_dirs:
    subj_id = subj_dir.split('-')[1]
    print(f"Processing subject {subj_id}...")
    
    func_input_dir = os.path.join(root_dir, 'raw', subj_dir)
    func_output_dir = os.path.join(root_dir, 'bids', subj_dir, 'func')

    func_dirs = [d for d in os.listdir(func_input_dir) if 'EP2D_BOLD' in d and not d.endswith('SBREF')]
    if len(func_dirs) == 0:
        print(f"No functional image directory found for subject {subj_id}!")
        continue
    func_dir = os.path.join(func_input_dir, func_dirs[0])
    print(f"Functional image directory found: {func_dir}")

    struct_input_dir = os.path.join(func_input_dir, [d for d in os.listdir(func_input_dir) if 'SUB' in d][0])
    struct_output_dir = os.path.join(root_dir, 'bids', subj_dir, 'anat')

    cmd = f"dcm2niix -b y -ba y -z y -o {func_output_dir} -f sub-{subj_id}_task-rest_bold {func_dir}"
    subprocess.call(cmd, shell=True)

    if not os.path.exists(struct_input_dir):
        print(f"No structural image directory found for subject {subj_id}!")
        continue
    struct_output_file = f"sub-{subj_id}_T1w"
    if not os.path.exists(struct_output_dir):
        os.makedirs(struct_output_dir)
    cmd = f"dcm2niix -b y -ba y -z y -o {struct_output_dir} -f {struct_output_file} {struct_input_dir}"
    subprocess.call(cmd, shell=True)
    print(f"Data conversion complete for subject {subj_id}!")

Automating Script Execution with Crontab

Crontab allows the time-scheduled execution of scripts and commands. The following crontab entry runs the batch conversion daily at 4 AM PST:

0 4 * * * cd ~/project-tiffany/projects/TIGER_K01 && python batch_dcm2niix.py >> batch_dcm2niix.log 2>&1

Processing Structural Data: process_struct.sh

#!/bin/bash

root_dir="/u/project/tiffany/psychmik/projects/TIGER_K01/raw"
folders=("TIGER-T1" "TIGER-T2" "TIGER-T3")

process_data() {
    input_path="$1"
    output_path="$2"
    echo "Processing ${input_path}..."

    mkdir -p "${output_path}"

    bet "${input_path}" "${output_path}/DTI_2mm_b2000_60dir_raw_bet.nii.gz"
    N4BiasFieldCorrection -i "${input_path}" -o "${output_path}/DTI_2mm_b2000_60dir_raw_n4.nii.gz"
    fast -t 1 -n 3 -g -o "${output_path}/segmentation" "${output_path}/DTI_2mm_b2000_60dir_raw_n4.nii.gz"
    fslmaths "${output_path}/segmentation_seg_0.nii.gz" -bin "${output_path}/binary_mask.nii.gz"
    echo "Finished processing ${input_path}."
}

for folder in "${folders[@]}"; do
    folder_path="${root_dir}/${folder}"
    subject_dirs=$(find "${folder_path}" -mindepth 1 -maxdepth 1 -type d)

    for subject_dir in $subject_dirs; do
        subject_name=$(basename "${subject_dir}")
        input_file=$(find "${subject_dir}" -name "DTI*.nii.gz" -type f)
        output_path="${folder_path}/processed/${subject_name}"

        if [ ! -f "${output_path}/binary_mask.nii.gz" ]; then
            if [ -f "${input_file}" ]; then
                process_data "${input_file}" "${output_path}"
            else
                echo "Skipping ${subject_name} because the input file does not exist."
            fi
        else
            echo "Skipping ${subject_name} because the output already exists."
        fi
    done
done

Processing Functional Data: process_func.sh

#!/bin/bash

root_dir="/u/project/tiffany/psychmik/projects/TIGER_K01/raw"
folders=("TIGER-T1" "TIGER-T2" "TIGER-T3")
smoothing_kernel="5"
lowpass_cutoff="0.08"
highpass_cutoff="0.01"

process_func_data() {
    input_path="$1"
    output_path="$2"
    echo "Processing ${input_path}..."

    mkdir -p "${output_path}"
    mcflirt -in "${input_path}" -out "${output_path}/func_data_mcf.nii.gz"
    slicetimer -i "${output_path}/func_data_mcf.nii.gz" -o "${output_path}/func_data_stc.nii.gz" -r TR
    fslmaths "${output_path}/func_data_stc.nii.gz" -s "${smoothing_kernel}" "${output_path}/func_data_smooth.nii.gz"
    fslmaths "${output_path}/func_data_smooth.nii.gz" -bptf "${highpass_cutoff}" "${lowpass_cutoff}" "${output_path}/func_data_filtered.nii.gz"
    echo "Finished processing ${input_path}."
}

for folder in "${folders[@]}"; do
    folder_path="${root_dir}/${folder}"
    subject_dirs=$(find "${folder_path}" -mindepth 1 -maxdepth 1 -type d)

    for subject_dir in $subject_dirs; do
        subject_name=$(basename "${subject_dir}")
        input_file=$(find "${subject_dir}" -name "func*.nii.gz" -type f)
        output_path="${folder_path}/processed_func/${subject_name}"

        if [ ! -f "${output_path}/func_data_filtered.nii.gz" ]; then
            process_func_data "${input_file}" "${output_path}"
        else
            echo "Skipping ${subject_name} because the output already exists."
        fi
    done
done