GenoTools is a suite of automated genotype data processing steps written in Python. The core pipeline was built for Quality Control and Ancestry estimation of data in the Global Parkinson's Genetics Program (GP2)

Setup just requires:

git clone https://github.com/dvitale199/GenoTools
cd GenoTools
pip install .

The core pipeline can be called as:

python3 run_qc_pipeline.py --geno <genotype file to be QC'd (plink format)> --ref <genotype file of reference panel (plink format)> --ref_labels <labels for reference panel ancestry> --out <path and prefix for output>

--geno: Path to genotype to be processed in Plink (.bed/.bim/.fam) format. Include everything up to .bed/.bim/.fam. MUST HAVE PHENOTYPES OR SEVERAL STEPS WILL FAIL

--ref: Path to reference panel genotype in Plink (.bed/.bim/.fam) format. Include everything up to .bed/.bim/.fam. For GP2, we use a combination of 1kGenomes + Ashkenazi Jewish Reference panel

--ref_labels: Path to a tab-separated (Plink-style) file containing ancestry labels for the reference panel with the following columns: FID IID label with NO HEADER.

--out: Path and prefix to output QC'd and ancestry-predicted genotype in Plink (.bed/.bim/.fam) format. Include everything up to .bed/.bim/.fam. For now, this just outputs a log file but will include function to move output files soon.

The core pipeline is broken down into 3 main pieces:

1. Sample-level Quality Control
2. Ancestry Estimation
3. Variant-level Quality Control

The quality control steps have been developed in large part by: Cornelis Blauwendraat, Mike Nalls, Hirotaka Iwaki, Sara Bandres-Ciga, Mary Makarious, Ruth Chia, Frank Grenn, Hampton Leonard, Monica Diez-Fairen, Jeff Kim of the Laboratory of Neurogenetics and Center for Alzheimer's and Related Dementias at the National Institute on Aging, NIH and has been adapted into an automated Python package by Dan Vitale.

1. callrate_prune(geno_path, out_path, mind=0.02)
2. sex_prune(geno_path, out_path, check_sex=[0.25,0.75])

Genotypes can be called from .idats in parallel as follows:

iaap-cli gencall {bpm} {cluster_file} {ped_dir} -f {idat} -p -t 8

iaap-cli can be found in directory: executables/iaap-cli-linux-x64-1.1.0-sha.80d7e5b3d9c1fdfc2e99b472a90652fd3848bbc7/iaap-cli/

bpm is the Illumina manifest file (.bpm)

cluster_file is the clusterfile included with the genotypes (.etg)

ped_dir is the directory which the .ped file will be output to

-f {idat_dir} is the directory that contains .idat files. In our case, each directory contains all of the idats for a single chip.

-p means "output .ped"

-t is number of threads

More information about how the iaap-cli works is in GenoTools/executables/iaap-cli-linux-x64-1.1.0-sha.80d7e5b3d9c1fdfc2e99b472a90652fd3848bbc7 directory.

The way this is used for the GP2 pipeline can be found in the GenoTools/GP2_data_processing/GP2_shulman_processing.ipynb notebook, in which I launch a swarm job in NIH's Biowulf slurm system like so:

with open(f'{swarm_scripts_dir}/idat_to_ped.swarm', 'w') as f:
    for code in manifest.SentrixBarcode_A.unique():
        idat_to_ped_cmd = f'\
{iaap} gencall \
{bpm} \
{cluster_file} \
{ped_dir}/ \
-f {idat_dir} \
-p \
-t 8'
        f.write(f'{idat_to_ped_cmd}\n')
f.close()

In this example, I write one command per sample (individual .idat) and launch a swarm job