HiFi DeepVariant + WhatsHap workflowHiFi DeepVariant + WhatsHap workflow

Last update: May 14, 2022

Overview

HiFi DeepVariant + WhatsHap workflow

Workflow steps

align HiFi reads to reference with pbmm2
call small variants with DeepVariant, using two-pass method (DeepVariant ➡️ WhatsHap phase ➡️ WhatsHap haplotag ➡️ DeepVariant)
phase small variants with WhatsHap
haplotag aligned BAMs with WhatsHap and merge

Directory structure within basedir

.
├── cluster_logs  # slurm stderr/stdout logs
├── reference
│   ├── reference.chr_lengths.txt  # cut -f1,2 reference.fasta > reference.chr_lengths.txt
│   ├── reference.fasta
│   └── reference.fasta.fai
├── samples
│   └── 
   
      # sample_id regex: r'[A-Za-z0-9_-]+'
│       ├── whatshap/  # phased small variants; merged haplotagged alignments
│       ├── logs/  # per-rule stdout/stderr logs
│       ├── aligned/  # intermediate
│       ├── deepvariant/  # intermediate
│       ├── deepvariant_intermediate/  # intermediate
│       └── whatshap_intermediate/  # intermediate
├── smrtcells
│   ├── done  # move folders from smrtcells/ready to smrtcells/done to prevent re-processing
│   └── ready
│       └── 
    
       # uBAMs or FASTQs per sample
│                        # filename regex: r'm\d{5}[Ue]?_\d{6}_\d{6}).(ccs|hifi_reads).bam' or r'm\d{5}[Ue]?_\d{6}_\d{6}).fastq.gz'
└── workflow  # clone of this repo

To run the pipeline

$ conda create \
    --channel bioconda \
    --channel conda-forge \
    --prefix ./conda_env \
    python=3 snakemake mamba lockfile

$ conda activate ./conda_env

$ sbatch workflow/run_snakemake.sh <sample_id>

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

Couler What is Couler? Couler aims to provide a unified interface for constructing and managing workflows on different workflow engines, such as Argo

781 Jan 3, 2023

Reusable workflow library for Django

django-viewflow Viewflow is a lightweight reusable workflow library that helps to organize people collaboration business logic in django applications.

2.3k Jan 8, 2023

Python Development Workflow for Humans.

Pipenv: Python Development Workflow for Humans [ ~ Dependency Scanning by PyUp.io ~ ] Pipenv is a tool that aims to bring the best of all packaging wo

23.5k Jan 1, 2023

An audio digital processing toolbox based on a workflow/pipeline principle

AudioTK Audio ToolKit is a set of audio filters. It helps assembling workflows for specific audio processing workloads. The audio workflow is split in

238 Oct 18, 2022

PyWorkflow(PyWF) - A Python Binding of C++ Workflow

PyWorkflow(PyWF) - A Python Binding of C++ Workflow 概览 C++ Workflow是一个高性能的异步引擎，本项目着力于实现一个Python版的Workflow，让Python用户也能享受Workflow带来的绝佳体验。

108 Dec 1, 2022

Git Plan - a better workflow for git

git plan A better workflow for git. Git plan inverts the git workflow so that you can write your commit message first, before you start writing code.

178 Dec 11, 2022

Python Development Workflow for Humans.

Pipenv: Python Development Workflow for Humans [ ~ Dependency Scanning by PyUp.io ~ ] Pipenv is a tool that aims to bring the best of all packaging wo

23.5k Jan 6, 2023

StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, security responses, troubleshooting, deployments, and more. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html. Questions? https://forum.stackstorm.com/.

StackStorm is a platform for integration and automation across services and tools, taking actions in response to events. Learn more at www.stackstorm.

5.3k Jan 2, 2023

Make tree planting a part of your daily workflow. 🌳

Continuous Reforestation Make tree planting a part of your daily workflow. 🌳 A GitHub Action for planting trees within your development workflow usin

168 Dec 22, 2022

This repository contains free labs for setting up an entire workflow and DevOps environment from a real-world perspective in AWS

DevOps-The-Hard-Way-AWS This tutorial contains a full, real-world solution for setting up an environment that is using DevOps technologies and practic

1.6k Jan 5, 2023

LOC-FLOW is an “hands-free” earthquake location workflow to process continuous seismic records

LOC-FLOW is an “hands-free” earthquake location workflow to process continuous seismic records: from raw waveforms to well located earthquakes with magnitude calculations. The package assembles several popular routines for sequential earthquake location refinements, suitable for catalog building ranging from local to regional scales.

71 Jan 9, 2023

An adaptable Snakemake workflow which uses GATKs best practice recommendations to perform germline mutation calling starting with BAM files

Germline Mutation Calling This Snakemake workflow follows the GATK best-practice recommandations to call small germline variants. The pipeline require

12 Dec 24, 2022

A project based example of Data pipelines, ML workflow management, API endpoints and Monitoring.

Comments

Add multithreaded whatshap haplotag output compression

In whatshap v1.3, an option was introduced to haplotag to allow additional threads for output compression (link). This PR updates to the most recent whatshap and provides 4 threads for haplotag.

opened by williamrowell 0

HiFi DeepVariant + WhatsHap workflowHiFi DeepVariant + WhatsHap workflow

Related tags

Overview

HiFi DeepVariant + WhatsHap workflow

Workflow steps

Directory structure within basedir

To run the pipeline

You might also like...

Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.

Reusable workflow library for Django

Python Development Workflow for Humans.

An audio digital processing toolbox based on a workflow/pipeline principle

PyWorkflow(PyWF) - A Python Binding of C++ Workflow

Git Plan - a better workflow for git

Python Development Workflow for Humans.

Make tree planting a part of your daily workflow. 🌳

This repository contains free labs for setting up an entire workflow and DevOps environment from a real-world perspective in AWS

LOC-FLOW is an “hands-free” earthquake location workflow to process continuous seismic records

An adaptable Snakemake workflow which uses GATKs best practice recommendations to perform germline mutation calling starting with BAM files

A project based example of Data pipelines, ML workflow management, API endpoints and Monitoring.

A frontend to ease the use of pulseaudio's routing capabilities, mimicking voicemeeter's workflow

Teaching end to end workflow of deep learning

A python based app to improve your presentation workflow

A Snakemake workflow for standardised sc/snRNAseq analysis

redun aims to be a more expressive and efficient workflow framework

SaltConf21: Adding Workflow Approval to Salt

Comments

Add multithreaded whatshap haplotag output compression

Owner

William Rowell

Unofficial Parallel WaveGAN (+ MelGAN & Multi-band MelGAN & HiFi-GAN & StyleMelGAN) with Pytorch

HiFi-GAN: High Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks

HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

Implementation of TTS with combination of Tacotron2 and HiFi-GAN

Combine Tacotron2 and Hifi GAN to generate speech from text

Unofficial Parallel WaveGAN (+ MelGAN & Multi-band MelGAN & HiFi-GAN & StyleMelGAN) with Pytorch

HiFi++: a Unified Framework for Neural Vocoding, Bandwidth Extension and Speech Enhancement

Unofficial implementation of HiFi-GAN+ from the paper "Bandwidth Extension is All You Need" by Su, et al.

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

A powerful workflow engine implemented in pure Python