Small variant workflow appendix¶
Compute resources (cpus, memory, queue)¶
Longer genes, or genes that contain a large number of variants, may require additional memory in the merge step. Merge resources are defined dynamically as
You can set these on the command line, e.g.,
--merge_cpus 4. If you encounter a resource problem for a particular gene, try using a larger value on the command line, while maintaining the recommended cpu:memory ratios (1 cpu per 16GB memory HPC; 1 cpu per 6GB memory CloudOS)
Set any Nextflow workflow parameter on the command line with
--<parameter_name> <value>. For a full list of defined parameters
nextflow config -flat -profile cluster main.nf | grep params. | grep --color=auto --color "="
Normalization and variant representation¶
The normalization and left-alignment happens in 3 steps (excerpt from workflow process
bcftools norm -m (or
--multiallelics) option "split(s) multiallelic sites into biallelic records (-) or joins biallelic sites into multiallelic records (+). An optional type string can follow which controls variant types which should be split or merged together: If only SNP records should be split or merged, specify snps; if both SNPs and indels should be merged separately into two records, specify both; if SNPs and indels should be merged into a single record, specify any." The
both option indicates "SNPs and indels should be merged separately into two records".
bcftools norm -m-both once produces duplicated variants for positions where both MNPs and SNPs are observed. Normalisation decomposes the MNP into separate SNPs that do not combine with canonical SNPs (not derived from MNPs), as they have different associated metrics (QUAL, DP, GQ, etc.). The alternative would be to have multiple entries for the same SNP, with different allele counts (AN, AC, AF).
See detailed documentation on variant duplication. (Note. The documentation describes characteristics of aggV2 in which
vt was used for the normalisation. The same considerations apply and
bcftools norm give identical output.)
Additional INFO tags¶
bcftools INFO tags are computed with the plugin
fill-tags. These per variant tags are derived from the merged single-sample VCF and combined with the VEP annotation output in the final annotated variant file.
|AN||Total number of alleles in called genotypes|
|AC||Allele count in genotypes|
|NS||Number of samples with data||The number of samples which have been included for a particular variant, and is equal to the number of samples with any variant at the given chromosomal position|
|AC_Hom||Allele counts in homozygous genotypes||The number of alleles for the given variant counted from the genotypes listed in the multi-sample VCF|
|AC_Het||Allele counts in heterozygous genotypes|
|AC_Hemi||Allele counts in hemizygous genotypes|
|MAF||Frequency of the second most common allele|
Annotated variant file columns¶
- *_variant columns are
- *_annotation columns are
- *_sample columns derived from filtering on bi-allelic genotype
|(...)||VEP annotation output|
|Het_samples||samples from output VCF selected with
|Hom_samples||samples from output VCF selected with
|Hemi_samples||samples from output VCF selected with
Het/Hemi variant and sample count inconsistency¶
We are currently investigating an inconsistency in the variant and sample counts in the annotated variant output file, for a small number of variants. This affects hemizygous and heterozygous variant (
AC_Hemi_variant) and sample (
For autosomal variants, the majority of samples will have diploid genotypes (e.g. 0/1). However, some samples will have haploid (hemizygous-like) calls (e.g. 1) for certain variants, where a variant lies within a deletion.
In the example above, position 2118754 is a 2 basepair deletion. Bases GA are in positions 2118755 and 2118756, respectively. Sample S1 is heterozygous REF/ALT (diploid) at position 2118754; they have the deletion on one chromosome. At position 2118755, S1 is REF (haploid). At position 2118756, S1 is ALT (haploid). These haploid calls are not an artefact of the workflow, but are seen in the single-sample VCFs, and will have
AC_Hemi_variant > 0 in the annotated variant output.
Chromosome X: cannot combine diploid with haploid genotype¶
For a small number of genes on chromosome X the workflow
second_round_merge process fails with the error: cannot combine diploid with haploid genotype.
The error arises because
bcftools norm join (
-m+both) cannot combine the decomposed variants from the previous split step (
-m-both), in the repeated normalisation. For example, DMD for 4 test samples (M, F, M, F) includes both haploid and diploid genotypes at chrX:31709584 after the first
This bug potentially only affects non-PAR chromosome X genes. We are currently working on a solution.
Memory allocation for VEP annotation on CloudOS¶
On CloudOS, the awsbatch executor copies the VEP cache (~15GB) to the compute instance for each Nextflow task (i.e. each query gene). With multiple tasks being assigned to the same compute instance, we have observed that the annotation process can complete, with all columns in the annotated variant file output, but with a "download failed" message and "cannot allocate memory" error reported in the log. We are currently exploring solutions to this issue.
Check the logs to be sure the VEP annotation has run succesfully.