nf-core/variantbenchmarking

Pipeline to evaluate and validate the accuracy of variant calling methods in genomic research

benchmarksmall-variantsstructural-variantsvariant-calling

Launch version 1.6.0 https://github.com/nf-core/variantbenchmarking

Version history

Download .zip Download .tar.gz View on GitHub

What’s Changed

nf-core/variantbenchmarking 1.6.0 – Nominal Eco

Highlights

New benchmarking method: Aardvark for small variants (--method aardvark). Results are merged into the summary tables with TP/FP/FN, precision, recall, F1 and genotype-level FN/FP counts (#315)
Interactive reports: all summary plots (metrics, upset, length distributions) are now interactive HTML instead of static PNG (#325)
New hap.py stratified HTML report, generated from stratified hap.py results (#326)
New --skip_reports parameter: comma-separated list (datavzrd, happy) to turn off optional report generation (#356)
Concordance analysis can be limited to specific truth/query pairs, so you no longer have to run every pairwise comparison (#348)
Visualising results in Depictio (#345)

Changed

hap.py now runs in a container that also includes rtgtools (#327); the Singularity image is gzip-enabled (#334)
Local VCF_VARIANT_FILTERING replaced by the nf-core subworkflow VCF_FILTER_SURVIVOR_BCFTOOLS (#349)
Local tabix module replaced by the nf-core htslib/bgziptabix module (#329)
BED files for small-variant analysis reorganised (#319)
Pipeline template updated to nf-core/tools v4.2.1, via v4.0.0, v4.0.2, v4.0.3 and v4.1.0 (#304, #305, #317, #322, #355)

Fixed

Liftover: target_bed was not passed through and is now included (#320)
Sample-ID conflicts when the same sample name occurs in several inputs (#336)
Sample IDs in truth files produced by rtgtools are now set correctly (#337)

Performance

VCF_TO_CSV streams records instead of keeping the whole VCF in memory, which lowers memory use on large VCFs (#331)
Upset plot generation optimised to avoid out-of-memory failures on large comparisons (#339)

Removed / deprecated

[!WARNING] These changes may require updating existing parameter files or downstream scripts.

pre.py preprocessing deprecated(#316)
Small-variant splitting subworkflow removed (#354)
Plot outputs are now .html instead of .png; adjust any downstream scripts that look for PNG files (#325)
Concordance runs only the requested pairs rather than all combinations (#348)

CI / infrastructure

Removed a vulnerable PR-comment artifact pattern (#318)
fix-linting switched to the hardened nf-core/actions reusable workflow (#333)
New nf-diff PR check and baseline workflows to catch output changes between versions (#344); Nextflow version pinned and results posted via pr-comment.yml (#346)

Software dependencies

Dependency	Old version	New version
pigz	2.8	deprecated
nf-core	3.5.1	4.1.0
prettier	3.6.2	3.8.3
prepy	0.3.15	deprecated
r-base	4.1.0	deprecated
ga4gh-happy-report	-	0.1.2
rtg-tools (hap.py)	-	3.13
tabix	1.21	deprecated
htslib	-	1.24
xz	-	5.8.3
nf-schema	2.5.1	2.7.3
Nextflow	25.04.0	25.10.4
multiqc	1.32	1.33

New parameters: skip_plots, skip_reports, high_conf, happy_comparison_engine
Depreciated parameters: falsepositive_bed

Contributors

Thanks to @kubranarci, @georgiakes, @edogiuili, @mashehu, @adamrtalbot and @nf-core-bot.

New contributors: @mashehu (#318), @adamrtalbot (#331), @edogiuili (#345) – welcome!

Full changelog: https://github.com/nf-core/variantbenchmarking/compare/1.5.0...1.6.0

Download .zip Download .tar.gz View on GitHub

What’s Changed

Replace local modules: SORT_BED, REFORMAT_HEADER, VARIANT_EXTRACTOR, bcftools, tabix, fix_vcf_prefix, create_datavzrd_input @georgiakes and @kubranarci https://github.com/nf-core/variantbenchmarking/pull/267, https://github.com/nf-core/variantbenchmarking/pull/268 https://github.com/nf-core/variantbenchmarking/pull/292 https://github.com/nf-core/variantbenchmarking/pull/287
Strict syntax health, versions to topics channels - by @kubranarci in https://github.com/nf-core/variantbenchmarking/pull/272 https://github.com/nf-core/variantbenchmarking/pull/277
Introducing a new subworkflow to generate truth vcf with an ensemble approach. Test VCFs are being merged and according to ensembl_truth rule (the minimum number of callers to agree) a new truth set is created. This apporach is especially important and needed for somatic benchmarks where truth is often missing. #276
Updating metromap by @kubranarci in https://github.com/nf-core/variantbenchmarking/pull/281
Issue # 256 bed sanity checking by @Qwertille42 in https://github.com/nf-core/variantbenchmarking/pull/290
Remove unnecessary reheader module form benchmarking by @georgiakes in https://github.com/nf-core/variantbenchmarking/pull/286
Adding tests to local modules by @kubranarci in https://github.com/nf-core/variantbenchmarking/pull/295
Adding tests and meta.yml subworkflows by @kubranarci in https://github.com/nf-core/variantbenchmarking/pull/296
Fixing bugs in bndeval outputs and ensemble analysis by @kubranarci in https://github.com/nf-core/variantbenchmarking/pull/297
Fixing strelka processing by @kubranarci in https://github.com/nf-core/variantbenchmarking/pull/279
Adding rtg cnveval as benchmark method for copynumber variants by @kubranarci in https://github.com/nf-core/variantbenchmarking/pull/298

`New Dependencies`

Dependency	Old version	New version
rtgtools	3.12.1	3.13
bcftools	1.2	1.22
tabix	1.19.1	1.21
bcftools/stats	1.18	1.22
happy	0.3.14	0.3.15
bedops	2.4.41	2.4.42
picard	3.2.0	3.4.0
ucsc	377	482
datavzrd	2.36.12	2.63.3
truvari	5.3.0	5.4.0
gawk	-	5.3.0
coreutils	-	9.5