GenotypeGVCFs issue when using --include-non-variant-sites parameter
Hi,
I am running similar issues as in this post :
https://github.com/broadinstitute/gatk/issues/8415
but I am reposting here as it seems this is a more convenient place to get an answer.
I am running gatk GenotypeGVCFs with gatk v 4.5.0.0 installed from the docker image on a cluster with Slurm manager.
I and other users have faced an odd issue where, when running GenotypeGVCFs with --include-non-variant-sites argument on multiple chromosomes (each in a separate GenomicDB), it only works for the first chromosome, then fails for all other chromosomes.
However when removing this argument it works for all chromosomes.
As other users seem to have faced this issue when running gatk with singularity I am suspecting that this error comes from certain clusters configuration.
The commands I have used :
#!/bin/bash
#Execute file : sbatch /home/hdenis/Coral-Genomics/Scripts/GATK_GenomicsDBimport_slurm.sh
##SBATCH --clusters=dell_r740xd
### partition
#SBATCH --nodelist=R740xd
#SBATCH --partition=local
### JOB NAME
#SBATCH -J GATK
### WALLTIME
#SBATCH -t 3-00:00:00
#MEMORY
#SBATCH --mem=200G # RAM per job given in megabytes (M), gigabytes (G), or terabytes (T)
#Output and error directory
#SBATCH -o /home/hdenis/Slurm/gatk_%A_%a.o #Standard output
#SBATCH -e /home/hdenis/Slurm/gatk_%A_%a.e # standard error
#SBATCH --nodes=1 # use 1 node
#SBATCH --ntasks-per-node=1 # use 1 for single and multi core jobs
#SBATCH --cpus-per-task=2 # number of cores per job
#SBATCH --array=1-14%1 # job array
INDIR="/nvme/disk0/lecellier_data/WGS_GBR_data/GATK_files/"
OUTDIR="/nvme/disk0/lecellier_data/WGS_GBR_data/GATK_files/Vcf_files/"
#A.millepora v3 reference genome
REF_3="/nvme/disk0/lecellier_data/WGS_GBR_data/Ref_genomes/Amil_scaffolds_final_v3.fa"
REF_NAME="Amilleporav3"
#Associate slurm array index with contig (chr) name
CHROMOSOME_FILE="/nvme/disk0/lecellier_data/WGS_GBR_data/ANGSD_files/chromosomes_header.txt"
CONTIG=`sed -n ${SLURM_ARRAY_TASK_ID}p $CHROMOSOME_FILE`
singularity exec --bind /nvme/disk0/lecellier_data:/nvme/disk0/lecellier_data /home/hdenis/gatk_latest.sif gatk --java-options "-Xmx100g -Xms4g" GenomicsDBImport --genomicsdb-workspace-path "${INDIR}GenomicDB/${CONTIG}" --batch-size 50 -L $CONTIG --sample-name-map "${INDIR}aspat_gvcf_clean.sample_map" --tmp-dir /nvme/disk0/lecellier_data/WGS_GBR_data/tmp --reader-threads 2 --genomicsdb-shared-posixfs-optimizations true
cd ${INDIR}GenomicDB/
singularity exec --bind /nvme/disk0/lecellier_data:/nvme/disk0/lecellier_data /home/hdenis/gatk_latest.sif gatk --java-options "-Xmx48g" GenotypeGVCFs -R $REF_3 -V "gendb://${CONTIG}" -O "${OUTDIR}aspat_clean_${CONTIG}.vcf.gz" --tmp-dir /nvme/disk0/lecellier_data/WGS_GBR_data/tmp --include-non-variant-sites true
Here is the log file for the first scaffold (chromosome), which outputs a complete vcf file as expected. Please not there was already a java.lang.IllegalStateException:, although the vcf file seems to contains all base positions and does not seem corrupted.
04:40:31.093 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.5.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
04:40:31.255 INFO GenotypeGVCFs - ------------------------------------------------------------
04:40:31.258 INFO GenotypeGVCFs - The Genome Analysis Toolkit (GATK) v4.5.0.0
04:40:31.258 INFO GenotypeGVCFs - For support and documentation go to https://software.broadinstitute.org/gatk/
04:40:31.262 INFO GenotypeGVCFs - Initializing engine
04:40:31.615 INFO GenomicsDBLibLoader - GenomicsDB native library version : 1.5.1-84e800e
15:40:31.666 INFO NativeGenomicsDB - pid=680078 tid=680079 No valid combination operation found for INFO field InbreedingCoeff - the field will NOT be part of INFO fields in the generated VCF records
15:40:31.666 INFO NativeGenomicsDB - pid=680078 tid=680079 No valid combination operation found for INFO field MLEAC - the field will NOT be part of INFO fields in the generated VCF records
15:40:31.666 INFO NativeGenomicsDB - pid=680078 tid=680079 No valid combination operation found for INFO field MLEAF - the field will NOT be part of INFO fields in the generated VCF records
04:40:31.744 INFO GenotypeGVCFs - Done initializing engine
04:40:31.788 INFO ProgressMeter - Starting traversal
04:40:31.789 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.411632535,Cpu time(s),0.38179858199999755
04:40:34.447 WARN InbreedingCoeff - InbreedingCoeff will not be calculated at position scaffold_1:172 and possibly subsequent; at least 10 samples must have called genotypes
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.322359918000002,Cpu time(s),0.273166717000001
04:40:43.914 INFO ProgressMeter - scaffold_1:201000 0.2 201000 994721.2
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2796300050000068,Cpu time(s),0.28137058400000087
04:40:53.924 INFO ProgressMeter - scaffold_1:452000 0.4 452000 1225264.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3266611720000012,Cpu time(s),0.3283720680000047
04:41:03.942 INFO ProgressMeter - scaffold_1:725000 0.5 725000 1352948.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.31141360600000473,Cpu time(s),0.3127274939999963
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3579390069999994,Cpu time(s),0.36000182499999783
04:41:15.907 INFO ProgressMeter - scaffold_1:1001000 0.7 1001000 1361349.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.1821918779999979,Cpu time(s),0.1836738459999994
04:41:25.918 INFO ProgressMeter - scaffold_1:1289000 0.9 1289000 1428809.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27046602799999736,Cpu time(s),0.27239400199999914
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.16872037299999973,Cpu time(s),0.16997418700000047
04:41:37.677 INFO ProgressMeter - scaffold_1:1601000 1.1 1601000 1457928.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.10341928200000111,Cpu time(s),0.10427671499999966
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.15803377599999857,Cpu time(s),0.15918165699999934
04:41:48.524 INFO ProgressMeter - scaffold_1:2001000 1.3 2001000 1564605.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.15190452399999918,Cpu time(s),0.15308379699999955
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.10847042200000073,Cpu time(s),0.10939113699999983
04:41:59.914 INFO ProgressMeter - scaffold_1:2401000 1.5 2401000 1634723.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.37515572000000275,Cpu time(s),0.32722164699999806
04:42:09.914 INFO ProgressMeter - scaffold_1:2735000 1.6 2735000 1672356.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.26556471299999784,Cpu time(s),0.26744696799999973
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2876433249999994,Cpu time(s),0.27329983799999835
04:42:20.291 INFO ProgressMeter - scaffold_1:3001000 1.8 3001000 1659508.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.13038837799999967,Cpu time(s),0.13139388700000007
04:42:30.299 INFO ProgressMeter - scaffold_1:3384000 2.0 3384000 1713273.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.11721459500000028,Cpu time(s),0.11815548699999964
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.12827026000000089,Cpu time(s),0.12932281799999965
04:42:40.300 INFO ProgressMeter - scaffold_1:3709000 2.1 3709000 1731680.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.22021793299999975,Cpu time(s),0.22164610099999998
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.24521596299999976,Cpu time(s),0.24700500400000217
04:42:50.309 INFO ProgressMeter - scaffold_1:4005000 2.3 4005000 1734767.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.13862809099999973,Cpu time(s),0.13967120300000024
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.1129488350000011,Cpu time(s),0.11383181600000045
04:43:02.067 INFO ProgressMeter - scaffold_1:4401000 2.5 4401000 1757143.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.12063961100000109,Cpu time(s),0.12165769200000011
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.13224236699999978,Cpu time(s),0.13338863099999887
04:43:12.631 INFO ProgressMeter - scaffold_1:4801000 2.7 4801000 1790950.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27186300300000066,Cpu time(s),0.27378760600000085
04:43:22.643 INFO ProgressMeter - scaffold_1:5139000 2.8 5139000 1804709.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.10329612600000079,Cpu time(s),0.10431216399999933
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3333567139999989,Cpu time(s),0.33539336499999817
04:43:32.644 INFO ProgressMeter - scaffold_1:5440000 3.0 5440000 1804760.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.14376514400000018,Cpu time(s),0.1448029540000005
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2596802610000012,Cpu time(s),0.26141970599999975
04:43:44.862 INFO ProgressMeter - scaffold_1:5801000 3.2 5801000 1802747.2
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2881398679999993,Cpu time(s),0.29005781900000205
04:43:54.880 INFO ProgressMeter - scaffold_1:6084000 3.4 6084000 1797420.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25496484199999747,Cpu time(s),0.2562914140000025
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2678788129999972,Cpu time(s),0.2696684719999984
04:44:06.093 INFO ProgressMeter - scaffold_1:6401000 3.6 6401000 1792127.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.17371115899999842,Cpu time(s),0.17530232600000112
04:44:16.106 INFO ProgressMeter - scaffold_1:6744000 3.7 6744000 1803875.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2576058720000002,Cpu time(s),0.25929825200000023
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.21414285799999946,Cpu time(s),0.21551755300000044
04:44:26.421 INFO ProgressMeter - scaffold_1:7001000 3.9 7001000 1790292.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.31207292999999847,Cpu time(s),0.28889831700000057
04:44:36.439 INFO ProgressMeter - scaffold_1:7311000 4.1 7311000 1793010.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3159128230000005,Cpu time(s),0.31799484799999606
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.30313246999999843,Cpu time(s),0.3050018690000013
04:44:48.386 INFO ProgressMeter - scaffold_1:7601000 4.3 7601000 1777346.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.1998059969999984,Cpu time(s),0.20135365800000093
04:44:58.387 INFO ProgressMeter - scaffold_1:7906000 4.4 7906000 1779308.2
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.12335151500000023,Cpu time(s),0.12430344299999983
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.12458211200000036,Cpu time(s),0.11192197899999914
04:45:08.391 INFO ProgressMeter - scaffold_1:8271000 4.6 8271000 1794130.2
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.11560800500000012,Cpu time(s),0.11645082700000003
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.10977641200000103,Cpu time(s),0.11081306099999982
04:45:18.407 INFO ProgressMeter - scaffold_1:8634000 4.8 8634000 1807429.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2683542739999995,Cpu time(s),0.2704120570000002
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.28571143499999996,Cpu time(s),0.2874725719999963
04:45:30.393 INFO ProgressMeter - scaffold_1:9001000 5.0 9001000 1808616.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2752968489999993,Cpu time(s),0.2770853990000012
04:45:40.405 INFO ProgressMeter - scaffold_1:9282000 5.1 9282000 1804572.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.26300380899999853,Cpu time(s),0.2646141600000038
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.290747247,Cpu time(s),0.2928229119999991
04:45:51.728 INFO ProgressMeter - scaffold_1:9601000 5.3 9601000 1800530.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.29550259000000034,Cpu time(s),0.29742784299999403
04:46:01.736 INFO ProgressMeter - scaffold_1:9872000 5.5 9872000 1795197.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2733998659999958,Cpu time(s),0.27759045799999776
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.22891349999999971,Cpu time(s),0.23059863500000188
04:46:13.765 INFO ProgressMeter - scaffold_1:10201000 5.7 10201000 1789774.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27549386899999695,Cpu time(s),0.27670454399999983
04:46:23.784 INFO ProgressMeter - scaffold_1:10496000 5.9 10496000 1789116.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3016341779999984,Cpu time(s),0.3035826030000004
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.30527084000000027,Cpu time(s),0.30751550700000024
04:46:35.553 INFO ProgressMeter - scaffold_1:10801000 6.1 10801000 1781539.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25619790400000203,Cpu time(s),0.2577051639999997
04:46:45.553 INFO ProgressMeter - scaffold_1:11073000 6.2 11073000 1777538.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.309869578999997,Cpu time(s),0.3121099769999961
04:46:55.561 INFO ProgressMeter - scaffold_1:11382000 6.4 11382000 1779494.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2569128530000023,Cpu time(s),0.2588401650000019
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25354489800000113,Cpu time(s),0.255280658000003
04:47:05.578 INFO ProgressMeter - scaffold_1:11612000 6.6 11612000 1769272.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.11362957700000025,Cpu time(s),0.11448609599999955
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.08340972600000021,Cpu time(s),0.08417236700000001
04:47:17.146 INFO ProgressMeter - scaffold_1:12001000 6.8 12001000 1776360.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2422813029999975,Cpu time(s),0.24394292300000425
04:47:27.148 INFO ProgressMeter - scaffold_1:12391000 6.9 12391000 1789921.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3184627520000022,Cpu time(s),0.32071319999999776
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.28334422100000306,Cpu time(s),0.2856379049999965
04:47:37.547 INFO ProgressMeter - scaffold_1:12601000 7.1 12601000 1775797.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27632141700000024,Cpu time(s),0.2781827929999983
04:47:47.571 INFO ProgressMeter - scaffold_1:12872000 7.3 12872000 1772262.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.14546003199999888,Cpu time(s),0.14649917500000145
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.308139296999998,Cpu time(s),0.3100686540000028
04:47:58.026 INFO ProgressMeter - scaffold_1:13201000 7.4 13201000 1774976.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3159120389999956,Cpu time(s),0.31825892200000144
04:48:08.071 INFO ProgressMeter - scaffold_1:13452000 7.6 13452000 1768906.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.18775758200000092,Cpu time(s),0.18915874499999993
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.32767551599999467,Cpu time(s),0.3297623329999956
04:48:20.173 INFO ProgressMeter - scaffold_1:13801000 7.8 13801000 1767908.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.31207831200000585,Cpu time(s),0.3115969590000011
04:48:30.173 INFO ProgressMeter - scaffold_1:14053000 8.0 14053000 1762558.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2867203739999987,Cpu time(s),0.28846604599999803
04:48:40.187 INFO ProgressMeter - scaffold_1:14349000 8.1 14349000 1762783.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.23580270899999864,Cpu time(s),0.23751065300000135
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.0772410220000006,Cpu time(s),0.07799677700000046
04:48:50.193 INFO ProgressMeter - scaffold_1:14645000 8.3 14645000 1763027.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.08859928500000157,Cpu time(s),0.08939590200000026
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.08575133200000033,Cpu time(s),0.08648309200000008
04:49:00.193 INFO ProgressMeter - scaffold_1:15057000 8.5 15057000 1776972.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2659558400000026,Cpu time(s),0.26788878500000063
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2971501109999962,Cpu time(s),0.2989893319999996
04:49:11.815 INFO ProgressMeter - scaffold_1:15401000 8.7 15401000 1776949.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.10040105200000088,Cpu time(s),0.10139144899999936
04:49:21.815 INFO ProgressMeter - scaffold_1:15753000 8.8 15753000 1783271.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.06918999699999952,Cpu time(s),0.0699540479999998
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.12527576400000098,Cpu time(s),0.12630379400000025
04:49:31.826 INFO ProgressMeter - scaffold_1:16168000 9.0 16168000 1796321.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.16403358100000015,Cpu time(s),0.16533393900000043
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25058666200000085,Cpu time(s),0.2524360150000008
04:49:41.854 INFO ProgressMeter - scaffold_1:16455000 9.2 16455000 1794878.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3087754000000027,Cpu time(s),0.31048791499999695
04:49:51.871 INFO ProgressMeter - scaffold_1:16737000 9.3 16737000 1792987.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.31389429599999813,Cpu time(s),0.31591289100000103
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27929805199999885,Cpu time(s),0.2812242040000004
04:50:03.134 INFO ProgressMeter - scaffold_1:17001000 9.5 17001000 1785366.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3072539419999976,Cpu time(s),0.3092819649999969
04:50:13.144 INFO ProgressMeter - scaffold_1:17281000 9.7 17281000 1783523.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.22721117100000218,Cpu time(s),0.2287650410000014
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2426692849999998,Cpu time(s),0.2443506360000025
04:50:24.538 INFO ProgressMeter - scaffold_1:17601000 9.9 17601000 1781631.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2753109789999955,Cpu time(s),0.27730226100000027
04:50:34.544 INFO ProgressMeter - scaffold_1:17892000 10.0 17892000 1781022.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.22250919199999952,Cpu time(s),0.2238179399999977
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.285072058999999,Cpu time(s),0.28708201899999736
04:50:45.629 INFO ProgressMeter - scaffold_1:18201000 10.2 18201000 1779062.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.264120485999997,Cpu time(s),0.25533977400000224
04:50:55.669 INFO ProgressMeter - scaffold_1:18483000 10.4 18483000 1777553.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3768040669999958,Cpu time(s),0.37923861
04:51:05.686 INFO ProgressMeter - scaffold_1:18740000 10.6 18740000 1773789.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27616466599999956,Cpu time(s),0.2779268949999999
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.28964474399999757,Cpu time(s),0.2918599070000017
04:51:16.031 INFO ProgressMeter - scaffold_1:19001000 10.7 19001000 1769614.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.23521994200000004,Cpu time(s),0.23656142599999977
04:51:26.037 INFO ProgressMeter - scaffold_1:19278000 10.9 19278000 1767953.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2385301030000004,Cpu time(s),0.24015210700000286
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.22640173400000257,Cpu time(s),0.22787413800000075
04:51:37.693 INFO ProgressMeter - scaffold_1:19601000 11.1 19601000 1766110.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2525786580000003,Cpu time(s),0.24312911100000079
04:51:47.695 INFO ProgressMeter - scaffold_1:19926000 11.3 19926000 1768825.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.32855676599999906,Cpu time(s),0.33070413800000176
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.18332806199999901,Cpu time(s),0.18455287499999973
04:51:58.919 INFO ProgressMeter - scaffold_1:20201000 11.5 20201000 1763945.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3157072469999993,Cpu time(s),0.31774817199999633
04:52:08.931 INFO ProgressMeter - scaffold_1:20490000 11.6 20490000 1763485.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2967802849999993,Cpu time(s),0.2986199030000004
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.21307945000000225,Cpu time(s),0.21466273400000466
04:52:20.970 INFO ProgressMeter - scaffold_1:20801000 11.8 20801000 1759861.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.28586596600000047,Cpu time(s),0.2878967530000021
04:52:30.971 INFO ProgressMeter - scaffold_1:21096000 12.0 21096000 1759999.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.29545835800000025,Cpu time(s),0.29754866599999985
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.28065302399999775,Cpu time(s),0.28264209000000245
04:52:42.704 INFO ProgressMeter - scaffold_1:21401000 12.2 21401000 1756784.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.32116579599999734,Cpu time(s),0.3081755270000053
04:52:52.734 INFO ProgressMeter - scaffold_1:21679000 12.3 21679000 1755514.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25541790799999553,Cpu time(s),0.2568532459999992
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2744228149999996,Cpu time(s),0.2767923369999994
04:53:04.906 INFO ProgressMeter - scaffold_1:22001000 12.6 22001000 1752795.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2873316589999993,Cpu time(s),0.2658942350000006
04:53:14.922 INFO ProgressMeter - scaffold_1:22285000 12.7 22285000 1752121.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.26823474000000186,Cpu time(s),0.270311125000003
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.30121818799999495,Cpu time(s),0.3035035540000012
04:53:26.473 INFO ProgressMeter - scaffold_1:22601000 12.9 22601000 1750468.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25503495900000234,Cpu time(s),0.25695861900000316
04:53:36.497 INFO ProgressMeter - scaffold_1:22869000 13.1 22869000 1748599.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.24200155400000184,Cpu time(s),0.2435153439999992
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2867543999999992,Cpu time(s),0.2840350190000005
04:53:48.539 INFO ProgressMeter - scaffold_1:23201000 13.3 23201000 1747172.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.1900074489999995,Cpu time(s),0.19127788899999884
04:53:58.540 INFO ProgressMeter - scaffold_1:23544000 13.4 23544000 1751025.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.18277025099999872,Cpu time(s),0.18389049100000127
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.23383687500000136,Cpu time(s),0.23532427400000003
04:54:08.568 INFO ProgressMeter - scaffold_1:23830000 13.6 23830000 1750536.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.26151621099999856,Cpu time(s),0.22811231800000084
04:54:18.567 INFO ProgressMeter - scaffold_1:24167000 13.8 24167000 1753820.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.17366457200000163,Cpu time(s),0.17493936900000148
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.23869892500000092,Cpu time(s),0.24042215099999945
04:54:28.592 INFO ProgressMeter - scaffold_1:24433000 13.9 24433000 1751881.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2520316890000015,Cpu time(s),0.253768121000001
04:54:38.595 INFO ProgressMeter - scaffold_1:24762000 14.1 24762000 1754498.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.09499459899999968,Cpu time(s),0.09583409899999969
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.11731998900000065,Cpu time(s),0.11816367400000007
04:54:48.596 INFO ProgressMeter - scaffold_1:25134000 14.3 25134000 1760069.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.20773532999999975,Cpu time(s),0.20917828400000105
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2393451270000017,Cpu time(s),0.24108032000000096
04:54:58.617 INFO ProgressMeter - scaffold_1:25411000 14.4 25411000 1758895.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.17451355600000082,Cpu time(s),0.1756520120000011
04:55:08.627 INFO ProgressMeter - scaffold_1:25761000 14.6 25761000 1762765.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.16899278100000115,Cpu time(s),0.16999687800000238
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2371467559999967,Cpu time(s),0.23877015099999904
04:55:18.643 INFO ProgressMeter - scaffold_1:26036000 14.8 26036000 1761462.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2923757269999974,Cpu time(s),0.2944610510000029
04:55:28.657 INFO ProgressMeter - scaffold_1:26337000 14.9 26337000 1761931.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.23294987200000092,Cpu time(s),0.2345516820000015
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2831113159999992,Cpu time(s),0.2849831140000002
04:55:38.661 INFO ProgressMeter - scaffold_1:26602000 15.1 26602000 1760027.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25837688799999664,Cpu time(s),0.25999888500000057
04:55:48.676 INFO ProgressMeter - scaffold_1:26888000 15.3 26888000 1759518.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.31104246999999907,Cpu time(s),0.3131865900000025
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.19632450600000054,Cpu time(s),0.19756631199999944
04:56:00.700 INFO ProgressMeter - scaffold_1:27201000 15.5 27201000 1756960.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3128972450000001,Cpu time(s),0.315147022
04:56:10.708 INFO ProgressMeter - scaffold_1:27491000 15.6 27491000 1756765.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.23917333000000016,Cpu time(s),0.2407320909999989
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.10993748300000056,Cpu time(s),0.11084323100000026
04:56:21.550 INFO ProgressMeter - scaffold_1:27801000 15.8 27801000 1756294.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2550608650000024,Cpu time(s),0.25716681900000377
04:56:31.554 INFO ProgressMeter - scaffold_1:28183000 16.0 28183000 1761868.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2966492860000012,Cpu time(s),0.29886223900000164
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2592804229999985,Cpu time(s),0.2611176550000019
04:56:41.569 INFO ProgressMeter - scaffold_1:28403000 16.2 28403000 1757285.2
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.20492232699999904,Cpu time(s),0.2061792880000005
04:56:51.589 INFO ProgressMeter - scaffold_1:28720000 16.3 28720000 1758726.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.24493402599999983,Cpu time(s),0.24661030500000028
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3053338000000032,Cpu time(s),0.30713536199999814
04:57:02.199 INFO ProgressMeter - scaffold_1:29001000 16.5 29001000 1756908.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27828650099999896,Cpu time(s),0.2802107709999975
04:57:12.217 INFO ProgressMeter - scaffold_1:29270000 16.7 29270000 1755448.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2772107419999986,Cpu time(s),0.27925507699999935
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.28148614399999905,Cpu time(s),0.2835892069999982
04:57:24.272 INFO ProgressMeter - scaffold_1:29601000 16.9 29601000 1754164.5
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.29018134999999884,Cpu time(s),0.29190009700000086
04:57:34.282 INFO ProgressMeter - scaffold_1:29881000 17.0 29881000 1753420.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.20686813700000045,Cpu time(s),0.20821015000000123
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.24796414399999506,Cpu time(s),0.24966437900000157
04:57:45.299 INFO ProgressMeter - scaffold_1:30201000 17.2 30201000 1753306.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.18049411899999904,Cpu time(s),0.18185181299999972
04:57:55.306 INFO ProgressMeter - scaffold_1:30562000 17.4 30562000 1757249.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.26469178800000015,Cpu time(s),0.27077938000000024
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.33284774299999753,Cpu time(s),0.3349479790000025
04:58:05.562 INFO ProgressMeter - scaffold_1:30801000 17.6 30801000 1753755.3
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27093011199999395,Cpu time(s),0.27278647799999867
04:58:15.590 INFO ProgressMeter - scaffold_1:31053000 17.7 31053000 1751436.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25839364200000037,Cpu time(s),0.2599447419999989
04:58:25.601 INFO ProgressMeter - scaffold_1:31389000 17.9 31389000 1753882.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2542186159999975,Cpu time(s),0.24959069099999973
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.24667794100000118,Cpu time(s),0.2485157590000049
04:58:35.630 INFO ProgressMeter - scaffold_1:31631000 18.1 31631000 1751050.2
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.26532961500000185,Cpu time(s),0.26683514000000014
04:58:45.642 INFO ProgressMeter - scaffold_1:31969000 18.2 31969000 1753562.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.25076040299999675,Cpu time(s),0.25239965600000036
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.19897787300000308,Cpu time(s),0.20030001300000144
04:58:55.645 INFO ProgressMeter - scaffold_1:32207000 18.4 32207000 1750608.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.24799227399999732,Cpu time(s),0.24939072400000084
04:59:05.664 INFO ProgressMeter - scaffold_1:32544000 18.6 32544000 1753015.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3283118630000001,Cpu time(s),0.3303895989999986
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.31511851499999566,Cpu time(s),0.31750325700000165
04:59:17.205 INFO ProgressMeter - scaffold_1:32801000 18.8 32801000 1748740.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.30728992699999946,Cpu time(s),0.3095138049999986
04:59:27.205 INFO ProgressMeter - scaffold_1:33063000 18.9 33063000 1747183.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2052157330000011,Cpu time(s),0.20672614800000144
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.28286867999999643,Cpu time(s),0.2846333119999999
04:59:38.247 INFO ProgressMeter - scaffold_1:33401000 19.1 33401000 1748044.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.21456374399999933,Cpu time(s),0.21596798900000128
04:59:48.249 INFO ProgressMeter - scaffold_1:33707000 19.3 33707000 1748802.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27524950999999803,Cpu time(s),0.2770334180000031
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.30744700999999486,Cpu time(s),0.30934397599999836
04:59:58.702 INFO ProgressMeter - scaffold_1:34001000 19.4 34001000 1748253.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.32086001100000333,Cpu time(s),0.3228151780000024
05:00:08.702 INFO ProgressMeter - scaffold_1:34264000 19.6 34264000 1746807.1
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3287624729999984,Cpu time(s),0.3310145959999985
05:00:18.706 INFO ProgressMeter - scaffold_1:34552000 19.8 34552000 1746642.8
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.23367892400000131,Cpu time(s),0.2354300320000019
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2769880330000018,Cpu time(s),0.27893538699999915
05:00:28.722 INFO ProgressMeter - scaffold_1:34824000 19.9 34824000 1745661.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2924649629999991,Cpu time(s),0.29435058799999936
05:00:38.739 INFO ProgressMeter - scaffold_1:35127000 20.1 35127000 1746236.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.18771303699999997,Cpu time(s),0.18896238700000043
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.26189974199999505,Cpu time(s),0.26388380799999533
05:00:48.765 INFO ProgressMeter - scaffold_1:35443000 20.3 35443000 1747429.7
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.35592166700000133,Cpu time(s),0.35809410599999775
05:00:58.773 INFO ProgressMeter - scaffold_1:35711000 20.4 35711000 1746281.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3186077659999976,Cpu time(s),0.3207936220000025
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27556668899999803,Cpu time(s),0.2774107529999979
05:01:08.999 INFO ProgressMeter - scaffold_1:36001000 20.6 36001000 1745913.6
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.34210229300000117,Cpu time(s),0.3442658129999996
05:01:19.002 INFO ProgressMeter - scaffold_1:36314000 20.8 36314000 1746967.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.27997390299999947,Cpu time(s),0.2818799329999979
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3126020589999999,Cpu time(s),0.31484275099999753
05:01:29.003 INFO ProgressMeter - scaffold_1:36610000 21.0 36610000 1747198.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.24849876800000195,Cpu time(s),0.24996252400000102
05:01:39.013 INFO ProgressMeter - scaffold_1:36965000 21.1 36965000 1750205.0
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.19995450899999873,Cpu time(s),0.2011818870000025
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2688014800000004,Cpu time(s),0.27052468599999696
05:01:49.017 INFO ProgressMeter - scaffold_1:37314000 21.3 37314000 1752889.9
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2840990890000007,Cpu time(s),0.2649249350000047
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.2791010069999977,Cpu time(s),0.28101516
05:01:59.046 INFO ProgressMeter - scaffold_1:37642000 21.5 37642000 1754521.4
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.3066701449999967,Cpu time(s),0.30869057700000085
GENOMICSDB_TIMER,GenomicsDB iterator next() timer,Wall-clock time(s),0.17638480700000014,Cpu time(s),0.15843300900000132
05:02:09.066 INFO ProgressMeter - scaffold_1:38009000 21.6 38009000 1757943.8
05:02:11.554 INFO GenotypeGVCFs - Shutting down engine
[June 26, 2024 at 5:02:11 AM GMT] org.broadinstitute.hellbender.tools.walkers.GenotypeGVCFs done. Elapsed time: 21.68 minutes.
Runtime.totalMemory()=3055550464
java.lang.IllegalStateException: There are no sources based on those query parameters
at org.genomicsdb.reader.GenomicsDBFeatureIterator.<init>(GenomicsDBFeatureIterator.java:167)
at org.genomicsdb.reader.GenomicsDBFeatureReader.query(GenomicsDBFeatureReader.java:149)
at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:622)
at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:591)
at org.broadinstitute.hellbender.engine.FeatureDataSource.query(FeatureDataSource.java:557)
at org.broadinstitute.hellbender.engine.VariantLocusWalker.lambda$traverse$2(VariantLocusWalker.java:149)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at java.base/java.util.stream.ReferencePipeline$Head.forEachOrdered(ReferencePipeline.java:772)
at org.broadinstitute.hellbender.engine.VariantLocusWalker.traverse(VariantLocusWalker.java:148)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1098)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:149)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:198)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:217)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:166)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:209)
at org.broadinstitute.hellbender.Main.main(Main.java:306)
Using GATK jar /gatk/gatk-package-4.5.0.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx4g -jar /gatk/gatk-package-4.5.0.0-local.jar GenotypeGVCFs -R /nvme/disk0/lecellier_data/WGS_GBR_data/Ref_genomes/Amil_scaffolds_final_v3.fa -V gendb://scaffold_1 -O /nvme/disk0/lecellier_data/WGS_GBR_data/GATK_files/Vcf_files/aspat_clean_scaffold_1.vcf.gz --include-non-variant-sites --tmp-dir /nvme/disk0/lecellier_data/WGS_GBR_data/tmp
And here is the log file for the second scaffold that did not work
05:04:16.405 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.5.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
05:04:16.556 INFO GenotypeGVCFs - ------------------------------------------------------------
05:04:16.559 INFO GenotypeGVCFs - The Genome Analysis Toolkit (GATK) v4.5.0.0
05:04:16.559 INFO GenotypeGVCFs - For support and documentation go to https://software.broadinstitute.org/gatk/
05:04:16.563 INFO GenotypeGVCFs - Initializing engine
05:04:16.929 INFO GenomicsDBLibLoader - GenomicsDB native library version : 1.5.1-84e800e
16:04:16.979 INFO NativeGenomicsDB - pid=680685 tid=680686 No valid combination operation found for INFO field InbreedingCoeff - the field will NOT be part of INFO fields in the generated VCF records
16:04:16.979 INFO NativeGenomicsDB - pid=680685 tid=680686 No valid combination operation found for INFO field MLEAC - the field will NOT be part of INFO fields in the generated VCF records
16:04:16.979 INFO NativeGenomicsDB - pid=680685 tid=680686 No valid combination operation found for INFO field MLEAF - the field will NOT be part of INFO fields in the generated VCF records
05:04:17.059 INFO GenotypeGVCFs - Done initializing engine
05:04:17.104 INFO ProgressMeter - Starting traversal
05:04:17.105 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute
05:04:17.124 INFO GenotypeGVCFs - Shutting down engine
[June 26, 2024 at 5:04:17 AM GMT] org.broadinstitute.hellbender.tools.walkers.GenotypeGVCFs done. Elapsed time: 0.01 minutes.
Runtime.totalMemory()=1126170624
java.lang.IllegalStateException: There are no sources based on those query parameters
at org.genomicsdb.reader.GenomicsDBFeatureIterator.<init>(GenomicsDBFeatureIterator.java:167)
at org.genomicsdb.reader.GenomicsDBFeatureReader.query(GenomicsDBFeatureReader.java:149)
at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:622)
at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:591)
at org.broadinstitute.hellbender.engine.FeatureDataSource.query(FeatureDataSource.java:557)
at org.broadinstitute.hellbender.engine.VariantLocusWalker.lambda$traverse$2(VariantLocusWalker.java:149)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at java.base/java.util.stream.ReferencePipeline$Head.forEachOrdered(ReferencePipeline.java:772)
at org.broadinstitute.hellbender.engine.VariantLocusWalker.traverse(VariantLocusWalker.java:148)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1098)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:149)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:198)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:217)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:166)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:209)
at org.broadinstitute.hellbender.Main.main(Main.java:306)
Using GATK jar /gatk/gatk-package-4.5.0.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx4g -jar /gatk/gatk-package-4.5.0.0-local.jar GenotypeGVCFs -R /nvme/disk0/lecellier_data/WGS_GBR_data/Ref_genomes/Amil_scaffolds_final_v3.fa -V gendb://scaffold_2 -O /nvme/disk0/lecellier_data/WGS_GBR_data/GATK_files/Vcf_files/aspat_clean_scaffold_2.vcf.gz --include-non-variant-sites --tmp-dir /nvme/disk0/lecellier_data/WGS_GBR_data/tmp
Of note I am facing this issue when running a first test on only two samples called with Haplotypecaller so I do not suspect a memory issue. I have also checked the gvcf files using ValidateVariants and it runned fine.
Thank you for your help
gatk ValidateVariants -V /nvme/disk0/lecellier_data/WGS_GBR_data/GATK_files/RRAP-ECT01-2022-Aspat-CBHE-1718_L1_pe_aln_Amilleporav3.g.vcf.gz --validation-type-to-exclude ALL
Using GATK jar /home/hdenis/Programs/gatk-4.5.0.0/gatk-package-4.5.0.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/hdenis/Programs/gatk-4.5.0.0/gatk-package-4.5.0.0-local.jar ValidateVariants -V /nvme/disk0/lecellier_data/WGS_GBR_data/GATK_files/RRAP-ECT01-2022-Aspat-CBHE-1718_L1_pe_aln_Amilleporav3.g.vcf.gz --validation-type-to-exclude ALL
20:49:48.492 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/hdenis/Programs/gatk-4.5.0.0/gatk-package-4.5.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
20:49:48.623 INFO ValidateVariants - ------------------------------------------------------------
20:49:48.625 INFO ValidateVariants - The Genome Analysis Toolkit (GATK) v4.5.0.0
20:49:48.625 INFO ValidateVariants - For support and documentation go to https://software.broadinstitute.org/gatk/
20:49:48.625 INFO ValidateVariants - Executing as hdenis@R740xd on Linux v5.14.0-362.13.1.el9_3.x86_64 amd64
20:49:48.625 INFO ValidateVariants - Java runtime: OpenJDK 64-Bit Server VM v17.0.2+8-86
20:49:48.626 INFO ValidateVariants - Start Date/Time: June 26, 2024 at 8:49:48 PM NCT
20:49:48.626 INFO ValidateVariants - ------------------------------------------------------------
20:49:48.626 INFO ValidateVariants - ------------------------------------------------------------
20:49:48.627 INFO ValidateVariants - HTSJDK Version: 4.1.0
20:49:48.627 INFO ValidateVariants - Picard Version: 3.1.1
20:49:48.627 INFO ValidateVariants - Built for Spark Version: 3.5.0
20:49:48.628 INFO ValidateVariants - HTSJDK Defaults.COMPRESSION_LEVEL : 2
20:49:48.628 INFO ValidateVariants - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
20:49:48.628 INFO ValidateVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
20:49:48.628 INFO ValidateVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
20:49:48.628 INFO ValidateVariants - Deflater: IntelDeflater
20:49:48.628 INFO ValidateVariants - Inflater: IntelInflater
20:49:48.628 INFO ValidateVariants - GCS max retries/reopens: 20
20:49:48.628 INFO ValidateVariants - Requester pays: disabled
20:49:48.629 INFO ValidateVariants - Initializing engine
20:49:48.717 INFO FeatureManager - Using codec VCFCodec to read file file:///nvme/disk0/lecellier_data/WGS_GBR_data/GATK_files/RRAP-ECT01-2022-Aspat-CBHE-1718_L1_pe_aln_Amilleporav3.g.vcf.gz
20:49:48.850 INFO ValidateVariants - Done initializing engine
20:49:48.850 INFO ProgressMeter - Starting traversal
20:49:48.851 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute
20:49:58.855 INFO ProgressMeter - scaffold_2:21742739 0.2 7021000 42126000.0
20:50:08.851 INFO ProgressMeter - scaffold_4:14183951 0.3 14390000 43170000.0
20:50:18.851 INFO ProgressMeter - scaffold_6:21876105 0.5 21730000 43460000.0
20:50:28.852 INFO ProgressMeter - scaffold_9:15550968 0.7 28988000 43482000.0
20:50:38.851 INFO ProgressMeter - scaffold_13:7792730 0.8 36267000 43520400.0
20:50:48.851 INFO ProgressMeter - scaffold_33:1221954 1.0 43493000 43493000.0
20:50:58.852 INFO ProgressMeter - scaffold_165:184601 1.2 50798000 43540520.8
20:51:02.678 INFO ProgressMeter - scaffold_745:685 1.2 53637621 43592464.2
20:51:02.678 INFO ProgressMeter - Traversal complete. Processed 53637621 total variants in 1.2 minutes.
20:51:02.678 INFO ValidateVariants - Shutting down engine
[June 26, 2024 at 8:51:02 PM NCT] org.broadinstitute.hellbender.tools.walkers.variantutils.ValidateVariants done. Elapsed time: 1.24 minutes.
Runtime.totalMemory()=2046820352
-
Hi Hugo DENIS
Can you add -L ${CONTIG} parameter also when you have --include-non-variant-sites true parameter as well.
EDIT: I have tried this and you might still get an error but a different one. The best solution would be to have a bed/interval_list file to cover the whole contig from 1st base to the last given as a parameter to GenotypeGVCFs -L.
This looks like the only current solution. I will consult with the team and we may have a corrective PR for this behavior and we may push more description and requirement for that parameter.
EDIT2: This second error only occurs if your GenomicsDBImport folder name is the same as your contig name. If not you can use -L ${CONTIG} as parameter
-
Dear Gökalp,
Thank you very much for your help. When specifying -L ${CONTIG} it works perfectly, as long as the working directory is not the same as the one where the GenomicsDBImport folder is located, because of name conflicts as you reported (or one would need to change the GenomicDBimport folder name).
I have now been able to run GenotypeGVCFs on all chromosome and outputting all sites.
Again thank you very much for your support,
Best
Please sign in to leave a comment.
2 comments