VariantRecalibrator failing because no data found/Standard deviation of normal must be > 0 error despite running on whole genome data from ~900 samples

Follow

Annie Forster

• June 06, 2023 12:28

Hi! I am trying to run VariantRecalibrator on vcfs I've just generated on about 900 samples of whole genome sequence data (from Plasmodium falciparum parasites - genome ~23kB). It is failing with either "no data found" or "Standard deviation of normal must be > 0" as below. The difference was I used FS and BaseQRankSum as annotations which got the no data found error where as I just used FS (shown below) which got the the standard deviation error. Looking at other forums usually this happens on smaller datasets and/or exome data but that shouldn't be a problem here - there are ~1.7million total variants. Any help would be greatly appreciated! Thanks.

a) GATK version used:

4.1.4.0

b) Exact command used:

/apps/well/gatk/4.1.4.0/gatk VariantRecalibrator \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_01_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_02_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_03_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_04_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_05_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_06_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_07_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_08_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_09_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_10_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_11_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_12_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_13_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_14_v3.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/Pf_M76611.GAMCC_full.combined.all.vcf.gz \
-V data/called_genotypes/B-VQSR_version/GAMCC_full/PF_apicoplast_genome_1.GAMCC_full.combined.all.vcf.gz \
-R /well/band/projects/pfsa/data/assemblies/Pf3D7_v3/Pf3D7_v3.fasta  \
--tranches-file results/GATK/GAMCC_full/VQSR/GAMCC_full.SNP.tranches \
--resource:3d7_hb3,known=false,training=true,truth=true,prior=15.0 /well/band/projects/pfsa/data/pf-crosses/1.0/3d7_hb3.combined.final.vcf.gz \
--resource:7g8_gb4,known=false,training=true,truth=true,prior=15.0 /well/band/projects/pfsa/data/pf-crosses/1.0/7g8_gb4.combined.final.vcf.gz \
--resource:hb3_dd2,known=false,training=true,truth=true,prior=15.0 /well/band/projects/pfsa/data/pf-crosses/1.0/hb3_dd2.combined.final.vcf.gz \
--intervals Pf3D7_01_v3:92901-457931 --intervals Pf3D7_01_v3:460312-575900 --intervals Pf3D7_02_v3:105801-447300 --intervals Pf3D7_02_v3:450451-862500 \
--intervals Pf3D7_03_v3:70631-597816 --intervals Pf3D7_03_v3:600276-1003060 --intervals Pf3D7_04_v3:91421-545800 --intervals Pf3D7_04_v3:614901-642003 \
--intervals Pf3D7_04_v3:644530-935030 --intervals Pf3D7_04_v3:983081-1143990 --intervals Pf3D7_05_v3:37901-455740 --intervals Pf3D7_05_v3:457253-1321390 \
--intervals Pf3D7_06_v3:72351-478652 --intervals Pf3D7_06_v3:480972-723117 --intervals Pf3D7_06_v3:742801-1294830 --intervals Pf3D7_07_v3:77101-508360 \
--intervals Pf3D7_07_v3:605651-809245 --intervals Pf3D7_07_v3:811717-1381600 --intervals Pf3D7_08_v3:73561-299079 --intervals Pf3D7_08_v3:301404-427430 \
--intervals Pf3D7_08_v3:467341-1365730 --intervals Pf3D7_09_v3:79101-1242137 --intervals Pf3D7_09_v3:1244484-1473560 --intervals Pf3D7_10_v3:68971-1571815 \
--intervals Pf3D7_11_v3:110001-831968 --intervals Pf3D7_11_v3:834246-2003320 --intervals Pf3D7_12_v3:60301-766654 --intervals Pf3D7_12_v3:780451-1282773 \
--intervals Pf3D7_12_v3:1285068-1688600 --intervals Pf3D7_12_v3:1745531-2163700 --intervals Pf3D7_13_v3:74414-1168127 --intervals Pf3D7_13_v3:1170426-2791900 \
--intervals Pf3D7_14_v3:35775-1071523 --intervals Pf3D7_14_v3:1075090-3255710 
-an FS         -mode SNP  \
-O results/GATK/GAMCC_full/VQSR/GAMCC_full.SNP.recal

 
c) Entire program log:

12:47:36.066 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gpfs3/apps/well/gatk/4.1.4.0/gatk-package-4.1.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so

Jun 06, 2023 12:47:36 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine

INFO: Failed to detect whether we are running on Google Compute Engine.

12:47:36.348 INFO  VariantRecalibrator - ------------------------------------------------------------

12:47:36.349 INFO  VariantRecalibrator - The Genome Analysis Toolkit (GATK) v4.1.4.0

12:47:36.349 INFO  VariantRecalibrator - For support and documentation go to https://software.broadinstitute.org/gatk/

12:47:36.349 INFO  VariantRecalibrator - Executing as ban349@compe008.hpc.in.bmrc.ox.ac.uk on Linux v3.10.0-1160.90.1.el7.x86_64 amd64

12:47:36.349 INFO  VariantRecalibrator - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_372-b07

12:47:36.349 INFO  VariantRecalibrator - Start Date/Time: June 6, 2023 12:47:36 PM BST

12:47:36.349 INFO  VariantRecalibrator - ------------------------------------------------------------

12:47:36.349 INFO  VariantRecalibrator - ------------------------------------------------------------

12:47:36.350 INFO  VariantRecalibrator - HTSJDK Version: 2.20.3

12:47:36.350 INFO  VariantRecalibrator - Picard Version: 2.21.1

12:47:36.350 INFO  VariantRecalibrator - HTSJDK Defaults.COMPRESSION_LEVEL : 2

12:47:36.350 INFO  VariantRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false

12:47:36.350 INFO  VariantRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true

12:47:36.350 INFO  VariantRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false

12:47:36.350 INFO  VariantRecalibrator - Deflater: IntelDeflater

12:47:36.350 INFO  VariantRecalibrator - Inflater: IntelInflater

12:47:36.350 INFO  VariantRecalibrator - GCS max retries/reopens: 20

12:47:36.350 INFO  VariantRecalibrator - Requester pays: disabled

12:47:36.350 INFO  VariantRecalibrator - Initializing engine

12:47:36.690 INFO  FeatureManager - Using codec VCFCodec to read file file:///well/band/projects/pfsa/data/pf-crosses/1.0/3d7_hb3.combined.final.vcf.gz

12:47:36.773 INFO  FeatureManager - Using codec VCFCodec to read file file:///well/band/projects/pfsa/data/pf-crosses/1.0/7g8_gb4.combined.final.vcf.gz

12:47:36.851 INFO  FeatureManager - Using codec VCFCodec to read file file:///well/band/projects/pfsa/data/pf-crosses/1.0/hb3_dd2.combined.final.vcf.gz

12:47:36.916 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_01_v3.GAMCC_full.combin

ed.all.vcf.gz

12:47:37.141 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_02_v3.GAMCC_full.combin

ed.all.vcf.gz

...

12:47:40.269 INFO  IntervalArgumentCollection - Processing 20782107 bp from intervals

12:47:40.297 INFO  VariantRecalibrator - Done initializing engine

12:47:40.342 INFO  TrainingSet - Found 3d7_hb3 track:   Known = false   Training = true         Truth = true    Prior = Q15.0

12:47:40.342 INFO  TrainingSet - Found 7g8_gb4 track:   Known = false   Training = true         Truth = true    Prior = Q15.0

12:47:40.342 INFO  TrainingSet - Found hb3_dd2 track:   Known = false   Training = true         Truth = true    Prior = Q15.0

12:47:40.347 WARN  GATKVariantContextUtils - Can't determine output variant file format from output file extension "recal". Defaulting to VCF.

12:47:40.420 INFO  ProgressMeter - Starting traversal

12:47:40.433 INFO  ProgressMeter -        Current Locus  Elapsed Minutes    Variants Processed  Variants/Minute

12:47:50.499 INFO  ProgressMeter -   Pf3D7_08_v3:184482              0.2                  9000          53645.9

12:48:00.801 INFO  ProgressMeter -   Pf3D7_08_v3:500003              0.3                 32000          94265.5

12:48:11.110 INFO  ProgressMeter -   Pf3D7_08_v3:852492              0.5                 62000         121263.5

12:48:21.132 INFO  ProgressMeter -  Pf3D7_08_v3:1188481              0.7                 92000         135629.9

12:48:31.363 INFO  ProgressMeter -   Pf3D7_01_v3:264236              0.8                123000         144904.8

12:48:41.466 INFO  ProgressMeter -   Pf3D7_09_v3:117322              1.0                153000         150410.4

12:48:51.755 INFO  ProgressMeter -   Pf3D7_09_v3:465896              1.2                186000         156473.5

...

12:57:20.090 INFO  ProgressMeter -  Pf3D7_07_v3:1185136              9.7               1758000         181969.7

12:57:24.208 INFO  ProgressMeter -  Pf3D7_07_v3:1371826              9.7               1771496         182073.5

12:57:24.208 INFO  ProgressMeter - Traversal complete. Processed 1771496 total variants in 9.7 minutes.

12:57:24.270 INFO  VariantDataManager - FS:      mean = 1.34     standard deviation = 6.34

12:57:24.749 INFO  VariantDataManager - Annotation order is: [FS]

12:57:24.773 INFO  VariantDataManager - Training with 19802 variants after standard deviation thresholding.

12:57:24.775 INFO  GaussianMixtureModel - Initializing model with 100 k-means iterations...

12:57:25.375 INFO  VariantRecalibratorEngine - Finished iteration 0.

12:57:25.945 INFO  VariantRecalibratorEngine - Finished iteration 5.    Current change in mixture coefficients = 0.13577

12:57:26.140 INFO  VariantRecalibratorEngine - Finished iteration 10.   Current change in mixture coefficients = 0.05343

12:57:26.287 INFO  VariantRecalibratorEngine - Finished iteration 15.   Current change in mixture coefficients = 0.05262

12:57:26.427 INFO  VariantRecalibratorEngine - Finished iteration 20.   Current change in mixture coefficients = 0.04896

12:57:26.588 INFO  VariantRecalibratorEngine - Finished iteration 25.   Current change in mixture coefficients = 0.05040

...

12:57:30.310 INFO  VariantRecalibratorEngine - Finished iteration 145.  Current change in mixture coefficients = 0.00791

12:57:30.472 INFO  VariantRecalibratorEngine - Finished iteration 150.  Current change in mixture coefficients = 0.00767

12:57:30.511 INFO  VariantRecalibratorEngine - Evaluating full set of 1112782 variants...

12:57:31.749 INFO  VariantDataManager - Selected worst 800 scoring variants --> variants with LOD <= -5.0000.

12:57:31.749 INFO  GaussianMixtureModel - Initializing model with 100 k-means iterations...

12:57:31.752 INFO  VariantRecalibratorEngine - Finished iteration 0.

12:57:31.754 INFO  VariantRecalibratorEngine - Convergence after 3 iterations!

12:57:31.819 INFO  VariantRecalibratorEngine - Evaluating full set of 1112782 variants...

12:57:31.820 WARN  VariantRecalibratorEngine - Evaluate datum returned a NaN.

12:57:31.827 INFO  VariantRecalibrator - Shutting down engine

[June 6, 2023 12:57:31 PM BST] org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator done. Elapsed time: 9.93 minutes.

Runtime.totalMemory()=563724288

java.lang.IllegalArgumentException: sd: Standard deviation of normal must be > 0

        at org.broadinstitute.hellbender.utils.Utils.validateArg(Utils.java:725)

        at org.broadinstitute.hellbender.utils.MathUtils.normalDistributionLog10(MathUtils.java:626)

        at org.broadinstitute.hellbender.tools.walkers.vqsr.GaussianMixtureModel.evaluateDatumInOneDimension(GaussianMixtureModel.java:215)

        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibratorEngine.calculateWorstPerformingAnnotation(VariantRecalibratorEngine.java:86)

        at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator.onTraversalSuccess(VariantRecalibrator.java:676)

        at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1050)

        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)

        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)

        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)

        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163)

        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206)

        at org.broadinstitute.hellbender.Main.main(Main.java:292)

Using GATK jar /gpfs3/apps/well/gatk/4.1.4.0/gatk-package-4.1.4.0-local.jar

 

0

• 

  Annie Forster

  • June 06, 2023 12:32

  Sorry not sure if error log is very readable. This might be better:
  
  12:21:56.188 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gpfs3/apps/well/gatk/4.1.4.0/gatk-package-4.1.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
  Jun 06, 2023 12:21:56 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
  INFO: Failed to detect whether we are running on Google Compute Engine.
  12:21:56.456 INFO  VariantRecalibrator - ------------------------------------------------------------
  12:21:56.457 INFO  VariantRecalibrator - The Genome Analysis Toolkit (GATK) v4.1.4.0
  12:21:56.457 INFO  VariantRecalibrator - For support and documentation go to https://software.broadinstitute.org/gatk/
  12:21:56.470 INFO  VariantRecalibrator - Executing as ban349@compe012.hpc.in.bmrc.ox.ac.uk on Linux v3.10.0-1160.90.1.el7.x86_64 amd64
  12:21:56.470 INFO  VariantRecalibrator - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_362-b08
  12:21:56.470 INFO  VariantRecalibrator - Start Date/Time: June 6, 2023 12:21:56 PM BST
  12:21:56.470 INFO  VariantRecalibrator - ------------------------------------------------------------
  12:21:56.470 INFO  VariantRecalibrator - ------------------------------------------------------------
  12:21:56.472 INFO  VariantRecalibrator - HTSJDK Version: 2.20.3
  12:21:56.472 INFO  VariantRecalibrator - Picard Version: 2.21.1
  12:21:56.472 INFO  VariantRecalibrator - HTSJDK Defaults.COMPRESSION_LEVEL : 2
  12:21:56.472 INFO  VariantRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
  12:21:56.472 INFO  VariantRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
  12:21:56.472 INFO  VariantRecalibrator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
  12:21:56.472 INFO  VariantRecalibrator - Deflater: IntelDeflater
  12:21:56.472 INFO  VariantRecalibrator - Inflater: IntelInflater
  12:21:56.472 INFO  VariantRecalibrator - GCS max retries/reopens: 20
  12:21:56.472 INFO  VariantRecalibrator - Requester pays: disabled
  12:21:56.472 INFO  VariantRecalibrator - Initializing engine
  12:21:56.870 INFO  FeatureManager - Using codec VCFCodec to read file file:///well/band/projects/pfsa/data/pf-crosses/1.0/3d7_hb3.combined.final.vcf.gz
  12:21:56.916 INFO  FeatureManager - Using codec VCFCodec to read file file:///well/band/projects/pfsa/data/pf-crosses/1.0/7g8_gb4.combined.final.vcf.gz
  12:21:56.956 INFO  FeatureManager - Using codec VCFCodec to read file file:///well/band/projects/pfsa/data/pf-crosses/1.0/hb3_dd2.combined.final.vcf.gz
  12:21:56.996 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_01_v3.GAMCC_full.combin
  ed.all.vcf.gz
  12:21:57.043 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_02_v3.GAMCC_full.combin
  ed.all.vcf.gz
  12:21:57.093 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_03_v3.GAMCC_full.combin
  ed.all.vcf.gz
  12:21:57.148 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_04_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.214 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_05_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.292 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_06_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.372 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_07_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.405 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_08_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.442 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_09_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.491 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_10_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.529 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_11_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.574 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_12_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.626 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_13_v3.GAMCC_full.combined.all.vcf.gz
  12:21:57.666 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf3D7_14_v3.GAMCC_full.combin
  ed.all.vcf.gz
  12:21:57.715 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/Pf_M76611.GAMCC_full.combined
  .all.vcf.gz
  12:21:57.749 INFO  FeatureManager - Using codec VCFCodec to read file file:///gpfs3/well/band/projects/pf-GAMCC/data/called_genotypes/B-VQSR_version/GAMCC_full/PF_apicoplast_genome_1.GAMCC_full.combined.all.vcf.gz
  12:21:59.236 INFO  IntervalArgumentCollection - Processing 20782107 bp from intervals
  12:21:59.259 INFO  VariantRecalibrator - Done initializing engine
  12:21:59.283 INFO  TrainingSet - Found 3d7_hb3 track:   Known = false   Training = true         Truth = true    Prior = Q15.0
  12:21:59.283 INFO  TrainingSet - Found 7g8_gb4 track:   Known = false   Training = true         Truth = true    Prior = Q15.0
  12:21:59.284 INFO  TrainingSet - Found hb3_dd2 track:   Known = false   Training = true         Truth = true    Prior = Q15.0
  12:21:59.289 WARN  GATKVariantContextUtils - Can't determine output variant file format from output file extension "recal". Defaulting to VCF.
  12:21:59.316 INFO  ProgressMeter - Starting traversal
  12:21:59.316 INFO  ProgressMeter -        Current Locus  Elapsed Minutes    Variants Processed  Variants/Minute
  12:22:09.960 INFO  ProgressMeter -   Pf3D7_08_v3:146111              0.2                  6000          33821.9
  12:22:20.746 INFO  ProgressMeter -   Pf3D7_08_v3:380738              0.4                 25000          69998.6
  12:22:31.195 INFO  ProgressMeter -   Pf3D7_08_v3:676513              0.5                 46000          86577.4
  12:22:41.392 INFO  ProgressMeter -   Pf3D7_08_v3:939399              0.7                 70000          99819.4
  12:22:51.411 INFO  ProgressMeter -  Pf3D7_08_v3:1207154              0.9                 94000         108265.8
  12:23:01.808 INFO  ProgressMeter -   Pf3D7_01_v3:213212              1.0                119000         114254.6
  12:23:12.128 INFO  ProgressMeter -   Pf3D7_01_v3:501335              1.2                142000         117015.3
  12:23:22.272 INFO  ProgressMeter -   Pf3D7_09_v3:297337              1.4                169000         122233.5
  12:23:32.344 INFO  ProgressMeter -   Pf3D7_09_v3:556551              1.6                194000         125123.6
  12:23:42.556 INFO  ProgressMeter -   Pf3D7_09_v3:823501              1.7                219000         127276.2
  12:23:52.797 INFO  ProgressMeter -  Pf3D7_09_v3:1096638              1.9                243000         128479.7
  12:24:02.852 INFO  ProgressMeter -  Pf3D7_09_v3:1366568              2.1                265000         128707.4
  12:24:13.005 INFO  ProgressMeter -   Pf3D7_03_v3:298334              2.2                291000         130601.6
  12:24:23.017 INFO  ProgressMeter -   Pf3D7_03_v3:553743              2.4                314000         131105.6
  12:24:33.371 INFO  ProgressMeter -   Pf3D7_03_v3:865524              2.6                338000         131641.3
  12:24:43.391 INFO  ProgressMeter -   Pf3D7_05_v3:204318              2.7                362000         132378.5
  12:24:53.718 INFO  ProgressMeter -   Pf3D7_05_v3:509495              2.9                386000         132796.6
  12:25:04.128 INFO  ProgressMeter -   Pf3D7_05_v3:795460              3.1                410000         133108.2
  12:25:14.435 INFO  ProgressMeter -  Pf3D7_05_v3:1113619              3.3                437000         134380.2
  12:25:24.542 INFO  ProgressMeter -   Pf3D7_02_v3:178753              3.4                463000         135363.0
  12:25:34.775 INFO  ProgressMeter -   Pf3D7_02_v3:494867              3.6                488000         135895.9
  12:25:45.092 INFO  ProgressMeter -   Pf3D7_02_v3:765602              3.8                512000         136064.1
  12:25:55.234 INFO  ProgressMeter -   Pf3D7_14_v3:224183              3.9                537000         136572.9
  12:26:05.291 INFO  ProgressMeter -   Pf3D7_14_v3:526402              4.1                563000         137331.0
  12:26:15.689 INFO  ProgressMeter -   Pf3D7_14_v3:823826              4.3                589000         137846.0
  12:26:25.765 INFO  ProgressMeter -  Pf3D7_14_v3:1104609              4.4                612000         137812.5
  12:26:35.780 INFO  ProgressMeter -  Pf3D7_14_v3:1402557              4.6                636000         138028.8
  12:26:45.863 INFO  ProgressMeter -  Pf3D7_14_v3:1736597              4.8                663000         138825.9
  12:26:55.962 INFO  ProgressMeter -  Pf3D7_14_v3:2039344              4.9                689000         139358.0
  12:27:06.058 INFO  ProgressMeter -  Pf3D7_14_v3:2351664              5.1                716000         140053.0
  12:27:16.368 INFO  ProgressMeter -  Pf3D7_14_v3:2645359              5.3                743000         140607.8
  12:27:26.546 INFO  ProgressMeter -  Pf3D7_14_v3:2948210              5.5                770000         141185.1
  12:27:36.636 INFO  ProgressMeter -  Pf3D7_14_v3:3235177              5.6                796000         141586.6
  12:27:46.683 INFO  ProgressMeter -   Pf3D7_11_v3:359680              5.8                821000         141809.7
  12:27:56.687 INFO  ProgressMeter -   Pf3D7_11_v3:660565              6.0                847000         142205.2
  12:28:07.059 INFO  ProgressMeter -   Pf3D7_11_v3:984787              6.1                872000         142273.3
  12:28:17.356 INFO  ProgressMeter -  Pf3D7_11_v3:1261258              6.3                896000         142207.2
  12:28:27.615 INFO  ProgressMeter -  Pf3D7_11_v3:1580495              6.5                923000         142622.1
  12:28:37.852 INFO  ProgressMeter -  Pf3D7_11_v3:1829873              6.6                948000         142722.4
  12:28:48.093 INFO  ProgressMeter -   Pf3D7_10_v3:264771              6.8                975000         143109.8
  12:28:58.154 INFO  ProgressMeter -   Pf3D7_10_v3:567638              7.0               1001000         143397.1
  12:29:08.555 INFO  ProgressMeter -   Pf3D7_10_v3:876443              7.2               1026000         143416.9
  12:29:18.619 INFO  ProgressMeter -  Pf3D7_10_v3:1183786              7.3               1050000         143409.0
  12:29:28.800 INFO  ProgressMeter -  Pf3D7_10_v3:1497421              7.5               1076000         143631.7
  12:29:39.015 INFO  ProgressMeter -   Pf3D7_04_v3:309939              7.7               1101000         143703.0
  12:29:49.207 INFO  ProgressMeter -   Pf3D7_04_v3:650358              7.8               1124000         143522.6
  12:29:59.576 INFO  ProgressMeter -   Pf3D7_04_v3:925180              8.0               1148000         143422.3
  12:30:09.680 INFO  ProgressMeter -   Pf3D7_12_v3:177396              8.2               1173000         143526.0
  12:30:19.830 INFO  ProgressMeter -   Pf3D7_12_v3:485246              8.3               1199000         143732.2
  12:30:30.119 INFO  ProgressMeter -   Pf3D7_12_v3:785168              8.5               1224000         143773.9
  12:30:40.524 INFO  ProgressMeter -  Pf3D7_12_v3:1133779              8.7               1251000         144011.6
  12:30:50.716 INFO  ProgressMeter -  Pf3D7_12_v3:1427300              8.9               1275000         143959.6
  12:31:00.826 INFO  ProgressMeter -  Pf3D7_12_v3:1752847              9.0               1297000         143709.3
  12:31:11.222 INFO  ProgressMeter -  Pf3D7_12_v3:2044652              9.2               1323000         143828.8
  12:31:21.548 INFO  ProgressMeter -   Pf3D7_13_v3:283411              9.4               1350000         144068.6
  12:31:31.616 INFO  ProgressMeter -   Pf3D7_13_v3:558322              9.5               1375000         144155.2
  12:31:41.644 INFO  ProgressMeter -   Pf3D7_13_v3:881952              9.7               1401000         144351.6
  12:31:52.153 INFO  ProgressMeter -  Pf3D7_13_v3:1203671              9.9               1426000         144323.2
  12:32:02.314 INFO  ProgressMeter -  Pf3D7_13_v3:1538895             10.0               1451000         144378.6
  12:32:12.423 INFO  ProgressMeter -  Pf3D7_13_v3:1839284             10.2               1478000         144640.3
  12:32:22.472 INFO  ProgressMeter -  Pf3D7_13_v3:2131509             10.4               1504000         144811.5
  12:32:32.671 INFO  ProgressMeter -  Pf3D7_13_v3:2448401             10.6               1532000         145131.9
  12:32:42.747 INFO  ProgressMeter -  Pf3D7_13_v3:2722969             10.7               1558000         145283.9
  12:32:53.194 INFO  ProgressMeter -   Pf3D7_06_v3:317093             10.9               1584000         145348.2
  12:33:03.466 INFO  ProgressMeter -   Pf3D7_06_v3:601005             11.1               1607000         145178.0
  12:33:13.710 INFO  ProgressMeter -   Pf3D7_06_v3:916493             11.2               1632000         145197.0
  12:33:24.083 INFO  ProgressMeter -  Pf3D7_06_v3:1222017             11.4               1659000         145363.3
  12:33:34.238 INFO  ProgressMeter -   Pf3D7_07_v3:286814             11.6               1685000         145484.0
  12:33:44.306 INFO  ProgressMeter -   Pf3D7_07_v3:610034             11.7               1707000         145278.7
  12:33:54.520 INFO  ProgressMeter -   Pf3D7_07_v3:909814             11.9               1732000         145301.2
  12:34:04.820 INFO  ProgressMeter -  Pf3D7_07_v3:1185136             12.1               1758000         145388.8
  12:34:09.677 INFO  ProgressMeter -  Pf3D7_07_v3:1371826             12.2               1771496         145530.4
  12:34:09.677 INFO  ProgressMeter - Traversal complete. Processed 1771496 total variants in 12.2 minutes.
  12:34:09.745 INFO  VariantDataManager - BaseQRankSum:    mean = 0.24     standard deviation = 1.70
  12:34:09.865 INFO  VariantDataManager - FS:      mean = 1.34     standard deviation = 6.34
  12:34:10.369 INFO  VariantDataManager - Annotation order is: [FS, BaseQRankSum]
  12:34:10.399 INFO  VariantDataManager - Training with 19802 variants after standard deviation thresholding.
  12:34:10.419 INFO  GaussianMixtureModel - Initializing model with 100 k-means iterations...
  12:34:11.271 INFO  VariantRecalibratorEngine - Finished iteration 0.
  12:34:12.380 INFO  VariantRecalibratorEngine - Finished iteration 5.    Current change in mixture coefficients = 0.28838
  12:34:12.707 INFO  VariantRecalibratorEngine - Finished iteration 10.   Current change in mixture coefficients = 0.87413
  12:34:13.043 INFO  VariantRecalibratorEngine - Finished iteration 15.   Current change in mixture coefficients = 0.79183
  12:34:13.397 INFO  VariantRecalibratorEngine - Finished iteration 20.   Current change in mixture coefficients = 0.08640
  12:34:13.720 INFO  VariantRecalibratorEngine - Finished iteration 25.   Current change in mixture coefficients = 0.03875
  12:34:14.084 INFO  VariantRecalibratorEngine - Finished iteration 30.   Current change in mixture coefficients = 0.02271
  12:34:14.425 INFO  VariantRecalibratorEngine - Finished iteration 35.   Current change in mixture coefficients = 0.01462
  12:34:14.783 INFO  VariantRecalibratorEngine - Finished iteration 40.   Current change in mixture coefficients = 0.01025
  12:34:15.133 INFO  VariantRecalibratorEngine - Finished iteration 45.   Current change in mixture coefficients = 0.00766
  12:34:15.659 INFO  VariantRecalibratorEngine - Finished iteration 50.   Current change in mixture coefficients = 0.00604
  12:34:16.011 INFO  VariantRecalibratorEngine - Finished iteration 55.   Current change in mixture coefficients = 0.00484
  12:34:16.371 INFO  VariantRecalibratorEngine - Finished iteration 60.   Current change in mixture coefficients = 0.00386
  12:34:16.707 INFO  VariantRecalibratorEngine - Finished iteration 65.   Current change in mixture coefficients = 0.00308
  12:34:17.049 INFO  VariantRecalibratorEngine - Finished iteration 70.   Current change in mixture coefficients = 0.00245
  12:34:17.399 INFO  VariantRecalibratorEngine - Finished iteration 75.   Current change in mixture coefficients = 0.00195
  12:34:17.399 INFO  VariantRecalibratorEngine - Convergence after 75 iterations!
  12:34:17.512 INFO  VariantRecalibratorEngine - Evaluating full set of 1112782 variants...
  12:34:17.512 WARN  VariantRecalibratorEngine - Evaluate datum returned a NaN.
  12:34:17.558 INFO  VariantDataManager - Selected worst 0 scoring variants --> variants with LOD <= -5.0000.
  12:34:17.568 INFO  VariantRecalibrator - Shutting down engine
  [June 6, 2023 12:34:17 PM BST] org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator done. Elapsed time: 12.36 minutes.
  Runtime.totalMemory()=599588864
  java.lang.IllegalArgumentException: No data found.
          at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibratorEngine.generateModel(VariantRecalibratorEngine.java:34)
          at org.broadinstitute.hellbender.tools.walkers.vqsr.VariantRecalibrator.onTraversalSuccess(VariantRecalibrator.java:655)
          at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1050)
          at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
          at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
          at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
          at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163)
          at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206)
          at org.broadinstitute.hellbender.Main.main(Main.java:292)
  Using GATK jar /gpfs3/apps/well/gatk/4.1.4.0/gatk-package-4.1.4.0-local.jar

  0

  Comment actions Permalink
• 

  Annie Forster

  • June 07, 2023 09:46

  Fixed by adding full set of annotations:

  -an QD -an MQ -an MQRankSum -an ReadPosRankSum -an FS -an SOR 

   

  0

  Comment actions Permalink

Please sign in to leave a comment.