HaplotypeCaller--Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
If you are seeing an error, please provide(REQUIRED) :
a) GATK version used:
b) Exact command used:
c) Entire error log:
If not an error, choose a category for your question(REQUIRED):
a)How do I (......)?
b) What does (......) mean?
c) Why do I see (......)?
d) Where do I find (......)?
e) Will (......) be in future releases?
gatk --java-options "-Xmx60G" HaplotypeCaller \
--emit-ref-confidence GVCF \
-R $ref_gen/ucsc.hg19.fasta \
-I $bamf/"$1".bqsr.bam \
-O $gvcf/"$1".HC.g.vcf.gz \
-bamout $bamf/"$1".realigned.bam \
--active-probability-threshold 0.0012 \
--assembly-region-padding 100 \
--bam-writer-type CALLED_HAPLOTYPES \
--linked-de-bruijn-graph \
-G StandardAnnotation \
-G AS_StandardAnnotation \
-G StandardHCAnnotation \
-OBI true
c) Entire error log:
( Log too long to paste it entirely here. Only paste the head part and end part.)
bwa/0.7.17 is loaded
samtools/1.9 is loaded
java/8.0_161 is loaded
TrimGalore/0.4.5 is loaded
cutadapt/2.3 is loaded
sample_ID = A200272
special_region_interval =
Directly run haplotypecaller
14:24:47.009 WARN GATKAnnotationPluginDescriptor - Redundant enabled annotation group (StandardAnnotation) is enabled for this tool by default
14:24:47.009 WARN GATKAnnotationPluginDescriptor - Redundant enabled annotation group (StandardHCAnnotation) is enabled for this tool by default
14:24:47.041 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/yangyxt/software/gatk-4.1.4.1/gatk-package-4.1.4.1-local.jar!/com/intel/gkl/native/libgkl_compression.so
Aug 22, 2020 2:24:47 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
14:24:47.211 INFO HaplotypeCaller - ------------------------------------------------------------
14:24:47.211 INFO HaplotypeCaller - The Genome Analysis Toolkit (GATK) v4.1.4.1
14:24:47.212 INFO HaplotypeCaller - For support and documentation go to https://software.broadinstitute.org/gatk/
14:24:47.212 INFO HaplotypeCaller - Executing as yangyxt@paedyl01 on Linux v3.10.0-1062.18.1.el7.x86_64 amd64
14:24:47.212 INFO HaplotypeCaller - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_161-b12
14:24:47.212 INFO HaplotypeCaller - Start Date/Time: August 22, 2020 2:24:47 PM HKT
14:24:47.212 INFO HaplotypeCaller - ------------------------------------------------------------
14:24:47.212 INFO HaplotypeCaller - ------------------------------------------------------------
14:24:47.212 INFO HaplotypeCaller - HTSJDK Version: 2.21.0
14:24:47.212 INFO HaplotypeCaller - Picard Version: 2.21.2
14:24:47.212 INFO HaplotypeCaller - HTSJDK Defaults.COMPRESSION_LEVEL : 2
14:24:47.212 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
14:24:47.212 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
14:24:47.212 INFO HaplotypeCaller - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
14:24:47.212 INFO HaplotypeCaller - Deflater: IntelDeflater
14:24:47.212 INFO HaplotypeCaller - Inflater: IntelInflater
14:24:47.212 INFO HaplotypeCaller - GCS max retries/reopens: 20
14:24:47.212 INFO HaplotypeCaller - Requester pays: disabled
14:24:47.212 INFO HaplotypeCaller - Initializing engine
14:24:47.483 INFO HaplotypeCaller - Done initializing engine
14:24:47.484 INFO HaplotypeCallerEngine - Tool is in reference confidence mode and the annotation, the following changes will be made to any specified annotations: 'StrandBiasBySample' will be enabled. 'ChromosomeCounts', 'FisherStrand', 'StrandOddsRatio' and 'QualByDepth' annotations have been disabled
14:24:47.491 INFO HaplotypeCallerEngine - Standard Emitting and Calling confidence set to 0.0 for reference-model confidence output
14:24:47.491 INFO HaplotypeCallerEngine - All sites annotated with PLs forced to true for reference-model confidence output
14:24:47.516 INFO NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/home/yangyxt/software/gatk-4.1.4.1/gatk-package-4.1.4.1-local.jar!/com/intel/gkl/native/libgkl_utils.so
14:24:47.518 INFO NativeLibraryLoader - Loading libgkl_pairhmm_omp.so from jar:file:/home/yangyxt/software/gatk-4.1.4.1/gatk-package-4.1.4.1-local.jar!/com/intel/gkl/native/libgkl_pairhmm_omp.so
14:24:47.556 INFO IntelPairHmm - Using CPU-supported AVX-512 instructions
14:24:47.556 INFO IntelPairHmm - Flush-to-zero (FTZ) is enabled when running PairHMM
14:24:47.557 INFO IntelPairHmm - Available threads: 10
14:24:47.557 INFO IntelPairHmm - Requested threads: 4
14:24:47.557 INFO PairHMM - Using the OpenMP multi-threaded AVX-accelerated native PairHMM implementation
14:24:47.582 INFO ProgressMeter - Starting traversal
14:24:47.582 INFO ProgressMeter - Current Locus Elapsed Minutes Regions Processed Regions/Minute
14:24:47.901 WARN InbreedingCoeff - InbreedingCoeff will not be calculated; at least 10 samples must have called genotypes
14:24:52.821 WARN DepthPerSampleHC - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
14:24:52.821 WARN StrandBiasBySample - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
Found nothing Queue has this many: 0
Path that failed to extend was junction tree: [MultiDeBruijnVertex_id_144251742_seq_TCTCCTGGAAGGTTTAGAGCCCAGCCTGGGAGTCTTTGGTGCTGAAACGGATCTGCTTAGGGGCAGCCTTGGATTAGCCCAGCTC]
Found nothing Queue has this many: 0
Path that failed to extend was junction tree: [MultiDeBruijnVertex_id_1682944867_seq_GTGGGCCAAGGGCGCCCACACCCACGCCACCCTTTCCGAAGGAACCGAGCCCCAGCCCCTCGTGGGCCAAGGGCGCCCACAGCCA]
14:24:57.629 INFO ProgressMeter - chr1:984099 0.2 3810 22753.1
Dangling End recovery killed because of a loop (findPath)
Dangling End recovery killed because of a loop (findPath)
14:25:03.209 WARN HaplotypeCallerGenotypingEngine - At position chr1:1247593-1247593 removed alt alleles where ploidy is 2 and original allele count is 73, whereas after trimming the allele count becomes 44. Alleles kept are:[G*, GCCCCCCCCCCCC, GCCCCCCCCCCCCC, GCCCCCCCCCCCCCC, GCCCCCCCCCCCCCCC, GGCCCCCCCCCCCC, GGCCCCCCCCCCCCC, GGGCCCCCCCCCCCC, GGCCCCCCCCCCCCCC, GGGCCCCCCCCCCCCC, GGGGCCCCCCCCCCCC, GGCCCCCCCCCCCCCCC, GGGCCCCCCCCCCCCCC, GGGGCCCCCCCCCCCCC, GGGGGCCCCCCCCCCCC, GGGCCCCCCCCCCCCCCC, GGGGCCCCCCCCCCCCCC, GGGGGCCCCCCCCCCCCC, GGGGGGCCCCCCCCCCCC, GGGGCCCCCCCCCCCCCCC, GGGGGGCCCCCCCCCCCCC, GCCCCCCCCCCCCCCCCCCCCCCCCCC, GCCCCCCCCCCCCCCCCCCCCCCCCCCC, GCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGCCCCCCCCCCCCCCCCCCCCCCCCCC, GCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGGCCCCCCCCCCCCCCCCCCCCCCCCCC, GCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGGCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGGGCCCCCCCCCCCCCCCCCCCCCCCCCC, GCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGGCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGGGCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGGCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC, GGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC]
Dangling End recovery killed because of a loop (findPath)
14:25:07.679 INFO ProgressMeter - chr1:1416224 0.3 5800 17316.0
Dangling End recovery killed because of a loop (findPath)
14:25:17.699 INFO ProgressMeter - chr1:1790711 0.5 7500 14941.7
Found nothing Queue has this many: 0
Path that failed to extend was junction tree: [MultiDeBruijnVertex_id_1949404492_seq_AACCTGCCATGGACCTGCAGACCAGGGCGTCACCATAGGTTGGGTCCATAAACCCACTGGATGGCCACAGGCCGGGACCCCATCG]
Found nothing Queue has this many: 0
Path that failed to extend was junction tree: [MultiDeBruijnVertex_id_1516057892_seq_ACATAGTGAGGCCTCGTGCACACCCTGCCTGGCTCCTGGCAACCTGCACTCCAGCCGATACCTGTGACTCTCAGCAAGCCCCTCT]
Found nothing Queue has this many: 0
Path that failed to extend was junction tree: [MultiDeBruijnVertex_id_379703715_seq_CTCCCTGGGGGCAGCCAAGGACCTAAAACCAATGGGTCCCAACCAAGAGGATCCCAGAGGTGAGACACAGAACGGCCAGGGCTGA]
Found nothing Queue has this many: 0
Path that failed to extend was junction tree: [MultiDeBruijnVertex_id_357277148_seq_GTCACGGCGGTGGTTAGGGTCACGGCGGTGGTTAGGGTCACGGTGGTGGTTAGGGTCACGGCGGTGGTTAGGGTCACGGTGGTGG]
Path that failed to extend was junction tree: [MultiDeBruijnVertex_id_1676112726_seq_TGTGTGTGTGTGTGTGCGCGCGCATCCAGGAGAGATGATTGCAGAGGGTGTGTGTGTGTGTGTGCACGCGCATCCAGGAGAGATG, MultiDeBruijnVertex_id_163943688_seq_GTGTGTGTGTGTGTGCGCGCGCATCCAGGAGAGATGATTGCAGAGGGTGTGTGTGTGTGTGTGCACGCGCATCCAGGAGAGATGA, MultiDeBruijnVertex_id_96402584_seq_TGTGTGTGTGTGTGCGCGCGCATCCAGGAGAGATGATTGCAGAGGGTGTGTGTGTGTGTGTGCACGCGCATCCAGGAGAGATGAC]
15:57:58.361 WARN StrandBiasBySample - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
15:57:58.367 WARN StrandBiasBySample - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
15:57:58.368 WARN StrandBiasBySample - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
15:57:58.466 WARN StrandBiasBySample - Annotation will not be calculated, genotype is not called or alleleLikelihoodMap is null
16:20:54.655 INFO VectorLoglessPairHMM - Time spent in setup for JNI call : 0.6850046510000001
16:20:54.655 INFO PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 34.145577074
16:20:54.656 INFO SmithWatermanAligner - Total compute time in java Smith-Waterman : 67.80 sec
WARNING 2020-08-22 16:20:58 SortingCollection There is not enough memory per file for buffering. Reading will be unbuffered.
16:21:00.802 INFO HaplotypeCaller - Shutting down engine
[August 22, 2020 4:21:00 PM HKT] org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller done. Elapsed time: 116.23 minutes.
Runtime.totalMemory()=23848288256
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.ArrayList.<init>(ArrayList.java:153)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.graphs.Path.<init>(Path.java:101)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.graphs.KBestHaplotype.<init>(KBestHaplotype.java:36)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.graphs.JTBestHaplotype.<init>(JTBestHaplotype.java:25)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.graphs.JunctionTreeKBestHaplotypeFinder.findBestHaplotypes(JunctionTreeKBestHaplotypeFinder.java:182)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.readthreading.ReadThreadingAssembler.findBestPaths(ReadThreadingAssembler.java:202)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.readthreading.ReadThreadingAssembler.runLocalAssembly(ReadThreadingAssembler.java:174)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.AssemblyBasedCallerUtils.assembleReads(AssemblyBasedCallerUtils.java:290)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCallerEngine.callRegion(HaplotypeCallerEngine.java:547)
at org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller.apply(HaplotypeCaller.java:240)
at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.processReadShard(AssemblyRegionWalker.java:308)
at org.broadinstitute.hellbender.engine.AssemblyRegionWalker.traverse(AssemblyRegionWalker.java:281)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1048)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:139)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:191)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:210)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:163)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:206)
at org.broadinstitute.hellbender.Main.main(Main.java:292)
Using GATK jar /home/yangyxt/software/gatk-4.1.4.1/gatk-package-4.1.4.1-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx40G -jar /home/yangyxt/software/gatk-4.1.4.1/gatk-package-4.1.4.1-local.jar HaplotypeCaller --emit-ref-confidence GVCF -R /paedwy/disk1/yangyxt/indexed_genome/ucsc.hg19.fasta -I /paedwy/disk1/yangyxt/wes/8_samples_20200819/aligned_results/A200272.bqsr.bam -O /paedwy/disk1/yangyxt/wes/8_samples_20200819/gvcfs/A200272.HC.g.vcf.gz -bamout /paedwy/disk1/yangyxt/wes/8_samples_20200819/aligned_results/A200272.realigned.bam --active-probability-threshold 0.001 --assembly-region-padding 100 --bam-writer-type CALLED_HAPLOTYPES --linked-de-bruijn-graph -G StandardAnnotation -G AS_StandardAnnotation -G StandardHCAnnotation -OBI true
real 116m18.640s
user 782m14.162s
sys 24m4.038s
If not an error, choose a category for your question(REQUIRED):
a)How do I (......)?
b) What does (......) mean?
c) Why do I see (......)?
d) Where do I find (......)?
e) Will (......) be in future releases?
-
If other users find this issue, there is a solution here
Please sign in to leave a comment.
1 comment