Out of memory error using OpenJDK vs Oracle JDK for CNV-LOH
Previously I have been able to run the CNV-LOH pipeline with no problems whatsoever. However recently Oracle JDK made a change to their license that has prevented our institution from using their software. We've switched to OpenJDK (OpenJ9) and we start to run into OOM errors. I can manually go in and fix this by setting the jvm memory to -Xmx24G.
However, such a hardcoded change is not ideal, as the demand changes depending on whether the input is WGS and number of samples (we are running CNV-LOH in parallel) .
I am hoping you could shed some light on how GATK previously tuned the amount of memory.
I did notice JAVA_TOOL_OPTIONS: -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/igm/temp were no longer being included in the arguments, however upon adding them I still ran into the same error.
Can you please provide
a) GATK version used
gatk 4.1.3
b) Exact GATK commands used
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /igm/apps/gatk/gatk-4.1.3.0/gatk-package-4.1.3.0-local.jar CollectAllelicCounts
c) The entire error log if applicable.
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3210)
at java.util.Arrays.copyOf(Arrays.java:3181)
at java.util.ArrayList.grow(ArrayList.java:265)
at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:239)
at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:231)
at java.util.ArrayList.addAll(ArrayList.java:583)
at htsjdk.samtools.BAMFileSpan.merge(BAMFileSpan.java:307)
at htsjdk.samtools.BAMFileReader.getFileSpan(BAMFileReader.java:940)
at htsjdk.samtools.BAMFileReader.createIndexIterator(BAMFileReader.java:952)
at htsjdk.samtools.BAMFileReader.query(BAMFileReader.java:612)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:533)
at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.queryOverlapping(SamReader.java:405)
at org.broadinstitute.hellbender.utils.iterators.SamReaderQueryingIterator.loadNextIterator(SamReaderQueryingIterator.java:125)
at org.broadinstitute.hellbender.utils.iterators.SamReaderQueryingIterator.<init>(SamReaderQueryingIterator.java:66)
at org.broadinstitute.hellbender.engine.ReadsDataSource.prepareIteratorsForTraversal(ReadsDataSource.java:404)
at org.broadinstitute.hellbender.engine.ReadsDataSource.iterator(ReadsDataSource.java:330)
at java.lang.Iterable.spliterator(Iterable.java:101)
at org.broadinstitute.hellbender.utils.Utils.stream(Utils.java:1098)
at org.broadinstitute.hellbender.engine.GATKTool.getTransformedReadStream(GATKTool.java:377)
at org.broadinstitute.hellbender.engine.LocusWalker.traverse(LocusWalker.java:164)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1048)
-
You may want to use OpenJDK Hotspot instead of OpenJ9.
Can you check with AdoptOpenJDK? -
Switching to OpenJDK Hotspot solved the issue!
Thanks you!
-
I am facing the same problem when switching from Oracle JDK to java-1.8.0-openjdk-1.8.0.372. I will try OpenJDK Hotspot.
Please sign in to leave a comment.
3 comments