java.lang.reflect.InaccessibleObjectException: Unable to make field transient java.lang.Object[] java.util.ArrayList.elementData accessible: module java.base does not "opens java.util" to unnamed module @3bf44630
when I use the MarkDuplicatesSpark,which happen a error: java.lang.reflect.InaccessibleObjectException: Unable to make field transient java.lang.Object[] java.util.ArrayList.elementData accessible: module java.base does not "opens java.util" to unnamed module @3bf44630
how can i deal with
-
Hello, for posts regarding GATK issues, we require three items to be included in the post.
- GATK version number
- Exact command used
- Complete program log & stack trace
Your post is missing these items that must be included so that we can thoroughly look into the problem. Please read our forum guidelines for more information.
-
Hi,
I encountered the same error as mentioned above while testing MarkDuplicatesSpark with some sample file. Please find below the GATK version, exact command used and error log.
GATK Version: 4.2.6.1
Exact command used: ./gatk MarkDuplicatesSpark -I /home/josh/GATK_Practice/TestFile_TO.bam -O /home/josh/GATK_Practice/TestFile_Output_TO.bamError Log:
./gatk MarkDuplicatesSpark -I /home/josh/GATK_Practice/TestFile_TO.bam -O /home/josh/GATK_Practice/TestFile_Output_TO.bam
Using GATK jar /home/josh/softwares/gatk-4.2.6.1/gatk-package-4.2.6.1-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/josh/softwares/gatk-4.2.6.1/gatk-package-4.2.6.1-local.jar MarkDuplicatesSpark -I /home/josh/GATK_Practice/TestFile_BS_TO.bam -O /home/josh/GATK_Practice/TestFile_Output_TO.bam
11:44:07.393 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/josh/softwares/gatk-4.2.6.1/gatk-package-4.2.6.1-local.jar!/com/intel/gkl/native/libgkl_compression.so
11:44:07.790 INFO MarkDuplicatesSpark - ------------------------------------------------------------
11:44:07.790 INFO MarkDuplicatesSpark - The Genome Analysis Toolkit (GATK) v4.2.6.1
11:44:07.790 INFO MarkDuplicatesSpark - For support and documentation go to https://software.broadinstitute.org/gatk/
11:44:07.790 INFO MarkDuplicatesSpark - Executing as josh@josh-Lenovo-V520S-08IKL-Desktop on Linux v5.15.0-56-generic amd64
11:44:07.790 INFO MarkDuplicatesSpark - Java runtime: OpenJDK 64-Bit Server VM v17.0.5+8-Ubuntu-2ubuntu122.04
11:44:07.790 INFO MarkDuplicatesSpark - Start Date/Time: 21 December 2022 at 11:44:07 am IST
11:44:07.790 INFO MarkDuplicatesSpark - ------------------------------------------------------------
11:44:07.791 INFO MarkDuplicatesSpark - ------------------------------------------------------------
11:44:07.791 INFO MarkDuplicatesSpark - HTSJDK Version: 2.24.1
11:44:07.791 INFO MarkDuplicatesSpark - Picard Version: 2.27.1
11:44:07.791 INFO MarkDuplicatesSpark - Built for Spark Version: 2.4.5
11:44:07.791 INFO MarkDuplicatesSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2
11:44:07.791 INFO MarkDuplicatesSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
11:44:07.791 INFO MarkDuplicatesSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
11:44:07.791 INFO MarkDuplicatesSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
11:44:07.791 INFO MarkDuplicatesSpark - Deflater: IntelDeflater
11:44:07.792 INFO MarkDuplicatesSpark - Inflater: IntelInflater
11:44:07.792 INFO MarkDuplicatesSpark - GCS max retries/reopens: 20
11:44:07.792 INFO MarkDuplicatesSpark - Requester pays: disabled
11:44:07.792 INFO MarkDuplicatesSpark - Initializing engine
11:44:07.792 INFO MarkDuplicatesSpark - Done initializing engine
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
22/12/21 11:44:08 WARN Utils: Your hostname, josh-Lenovo-V520S-08IKL-Desktop resolves to a loopback address: 127.0.1.1; using 192.168.25.72 instead (on interface enp1s0)
22/12/21 11:44:08 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address
22/12/21 11:44:08 INFO SparkContext: Running Spark version 2.4.5
22/12/21 11:44:08 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
22/12/21 11:44:08 INFO SparkContext: Submitted application: MarkDuplicatesSpark
22/12/21 11:44:08 INFO SecurityManager: Changing view acls to: josh
22/12/21 11:44:08 INFO SecurityManager: Changing modify acls to: josh
22/12/21 11:44:08 INFO SecurityManager: Changing view acls groups to:
22/12/21 11:44:08 INFO SecurityManager: Changing modify acls groups to:
22/12/21 11:44:08 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(josh); groups with view permissions: Set(); users with modify permissions: Set(josh); groups with modify permissions: Set()
22/12/21 11:44:09 INFO Utils: Successfully started service 'sparkDriver' on port 34327.
22/12/21 11:44:09 INFO SparkEnv: Registering MapOutputTracker
22/12/21 11:44:09 INFO SparkEnv: Registering BlockManagerMaster
22/12/21 11:44:09 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
22/12/21 11:44:09 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
22/12/21 11:44:09 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-566f7309-7a78-4f4f-ab00-fe0ccf3fbed2
22/12/21 11:44:09 INFO MemoryStore: MemoryStore started with capacity 2.7 GB
22/12/21 11:44:09 INFO SparkEnv: Registering OutputCommitCoordinator
22/12/21 11:44:09 INFO Utils: Successfully started service 'SparkUI' on port 4040.
22/12/21 11:44:09 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://192.168.25.72:4040
22/12/21 11:44:09 INFO Executor: Starting executor ID driver on host localhost
22/12/21 11:44:09 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 33223.
22/12/21 11:44:09 INFO NettyBlockTransferService: Server created on 192.168.25.72:33223
22/12/21 11:44:09 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
22/12/21 11:44:09 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 192.168.25.72, 33223, None)
22/12/21 11:44:09 INFO BlockManagerMasterEndpoint: Registering block manager 192.168.25.72:33223 with 2.7 GB RAM, BlockManagerId(driver, 192.168.25.72, 33223, None)
22/12/21 11:44:09 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 192.168.25.72, 33223, None)
22/12/21 11:44:09 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, 192.168.25.72, 33223, None)
11:44:10.010 INFO MarkDuplicatesSpark - Spark verbosity set to INFO (see --spark-verbosity argument)
22/12/21 11:44:10 INFO GoogleHadoopFileSystemBase: GHFS version: 1.9.4-hadoop3
22/12/21 11:44:10 WARN BlockManager: Putting block broadcast_0 failed due to exception java.lang.reflect.InaccessibleObjectException: Unable to make field transient java.lang.Object[] java.util.ArrayList.elementData accessible: module java.base does not "opens java.util" to unnamed module @62bd2070.
22/12/21 11:44:10 WARN BlockManager: Block broadcast_0 could not be removed as it was not found on disk or in memory
22/12/21 11:44:10 INFO SparkUI: Stopped Spark web UI at http://192.168.25.72:4040
22/12/21 11:44:10 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
22/12/21 11:44:10 INFO MemoryStore: MemoryStore cleared
22/12/21 11:44:10 INFO BlockManager: BlockManager stopped
22/12/21 11:44:10 INFO BlockManagerMaster: BlockManagerMaster stopped
22/12/21 11:44:10 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
22/12/21 11:44:10 INFO SparkContext: Successfully stopped SparkContext
11:44:10.932 INFO MarkDuplicatesSpark - Shutting down engine
[21 December 2022 at 11:44:10 am IST] org.broadinstitute.hellbender.tools.spark.transforms.markduplicates.MarkDuplicatesSpark done. Elapsed time: 0.06 minutes.
Runtime.totalMemory()=155189248
java.lang.reflect.InaccessibleObjectException: Unable to make field transient java.lang.Object[] java.util.ArrayList.elementData accessible: module java.base does not "opens java.util" to unnamed module @62bd2070
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
at org.apache.spark.util.SizeEstimator$$anonfun$getClassInfo$3.apply(SizeEstimator.scala:336)
at org.apache.spark.util.SizeEstimator$$anonfun$getClassInfo$3.apply(SizeEstimator.scala:330)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
at org.apache.spark.util.SizeEstimator$.getClassInfo(SizeEstimator.scala:330)
at org.apache.spark.util.SizeEstimator$.visitSingleObject(SizeEstimator.scala:222)
at org.apache.spark.util.SizeEstimator$.org$apache$spark$util$SizeEstimator$$estimate(SizeEstimator.scala:201)
at org.apache.spark.util.SizeEstimator$.estimate(SizeEstimator.scala:69)
at org.apache.spark.util.collection.SizeTracker$class.takeSample(SizeTracker.scala:78)
at org.apache.spark.util.collection.SizeTracker$class.afterUpdate(SizeTracker.scala:70)
at org.apache.spark.util.collection.SizeTrackingVector.$plus$eq(SizeTrackingVector.scala:31)
at org.apache.spark.storage.memory.DeserializedValuesHolder.storeValue(MemoryStore.scala:665)
at org.apache.spark.storage.memory.MemoryStore.putIterator(MemoryStore.scala:222)
at org.apache.spark.storage.memory.MemoryStore.putIteratorAsValues(MemoryStore.scala:299)
at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1165)
at org.apache.spark.storage.BlockManager$$anonfun$doPutIterator$1.apply(BlockManager.scala:1156)
at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:1091)
at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:1156)
at org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:914)
at org.apache.spark.storage.BlockManager.putSingle(BlockManager.scala:1481)
at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:123)
at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:88)
at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62)
at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1489)
at org.apache.spark.rdd.NewHadoopRDD.<init>(NewHadoopRDD.scala:79)
at org.apache.spark.SparkContext$$anonfun$newAPIHadoopFile$2.apply(SparkContext.scala:1160)
at org.apache.spark.SparkContext$$anonfun$newAPIHadoopFile$2.apply(SparkContext.scala:1146)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
at org.apache.spark.SparkContext.withScope(SparkContext.scala:699)
at org.apache.spark.SparkContext.newAPIHadoopFile(SparkContext.scala:1146)
at org.apache.spark.api.java.JavaSparkContext.newAPIHadoopFile(JavaSparkContext.scala:478)
at org.disq_bio.disq.impl.file.PathSplitSource.getPathSplits(PathSplitSource.java:96)
at org.disq_bio.disq.impl.formats.bgzf.BgzfBlockSource.getBgzfBlocks(BgzfBlockSource.java:66)
at org.disq_bio.disq.impl.formats.bam.BamSource.getPathChunks(BamSource.java:125)
at org.disq_bio.disq.impl.formats.sam.AbstractBinarySamSource.getReads(AbstractBinarySamSource.java:86)
at org.disq_bio.disq.HtsjdkReadsRddStorage.read(HtsjdkReadsRddStorage.java:166)
at org.disq_bio.disq.HtsjdkReadsRddStorage.read(HtsjdkReadsRddStorage.java:127)
at org.broadinstitute.hellbender.engine.spark.datasources.ReadsSparkSource.getHeader(ReadsSparkSource.java:188)
at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.initializeReads(GATKSparkTool.java:575)
at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.initializeToolInputs(GATKSparkTool.java:554)
at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.runPipeline(GATKSparkTool.java:544)
at org.broadinstitute.hellbender.engine.spark.SparkCommandLineProgram.doWork(SparkCommandLineProgram.java:31)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
at org.broadinstitute.hellbender.Main.main(Main.java:289)
22/12/21 11:44:10 INFO ShutdownHookManager: Shutdown hook called
22/12/21 11:44:10 INFO ShutdownHookManager: Deleting directory /tmp/spark-4406d1a9-755c-4b3c-b795-32885fbcd838 -
I got the same error using gatk 4.3.0.0 with Java 19 on a new machine. The error is resolved when I used Java 8 instead.
As per the docs, gatk should be run with Java 8: https://github.com/broadinstitute/gatk
-
This kind of error is definitely caused by using newer versions of Java (i.e., 17+). The newest version that GATK is tested with currently is Java 11. So Java 8 or Java 11 are required; newer versions are likely to cause problems.
Please sign in to leave a comment.
4 comments