Fatal error occuring during GATK4 PathSeqPipeline tool run
Dear GATK guru's,
I am encountering a fatal error when I run the PathSeqPipeline tool using GATK 4.1.4.1 and the following command:
/Users/jude/gatk-4.1.4.1/gatk-package-4.1.4.1-local.jar PathSeqPipelineSpark -I SRR1839791.raw.bam --filter-bwa-image //Users/jude/GRCh38.primary_assembly.genome.fa --kmer-file //Users/jude/GRCh38.hss --min-clipped-read-length 70 --microbe-fasta //Users/jude//gatk-4.1.4.1/pathseq_microbe.fa --microbe-bwa-image //Users/jude//gatk-4.1.4.1/pathseq_microbe.fa.img --taxonomy-file //Users/jude//gatk-4.1.4.1/pathseq_taxonomy.db --output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.bam --scores-output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.txt
SRR1839791.raw.bam is derived from paired read fastq files converted to an unaligned bam using FastqToSam. I had edited the fastq headers to the standard format prior using awk and tr.
The fatal error is:
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x0000000e17a8cb25, pid=3931, tid=0x000000000000ec03
#
# JRE version: Java(TM) SE Runtime Environment (8.0_161-b12) (build 1.8.0_161-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.161-b12 mixed mode bsd-amd64 )
# Problematic frame:
# C [libbwa.8008850932864317462.jnilib+0xeb25] bwa_mem2idx+0xa5
#
# Core dump written. Default location: /cores/core or core.3931
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
I have done a hardware test on my RAM which passed and tried varying "-XmxG", if this allocation is set too low there is a different Java stack heap error related to insufficient memory.
I have attached an excerpt of the error log which suggests a potential 'Register to memory mapping' problem:
Register to memory mapping:
RAX=0x00007f9fdd226030 is an unknown value
RBX=0xabababababa99008 is an unknown value
RCX=0x0000000000000001 is an unknown value
RDX=0x0000000000000200 is an unknown value
RSP=0x0000700011f286a0 is pointing into the stack for thread: 0x00007f9fd607e800
RBP=0x0000700011f286f0 is pointing into the stack for thread: 0x00007f9fd607e800
RSI=0x0000000000000040 is an unknown value
RDI=0x0000000000000261 is an unknown value
R8 =0x0000000000000030 is an unknown value
R9 =0x00007f9fdd226030 is an unknown value
R10=0x00000000ffffffc0 is an unknown value
R11=0x00007f91be05dc00 is an unknown value
R12=0x00007f9fdd2248c0 is an unknown value
R13=0x0000000e17aa68fc: induceSA+0xf2c in /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libbwa.8008850932864317462.jnilib at 0x0000000e17a7e000
R14=0x0000000e207fd000 is an unknown value
R15=0x0000000e17aa6a26: induceSA+0x1056 in /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libbwa.8008850932864317462.jnilib at 0x0000000e17a7e000
and the excerpt with my Mac Pro system details running OsX Sierra 10.12:
--------------- S Y S T E M ---------------
OS:Bsduname:Darwin 16.7.0 Darwin Kernel Version 16.7.0: Sun Jun 2 20:26:31 PDT 2019; root:xnu-3789.73.50~1/RELEASE_X86_64 x86_64
rlimit: STACK 8192k, CORE infinity, NPROC 709, NOFILE 10240, AS infinity
load average:6.88 6.84 5.74
CPU:total 24 (initial active 24) (6 cores per cpu, 2 threads per core) family 6 model 44 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, aes, clmul, ht, tsc, tscinvbit, tscinv
Memory: 4k page, physical 83886080k(935068k free)
/proc/meminfo:
vm_info: Java HotSpot(TM) 64-Bit Server VM (25.161-b12) for bsd-amd64 JRE (1.8.0_161-b12), built on Dec 19 2017 16:22:20 by "java_re" with gcc 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)
I am not ruling out a hardware issue and I can provide the full log if that will help identify the cause assuming it is a bug.
thanks very much for your help
Dr Jude Taylor
Torrens University
Sydney
-
Hi Jude Taylor, yes, could you provide the entire error log?
-
The whole error log:
Thanks Jude
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x0000000e17a8cb25, pid=3931, tid=0x000000000000ec03
#
# JRE version: Java(TM) SE Runtime Environment (8.0_161-b12) (build 1.8.0_161-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.161-b12 mixed mode bsd-amd64 )
# Problematic frame:
# C [libbwa.8008850932864317462.jnilib+0xeb25] bwa_mem2idx+0xa5
#
# Core dump written. Default location: /cores/core or core.3931
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- T H R E A D ---------------
Current thread (0x00007f9fd607e800): JavaThread "Executor task launch worker for task 634" daemon [_thread_in_native, id=60419, stack(0x0000700011e2a000,0x0000700011f2a000)]
siginfo: si_signo: 11 (SIGSEGV), si_code: 0 (unknown), si_addr: 0x0000000000000000
Registers:
RAX=0x00007f9fdd226030, RBX=0xabababababa99008, RCX=0x0000000000000001, RDX=0x0000000000000200
RSP=0x0000700011f286a0, RBP=0x0000700011f286f0, RSI=0x0000000000000040, RDI=0x0000000000000261
R8 =0x0000000000000030, R9 =0x00007f9fdd226030, R10=0x00000000ffffffc0, R11=0x00007f91be05dc00
R12=0x00007f9fdd2248c0, R13=0x0000000e17aa68fc, R14=0x0000000e207fd000, R15=0x0000000e17aa6a26
RIP=0x0000000e17a8cb25, EFLAGS=0x0000000000010202, ERR=0x0000000000000000
TRAPNO=0x000000000000000d
Top of Stack: (sp=0x0000700011f286a0)
0x0000700011f286a0: 0000700011f286d0 00007fffc44dd4d6
0x0000700011f286b0: 00000000bbd6d067 00007f9fdd2248c0
0x0000700011f286c0: 0000000000000083 00007f9fdd2248c0
0x0000700011f286d0: 00007f9fdd226030 00007f9fdd2248c0
0x0000700011f286e0: 0000000000000083 0000000e207fd000
0x0000700011f286f0: 0000700011f287b0 0000000e17a7f66d
0x0000700011f28700: 0000000000000030 000181a401000002
0x0000700011f28710: 00000000000eec5a 00000014000001f5
0x0000700011f28720: 0000000000000000 000000005f0d7611
0x0000700011f28730: 0000000000000000 000000005e05d67e
0x0000700011f28740: 0000000000000000 000000005e3fa92a
0x0000700011f28750: 0000000000000000 000000005e05d664
0x0000700011f28760: 0000000000000000 00000000bbd6d067
0x0000700011f28770: 00000000005deb70 0000000000001000
0x0000700011f28780: 0000000000000000 0000000000000000
0x0000700011f28790: 0000000000000000 0000000000000083
0x0000700011f287a0: 0000000000000000 0000700011f28868
0x0000700011f287b0: 0000700011f287f0 0000000e17a7f337
0x0000700011f287c0: 0000000e17a7f2d0 0000000e1d2439b0
0x0000700011f287d0: 0000000000000000 0000000e1d2439b0
0x0000700011f287e0: 0000700011f28868 00007f9fd607e800
0x0000700011f287f0: 0000700011f28848 0000000112a5b0a7
0x0000700011f28800: 0000000112a5adc2 0000700011f28808
0x0000700011f28810: 0000000e1d2439b0 0000700011f28868
0x0000700011f28820: 0000000e2065ba78 0000000000000000
0x0000700011f28830: 0000000e1d2439b0 0000000000000000
0x0000700011f28840: 0000700011f28868 0000700011f288b8
0x0000700011f28850: 0000000112a4adb0 000000012223b8b8
0x0000700011f28860: 0000000112a54cf7 0000000ab158deb8
0x0000700011f28870: 0000000ab1a400d8 0000700011f28878
0x0000700011f28880: 0000000e1d2429a6 0000700011f288f8
0x0000700011f28890: 0000000e2065ba78 0000000000000000
Instructions: (pc=0x0000000e17a8cb25)
0x0000000e17a8cb05: 9c d1 60 04 00 00 bf 30 00 00 00 ba 41 01 00 00
0x0000000e17a8cb15: 4c 89 ee 4c 89 f9 e8 a0 3c 01 00 49 89 44 24 08
0x0000000e17a8cb25: 49 8b 4c 1e 28 48 89 48 28 49 8b 4c 1e 20 48 89
0x0000000e17a8cb35: 48 20 49 8b 4c 1e 18 48 89 48 18 49 8b 4c 1e 10
Register to memory mapping:
RAX=0x00007f9fdd226030 is an unknown value
RBX=0xabababababa99008 is an unknown value
RCX=0x0000000000000001 is an unknown value
RDX=0x0000000000000200 is an unknown value
RSP=0x0000700011f286a0 is pointing into the stack for thread: 0x00007f9fd607e800
RBP=0x0000700011f286f0 is pointing into the stack for thread: 0x00007f9fd607e800
RSI=0x0000000000000040 is an unknown value
RDI=0x0000000000000261 is an unknown value
R8 =0x0000000000000030 is an unknown value
R9 =0x00007f9fdd226030 is an unknown value
R10=0x00000000ffffffc0 is an unknown value
R11=0x00007f91be05dc00 is an unknown value
R12=0x00007f9fdd2248c0 is an unknown value
R13=0x0000000e17aa68fc: induceSA+0xf2c in /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libbwa.8008850932864317462.jnilib at 0x0000000e17a7e000
R14=0x0000000e207fd000 is an unknown value
R15=0x0000000e17aa6a26: induceSA+0x1056 in /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libbwa.8008850932864317462.jnilib at 0x0000000e17a7e000
Stack: [0x0000700011e2a000,0x0000700011f2a000], sp=0x0000700011f286a0, free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libbwa.8008850932864317462.jnilib+0xeb25] bwa_mem2idx+0xa5
C [libbwa.8008850932864317462.jnilib+0x166d] jnibwa_openIndex+0x6d
C [libbwa.8008850932864317462.jnilib+0x1337] Java_org_broadinstitute_hellbender_utils_bwa_BwaMemIndex_openIndex+0x67
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndex.openIndex(Ljava/lang/String;)J+0
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndex.<init>(Ljava/lang/String;)V+30
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndexCache.getInstance(Ljava/lang/String;)Lorg/broadinstitute/hellbender/utils/bwa/BwaMemIndex;+28
j org.broadinstitute.hellbender.tools.spark.pathseq.PSBwaFilter.<init>(Ljava/lang/String;IIIZ)V+6
j org.broadinstitute.hellbender.tools.spark.pathseq.PSFilter.lambda$doBwaFilter$9ded5e08$1(Ljava/lang/String;IIILjava/util/Iterator;)Ljava/util/Iterator;+9
j org.broadinstitute.hellbender.tools.spark.pathseq.PSFilter$$Lambda$214.call(Ljava/lang/Object;)Ljava/util/Iterator;+20
J 7476 C1 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(Ljava/lang/Object;)Ljava/lang/Object; (9 bytes) @ 0x0000000113ed6254 [0x0000000113ed5ac0+0x794]
J 7484 C1 org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (17 bytes) @ 0x0000000113edabdc [0x0000000113eda640+0x59c]
j org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator;+27
J 11215 C2 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x00000001147f44b8 [0x00000001147f4400+0xb8]
j org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator;+24
J 11215 C2 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x00000001147f44b8 [0x00000001147f4400+0xb8]
j org.apache.spark.scheduler.ShuffleMapTask.runTask(Lorg/apache/spark/TaskContext;)Lorg/apache/spark/scheduler/MapStatus;+219
j org.apache.spark.scheduler.ShuffleMapTask.runTask(Lorg/apache/spark/TaskContext;)Ljava/lang/Object;+2
J 11375 C1 org.apache.spark.scheduler.Task.run(JILorg/apache/spark/metrics/MetricsSystem;)Ljava/lang/Object; (422 bytes) @ 0x00000001148b5934 [0x00000001148b3380+0x25b4]
J 10615 C1 org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply()Ljava/lang/Object; (51 bytes) @ 0x00000001138d3cf4 [0x00000001138d3aa0+0x254]
J 7832 C1 org.apache.spark.util.Utils$.tryWithSafeFinally(Lscala/Function0;Lscala/Function0;)Ljava/lang/Object; (182 bytes) @ 0x0000000112d9e014 [0x0000000112d9df00+0x114]
J 10593 C1 org.apache.spark.executor.Executor$TaskRunner.run()V (2494 bytes) @ 0x000000011467ce14 [0x00000001146794c0+0x3954]
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
V [libjvm.dylib+0x2ef9aa] JavaCalls::call_helper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x6ae
V [libjvm.dylib+0x2f014e] JavaCalls::call_virtual(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x164
V [libjvm.dylib+0x2f02fa] JavaCalls::call_virtual(JavaValue*, Handle, KlassHandle, Symbol*, Symbol*, Thread*)+0x4a
V [libjvm.dylib+0x34aae5] thread_entry(JavaThread*, Thread*)+0x7c
V [libjvm.dylib+0x56edef] JavaThread::thread_main_inner()+0x9b
V [libjvm.dylib+0x5704ea] JavaThread::run()+0x1c2
V [libjvm.dylib+0x48d83a] java_start(Thread*)+0xf6
C [libsystem_pthread.dylib+0x393b] _pthread_body+0xb4
C [libsystem_pthread.dylib+0x3887] _pthread_body+0x0
C [libsystem_pthread.dylib+0x308d] thread_start+0xd
C 0x0000000000000000
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndex.openIndex(Ljava/lang/String;)J+0
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndex.<init>(Ljava/lang/String;)V+30
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndexCache.getInstance(Ljava/lang/String;)Lorg/broadinstitute/hellbender/utils/bwa/BwaMemIndex;+28
j org.broadinstitute.hellbender.tools.spark.pathseq.PSBwaFilter.<init>(Ljava/lang/String;IIIZ)V+6
j org.broadinstitute.hellbender.tools.spark.pathseq.PSFilter.lambda$doBwaFilter$9ded5e08$1(Ljava/lang/String;IIILjava/util/Iterator;)Ljava/util/Iterator;+9
j org.broadinstitute.hellbender.tools.spark.pathseq.PSFilter$$Lambda$214.call(Ljava/lang/Object;)Ljava/util/Iterator;+20
J 7476 C1 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(Ljava/lang/Object;)Ljava/lang/Object; (9 bytes) @ 0x0000000113ed6254 [0x0000000113ed5ac0+0x794]
J 7484 C1 org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (17 bytes) @ 0x0000000113edabdc [0x0000000113eda640+0x59c]
j org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator;+27
J 11215 C2 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x00000001147f44b8 [0x00000001147f4400+0xb8]
j org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator;+24
J 11215 C2 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x00000001147f44b8 [0x00000001147f4400+0xb8]
j org.apache.spark.scheduler.ShuffleMapTask.runTask(Lorg/apache/spark/TaskContext;)Lorg/apache/spark/scheduler/MapStatus;+219
j org.apache.spark.scheduler.ShuffleMapTask.runTask(Lorg/apache/spark/TaskContext;)Ljava/lang/Object;+2
J 11375 C1 org.apache.spark.scheduler.Task.run(JILorg/apache/spark/metrics/MetricsSystem;)Ljava/lang/Object; (422 bytes) @ 0x00000001148b5934 [0x00000001148b3380+0x25b4]
J 10615 C1 org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply()Ljava/lang/Object; (51 bytes) @ 0x00000001138d3cf4 [0x00000001138d3aa0+0x254]
J 7832 C1 org.apache.spark.util.Utils$.tryWithSafeFinally(Lscala/Function0;Lscala/Function0;)Ljava/lang/Object; (182 bytes) @ 0x0000000112d9e014 [0x0000000112d9df00+0x114]
J 10593 C1 org.apache.spark.executor.Executor$TaskRunner.run()V (2494 bytes) @ 0x000000011467ce14 [0x00000001146794c0+0x3954]
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x00007f9fd8b1d000 JavaThread "task-result-getter-3" daemon [_thread_blocked, id=79363, stack(0x0000700014399000,0x0000700014499000)]
0x00007f9fd2a72800 JavaThread "task-result-getter-2" daemon [_thread_blocked, id=78851, stack(0x0000700014296000,0x0000700014396000)]
0x00007f9fd6990000 JavaThread "task-result-getter-1" daemon [_thread_blocked, id=75267, stack(0x0000700013b81000,0x0000700013c81000)]
0x00007f9fda1d7800 JavaThread "Executor task launch worker for task 630" daemon [_thread_blocked, id=71683, stack(0x000070001346c000,0x000070001356c000)]
0x00007f9fdc002000 JavaThread "Executor task launch worker for task 635" daemon [_thread_blocked, id=71171, stack(0x0000700013369000,0x0000700013469000)]
0x00007f9fdc801000 JavaThread "Executor task launch worker for task 644" daemon [_thread_blocked, id=70659, stack(0x0000700013266000,0x0000700013366000)]
0x00007f9fd9876000 JavaThread "Executor task launch worker for task 628" daemon [_thread_blocked, id=70147, stack(0x0000700013163000,0x0000700013263000)]
0x00007f9fd4609000 JavaThread "Executor task launch worker for task 625" daemon [_thread_blocked, id=69635, stack(0x0000700013060000,0x0000700013160000)]
0x00007f9fd5b4c800 JavaThread "Executor task launch worker for task 641" daemon [_thread_blocked, id=69123, stack(0x0000700012f5d000,0x000070001305d000)]
0x00007f9fd4608800 JavaThread "Executor task launch worker for task 646" daemon [_thread_blocked, id=68611, stack(0x0000700012e5a000,0x0000700012f5a000)]
0x00007f9fd6930000 JavaThread "Executor task launch worker for task 645" daemon [_thread_blocked, id=68099, stack(0x0000700012d57000,0x0000700012e57000)]
0x00007f9fd50e4000 JavaThread "Executor task launch worker for task 640" daemon [_thread_blocked, id=67587, stack(0x0000700012c54000,0x0000700012d54000)]
0x00007f9fd91a5000 JavaThread "Executor task launch worker for task 643" daemon [_thread_blocked, id=67075, stack(0x0000700012b51000,0x0000700012c51000)]
0x00007f9fda1d7000 JavaThread "Executor task launch worker for task 623" daemon [_thread_blocked, id=66563, stack(0x0000700012a4e000,0x0000700012b4e000)]
0x00007f9fda124800 JavaThread "Executor task launch worker for task 627" daemon [_thread_blocked, id=66051, stack(0x000070001294b000,0x0000700012a4b000)]
0x00007f9fd28ea800 JavaThread "Executor task launch worker for task 633" daemon [_thread_blocked, id=65539, stack(0x0000700012848000,0x0000700012948000)]
0x00007f9fd5ae1800 JavaThread "Executor task launch worker for task 626" daemon [_thread_blocked, id=65027, stack(0x0000700012745000,0x0000700012845000)]
0x00007f9fd6080800 JavaThread "Executor task launch worker for task 624" daemon [_thread_blocked, id=64515, stack(0x0000700012642000,0x0000700012742000)]
0x00007f9fd4616000 JavaThread "Executor task launch worker for task 629" daemon [_thread_blocked, id=64003, stack(0x000070001253f000,0x000070001263f000)]
0x00007f9fda127800 JavaThread "Executor task launch worker for task 637" daemon [_thread_blocked, id=63491, stack(0x000070001243c000,0x000070001253c000)]
0x00007f9fd91a4800 JavaThread "Executor task launch worker for task 638" daemon [_thread_blocked, id=62979, stack(0x0000700012339000,0x0000700012439000)]
0x00007f9fd2b2d000 JavaThread "Executor task launch worker for task 632" daemon [_thread_blocked, id=62467, stack(0x0000700012236000,0x0000700012336000)]
0x00007f9fd8927800 JavaThread "Executor task launch worker for task 631" daemon [_thread_blocked, id=61955, stack(0x0000700012133000,0x0000700012233000)]
0x00007f9fd6080000 JavaThread "Executor task launch worker for task 642" daemon [_thread_blocked, id=61443, stack(0x0000700012030000,0x0000700012130000)]
0x00007f9fd607f000 JavaThread "Executor task launch worker for task 636" daemon [_thread_blocked, id=60931, stack(0x0000700011f2d000,0x000070001202d000)]
=>0x00007f9fd607e800 JavaThread "Executor task launch worker for task 634" daemon [_thread_in_native, id=60419, stack(0x0000700011e2a000,0x0000700011f2a000)]
0x00007f9fd9176000 JavaThread "task-result-getter-0" daemon [_thread_blocked, id=4103, stack(0x0000700011d27000,0x0000700011e27000)]
0x00007f9fd4605000 JavaThread "Executor task launch worker for task 639" daemon [_thread_blocked, id=5127, stack(0x0000700011c24000,0x0000700011d24000)]
0x00007f9fd4243800 JavaThread "org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner" daemon [_thread_blocked, id=59907, stack(0x0000700011b21000,0x0000700011c21000)]
0x00007f9fd35b8800 JavaThread "element-tracking-store-worker" daemon [_thread_blocked, id=59395, stack(0x0000700011a1e000,0x0000700011b1e000)]
0x00007f9fd9811800 JavaThread "spark-listener-group-executorManagement" daemon [_thread_blocked, id=58883, stack(0x000070001191b000,0x0000700011a1b000)]
0x00007f9fd9811000 JavaThread "spark-listener-group-appStatus" daemon [_thread_blocked, id=58371, stack(0x0000700011818000,0x0000700011918000)]
0x00007f9fd17b6000 JavaThread "context-cleaner-periodic-gc" daemon [_thread_blocked, id=57859, stack(0x0000700011715000,0x0000700011815000)]
0x00007f9fd5222000 JavaThread "Spark Context Cleaner" daemon [_thread_blocked, id=57347, stack(0x0000700011612000,0x0000700011712000)]
0x00007f9fd5215000 JavaThread "shuffle-server-5-1" daemon [_thread_in_native, id=56835, stack(0x000070001150f000,0x000070001160f000)]
0x00007f9fd632c000 JavaThread "driver-heartbeater" daemon [_thread_blocked, id=56323, stack(0x000070001140c000,0x000070001150c000)]
0x00007f9fd3282800 JavaThread "dag-scheduler-event-loop" daemon [_thread_blocked, id=55811, stack(0x0000700011309000,0x0000700011409000)]
0x00007f9fd0f34800 JavaThread "netty-rpc-env-timeout" daemon [_thread_blocked, id=55299, stack(0x0000700011206000,0x0000700011306000)]
0x00007f9fd2b3c000 JavaThread "Timer-1" daemon [_thread_blocked, id=54787, stack(0x0000700011103000,0x0000700011203000)]
0x00007f9fd2afa800 JavaThread "Timer-0" daemon [_thread_blocked, id=54275, stack(0x0000700011000000,0x0000700011100000)]
0x00007f9fd1c2d000 JavaThread "heartbeat-receiver-event-loop-thread" daemon [_thread_blocked, id=53763, stack(0x0000700010efd000,0x0000700010ffd000)]
0x00007f9fd9812800 JavaThread "SparkUI-63" daemon [_thread_blocked, id=53251, stack(0x0000700010dfa000,0x0000700010efa000)]
0x00007f9fd896b800 JavaThread "SparkUI-62-acceptor-2@55688591-ServerConnector@1eeea7c8{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}" daemon [_thread_blocked, id=52739, stack(0x0000700010cf7000,0x0000700010df7000)]
0x00007f9fd896a800 JavaThread "SparkUI-61-acceptor-1@52193822-ServerConnector@1eeea7c8{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}" daemon [_thread_in_native, id=52227, stack(0x0000700010bf4000,0x0000700010cf4000)]
0x00007f9fd896a000 JavaThread "SparkUI-60-acceptor-0@10294bbd-ServerConnector@1eeea7c8{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}" daemon [_thread_blocked, id=51715, stack(0x0000700010af1000,0x0000700010bf1000)]
0x00007f9fd8969000 JavaThread "SparkUI-59" daemon [_thread_in_native, id=51203, stack(0x00007000109ee000,0x0000700010aee000)]
0x00007f9fd8968800 JavaThread "SparkUI-58" daemon [_thread_in_native, id=50691, stack(0x00007000108eb000,0x00007000109eb000)]
0x00007f9fd8967800 JavaThread "SparkUI-57" daemon [_thread_in_native, id=50179, stack(0x00007000107e8000,0x00007000108e8000)]
0x00007f9fd8962800 JavaThread "SparkUI-56" daemon [_thread_in_native, id=49667, stack(0x00007000106e5000,0x00007000107e5000)]
0x00007f9fd3281800 JavaThread "RemoteBlock-temp-file-clean-thread" daemon [_thread_blocked, id=49155, stack(0x00007000105e2000,0x00007000106e2000)]
0x00007f9fd5264800 JavaThread "map-output-dispatcher-7" daemon [_thread_blocked, id=48643, stack(0x00007000104df000,0x00007000105df000)]
0x00007f9fd5264000 JavaThread "map-output-dispatcher-6" daemon [_thread_blocked, id=48131, stack(0x00007000103dc000,0x00007000104dc000)]
0x00007f9fd0f88000 JavaThread "map-output-dispatcher-5" daemon [_thread_blocked, id=47619, stack(0x00007000102d9000,0x00007000103d9000)]
0x00007f9fd168f800 JavaThread "map-output-dispatcher-4" daemon [_thread_blocked, id=47107, stack(0x00007000101d6000,0x00007000102d6000)]
0x00007f9fd168f000 JavaThread "map-output-dispatcher-3" daemon [_thread_blocked, id=46595, stack(0x00007000100d3000,0x00007000101d3000)]
0x00007f9fd16f0000 JavaThread "map-output-dispatcher-2" daemon [_thread_blocked, id=46083, stack(0x000070000ffd0000,0x00007000100d0000)]
0x00007f9fd16ef800 JavaThread "map-output-dispatcher-1" daemon [_thread_blocked, id=45571, stack(0x000070000fecd000,0x000070000ffcd000)]
0x00007f9fd16ee800 JavaThread "map-output-dispatcher-0" daemon [_thread_blocked, id=45059, stack(0x000070000fdca000,0x000070000feca000)]
0x00007f9fd50d4800 JavaThread "rpc-server-3-1" daemon [_thread_in_native, id=44547, stack(0x000070000fcc7000,0x000070000fdc7000)]
0x00007f9fd2b63800 JavaThread "dispatcher-event-loop-23" daemon [_thread_blocked, id=44035, stack(0x000070000fbc4000,0x000070000fcc4000)]
0x00007f9fd2b63000 JavaThread "dispatcher-event-loop-22" daemon [_thread_blocked, id=43523, stack(0x000070000fac1000,0x000070000fbc1000)]
0x00007f9fd2b62000 JavaThread "dispatcher-event-loop-21" daemon [_thread_blocked, id=43011, stack(0x000070000f9be000,0x000070000fabe000)]
0x00007f9fd1fec800 JavaThread "dispatcher-event-loop-20" daemon [_thread_blocked, id=42499, stack(0x000070000f8bb000,0x000070000f9bb000)]
0x00007f9fd1fe9800 JavaThread "dispatcher-event-loop-19" daemon [_thread_blocked, id=41987, stack(0x000070000f7b8000,0x000070000f8b8000)]
0x00007f9fd627b000 JavaThread "dispatcher-event-loop-18" daemon [_thread_blocked, id=41475, stack(0x000070000f6b5000,0x000070000f7b5000)]
0x00007f9fd16ec800 JavaThread "dispatcher-event-loop-17" daemon [_thread_blocked, id=40963, stack(0x000070000f5b2000,0x000070000f6b2000)]
0x00007f9fd16eb800 JavaThread "dispatcher-event-loop-16" daemon [_thread_blocked, id=40451, stack(0x000070000f4af000,0x000070000f5af000)]
0x00007f9fd16eb000 JavaThread "dispatcher-event-loop-15" daemon [_thread_blocked, id=39939, stack(0x000070000f3ac000,0x000070000f4ac000)]
0x00007f9fd16ea000 JavaThread "dispatcher-event-loop-14" daemon [_thread_blocked, id=39427, stack(0x000070000f2a9000,0x000070000f3a9000)]
0x00007f9fd16e9800 JavaThread "dispatcher-event-loop-13" daemon [_thread_blocked, id=38915, stack(0x000070000f1a6000,0x000070000f2a6000)]
0x00007f9fd118d800 JavaThread "dispatcher-event-loop-12" daemon [_thread_blocked, id=38403, stack(0x000070000f0a3000,0x000070000f1a3000)]
0x00007f9fd118c800 JavaThread "dispatcher-event-loop-11" daemon [_thread_blocked, id=37891, stack(0x000070000efa0000,0x000070000f0a0000)]
0x00007f9fd3370000 JavaThread "dispatcher-event-loop-10" daemon [_thread_blocked, id=37379, stack(0x000070000ee9d000,0x000070000ef9d000)]
0x00007f9fd336f000 JavaThread "dispatcher-event-loop-9" daemon [_thread_blocked, id=36867, stack(0x000070000ed9a000,0x000070000ee9a000)]
0x00007f9fd336e800 JavaThread "dispatcher-event-loop-8" daemon [_thread_blocked, id=36355, stack(0x000070000ec97000,0x000070000ed97000)]
0x00007f9fd336d800 JavaThread "dispatcher-event-loop-7" daemon [_thread_blocked, id=35843, stack(0x000070000eb94000,0x000070000ec94000)]
0x00007f9fd336d000 JavaThread "dispatcher-event-loop-6" daemon [_thread_blocked, id=35331, stack(0x000070000ea91000,0x000070000eb91000)]
0x00007f9fd3637000 JavaThread "dispatcher-event-loop-5" daemon [_thread_blocked, id=34819, stack(0x000070000e98e000,0x000070000ea8e000)]
0x00007f9fd2b61800 JavaThread "dispatcher-event-loop-4" daemon [_thread_blocked, id=34307, stack(0x000070000e88b000,0x000070000e98b000)]
0x00007f9fd1fe8800 JavaThread "dispatcher-event-loop-3" daemon [_thread_blocked, id=33795, stack(0x000070000e788000,0x000070000e888000)]
0x00007f9fd1fe8000 JavaThread "dispatcher-event-loop-2" daemon [_thread_blocked, id=33283, stack(0x000070000e685000,0x000070000e785000)]
0x00007f9fd882e800 JavaThread "dispatcher-event-loop-1" daemon [_thread_blocked, id=32771, stack(0x000070000e582000,0x000070000e682000)]
0x00007f9fd0a2c000 JavaThread "dispatcher-event-loop-0" daemon [_thread_blocked, id=32263, stack(0x000070000e47f000,0x000070000e57f000)]
0x00007f9fd4821800 JavaThread "Service Thread" daemon [_thread_blocked, id=30467, stack(0x000070000e279000,0x000070000e379000)]
0x00007f9fd080e000 JavaThread "C1 CompilerThread11" daemon [_thread_blocked, id=29955, stack(0x000070000e176000,0x000070000e276000)]
0x00007f9fd080a800 JavaThread "C1 CompilerThread10" daemon [_thread_blocked, id=29443, stack(0x000070000e073000,0x000070000e173000)]
0x00007f9fd1849000 JavaThread "C1 CompilerThread9" daemon [_thread_blocked, id=28931, stack(0x000070000df70000,0x000070000e070000)]
0x00007f9fd480e800 JavaThread "C1 CompilerThread8" daemon [_thread_blocked, id=28419, stack(0x000070000de6d000,0x000070000df6d000)]
0x00007f9fd480d800 JavaThread "C2 CompilerThread7" daemon [_thread_blocked, id=27907, stack(0x000070000dd6a000,0x000070000de6a000)]
0x00007f9fd480d000 JavaThread "C2 CompilerThread6" daemon [_thread_blocked, id=27395, stack(0x000070000dc67000,0x000070000dd67000)]
0x00007f9fd480c000 JavaThread "C2 CompilerThread5" daemon [_thread_blocked, id=26883, stack(0x000070000db64000,0x000070000dc64000)]
0x00007f9fd4803800 JavaThread "C2 CompilerThread4" daemon [_thread_blocked, id=26371, stack(0x000070000da61000,0x000070000db61000)]
0x00007f9fd4802800 JavaThread "C2 CompilerThread3" daemon [_thread_blocked, id=25859, stack(0x000070000d95e000,0x000070000da5e000)]
0x00007f9fd4802000 JavaThread "C2 CompilerThread2" daemon [_thread_blocked, id=25347, stack(0x000070000d85b000,0x000070000d95b000)]
0x00007f9fd4801000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=24835, stack(0x000070000d758000,0x000070000d858000)]
0x00007f9fd183e000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=24323, stack(0x000070000d655000,0x000070000d755000)]
0x00007f9fd1839000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=23811, stack(0x000070000d552000,0x000070000d652000)]
0x00007f9fd3805000 JavaThread "Finalizer" daemon [_thread_blocked, id=19715, stack(0x000070000d44f000,0x000070000d54f000)]
0x00007f9fd3802000 JavaThread "Reference Handler" daemon [_thread_blocked, id=19203, stack(0x000070000d34c000,0x000070000d44c000)]
0x00007f9fd1802800 JavaThread "main" [_thread_blocked, id=7171, stack(0x000070000bf10000,0x000070000c010000)]
Other Threads:
0x00007f9fd305e800 VMThread [stack: 0x000070000d249000,0x000070000d349000] [id=18691]
0x00007f9fd2800800 WatcherThread [stack: 0x000070000e37c000,0x000070000e47c000] [id=30979]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
Heap:
PSYoungGen total 17376256K, used 10157622K [0x0000000977180000, 0x0000000da1c00000, 0x0000000da1c00000)
eden space 17278464K, 58% used [0x0000000977180000,0x0000000bde903da0,0x0000000d95b00000)
from space 97792K, 75% used [0x0000000d9bb80000,0x0000000da0389d68,0x0000000da1b00000)
to space 98816K, 0% used [0x0000000d95b00000,0x0000000d95b00000,0x0000000d9bb80000)
ParOldGen total 33457664K, used 28380046K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5ee3868,0x000000091bd80000)
Metaspace used 78805K, capacity 80392K, committed 80640K, reserved 81920K
Card table byte_map: [0x0000000da1c00000,0x0000000da8001000] byte_map_base: 0x0000000da12f2000
Marking Bits: (ParMarkBitMap*) 0x000000010d0f76c0
Begin Bits: [0x0000000da8001000, 0x0000000dda001000)
End Bits: [0x0000000dda001000, 0x0000000e0c001000)
Polling page: 0x00000001096a5000
CodeCache: size=245760Kb used=33116Kb max_used=33136Kb free=212644Kb
bounds [0x0000000112a43000, 0x0000000114af3000, 0x0000000121a43000]
total_blobs=10120 nmethods=9470 adapters=560
compilation: enabled
Compilation events (10 events):
Event: 540.624 Thread 0x00007f9fd4801000 nmethod 12157 0x0000000114a97d10 code [0x0000000114a97e80, 0x0000000114a98108]
Event: 540.624 Thread 0x00007f9fd4801000 12163 ! 4 sun.nio.ch.ChannelInputStream::read (95 bytes)
Event: 540.625 Thread 0x00007f9fd4802800 nmethod 12141 0x0000000114ab4250 code [0x0000000114ab4460, 0x0000000114ab5138]
Event: 540.630 Thread 0x00007f9fd4803800 nmethod 12143 0x0000000114ab8750 code [0x0000000114ab8980, 0x0000000114ab99b8]
Event: 540.632 Thread 0x00007f9fd480c000 nmethod 12142 0x0000000114abdf10 code [0x0000000114abe140, 0x0000000114abf538]
Event: 540.632 Thread 0x00007f9fd480d000 nmethod 12153 0x0000000114abb690 code [0x0000000114abb900, 0x0000000114abc7b0]
Event: 540.649 Thread 0x00007f9fd183e000 nmethod 12149 0x0000000114ac4bd0 code [0x0000000114ac4f00, 0x0000000114ac6fc0]
Event: 540.654 Thread 0x00007f9fd480d800 nmethod 12160 0x0000000114acc910 code [0x0000000114accc20, 0x0000000114acf070]
Event: 540.660 Thread 0x00007f9fd4802000 nmethod 12159 0x0000000114ad60d0 code [0x0000000114ad6440, 0x0000000114ad9210]
Event: 540.662 Thread 0x00007f9fd4801000 nmethod 12163 0x0000000114adda90 code [0x0000000114adddc0, 0x0000000114ae01f0]
GC Heap History (10 events):
Event: 514.242 GC heap before
{Heap before GC invocations=118 (full 13):
PSYoungGen total 17374720K, used 17359299K [0x0000000977180000, 0x0000000da1a00000, 0x0000000da1c00000)
eden space 17275392K, 100% used [0x0000000977180000,0x0000000d95800000,0x0000000d95800000)
from space 99328K, 84% used [0x0000000d95800000,0x0000000d9a9f0f00,0x0000000d9b900000)
to space 98304K, 0% used [0x0000000d9ba00000,0x0000000d9ba00000,0x0000000da1a00000)
ParOldGen total 33457664K, used 28379506K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5e5cb48,0x000000091bd80000)
Metaspace used 78214K, capacity 79634K, committed 79872K, reserved 79872K
Event: 514.295 GC heap after
Heap after GC invocations=118 (full 13):
PSYoungGen total 17375744K, used 89924K [0x0000000977180000, 0x0000000da1b00000, 0x0000000da1c00000)
eden space 17277440K, 0% used [0x0000000977180000,0x0000000977180000,0x0000000d95a00000)
from space 98304K, 91% used [0x0000000d9ba00000,0x0000000da11d11e8,0x0000000da1a00000)
to space 98304K, 0% used [0x0000000d95a00000,0x0000000d95a00000,0x0000000d9ba00000)
ParOldGen total 33457664K, used 28379626K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5e7ab48,0x000000091bd80000)
Metaspace used 78214K, capacity 79634K, committed 79872K, reserved 79872K
}
Event: 519.734 GC heap before
{Heap before GC invocations=119 (full 13):
PSYoungGen total 17375744K, used 17367364K [0x0000000977180000, 0x0000000da1b00000, 0x0000000da1c00000)
eden space 17277440K, 100% used [0x0000000977180000,0x0000000d95a00000,0x0000000d95a00000)
from space 98304K, 91% used [0x0000000d9ba00000,0x0000000da11d11e8,0x0000000da1a00000)
to space 98304K, 0% used [0x0000000d95a00000,0x0000000d95a00000,0x0000000d9ba00000)
ParOldGen total 33457664K, used 28379651K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5e80d58,0x000000091bd80000)
Metaspace used 78255K, capacity 79684K, committed 79872K, reserved 79872K
Event: 519.784 GC heap after
Heap after GC invocations=119 (full 13):
PSYoungGen total 17375744K, used 89700K [0x0000000977180000, 0x0000000da1c00000, 0x0000000da1c00000)
eden space 17277440K, 0% used [0x0000000977180000,0x0000000977180000,0x0000000d95a00000)
from space 98304K, 91% used [0x0000000d95a00000,0x0000000d9b1991e8,0x0000000d9ba00000)
to space 98304K, 0% used [0x0000000d9bc00000,0x0000000d9bc00000,0x0000000da1c00000)
ParOldGen total 33457664K, used 28379843K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5eb0d58,0x000000091bd80000)
Metaspace used 78255K, capacity 79684K, committed 79872K, reserved 79872K
}
Event: 525.255 GC heap before
{Heap before GC invocations=120 (full 13):
PSYoungGen total 17375744K, used 17367140K [0x0000000977180000, 0x0000000da1c00000, 0x0000000da1c00000)
eden space 17277440K, 100% used [0x0000000977180000,0x0000000d95a00000,0x0000000d95a00000)
from space 98304K, 91% used [0x0000000d95a00000,0x0000000d9b1991e8,0x0000000d9ba00000)
to space 98304K, 0% used [0x0000000d9bc00000,0x0000000d9bc00000,0x0000000da1c00000)
ParOldGen total 33457664K, used 28379846K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5eb1868,0x000000091bd80000)
Metaspace used 78267K, capacity 79699K, committed 79872K, reserved 79872K
Event: 525.302 GC heap after
Heap after GC invocations=120 (full 13):
PSYoungGen total 17377792K, used 82255K [0x0000000977180000, 0x0000000da1c00000, 0x0000000da1c00000)
eden space 17279488K, 0% used [0x0000000977180000,0x0000000977180000,0x0000000d95c00000)
from space 98304K, 83% used [0x0000000d9bc00000,0x0000000da0c53f50,0x0000000da1c00000)
to space 98304K, 0% used [0x0000000d95c00000,0x0000000d95c00000,0x0000000d9bc00000)
ParOldGen total 33457664K, used 28379990K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5ed5868,0x000000091bd80000)
Metaspace used 78267K, capacity 79699K, committed 79872K, reserved 79872K
}
Event: 530.624 GC heap before
{Heap before GC invocations=121 (full 13):
PSYoungGen total 17377792K, used 17361743K [0x0000000977180000, 0x0000000da1c00000, 0x0000000da1c00000)
eden space 17279488K, 100% used [0x0000000977180000,0x0000000d95c00000,0x0000000d95c00000)
from space 98304K, 83% used [0x0000000d9bc00000,0x0000000da0c53f50,0x0000000da1c00000)
to space 98304K, 0% used [0x0000000d95c00000,0x0000000d95c00000,0x0000000d9bc00000)
ParOldGen total 33457664K, used 28379990K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5ed5868,0x000000091bd80000)
Metaspace used 78267K, capacity 79699K, committed 79872K, reserved 79872K
Event: 530.667 GC heap after
Heap after GC invocations=121 (full 13):
PSYoungGen total 17360384K, used 80815K [0x0000000977180000, 0x0000000da1b00000, 0x0000000da1c00000)
eden space 17279488K, 0% used [0x0000000977180000,0x0000000977180000,0x0000000d95c00000)
from space 80896K, 99% used [0x0000000d95c00000,0x0000000d9aaebea8,0x0000000d9ab00000)
to space 97792K, 0% used [0x0000000d9bb80000,0x0000000d9bb80000,0x0000000da1b00000)
ParOldGen total 33457664K, used 28379998K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5ed7868,0x000000091bd80000)
Metaspace used 78267K, capacity 79699K, committed 79872K, reserved 79872K
}
Event: 536.445 GC heap before
{Heap before GC invocations=122 (full 13):
PSYoungGen total 17360384K, used 17360303K [0x0000000977180000, 0x0000000da1b00000, 0x0000000da1c00000)
eden space 17279488K, 100% used [0x0000000977180000,0x0000000d95c00000,0x0000000d95c00000)
from space 80896K, 99% used [0x0000000d95c00000,0x0000000d9aaebea8,0x0000000d9ab00000)
to space 97792K, 0% used [0x0000000d9bb80000,0x0000000d9bb80000,0x0000000da1b00000)
ParOldGen total 33457664K, used 28379998K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5ed7868,0x000000091bd80000)
Metaspace used 78267K, capacity 79699K, committed 79872K, reserved 79872K
Event: 536.489 GC heap after
Heap after GC invocations=122 (full 13):
PSYoungGen total 17376256K, used 73767K [0x0000000977180000, 0x0000000da1c00000, 0x0000000da1c00000)
eden space 17278464K, 0% used [0x0000000977180000,0x0000000977180000,0x0000000d95b00000)
from space 97792K, 75% used [0x0000000d9bb80000,0x0000000da0389d68,0x0000000da1b00000)
to space 98816K, 0% used [0x0000000d95b00000,0x0000000d95b00000,0x0000000d9bb80000)
ParOldGen total 33457664K, used 28380046K [0x0000000121c00000, 0x000000091bd80000, 0x0000000977180000)
object space 33457664K, 84% used [0x0000000121c00000,0x00000007e5ee3868,0x000000091bd80000)
Metaspace used 78267K, capacity 79699K, committed 79872K, reserved 79872K
}
Deoptimization events (10 events):
Event: 540.426 Thread 0x00007f9fd6080000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000113587184 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 540.426 Thread 0x00007f9fd4609000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000113587184 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 540.426 Thread 0x00007f9fda1d7800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000113587184 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 540.426 Thread 0x00007f9fdc002000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x0000000113587184 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 540.576 Thread 0x00007f9fd607e800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000114056064 method=java.io.File.getName()Ljava/lang/String; @ 16
Event: 540.577 Thread 0x00007f9fd607e800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x00000001149d78d8 method=java.lang.Long.stringSize(J)I @ 11
Event: 540.648 Thread 0x00007f9fd607e800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000114a8b530 method=sun.nio.fs.UnixNativeDispatcher.copyToNativeBuffer(Lsun/nio/fs/UnixPath;)Lsun/nio/fs/NativeBuffer; @ 32
Event: 540.650 Thread 0x00007f9fd607e800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000114a98edc method=sun.nio.ch.Util$BufferCache.get(I)Ljava/nio/ByteBuffer; @ 47
Event: 540.651 Thread 0x00007f9fd607e800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x0000000113634334 method=java.nio.Buffer.limit(I)Ljava/nio/Buffer; @ 33
Event: 540.749 Thread 0x00007f9fd607e800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000011384d76c method=java.io.UnixFileSystem.normalize(Ljava/lang/String;)Ljava/lang/String; @ 34
Classes redefined (0 events):
No events
Internal exceptions (10 events):
Event: 15.136 Thread 0x00007f9fd3282800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x00000009b689a970) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 15.136 Thread 0x00007f9fd3282800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x00000009b689c890) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 15.136 Thread 0x00007f9fd3282800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x00000009b689eaf8) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 15.157 Thread 0x00007f9fd1fec800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000996aa7f58) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 15.157 Thread 0x00007f9fd1fec800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000996ac3ad8) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 15.160 Thread 0x00007f9fd1fec800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000996b33248) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 216.115 Thread 0x00007f9fd6080000 Implicit null exception at 0x000000011313f736 to 0x000000011313ff3d
Event: 216.235 Thread 0x00007f9fd6930000 Implicit null exception at 0x0000000112e48e22 to 0x0000000112e49125
Event: 521.120 Thread 0x00007f9fd2b63800 Implicit null exception at 0x00000001132b7159 to 0x00000001132b7c29
Event: 540.334 Thread 0x00007f9fd1fe8000 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000bd5a88d80) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Events (10 events):
Event: 540.650 loading class java/nio/channels/Channels$1 done
Event: 540.650 Thread 0x00007f9fd607e800 Uncommon trap: trap_request=0xffffff65 fr.pc=0x0000000114a98edc
Event: 540.650 Thread 0x00007f9fd607e800 DEOPT PACKING pc=0x0000000114a98edc sp=0x0000700011f28300
Event: 540.650 Thread 0x00007f9fd607e800 DEOPT UNPACKING pc=0x0000000112a8873a sp=0x0000700011f28260 mode 2
Event: 540.651 Thread 0x00007f9fd607e800 Uncommon trap: trap_request=0xffffff65 fr.pc=0x0000000113634334
Event: 540.651 Thread 0x00007f9fd607e800 DEOPT PACKING pc=0x0000000113634334 sp=0x0000700011f285b0
Event: 540.651 Thread 0x00007f9fd607e800 DEOPT UNPACKING pc=0x0000000112a8873a sp=0x0000700011f28570 mode 2
Event: 540.749 Thread 0x00007f9fd607e800 Uncommon trap: trap_request=0xffffff65 fr.pc=0x000000011384d76c
Event: 540.749 Thread 0x00007f9fd607e800 DEOPT PACKING pc=0x000000011384d76c sp=0x0000700011f28620
Event: 540.749 Thread 0x00007f9fd607e800 DEOPT UNPACKING pc=0x0000000112a8873a sp=0x0000700011f285d0 mode 2
Dynamic libraries:
0x00000000391b0000 /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa
0x00000000391b0000 /System/Library/Frameworks/Security.framework/Versions/A/Security
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
0x00000000391b0000 /usr/lib/libz.1.dylib
0x00000000391b0000 /usr/lib/libSystem.B.dylib
0x00000000391b0000 /usr/lib/libobjc.A.dylib
0x00000000391b0000 /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
0x00000000391b0000 /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
0x00000000391b0000 /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
0x00000000391b0000 /System/Library/Frameworks/CoreData.framework/Versions/A/CoreData
0x00000000391b0000 /System/Library/PrivateFrameworks/RemoteViewServices.framework/Versions/A/RemoteViewServices
0x00000000391b0000 /System/Library/PrivateFrameworks/UIFoundation.framework/Versions/A/UIFoundation
0x00000000391b0000 /System/Library/PrivateFrameworks/DFRFoundation.framework/Versions/A/DFRFoundation
0x00000000391b0000 /usr/lib/libenergytrace.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/SkyLight.framework/Versions/A/SkyLight
0x00000000391b0000 /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
0x00000000391b0000 /usr/lib/libScreenReader.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate
0x00000000391b0000 /System/Library/Frameworks/IOSurface.framework/Versions/A/IOSurface
0x00000000391b0000 /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
0x00000000391b0000 /System/Library/Frameworks/AudioUnit.framework/Versions/A/AudioUnit
0x00000000391b0000 /System/Library/PrivateFrameworks/DataDetectorsCore.framework/Versions/A/DataDetectorsCore
0x00000000391b0000 /System/Library/PrivateFrameworks/DesktopServicesPriv.framework/Versions/A/DesktopServicesPriv
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox
0x00000000391b0000 /usr/lib/libicucore.A.dylib
0x00000000391b0000 /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SpeechRecognition.framework/Versions/A/SpeechRecognition
0x00000000391b0000 /usr/lib/libauto.dylib
0x00000000391b0000 /usr/lib/libxml2.2.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreUI.framework/Versions/A/CoreUI
0x00000000391b0000 /System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio
0x00000000391b0000 /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
0x00000000391b0000 /usr/lib/liblangid.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/MultitouchSupport.framework/Versions/A/MultitouchSupport
0x00000000391b0000 /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
0x00000000391b0000 /usr/lib/libDiagnosticMessagesClient.dylib
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
0x00000000391b0000 /System/Library/PrivateFrameworks/PerformanceAnalysis.framework/Versions/A/PerformanceAnalysis
0x00000000391b0000 /System/Library/PrivateFrameworks/GenerationalStorage.framework/Versions/A/GenerationalStorage
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
0x00000000391b0000 /System/Library/Frameworks/CoreImage.framework/Versions/A/CoreImage
0x00000000391b0000 /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
0x00000000391b0000 /System/Library/PrivateFrameworks/Backup.framework/Versions/A/Backup
0x00000000391b0000 /usr/lib/libarchive.2.dylib
0x00000000391b0000 /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
0x00000000391b0000 /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
0x00000000391b0000 /usr/lib/libCRFSuite.dylib
0x00000000391b0000 /usr/lib/libc++.1.dylib
0x00000000391b0000 /usr/lib/libc++abi.dylib
0x00000000391b0000 /usr/lib/system/libcache.dylib
0x00000000391b0000 /usr/lib/system/libcommonCrypto.dylib
0x00000000391b0000 /usr/lib/system/libcompiler_rt.dylib
0x00000000391b0000 /usr/lib/system/libcopyfile.dylib
0x00000000391b0000 /usr/lib/system/libcorecrypto.dylib
0x00000000391b0000 /usr/lib/system/libdispatch.dylib
0x00000000391b0000 /usr/lib/system/libdyld.dylib
0x00000000391b0000 /usr/lib/system/libkeymgr.dylib
0x00000000391b0000 /usr/lib/system/liblaunch.dylib
0x00000000391b0000 /usr/lib/system/libmacho.dylib
0x00000000391b0000 /usr/lib/system/libquarantine.dylib
0x00000000391b0000 /usr/lib/system/libremovefile.dylib
0x00000000391b0000 /usr/lib/system/libsystem_asl.dylib
0x00000000391b0000 /usr/lib/system/libsystem_blocks.dylib
0x00000000391b0000 /usr/lib/system/libsystem_c.dylib
0x00000000391b0000 /usr/lib/system/libsystem_configuration.dylib
0x00000000391b0000 /usr/lib/system/libsystem_coreservices.dylib
0x00000000391b0000 /usr/lib/system/libsystem_coretls.dylib
0x00000000391b0000 /usr/lib/system/libsystem_dnssd.dylib
0x00000000391b0000 /usr/lib/system/libsystem_info.dylib
0x00000000391b0000 /usr/lib/system/libsystem_kernel.dylib
0x00000000391b0000 /usr/lib/system/libsystem_m.dylib
0x00000000391b0000 /usr/lib/system/libsystem_malloc.dylib
0x00000000391b0000 /usr/lib/system/libsystem_network.dylib
0x00000000391b0000 /usr/lib/system/libsystem_networkextension.dylib
0x00000000391b0000 /usr/lib/system/libsystem_notify.dylib
0x00000000391b0000 /usr/lib/system/libsystem_platform.dylib
0x00000000391b0000 /usr/lib/system/libsystem_pthread.dylib
0x00000000391b0000 /usr/lib/system/libsystem_sandbox.dylib
0x00000000391b0000 /usr/lib/system/libsystem_secinit.dylib
0x00000000391b0000 /usr/lib/system/libsystem_symptoms.dylib
0x00000000391b0000 /usr/lib/system/libsystem_trace.dylib
0x00000000391b0000 /usr/lib/system/libunwind.dylib
0x00000000391b0000 /usr/lib/system/libxpc.dylib
0x00000000391b0000 /usr/lib/libbsm.0.dylib
0x00000000391b0000 /usr/lib/system/libkxld.dylib
0x00000000391b0000 /usr/lib/libcoretls.dylib
0x00000000391b0000 /usr/lib/libcoretls_cfhelpers.dylib
0x00000000391b0000 /usr/lib/libOpenScriptingUtil.dylib
0x00000000391b0000 /usr/lib/libpam.2.dylib
0x00000000391b0000 /usr/lib/libsqlite3.dylib
0x00000000391b0000 /usr/lib/libxar.1.dylib
0x00000000391b0000 /usr/lib/libbz2.1.0.dylib
0x00000000391b0000 /usr/lib/liblzma.5.dylib
0x00000000391b0000 /usr/lib/libnetwork.dylib
0x00000000391b0000 /usr/lib/libpcap.A.dylib
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList
0x00000000391b0000 /System/Library/Frameworks/NetFS.framework/Versions/A/NetFS
0x00000000391b0000 /System/Library/PrivateFrameworks/NetAuth.framework/Versions/A/NetAuth
0x00000000391b0000 /System/Library/PrivateFrameworks/login.framework/Versions/A/Frameworks/loginsupport.framework/Versions/A/loginsupport
0x00000000391b0000 /System/Library/PrivateFrameworks/TCC.framework/Versions/A/TCC
0x00000000391b0000 /usr/lib/libmecabra.dylib
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/ATS
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ColorSync.framework/Versions/A/ColorSync
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/HIServices
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LangAnalysis.framework/Versions/A/LangAnalysis
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/PrintCore.framework/Versions/A/PrintCore
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/QD.framework/Versions/A/QD
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/SpeechSynthesis.framework/Versions/A/SpeechSynthesis
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImage.framework/Versions/A/vImage
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvDSP.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBNNS.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libQuadrature.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvMisc.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLinearAlgebra.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libSparseBLAS.dylib
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/Resources/libFontParser.dylib
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/Resources/libFontRegistry.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libJPEG.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libTIFF.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libPng.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libGIF.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libJP2.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libRadiance.dylib
0x00000000391b0000 /usr/lib/libcompression.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/AppleJPEG.framework/Versions/A/AppleJPEG
0x00000000391b0000 /usr/lib/libcups.2.dylib
0x00000000391b0000 /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos
0x00000000391b0000 /System/Library/Frameworks/GSS.framework/Versions/A/GSS
0x00000000391b0000 /usr/lib/libresolv.9.dylib
0x00000000391b0000 /usr/lib/libiconv.2.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/Heimdal.framework/Versions/A/Heimdal
0x00000000391b0000 /usr/lib/libheimdal-asn1.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenDirectory.framework/Versions/A/OpenDirectory
0x00000000391b0000 /System/Library/PrivateFrameworks/CommonAuth.framework/Versions/A/CommonAuth
0x00000000391b0000 /System/Library/Frameworks/OpenDirectory.framework/Versions/A/Frameworks/CFOpenDirectory.framework/Versions/A/CFOpenDirectory
0x00000000391b0000 /System/Library/Frameworks/SecurityFoundation.framework/Versions/A/SecurityFoundation
0x00000000391b0000 /System/Library/PrivateFrameworks/LanguageModeling.framework/Versions/A/LanguageModeling
0x00000000391b0000 /usr/lib/libmarisa.dylib
0x00000000391b0000 /usr/lib/libChineseTokenizer.dylib
0x00000000391b0000 /usr/lib/libcmph.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreEmoji.framework/Versions/A/CoreEmoji
0x00000000391b0000 /System/Library/Frameworks/ServiceManagement.framework/Versions/A/ServiceManagement
0x00000000391b0000 /usr/lib/libxslt.1.dylib
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Ink.framework/Versions/A/Ink
0x00000000391b0000 /System/Library/PrivateFrameworks/TextureIO.framework/Versions/A/TextureIO
0x00000000391b0000 /System/Library/Frameworks/Metal.framework/Versions/A/Metal
0x00000000391b0000 /usr/lib/libate.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/GPUCompiler.framework/libmetal_timestamp.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCoreFSCache.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/IOAccelerator.framework/Versions/A/IOAccelerator
0x00000000391b0000 /System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo
0x00000000391b0000 /usr/lib/libFosl_dynamic.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/MetalPerformanceShaders.framework/Versions/A/MetalPerformanceShaders
0x00000000391b0000 /System/Library/PrivateFrameworks/FaceCore.framework/Versions/A/FaceCore
0x00000000391b0000 /System/Library/Frameworks/OpenCL.framework/Versions/A/OpenCL
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGLU.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGFXShared.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGL.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGLImage.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCVMSPluginSupport.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCoreVMClient.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/CrashReporterSupport.framework/Versions/A/CrashReporterSupport
0x00000000391b0000 /System/Library/PrivateFrameworks/Sharing.framework/Versions/A/Sharing
0x00000000391b0000 /System/Library/PrivateFrameworks/IconServices.framework/Versions/A/IconServices
0x00000000391b0000 /System/Library/PrivateFrameworks/ProtocolBuffer.framework/Versions/A/ProtocolBuffer
0x00000000391b0000 /System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Apple80211
0x00000000391b0000 /System/Library/Frameworks/CoreWLAN.framework/Versions/A/CoreWLAN
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreUtils.framework/Versions/A/CoreUtils
0x00000000391b0000 /System/Library/Frameworks/IOBluetooth.framework/Versions/A/IOBluetooth
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreWiFi.framework/Versions/A/CoreWiFi
0x00000000391b0000 /System/Library/Frameworks/CoreBluetooth.framework/Versions/A/CoreBluetooth
0x00000000391b0000 /System/Library/Frameworks/CoreDisplay.framework/Versions/A/CoreDisplay
0x00000000391b0000 /System/Library/PrivateFrameworks/IOPresentment.framework/Versions/A/IOPresentment
0x00000000391b0000 /System/Library/PrivateFrameworks/DSExternalDisplay.framework/Versions/A/DSExternalDisplay
0x00000000391b0000 /System/Library/PrivateFrameworks/DebugSymbols.framework/Versions/A/DebugSymbols
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreSymbolication.framework/Versions/A/CoreSymbolication
0x00000000391b0000 /System/Library/PrivateFrameworks/Symbolication.framework/Versions/A/Symbolication
0x00000000391b0000 /System/Library/PrivateFrameworks/AppleFSCompression.framework/Versions/A/AppleFSCompression
0x00000000391b0000 /System/Library/PrivateFrameworks/SpeechRecognitionCore.framework/Versions/A/SpeechRecognitionCore
0x00000000391b0000 /System/Library/PrivateFrameworks/ChunkingLibrary.framework/Versions/A/ChunkingLibrary
0x000000010c800000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/server/libjvm.dylib
0x00000000391b0000 /usr/lib/libstdc++.6.dylib
0x0000000109661000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libverify.dylib
0x000000010966f000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libjava.dylib
0x00000001096af000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libzip.dylib
0x0000000121be3000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libmanagement.dylib
0x0000000e17996000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libnet.dylib
0x0000000e179ad000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libnio.dylib
0x0000000e179d0000 /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libgkl_compression4738028315131812363.dylib
0x0000000e17a0d000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libosx.dylib
0x0000000e17a1b000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/./libosxapp.dylib
0x0000000e17a5d000 /System/Library/Frameworks/JavaVM.framework/Versions/A/Frameworks/JavaNativeFoundation.framework/Versions/A/JavaNativeFoundation
0x0000000e17ac0000 /System/Library/Frameworks/JavaVM.framework/Versions/A/Frameworks/JavaRuntimeSupport.framework/Versions/A/JavaRuntimeSupport
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
0x00000000391b0000 /System/Library/Frameworks/ExceptionHandling.framework/Versions/A/ExceptionHandling
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/CommonPanels.framework/Versions/A/CommonPanels
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Help.framework/Versions/A/Help
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/ImageCapture.framework/Versions/A/ImageCapture
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/OpenScripting.framework/Versions/A/OpenScripting
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Print.framework/Versions/A/Print
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SecurityHI.framework/Versions/A/SecurityHI
0x0000000e17ada000 /System/Library/Frameworks/JavaVM.framework/Versions/A/JavaVM
0x0000000e17a72000 /System/Library/PrivateFrameworks/JavaLaunching.framework/Versions/A/JavaLaunching
0x0000000121bfb000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libjaas_unix.dylib
0x0000000e17a7e000 /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libbwa.8008850932864317462.jnilib
VM Arguments:
jvm_args: -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx50G
java_command: /Users/jude/gatk-4.1.4.1/gatk-package-4.1.4.1-local.jar PathSeqPipelineSpark -I SRR1839791.raw.bam --filter-bwa-image //Users/jude/GRCh38.primary_assembly.genome.fa --kmer-file //Users/jude/GRCh38.hss --min-clipped-read-length 70 --microbe-fasta //Users/jude//gatk-4.1.4.1/pathseq_microbe.fa --microbe-bwa-image //Users/jude//gatk-4.1.4.1/pathseq_microbe.fa.img --taxonomy-file //Users/jude//gatk-4.1.4.1/pathseq_taxonomy.db --output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.bam --scores-output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.txt
java_class_path (initial): /Users/jude/gatk-4.1.4.1/gatk-package-4.1.4.1-local.jar
Launcher Type: SUN_STANDARD
Environment Variables:
PATH=/Users/jude/.pyenv/versions/3.7.3/bin:/usr/local/Cellar/pyenv/1.2.18/libexec:/Users/jude/.pyenv/shims:/Library/Frameworks/Python.framework/Versions/3.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
SHELL=/bin/bash
DISPLAY=/private/tmp/com.apple.launchd.2j2S8w567Z/org.macosforge.xquartz:0
Signal Handlers:
SIGSEGV: [libjvm.dylib+0x5b5439], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_ONSTACK|SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.dylib+0x5b5439], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.dylib+0x48ae24], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.dylib+0x48ae24], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.dylib+0x48ae24], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.dylib+0x48ae24], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR1: SIG_DFL, sa_mask[0]=11011111011111100000000111000110, sa_flags=none
SIGUSR2: [libjvm.dylib+0x48a942], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: [libjvm.dylib+0x488f0d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGINT: [libjvm.dylib+0x488f0d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGTERM: [libjvm.dylib+0x488f0d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGQUIT: [libjvm.dylib+0x488f0d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
--------------- S Y S T E M ---------------
OS:Bsduname:Darwin 16.7.0 Darwin Kernel Version 16.7.0: Sun Jun 2 20:26:31 PDT 2019; root:xnu-3789.73.50~1/RELEASE_X86_64 x86_64
rlimit: STACK 8192k, CORE infinity, NPROC 709, NOFILE 10240, AS infinity
load average:6.88 6.84 5.74
CPU:total 24 (initial active 24) (6 cores per cpu, 2 threads per core) family 6 model 44 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, aes, clmul, ht, tsc, tscinvbit, tscinv
Memory: 4k page, physical 83886080k(935068k free)
/proc/meminfo:
vm_info: Java HotSpot(TM) 64-Bit Server VM (25.161-b12) for bsd-amd64 JRE (1.8.0_161-b12), built on Dec 19 2017 16:22:20 by "java_re" with gcc 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)
time: Tue Jul 14 19:08:33 2020
elapsed time: 540 seconds (0d 0h 9m 0s) -
Hi Jude Taylor, thanks for sending the entire error message. I was looking more into this tool, and I found that we actually made a lot of improvements with this tool during our 4.1.8.0 release. Here is a blog post about it: https://gatk.broadinstitute.org/hc/en-us/articles/360045673592-GATK-4-1-8-0-release. Are you able to update GATK? We might find this error resolved after updating, here is the new documentation to look over: https://gatk.broadinstitute.org/hc/en-us/articles/360046221351-PathSeqPipelineSpark
-
Hi Genevieve,
Thanks for the reply. Unfortunately GATK 4.1.8.0 reports the same error with the following command:
~/gatk-4.1.8.0/gatk PathSeqPipelineSpark -I SRR1839791.raw.bam --filter-bwa-image //Users/jude/GRCh38.primary_assembly.genome.fa --kmer-file //Users/jude/GRCh38.bfi --min-clipped-read-length 60 --min-score-identity 0.90 --identity-margin 0.02 --microbe-dict //Users/jude//gatk-4.1.8.0/pathseq_microbe.dict --microbe-bwa-image //Users/jude//gatk-4.1.8.0/pathseq_microbe.fa.img --taxonomy-file //Users/jude//gatk-4.1.8.0/pathseq_taxonomy.db --output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.bam --scores-output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_scores.txt --filter-metrics //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_filter_metrics.txt --score-metrics //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_score_metrics.txt --java-options "-Xmx60G"
I have enclosed the output of the error log below, would appreciate any suggestions. thanks a lot J.
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00000010ac9a2b25, pid=20670, tid=0x000000000000fc03
#
# JRE version: Java(TM) SE Runtime Environment (8.0_161-b12) (build 1.8.0_161-b12)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.161-b12 mixed mode bsd-amd64 )
# Problematic frame:
# C [libbwa.4140429772138536592.jnilib+0xeb25] bwa_mem2idx+0xa5
#
# Core dump written. Default location: /cores/core or core.20670
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
--------------- T H R E A D ---------------
Current thread (0x00007f853717d000): JavaThread "Executor task launch worker for task 1564" daemon [_thread_in_native, id=64515, stack(0x00007000070be000,0x00007000071be000)]
siginfo: si_signo: 11 (SIGSEGV), si_code: 0 (unknown), si_addr: 0x0000000000000000
Registers:
RAX=0x00007f8537df6a20, RBX=0xabababababa99008, RCX=0x0000000000000001, RDX=0x0000000000000f00
RSP=0x00007000071bc770, RBP=0x00007000071bc7c0, RSI=0x0000000000000020, RDI=0x0000000000000f6b
R8 =0x0000000000000030, R9 =0x00007f8537df6a20, R10=0x00000000ffffffe0, R11=0x00007f746c203400
R12=0x00007f8537df7780, R13=0x00000010ac9bc8fc, R14=0x00000010b69e5000, R15=0x00000010ac9bca26
RIP=0x00000010ac9a2b25, EFLAGS=0x0000000000010202, ERR=0x0000000000000000
TRAPNO=0x000000000000000d
Top of Stack: (sp=0x00007000071bc770)
0x00007000071bc770: 00007000071bc7a0 00007fffc44dd4d6
0x00007000071bc780: 00000000bbd6d067 00007f8537df7780
0x00007000071bc790: 0000000000000089 00007f8537df7780
0x00007000071bc7a0: 00007f8537df6a20 00007f8537df7780
0x00007000071bc7b0: 0000000000000089 00000010b69e5000
0x00007000071bc7c0: 00007000071bc880 00000010ac99566d
0x00007000071bc7d0: 0000000000000030 000181a401000002
0x00007000071bc7e0: 00000000000eec5a 00000014000001f5
0x00007000071bc7f0: 0000000000000000 000000005f12756e
0x00007000071bc800: 0000000000000000 000000005e05d67e
0x00007000071bc810: 0000000000000000 000000005e3fa92a
0x00007000071bc820: 0000000000000000 000000005e05d664
0x00007000071bc830: 0000000000000000 00000000bbd6d067
0x00007000071bc840: 00000000005deb70 0000000000001000
0x00007000071bc850: 0000000000000000 0000000000000000
0x00007000071bc860: 0000000000000000 0000000000000089
0x00007000071bc870: 0000000000000000 00007000071bc938
0x00007000071bc880: 00007000071bc8c0 00000010ac995337
0x00007000071bc890: 00000010ac9952d0 00000010b1639ac0
0x00007000071bc8a0: 0000000000000000 00000010b1639ac0
0x00007000071bc8b0: 00007000071bc938 00007f853717d000
0x00007000071bc8c0: 00007000071bc918 000000010af5d0a7
0x00007000071bc8d0: 000000010af5cdc2 00007000071bc8d8
0x00007000071bc8e0: 00000010b1639ac0 00007000071bc938
0x00007000071bc8f0: 00000010b4f4e800 0000000000000000
0x00007000071bc900: 00000010b1639ac0 0000000000000000
0x00007000071bc910: 00007000071bc938 00007000071bc988
0x00007000071bc920: 000000010af4cdb0 000000011ab01600
0x00007000071bc930: 000000010af56cf7 0000000e59412fb0
0x00007000071bc940: 0000000e59667850 00007000071bc948
0x00007000071bc950: 00000010b1638ab6 00007000071bc9c8
0x00007000071bc960: 00000010b4f4e800 0000000000000000
Instructions: (pc=0x00000010ac9a2b25)
0x00000010ac9a2b05: 9c d1 60 04 00 00 bf 30 00 00 00 ba 41 01 00 00
0x00000010ac9a2b15: 4c 89 ee 4c 89 f9 e8 a0 3c 01 00 49 89 44 24 08
0x00000010ac9a2b25: 49 8b 4c 1e 28 48 89 48 28 49 8b 4c 1e 20 48 89
0x00000010ac9a2b35: 48 20 49 8b 4c 1e 18 48 89 48 18 49 8b 4c 1e 10
Register to memory mapping:
RAX=0x00007f8537df6a20 is an unknown value
RBX=0xabababababa99008 is an unknown value
RCX=0x0000000000000001 is an unknown value
RDX=0x0000000000000f00 is an unknown value
RSP=0x00007000071bc770 is pointing into the stack for thread: 0x00007f853717d000
RBP=0x00007000071bc7c0 is pointing into the stack for thread: 0x00007f853717d000
RSI=0x0000000000000020 is an unknown value
RDI=0x0000000000000f6b is an unknown value
R8 =0x0000000000000030 is an unknown value
R9 =0x00007f8537df6a20 is an unknown value
R10=0x00000000ffffffe0 is an unknown value
R11=0x00007f746c203400 is an unknown value
R12=0x00007f8537df7780 is an unknown value
R13=0x00000010ac9bc8fc: induceSA+0xf2c in /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libbwa.4140429772138536592.jnilib at 0x00000010ac994000
R14=0x00000010b69e5000 is an unknown value
R15=0x00000010ac9bca26: induceSA+0x1056 in /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libbwa.4140429772138536592.jnilib at 0x00000010ac994000
Stack: [0x00007000070be000,0x00007000071be000], sp=0x00007000071bc770, free space=1017k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C [libbwa.4140429772138536592.jnilib+0xeb25] bwa_mem2idx+0xa5
C [libbwa.4140429772138536592.jnilib+0x166d] jnibwa_openIndex+0x6d
C [libbwa.4140429772138536592.jnilib+0x1337] Java_org_broadinstitute_hellbender_utils_bwa_BwaMemIndex_openIndex+0x67
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndex.openIndex(Ljava/lang/String;)J+0
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndex.<init>(Ljava/lang/String;)V+30
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndexCache.getInstance(Ljava/lang/String;)Lorg/broadinstitute/hellbender/utils/bwa/BwaMemIndex;+28
j org.broadinstitute.hellbender.tools.spark.pathseq.PSBwaFilter.<init>(Ljava/lang/String;IIIZ)V+6
j org.broadinstitute.hellbender.tools.spark.pathseq.PSFilter.lambda$doBwaFilter$9ded5e08$1(Ljava/lang/String;IIILjava/util/Iterator;)Ljava/util/Iterator;+9
j org.broadinstitute.hellbender.tools.spark.pathseq.PSFilter$$Lambda$195.call(Ljava/lang/Object;)Ljava/util/Iterator;+20
J 7394 C1 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(Ljava/lang/Object;)Ljava/lang/Object; (9 bytes) @ 0x000000010c4151d4 [0x000000010c414a40+0x794]
J 7395 C1 org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (17 bytes) @ 0x000000010c402a5c [0x000000010c4024c0+0x59c]
j org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator;+27
J 10613 C2 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x000000010bc8cd78 [0x000000010bc8ccc0+0xb8]
J 9389 C1 org.apache.spark.scheduler.ResultTask.runTask(Lorg/apache/spark/TaskContext;)Ljava/lang/Object; (206 bytes) @ 0x000000010c36d694 [0x000000010c36b3e0+0x22b4]
J 9386 C1 org.apache.spark.scheduler.Task.run(JILorg/apache/spark/metrics/MetricsSystem;)Ljava/lang/Object; (476 bytes) @ 0x000000010ba20fd4 [0x000000010ba1e820+0x27b4]
J 9443 C1 org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply()Ljava/lang/Object; (51 bytes) @ 0x000000010b6e2ff4 [0x000000010b6e2da0+0x254]
J 7729 C1 org.apache.spark.util.Utils$.tryWithSafeFinally(Lscala/Function0;Lscala/Function0;)Ljava/lang/Object; (182 bytes) @ 0x000000010c50a994 [0x000000010c50a880+0x114]
J 9373 C1 org.apache.spark.executor.Executor$TaskRunner.run()V (2541 bytes) @ 0x000000010c96bcd4 [0x000000010c968380+0x3954]
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
V [libjvm.dylib+0x2ef9aa]
V [libjvm.dylib+0x2f014e]
V [libjvm.dylib+0x2f02fa]
V [libjvm.dylib+0x34aae5]
V [libjvm.dylib+0x56edef]
V [libjvm.dylib+0x5704ea]
V [libjvm.dylib+0x48d83a]
C [libsystem_pthread.dylib+0x393b] _pthread_body+0xb4
C [libsystem_pthread.dylib+0x3887] _pthread_body+0x0
C [libsystem_pthread.dylib+0x308d] thread_start+0xd
C 0x0000000000000000
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndex.openIndex(Ljava/lang/String;)J+0
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndex.<init>(Ljava/lang/String;)V+30
j org.broadinstitute.hellbender.utils.bwa.BwaMemIndexCache.getInstance(Ljava/lang/String;)Lorg/broadinstitute/hellbender/utils/bwa/BwaMemIndex;+28
j org.broadinstitute.hellbender.tools.spark.pathseq.PSBwaFilter.<init>(Ljava/lang/String;IIIZ)V+6
j org.broadinstitute.hellbender.tools.spark.pathseq.PSFilter.lambda$doBwaFilter$9ded5e08$1(Ljava/lang/String;IIILjava/util/Iterator;)Ljava/util/Iterator;+9
j org.broadinstitute.hellbender.tools.spark.pathseq.PSFilter$$Lambda$195.call(Ljava/lang/Object;)Ljava/util/Iterator;+20
J 7394 C1 org.apache.spark.api.java.JavaRDDLike$$anonfun$fn$4$1.apply(Ljava/lang/Object;)Ljava/lang/Object; (9 bytes) @ 0x000000010c4151d4 [0x000000010c414a40+0x794]
J 7395 C1 org.apache.spark.rdd.RDD$$anonfun$mapPartitions$1$$anonfun$apply$23.apply(Ljava/lang/Object;Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; (17 bytes) @ 0x000000010c402a5c [0x000000010c4024c0+0x59c]
j org.apache.spark.rdd.MapPartitionsRDD.compute(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator;+27
J 10613 C2 org.apache.spark.rdd.RDD.iterator(Lorg/apache/spark/Partition;Lorg/apache/spark/TaskContext;)Lscala/collection/Iterator; (46 bytes) @ 0x000000010bc8cd78 [0x000000010bc8ccc0+0xb8]
J 9389 C1 org.apache.spark.scheduler.ResultTask.runTask(Lorg/apache/spark/TaskContext;)Ljava/lang/Object; (206 bytes) @ 0x000000010c36d694 [0x000000010c36b3e0+0x22b4]
J 9386 C1 org.apache.spark.scheduler.Task.run(JILorg/apache/spark/metrics/MetricsSystem;)Ljava/lang/Object; (476 bytes) @ 0x000000010ba20fd4 [0x000000010ba1e820+0x27b4]
J 9443 C1 org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply()Ljava/lang/Object; (51 bytes) @ 0x000000010b6e2ff4 [0x000000010b6e2da0+0x254]
J 7729 C1 org.apache.spark.util.Utils$.tryWithSafeFinally(Lscala/Function0;Lscala/Function0;)Ljava/lang/Object; (182 bytes) @ 0x000000010c50a994 [0x000000010c50a880+0x114]
J 9373 C1 org.apache.spark.executor.Executor$TaskRunner.run()V (2541 bytes) @ 0x000000010c96bcd4 [0x000000010c968380+0x3954]
j java.util.concurrent.ThreadPoolExecutor.runWorker(Ljava/util/concurrent/ThreadPoolExecutor$Worker;)V+95
j java.util.concurrent.ThreadPoolExecutor$Worker.run()V+5
j java.lang.Thread.run()V+11
v ~StubRoutines::call_stub
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x00007f8538e80800 JavaThread "task-result-getter-3" daemon [_thread_blocked, id=79363, stack(0x0000700008e15000,0x0000700008f15000)]
0x00007f8538427000 JavaThread "task-result-getter-2" daemon [_thread_blocked, id=78851, stack(0x0000700008d12000,0x0000700008e12000)]
0x00007f853722a000 JavaThread "task-result-getter-1" daemon [_thread_blocked, id=78339, stack(0x0000700008c0f000,0x0000700008d0f000)]
0x00007f853d800000 JavaThread "Executor task launch worker for task 1561" daemon [_thread_blocked, id=71683, stack(0x0000700007ee8000,0x0000700007fe8000)]
0x00007f853b800800 JavaThread "Executor task launch worker for task 1567" daemon [_thread_blocked, id=71171, stack(0x0000700007de5000,0x0000700007ee5000)]
0x00007f8531af0000 JavaThread "Executor task launch worker for task 1565" daemon [_thread_blocked, id=70659, stack(0x0000700007ce2000,0x0000700007de2000)]
0x00007f853b009000 JavaThread "Executor task launch worker for task 1557" daemon [_thread_blocked, id=70147, stack(0x0000700007bdf000,0x0000700007cdf000)]
0x00007f85362af800 JavaThread "Executor task launch worker for task 1566" daemon [_thread_blocked, id=69635, stack(0x0000700007adc000,0x0000700007bdc000)]
0x00007f8533c1f000 JavaThread "Executor task launch worker for task 1559" daemon [_thread_blocked, id=69123, stack(0x00007000079d9000,0x0000700007ad9000)]
0x00007f85362af000 JavaThread "Executor task launch worker for task 1577" daemon [_thread_blocked, id=68611, stack(0x00007000078d6000,0x00007000079d6000)]
0x00007f8534467800 JavaThread "Executor task launch worker for task 1558" daemon [_thread_blocked, id=68099, stack(0x00007000077d3000,0x00007000078d3000)]
0x00007f8532317800 JavaThread "Executor task launch worker for task 1560" daemon [_thread_blocked, id=67587, stack(0x00007000076d0000,0x00007000077d0000)]
0x00007f853717d800 JavaThread "Executor task launch worker for task 1570" daemon [_thread_blocked, id=67075, stack(0x00007000075cd000,0x00007000076cd000)]
0x00007f8534a1e000 JavaThread "Executor task launch worker for task 1578" daemon [_thread_blocked, id=66563, stack(0x00007000074ca000,0x00007000075ca000)]
0x00007f8532316800 JavaThread "Executor task launch worker for task 1575" daemon [_thread_blocked, id=66051, stack(0x00007000073c7000,0x00007000074c7000)]
0x00007f85382cd000 JavaThread "Executor task launch worker for task 1574" daemon [_thread_blocked, id=65539, stack(0x00007000072c4000,0x00007000073c4000)]
0x00007f85310e7000 JavaThread "Executor task launch worker for task 1576" daemon [_thread_blocked, id=65027, stack(0x00007000071c1000,0x00007000072c1000)]
=>0x00007f853717d000 JavaThread "Executor task launch worker for task 1564" daemon [_thread_in_native, id=64515, stack(0x00007000070be000,0x00007000071be000)]
0x00007f85310e6000 JavaThread "Executor task launch worker for task 1579" daemon [_thread_blocked, id=64003, stack(0x0000700006fbb000,0x00007000070bb000)]
0x00007f853521c000 JavaThread "Executor task launch worker for task 1572" daemon [_thread_blocked, id=63491, stack(0x0000700006eb8000,0x0000700006fb8000)]
0x00007f853717c000 JavaThread "Executor task launch worker for task 1569" daemon [_thread_blocked, id=62979, stack(0x0000700006db5000,0x0000700006eb5000)]
0x00007f85362ae000 JavaThread "Executor task launch worker for task 1562" daemon [_thread_blocked, id=62467, stack(0x0000700006cb2000,0x0000700006db2000)]
0x00007f853717b800 JavaThread "Executor task launch worker for task 1571" daemon [_thread_blocked, id=61955, stack(0x0000700006baf000,0x0000700006caf000)]
0x00007f8533c4e000 JavaThread "Executor task launch worker for task 1573" daemon [_thread_blocked, id=61443, stack(0x0000700006aac000,0x0000700006bac000)]
0x00007f8536a4f800 JavaThread "Executor task launch worker for task 1556" daemon [_thread_blocked, id=60931, stack(0x00007000069a9000,0x0000700006aa9000)]
0x00007f8536a35800 JavaThread "Executor task launch worker for task 1563" daemon [_thread_blocked, id=60419, stack(0x00007000068a6000,0x00007000069a6000)]
0x00007f8536063800 JavaThread "task-result-getter-0" daemon [_thread_blocked, id=4103, stack(0x00007000067a3000,0x00007000068a3000)]
0x00007f8533955800 JavaThread "Executor task launch worker for task 1568" daemon [_thread_blocked, id=5127, stack(0x00007000066a0000,0x00007000067a0000)]
0x00007f8533d2a000 JavaThread "org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner" daemon [_thread_blocked, id=59907, stack(0x000070000659d000,0x000070000669d000)]
0x00007f8531abc800 JavaThread "element-tracking-store-worker" daemon [_thread_blocked, id=59395, stack(0x000070000649a000,0x000070000659a000)]
0x00007f8534057800 JavaThread "spark-listener-group-executorManagement" daemon [_thread_blocked, id=58883, stack(0x0000700006397000,0x0000700006497000)]
0x00007f8534057000 JavaThread "spark-listener-group-appStatus" daemon [_thread_blocked, id=58371, stack(0x0000700006294000,0x0000700006394000)]
0x00007f85340f0000 JavaThread "context-cleaner-periodic-gc" daemon [_thread_blocked, id=57859, stack(0x0000700006191000,0x0000700006291000)]
0x00007f85340d9000 JavaThread "Spark Context Cleaner" daemon [_thread_blocked, id=57347, stack(0x000070000608e000,0x000070000618e000)]
0x00007f85335e0800 JavaThread "shuffle-boss-6-1" daemon [_thread_in_native, id=56835, stack(0x0000700005f8b000,0x000070000608b000)]
0x00007f8532245000 JavaThread "driver-heartbeater" daemon [_thread_blocked, id=56323, stack(0x0000700005e88000,0x0000700005f88000)]
0x00007f853222f800 JavaThread "dag-scheduler-event-loop" daemon [_thread_blocked, id=55811, stack(0x0000700005d85000,0x0000700005e85000)]
0x00007f8538d73000 JavaThread "netty-rpc-env-timeout" daemon [_thread_blocked, id=55299, stack(0x0000700005c82000,0x0000700005d82000)]
0x00007f8534aae000 JavaThread "Timer-1" daemon [_thread_blocked, id=54787, stack(0x0000700005b7f000,0x0000700005c7f000)]
0x00007f8534aac800 JavaThread "Timer-0" daemon [_thread_blocked, id=54275, stack(0x0000700005a7c000,0x0000700005b7c000)]
0x00007f8535185000 JavaThread "heartbeat-receiver-event-loop-thread" daemon [_thread_blocked, id=53763, stack(0x0000700005979000,0x0000700005a79000)]
0x00007f8538d71800 JavaThread "SparkUI-63" daemon [_thread_blocked, id=53251, stack(0x0000700005876000,0x0000700005976000)]
0x00007f8538d70800 JavaThread "SparkUI-62-acceptor-2@1c676230-ServerConnector@3435a78f{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}" daemon [_thread_blocked, id=52739, stack(0x0000700005773000,0x0000700005873000)]
0x00007f8538d6b000 JavaThread "SparkUI-61-acceptor-1@1150dbe4-ServerConnector@3435a78f{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}" daemon [_thread_in_native, id=52227, stack(0x0000700005670000,0x0000700005770000)]
0x00007f8538d6a000 JavaThread "SparkUI-60-acceptor-0@3af25ea1-ServerConnector@3435a78f{HTTP/1.1,[http/1.1]}{0.0.0.0:4040}" daemon [_thread_blocked, id=51715, stack(0x000070000556d000,0x000070000566d000)]
0x00007f8533074800 JavaThread "SparkUI-59" daemon [_thread_in_native, id=51203, stack(0x000070000546a000,0x000070000556a000)]
0x00007f853361c800 JavaThread "SparkUI-58" daemon [_thread_in_native, id=50691, stack(0x0000700005367000,0x0000700005467000)]
0x00007f8533620000 JavaThread "SparkUI-57" daemon [_thread_in_native, id=50179, stack(0x0000700005264000,0x0000700005364000)]
0x00007f8533428800 JavaThread "SparkUI-56" daemon [_thread_in_native, id=49667, stack(0x0000700005161000,0x0000700005261000)]
0x00007f8533ba5800 JavaThread "RemoteBlock-temp-file-clean-thread" daemon [_thread_blocked, id=49155, stack(0x000070000505e000,0x000070000515e000)]
0x00007f853397b000 JavaThread "map-output-dispatcher-7" daemon [_thread_blocked, id=48643, stack(0x0000700004f5b000,0x000070000505b000)]
0x00007f8531221800 JavaThread "map-output-dispatcher-6" daemon [_thread_blocked, id=48131, stack(0x0000700004e58000,0x0000700004f58000)]
0x00007f853397a800 JavaThread "map-output-dispatcher-5" daemon [_thread_blocked, id=47619, stack(0x0000700004d55000,0x0000700004e55000)]
0x00007f85340ee000 JavaThread "map-output-dispatcher-4" daemon [_thread_blocked, id=47107, stack(0x0000700004c52000,0x0000700004d52000)]
0x00007f8531221000 JavaThread "map-output-dispatcher-3" daemon [_thread_blocked, id=46595, stack(0x0000700004b4f000,0x0000700004c4f000)]
0x00007f8531216000 JavaThread "map-output-dispatcher-2" daemon [_thread_blocked, id=46083, stack(0x0000700004a4c000,0x0000700004b4c000)]
0x00007f85311ff800 JavaThread "map-output-dispatcher-1" daemon [_thread_blocked, id=45571, stack(0x0000700004949000,0x0000700004a49000)]
0x00007f8533980800 JavaThread "map-output-dispatcher-0" daemon [_thread_blocked, id=45059, stack(0x0000700004846000,0x0000700004946000)]
0x00007f8534d53000 JavaThread "rpc-boss-3-1" daemon [_thread_in_native, id=44547, stack(0x0000700004743000,0x0000700004843000)]
0x00007f8534d50000 JavaThread "dispatcher-event-loop-23" daemon [_thread_blocked, id=44035, stack(0x0000700004640000,0x0000700004740000)]
0x00007f8534d4f800 JavaThread "dispatcher-event-loop-22" daemon [_thread_blocked, id=43523, stack(0x000070000453d000,0x000070000463d000)]
0x00007f8534d4e800 JavaThread "dispatcher-event-loop-21" daemon [_thread_blocked, id=43011, stack(0x000070000443a000,0x000070000453a000)]
0x00007f8534d4e000 JavaThread "dispatcher-event-loop-20" daemon [_thread_blocked, id=42499, stack(0x0000700004337000,0x0000700004437000)]
0x00007f8534b6c000 JavaThread "dispatcher-event-loop-19" daemon [_thread_blocked, id=41987, stack(0x0000700004234000,0x0000700004334000)]
0x00007f8533428000 JavaThread "dispatcher-event-loop-18" daemon [_thread_blocked, id=41475, stack(0x0000700004131000,0x0000700004231000)]
0x00007f8533427000 JavaThread "dispatcher-event-loop-17" daemon [_thread_blocked, id=40963, stack(0x000070000402e000,0x000070000412e000)]
0x00007f8533426800 JavaThread "dispatcher-event-loop-16" daemon [_thread_blocked, id=40451, stack(0x0000700003f2b000,0x000070000402b000)]
0x00007f8533425800 JavaThread "dispatcher-event-loop-15" daemon [_thread_blocked, id=39939, stack(0x0000700003e28000,0x0000700003f28000)]
0x00007f8533425000 JavaThread "dispatcher-event-loop-14" daemon [_thread_blocked, id=39427, stack(0x0000700003d25000,0x0000700003e25000)]
0x00007f8533424000 JavaThread "dispatcher-event-loop-13" daemon [_thread_blocked, id=38915, stack(0x0000700003c22000,0x0000700003d22000)]
0x00007f8531ab1800 JavaThread "dispatcher-event-loop-12" daemon [_thread_blocked, id=38403, stack(0x0000700003b1f000,0x0000700003c1f000)]
0x00007f8533423800 JavaThread "dispatcher-event-loop-11" daemon [_thread_blocked, id=37891, stack(0x0000700003a1c000,0x0000700003b1c000)]
0x00007f8533198000 JavaThread "dispatcher-event-loop-10" daemon [_thread_blocked, id=37379, stack(0x0000700003919000,0x0000700003a19000)]
0x00007f8533197000 JavaThread "dispatcher-event-loop-9" daemon [_thread_blocked, id=36867, stack(0x0000700003816000,0x0000700003916000)]
0x00007f8531ab1000 JavaThread "dispatcher-event-loop-8" daemon [_thread_blocked, id=36355, stack(0x0000700003713000,0x0000700003813000)]
0x00007f85333e4000 JavaThread "dispatcher-event-loop-7" daemon [_thread_blocked, id=35843, stack(0x0000700003610000,0x0000700003710000)]
0x00007f85333e3000 JavaThread "dispatcher-event-loop-6" daemon [_thread_blocked, id=35331, stack(0x000070000350d000,0x000070000360d000)]
0x00007f8533430000 JavaThread "dispatcher-event-loop-5" daemon [_thread_blocked, id=34819, stack(0x000070000340a000,0x000070000350a000)]
0x00007f85332b6800 JavaThread "dispatcher-event-loop-4" daemon [_thread_blocked, id=34307, stack(0x0000700003307000,0x0000700003407000)]
0x00007f8531ab0000 JavaThread "dispatcher-event-loop-3" daemon [_thread_blocked, id=33795, stack(0x0000700003204000,0x0000700003304000)]
0x00007f8531aaf800 JavaThread "dispatcher-event-loop-2" daemon [_thread_blocked, id=33283, stack(0x0000700003101000,0x0000700003201000)]
0x00007f8531aaa000 JavaThread "dispatcher-event-loop-1" daemon [_thread_blocked, id=32771, stack(0x0000700002ffe000,0x00007000030fe000)]
0x00007f8531aa9800 JavaThread "dispatcher-event-loop-0" daemon [_thread_blocked, id=32263, stack(0x0000700002efb000,0x0000700002ffb000)]
0x00007f8533032800 JavaThread "Service Thread" daemon [_thread_blocked, id=30467, stack(0x0000700002cf5000,0x0000700002df5000)]
0x00007f85318a6800 JavaThread "C1 CompilerThread11" daemon [_thread_blocked, id=29955, stack(0x0000700002bf2000,0x0000700002cf2000)]
0x00007f85318a6000 JavaThread "C1 CompilerThread10" daemon [_thread_blocked, id=29443, stack(0x0000700002aef000,0x0000700002bef000)]
0x00007f853200a800 JavaThread "C1 CompilerThread9" daemon [_thread_blocked, id=28931, stack(0x00007000029ec000,0x0000700002aec000)]
0x00007f853200a000 JavaThread "C1 CompilerThread8" daemon [_thread_blocked, id=28419, stack(0x00007000028e9000,0x00007000029e9000)]
0x00007f8532009000 JavaThread "C2 CompilerThread7" daemon [_thread_blocked, id=27907, stack(0x00007000027e6000,0x00007000028e6000)]
0x00007f8535012800 JavaThread "C2 CompilerThread6" daemon [_thread_blocked, id=27395, stack(0x00007000026e3000,0x00007000027e3000)]
0x00007f8535011800 JavaThread "C2 CompilerThread5" daemon [_thread_blocked, id=26883, stack(0x00007000025e0000,0x00007000026e0000)]
0x00007f8535009000 JavaThread "C2 CompilerThread4" daemon [_thread_blocked, id=26371, stack(0x00007000024dd000,0x00007000025dd000)]
0x00007f8535008000 JavaThread "C2 CompilerThread3" daemon [_thread_blocked, id=25859, stack(0x00007000023da000,0x00007000024da000)]
0x00007f8531862800 JavaThread "C2 CompilerThread2" daemon [_thread_blocked, id=25347, stack(0x00007000022d7000,0x00007000023d7000)]
0x00007f8533032000 JavaThread "C2 CompilerThread1" daemon [_thread_blocked, id=24835, stack(0x00007000021d4000,0x00007000022d4000)]
0x00007f8533027000 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=24323, stack(0x00007000020d1000,0x00007000021d1000)]
0x00007f8533026000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=23811, stack(0x0000700001fce000,0x00007000020ce000)]
0x00007f8534002000 JavaThread "Finalizer" daemon [_thread_blocked, id=19715, stack(0x0000700001ecb000,0x0000700001fcb000)]
0x00007f8531862000 JavaThread "Reference Handler" daemon [_thread_blocked, id=19203, stack(0x0000700001dc8000,0x0000700001ec8000)]
0x00007f8533006800 JavaThread "main" [_thread_blocked, id=7171, stack(0x000070000098c000,0x0000700000a8c000)]
Other Threads:
0x00007f853185d800 VMThread [stack: 0x0000700001cc5000,0x0000700001dc5000] [id=18691]
0x00007f853300f800 WatcherThread [stack: 0x0000700002df8000,0x0000700002ef8000] [id=30979]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
Heap:
PSYoungGen total 20813312K, used 16003998K [0x0000000b1a400000, 0x0000001019a00000, 0x000000101a400000)
eden space 20668928K, 77% used [0x0000000b1a400000,0x0000000ee7b00bb8,0x0000001007c80000)
from space 144384K, 38% used [0x0000001007c80000,0x000000100b266e50,0x0000001010980000)
to space 138240K, 0% used [0x0000001011300000,0x0000001011300000,0x0000001019a00000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 79470K, capacity 81057K, committed 85504K, reserved 86016K
Card table byte_map: [0x000000101a400000,0x0000001021c01000] byte_map_base: 0x0000001019b2e000
Marking Bits: (ParMarkBitMap*) 0x000000010a4f76c0
Begin Bits: [0x0000001026c01000, 0x0000001062c01000)
End Bits: [0x0000001062c01000, 0x000000109ec01000)
Polling page: 0x0000000106b05000
CodeCache: size=245760Kb used=32214Kb max_used=32235Kb free=213545Kb
bounds [0x000000010af45000, 0x000000010cf05000, 0x0000000119f45000]
total_blobs=10138 nmethods=9489 adapters=559
compilation: enabled
Compilation events (10 events):
Event: 614.999 Thread 0x00007f8532009000 nmethod 12468 0x000000010cebbb90 code [0x000000010cebbce0, 0x000000010cebbd38]
Event: 614.999 Thread 0x00007f8532009000 12479 4 sun.nio.ch.FileChannelImpl::<init> (74 bytes)
Event: 615.002 Thread 0x00007f8531862800 nmethod 12471 0x000000010ced3410 code [0x000000010ced3680, 0x000000010ced4530]
Event: 615.002 Thread 0x00007f8532009000 nmethod 12479 0x000000010cec9f10 code [0x000000010ceca080, 0x000000010ceca3b8]
Event: 615.003 Thread 0x00007f8535012800 nmethod 12466 0x000000010ced09d0 code [0x000000010ced0b60, 0x000000010ced0e68]
Event: 615.004 Thread 0x00007f8533027000 nmethod 12458 0x000000010ced8450 code [0x000000010ced8680, 0x000000010ced96b8]
Event: 615.027 Thread 0x00007f8535011800 nmethod 12461 0x000000010cedec50 code [0x000000010cedef80, 0x000000010cee13c0]
Event: 615.034 Thread 0x00007f8535008000 nmethod 12474 0x000000010ceef590 code [0x000000010ceef900, 0x000000010cef2870]
Event: 615.034 Thread 0x00007f8533032000 nmethod 12475 0x000000010cef7410 code [0x000000010cef7760, 0x000000010cef9de8]
Event: 615.035 Thread 0x00007f8535009000 nmethod 12476 0x000000010cee5390 code [0x000000010cee5700, 0x000000010cee7da8]
GC Heap History (10 events):
Event: 600.488 GC heap before
{Heap before GC invocations=154 (full 5):
PSYoungGen total 20806144K, used 20706307K [0x0000000b1a400000, 0x0000001019e00000, 0x000000101a400000)
eden space 20646400K, 100% used [0x0000000b1a400000,0x0000001006680000,0x0000001006680000)
from space 159744K, 37% used [0x000000100ff80000,0x0000001013a00fc0,0x0000001019b80000)
to space 156672K, 0% used [0x0000001006680000,0x0000001006680000,0x000000100ff80000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78832K, capacity 80180K, committed 84992K, reserved 86016K
Event: 600.548 GC heap after
Heap after GC invocations=154 (full 5):
PSYoungGen total 20803072K, used 59843K [0x0000000b1a400000, 0x0000001019b00000, 0x000000101a400000)
eden space 20646400K, 0% used [0x0000000b1a400000,0x0000000b1a400000,0x0000001006680000)
from space 156672K, 38% used [0x0000001006680000,0x000000100a0f0f90,0x000000100ff80000)
to space 153600K, 0% used [0x0000001010500000,0x0000001010500000,0x0000001019b00000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78832K, capacity 80180K, committed 84992K, reserved 86016K
}
Event: 603.246 GC heap before
{Heap before GC invocations=155 (full 5):
PSYoungGen total 20803072K, used 20706243K [0x0000000b1a400000, 0x0000001019b00000, 0x000000101a400000)
eden space 20646400K, 100% used [0x0000000b1a400000,0x0000001006680000,0x0000001006680000)
from space 156672K, 38% used [0x0000001006680000,0x000000100a0f0f90,0x000000100ff80000)
to space 153600K, 0% used [0x0000001010500000,0x0000001010500000,0x0000001019b00000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78881K, capacity 80245K, committed 84992K, reserved 86016K
Event: 603.316 GC heap after
Heap after GC invocations=155 (full 5):
PSYoungGen total 20809728K, used 67268K [0x0000000b1a400000, 0x0000001019f00000, 0x000000101a400000)
eden space 20656128K, 0% used [0x0000000b1a400000,0x0000000b1a400000,0x0000001007000000)
from space 153600K, 43% used [0x0000001010500000,0x00000010146b1320,0x0000001019b00000)
to space 152576K, 0% used [0x0000001007000000,0x0000001007000000,0x0000001010500000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78881K, capacity 80245K, committed 84992K, reserved 86016K
}
Event: 606.007 GC heap before
{Heap before GC invocations=156 (full 5):
PSYoungGen total 20809728K, used 20723396K [0x0000000b1a400000, 0x0000001019f00000, 0x000000101a400000)
eden space 20656128K, 100% used [0x0000000b1a400000,0x0000001007000000,0x0000001007000000)
from space 153600K, 43% used [0x0000001010500000,0x00000010146b1320,0x0000001019b00000)
to space 152576K, 0% used [0x0000001007000000,0x0000001007000000,0x0000001010500000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78918K, capacity 80295K, committed 84992K, reserved 86016K
Event: 606.073 GC heap after
Heap after GC invocations=156 (full 5):
PSYoungGen total 20808704K, used 63376K [0x0000000b1a400000, 0x0000001019e00000, 0x000000101a400000)
eden space 20656128K, 0% used [0x0000000b1a400000,0x0000000b1a400000,0x0000001007000000)
from space 152576K, 41% used [0x0000001007000000,0x000000100ade41d8,0x0000001010500000)
to space 150016K, 0% used [0x0000001010b80000,0x0000001010b80000,0x0000001019e00000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78918K, capacity 80295K, committed 84992K, reserved 86016K
}
Event: 608.821 GC heap before
{Heap before GC invocations=157 (full 5):
PSYoungGen total 20808704K, used 20719504K [0x0000000b1a400000, 0x0000001019e00000, 0x000000101a400000)
eden space 20656128K, 100% used [0x0000000b1a400000,0x0000001007000000,0x0000001007000000)
from space 152576K, 41% used [0x0000001007000000,0x000000100ade41d8,0x0000001010500000)
to space 150016K, 0% used [0x0000001010b80000,0x0000001010b80000,0x0000001019e00000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78918K, capacity 80295K, committed 84992K, reserved 86016K
Event: 608.876 GC heap after
Heap after GC invocations=157 (full 5):
PSYoungGen total 20811264K, used 57455K [0x0000000b1a400000, 0x0000001019680000, 0x000000101a400000)
eden space 20668928K, 0% used [0x0000000b1a400000,0x0000000b1a400000,0x0000001007c80000)
from space 142336K, 40% used [0x0000001010b80000,0x000000101439bed8,0x0000001019680000)
to space 144384K, 0% used [0x0000001007c80000,0x0000001007c80000,0x0000001010980000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78918K, capacity 80295K, committed 84992K, reserved 86016K
}
Event: 611.634 GC heap before
{Heap before GC invocations=158 (full 5):
PSYoungGen total 20811264K, used 20726383K [0x0000000b1a400000, 0x0000001019680000, 0x000000101a400000)
eden space 20668928K, 100% used [0x0000000b1a400000,0x0000001007c80000,0x0000001007c80000)
from space 142336K, 40% used [0x0000001010b80000,0x000000101439bed8,0x0000001019680000)
to space 144384K, 0% used [0x0000001007c80000,0x0000001007c80000,0x0000001010980000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78918K, capacity 80295K, committed 84992K, reserved 86016K
Event: 611.694 GC heap after
Heap after GC invocations=158 (full 5):
PSYoungGen total 20813312K, used 55195K [0x0000000b1a400000, 0x0000001019a00000, 0x000000101a400000)
eden space 20668928K, 0% used [0x0000000b1a400000,0x0000000b1a400000,0x0000001007c80000)
from space 144384K, 38% used [0x0000001007c80000,0x000000100b266e50,0x0000001010980000)
to space 138240K, 0% used [0x0000001011300000,0x0000001011300000,0x0000001019a00000)
ParOldGen total 14141440K, used 10843984K [0x000000011a400000, 0x0000000479600000, 0x0000000b1a400000)
object space 14141440K, 76% used [0x000000011a400000,0x00000003b01d4028,0x0000000479600000)
Metaspace used 78918K, capacity 80295K, committed 84992K, reserved 86016K
}
Deoptimization events (10 events):
Event: 614.797 Thread 0x00007f8531af0000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x000000010c772004 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 614.797 Thread 0x00007f853b800800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x000000010c772004 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 614.797 Thread 0x00007f853717d800 Uncommon trap: reason=class_check action=maybe_recompile pc=0x000000010c772004 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 614.797 Thread 0x00007f85382cd000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x000000010c772004 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 614.797 Thread 0x00007f853521c000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x000000010c772004 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 614.797 Thread 0x00007f85362af000 Uncommon trap: reason=class_check action=maybe_recompile pc=0x000000010c772004 method=scala.collection.mutable.WrappedArray$.make(Ljava/lang/Object;)Lscala/collection/mutable/WrappedArray; @ 11
Event: 614.844 Thread 0x00007f8532317800 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000010c484dfc method=sun.nio.ch.FileChannelImpl.position()J @ 97
Event: 614.983 Thread 0x00007f853717d000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000010cc04850 method=java.lang.Long.stringSize(J)I @ 11
Event: 614.987 Thread 0x00007f853717d000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000010bb2a2b4 method=java.nio.Buffer.limit(I)Ljava/nio/Buffer; @ 33
Event: 615.055 Thread 0x00007f853717d000 Uncommon trap: reason=unstable_if action=reinterpret pc=0x000000010be3dfec method=java.io.UnixFileSystem.normalize(Ljava/lang/String;)Ljava/lang/String; @ 34
Classes redefined (0 events):
No events
Internal exceptions (10 events):
Event: 404.294 Thread 0x00007f853222f800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000d925e7578) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 404.294 Thread 0x00007f853222f800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000d925f2c48) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 404.295 Thread 0x00007f853222f800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000d9260c7c8) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 404.295 Thread 0x00007f853222f800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000d9260e6d0) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 404.295 Thread 0x00007f853222f800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000d92610a58) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 404.310 Thread 0x00007f8534d4e800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000d92c2dea8) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 483.075 Thread 0x00007f85362af800 Implicit null exception at 0x000000010c3787f6 to 0x000000010c378ffd
Event: 483.080 Thread 0x00007f85310e6000 Implicit null exception at 0x000000010bfb4322 to 0x000000010bfb4625
Event: 614.697 Thread 0x00007f853222f800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000edf0bc568) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Event: 614.720 Thread 0x00007f8534d4f800 Exception <a 'java/lang/NoSuchMethodError': <clinit>> (0x0000000edf6a0cd0) thrown at [/Users/java_re/workspace/8-2-build-macosx-x86_64/jdk8u161/10277/hotspot/src/share/vm/prims/jni.cpp, line 1613]
Events (10 events):
Event: 614.986 loading class sun/nio/fs/UnixChannelFactory$1
Event: 614.986 loading class sun/nio/fs/UnixChannelFactory$1 done
Event: 614.986 loading class java/nio/channels/Channels$1
Event: 614.986 loading class java/nio/channels/Channels$1 done
Event: 614.987 Thread 0x00007f853717d000 Uncommon trap: trap_request=0xffffff65 fr.pc=0x000000010bb2a2b4
Event: 614.987 Thread 0x00007f853717d000 DEOPT PACKING pc=0x000000010bb2a2b4 sp=0x00007000071bc680
Event: 614.987 Thread 0x00007f853717d000 DEOPT UNPACKING pc=0x000000010af8a73a sp=0x00007000071bc640 mode 2
Event: 615.055 Thread 0x00007f853717d000 Uncommon trap: trap_request=0xffffff65 fr.pc=0x000000010be3dfec
Event: 615.055 Thread 0x00007f853717d000 DEOPT PACKING pc=0x000000010be3dfec sp=0x00007000071bc6f0
Event: 615.055 Thread 0x00007f853717d000 DEOPT UNPACKING pc=0x000000010af8a73a sp=0x00007000071bc6a0 mode 2
Dynamic libraries:
0x00000000391b0000 /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa
0x00000000391b0000 /System/Library/Frameworks/Security.framework/Versions/A/Security
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
0x00000000391b0000 /usr/lib/libz.1.dylib
0x00000000391b0000 /usr/lib/libSystem.B.dylib
0x00000000391b0000 /usr/lib/libobjc.A.dylib
0x00000000391b0000 /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
0x00000000391b0000 /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
0x00000000391b0000 /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
0x00000000391b0000 /System/Library/Frameworks/CoreData.framework/Versions/A/CoreData
0x00000000391b0000 /System/Library/PrivateFrameworks/RemoteViewServices.framework/Versions/A/RemoteViewServices
0x00000000391b0000 /System/Library/PrivateFrameworks/UIFoundation.framework/Versions/A/UIFoundation
0x00000000391b0000 /System/Library/PrivateFrameworks/DFRFoundation.framework/Versions/A/DFRFoundation
0x00000000391b0000 /usr/lib/libenergytrace.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/SkyLight.framework/Versions/A/SkyLight
0x00000000391b0000 /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
0x00000000391b0000 /usr/lib/libScreenReader.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate
0x00000000391b0000 /System/Library/Frameworks/IOSurface.framework/Versions/A/IOSurface
0x00000000391b0000 /System/Library/Frameworks/AudioToolbox.framework/Versions/A/AudioToolbox
0x00000000391b0000 /System/Library/Frameworks/AudioUnit.framework/Versions/A/AudioUnit
0x00000000391b0000 /System/Library/PrivateFrameworks/DataDetectorsCore.framework/Versions/A/DataDetectorsCore
0x00000000391b0000 /System/Library/PrivateFrameworks/DesktopServicesPriv.framework/Versions/A/DesktopServicesPriv
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/HIToolbox.framework/Versions/A/HIToolbox
0x00000000391b0000 /usr/lib/libicucore.A.dylib
0x00000000391b0000 /System/Library/Frameworks/QuartzCore.framework/Versions/A/QuartzCore
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SpeechRecognition.framework/Versions/A/SpeechRecognition
0x00000000391b0000 /usr/lib/libauto.dylib
0x00000000391b0000 /usr/lib/libxml2.2.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreUI.framework/Versions/A/CoreUI
0x00000000391b0000 /System/Library/Frameworks/CoreAudio.framework/Versions/A/CoreAudio
0x00000000391b0000 /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
0x00000000391b0000 /usr/lib/liblangid.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/MultitouchSupport.framework/Versions/A/MultitouchSupport
0x00000000391b0000 /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
0x00000000391b0000 /usr/lib/libDiagnosticMessagesClient.dylib
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
0x00000000391b0000 /System/Library/PrivateFrameworks/PerformanceAnalysis.framework/Versions/A/PerformanceAnalysis
0x00000000391b0000 /System/Library/PrivateFrameworks/GenerationalStorage.framework/Versions/A/GenerationalStorage
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
0x00000000391b0000 /System/Library/Frameworks/CoreImage.framework/Versions/A/CoreImage
0x00000000391b0000 /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
0x00000000391b0000 /System/Library/PrivateFrameworks/Backup.framework/Versions/A/Backup
0x00000000391b0000 /usr/lib/libarchive.2.dylib
0x00000000391b0000 /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
0x00000000391b0000 /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
0x00000000391b0000 /usr/lib/libCRFSuite.dylib
0x00000000391b0000 /usr/lib/libc++.1.dylib
0x00000000391b0000 /usr/lib/libc++abi.dylib
0x00000000391b0000 /usr/lib/system/libcache.dylib
0x00000000391b0000 /usr/lib/system/libcommonCrypto.dylib
0x00000000391b0000 /usr/lib/system/libcompiler_rt.dylib
0x00000000391b0000 /usr/lib/system/libcopyfile.dylib
0x00000000391b0000 /usr/lib/system/libcorecrypto.dylib
0x00000000391b0000 /usr/lib/system/libdispatch.dylib
0x00000000391b0000 /usr/lib/system/libdyld.dylib
0x00000000391b0000 /usr/lib/system/libkeymgr.dylib
0x00000000391b0000 /usr/lib/system/liblaunch.dylib
0x00000000391b0000 /usr/lib/system/libmacho.dylib
0x00000000391b0000 /usr/lib/system/libquarantine.dylib
0x00000000391b0000 /usr/lib/system/libremovefile.dylib
0x00000000391b0000 /usr/lib/system/libsystem_asl.dylib
0x00000000391b0000 /usr/lib/system/libsystem_blocks.dylib
0x00000000391b0000 /usr/lib/system/libsystem_c.dylib
0x00000000391b0000 /usr/lib/system/libsystem_configuration.dylib
0x00000000391b0000 /usr/lib/system/libsystem_coreservices.dylib
0x00000000391b0000 /usr/lib/system/libsystem_coretls.dylib
0x00000000391b0000 /usr/lib/system/libsystem_dnssd.dylib
0x00000000391b0000 /usr/lib/system/libsystem_info.dylib
0x00000000391b0000 /usr/lib/system/libsystem_kernel.dylib
0x00000000391b0000 /usr/lib/system/libsystem_m.dylib
0x00000000391b0000 /usr/lib/system/libsystem_malloc.dylib
0x00000000391b0000 /usr/lib/system/libsystem_network.dylib
0x00000000391b0000 /usr/lib/system/libsystem_networkextension.dylib
0x00000000391b0000 /usr/lib/system/libsystem_notify.dylib
0x00000000391b0000 /usr/lib/system/libsystem_platform.dylib
0x00000000391b0000 /usr/lib/system/libsystem_pthread.dylib
0x00000000391b0000 /usr/lib/system/libsystem_sandbox.dylib
0x00000000391b0000 /usr/lib/system/libsystem_secinit.dylib
0x00000000391b0000 /usr/lib/system/libsystem_symptoms.dylib
0x00000000391b0000 /usr/lib/system/libsystem_trace.dylib
0x00000000391b0000 /usr/lib/system/libunwind.dylib
0x00000000391b0000 /usr/lib/system/libxpc.dylib
0x00000000391b0000 /usr/lib/libbsm.0.dylib
0x00000000391b0000 /usr/lib/system/libkxld.dylib
0x00000000391b0000 /usr/lib/libcoretls.dylib
0x00000000391b0000 /usr/lib/libcoretls_cfhelpers.dylib
0x00000000391b0000 /usr/lib/libOpenScriptingUtil.dylib
0x00000000391b0000 /usr/lib/libpam.2.dylib
0x00000000391b0000 /usr/lib/libsqlite3.dylib
0x00000000391b0000 /usr/lib/libxar.1.dylib
0x00000000391b0000 /usr/lib/libbz2.1.0.dylib
0x00000000391b0000 /usr/lib/liblzma.5.dylib
0x00000000391b0000 /usr/lib/libnetwork.dylib
0x00000000391b0000 /usr/lib/libpcap.A.dylib
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/FSEvents.framework/Versions/A/FSEvents
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/CarbonCore.framework/Versions/A/CarbonCore
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/Metadata.framework/Versions/A/Metadata
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/OSServices.framework/Versions/A/OSServices
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SearchKit.framework/Versions/A/SearchKit
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/AE.framework/Versions/A/AE
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/LaunchServices.framework/Versions/A/LaunchServices
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/DictionaryServices.framework/Versions/A/DictionaryServices
0x00000000391b0000 /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList
0x00000000391b0000 /System/Library/Frameworks/NetFS.framework/Versions/A/NetFS
0x00000000391b0000 /System/Library/PrivateFrameworks/NetAuth.framework/Versions/A/NetAuth
0x00000000391b0000 /System/Library/PrivateFrameworks/login.framework/Versions/A/Frameworks/loginsupport.framework/Versions/A/loginsupport
0x00000000391b0000 /System/Library/PrivateFrameworks/TCC.framework/Versions/A/TCC
0x00000000391b0000 /usr/lib/libmecabra.dylib
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/ATS
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ColorSync.framework/Versions/A/ColorSync
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/HIServices.framework/Versions/A/HIServices
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/LangAnalysis.framework/Versions/A/LangAnalysis
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/PrintCore.framework/Versions/A/PrintCore
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/QD.framework/Versions/A/QD
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/SpeechSynthesis.framework/Versions/A/SpeechSynthesis
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vImage.framework/Versions/A/vImage
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/vecLib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvDSP.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBNNS.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libQuadrature.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libvMisc.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLinearAlgebra.dylib
0x00000000391b0000 /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libSparseBLAS.dylib
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/Resources/libFontParser.dylib
0x00000000391b0000 /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ATS.framework/Versions/A/Resources/libFontRegistry.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libJPEG.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libTIFF.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libPng.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libGIF.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libJP2.dylib
0x00000000391b0000 /System/Library/Frameworks/ImageIO.framework/Versions/A/Resources/libRadiance.dylib
0x00000000391b0000 /usr/lib/libcompression.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/AppleJPEG.framework/Versions/A/AppleJPEG
0x00000000391b0000 /usr/lib/libcups.2.dylib
0x00000000391b0000 /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos
0x00000000391b0000 /System/Library/Frameworks/GSS.framework/Versions/A/GSS
0x00000000391b0000 /usr/lib/libresolv.9.dylib
0x00000000391b0000 /usr/lib/libiconv.2.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/Heimdal.framework/Versions/A/Heimdal
0x00000000391b0000 /usr/lib/libheimdal-asn1.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenDirectory.framework/Versions/A/OpenDirectory
0x00000000391b0000 /System/Library/PrivateFrameworks/CommonAuth.framework/Versions/A/CommonAuth
0x00000000391b0000 /System/Library/Frameworks/OpenDirectory.framework/Versions/A/Frameworks/CFOpenDirectory.framework/Versions/A/CFOpenDirectory
0x00000000391b0000 /System/Library/Frameworks/SecurityFoundation.framework/Versions/A/SecurityFoundation
0x00000000391b0000 /System/Library/PrivateFrameworks/LanguageModeling.framework/Versions/A/LanguageModeling
0x00000000391b0000 /usr/lib/libmarisa.dylib
0x00000000391b0000 /usr/lib/libChineseTokenizer.dylib
0x00000000391b0000 /usr/lib/libcmph.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreEmoji.framework/Versions/A/CoreEmoji
0x00000000391b0000 /System/Library/Frameworks/ServiceManagement.framework/Versions/A/ServiceManagement
0x00000000391b0000 /usr/lib/libxslt.1.dylib
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Ink.framework/Versions/A/Ink
0x00000000391b0000 /System/Library/PrivateFrameworks/TextureIO.framework/Versions/A/TextureIO
0x00000000391b0000 /System/Library/Frameworks/Metal.framework/Versions/A/Metal
0x00000000391b0000 /usr/lib/libate.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/GPUCompiler.framework/libmetal_timestamp.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCoreFSCache.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/IOAccelerator.framework/Versions/A/IOAccelerator
0x00000000391b0000 /System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo
0x00000000391b0000 /usr/lib/libFosl_dynamic.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/MetalPerformanceShaders.framework/Versions/A/MetalPerformanceShaders
0x00000000391b0000 /System/Library/PrivateFrameworks/FaceCore.framework/Versions/A/FaceCore
0x00000000391b0000 /System/Library/Frameworks/OpenCL.framework/Versions/A/OpenCL
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGLU.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGFXShared.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGL.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libGLImage.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCVMSPluginSupport.dylib
0x00000000391b0000 /System/Library/Frameworks/OpenGL.framework/Versions/A/Libraries/libCoreVMClient.dylib
0x00000000391b0000 /System/Library/PrivateFrameworks/CrashReporterSupport.framework/Versions/A/CrashReporterSupport
0x00000000391b0000 /System/Library/PrivateFrameworks/Sharing.framework/Versions/A/Sharing
0x00000000391b0000 /System/Library/PrivateFrameworks/IconServices.framework/Versions/A/IconServices
0x00000000391b0000 /System/Library/PrivateFrameworks/ProtocolBuffer.framework/Versions/A/ProtocolBuffer
0x00000000391b0000 /System/Library/PrivateFrameworks/Apple80211.framework/Versions/A/Apple80211
0x00000000391b0000 /System/Library/Frameworks/CoreWLAN.framework/Versions/A/CoreWLAN
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreUtils.framework/Versions/A/CoreUtils
0x00000000391b0000 /System/Library/Frameworks/IOBluetooth.framework/Versions/A/IOBluetooth
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreWiFi.framework/Versions/A/CoreWiFi
0x00000000391b0000 /System/Library/Frameworks/CoreBluetooth.framework/Versions/A/CoreBluetooth
0x00000000391b0000 /System/Library/Frameworks/CoreDisplay.framework/Versions/A/CoreDisplay
0x00000000391b0000 /System/Library/PrivateFrameworks/IOPresentment.framework/Versions/A/IOPresentment
0x00000000391b0000 /System/Library/PrivateFrameworks/DSExternalDisplay.framework/Versions/A/DSExternalDisplay
0x00000000391b0000 /System/Library/PrivateFrameworks/DebugSymbols.framework/Versions/A/DebugSymbols
0x00000000391b0000 /System/Library/PrivateFrameworks/CoreSymbolication.framework/Versions/A/CoreSymbolication
0x00000000391b0000 /System/Library/PrivateFrameworks/Symbolication.framework/Versions/A/Symbolication
0x00000000391b0000 /System/Library/PrivateFrameworks/AppleFSCompression.framework/Versions/A/AppleFSCompression
0x00000000391b0000 /System/Library/PrivateFrameworks/SpeechRecognitionCore.framework/Versions/A/SpeechRecognitionCore
0x00000000391b0000 /System/Library/PrivateFrameworks/ChunkingLibrary.framework/Versions/A/ChunkingLibrary
0x0000000109c00000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/server/libjvm.dylib
0x00000000391b0000 /usr/lib/libstdc++.6.dylib
0x0000000106ac1000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libverify.dylib
0x0000000106acf000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libjava.dylib
0x0000000106b0f000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libzip.dylib
0x000000010aea5000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libmanagement.dylib
0x000000011a3e5000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libnet.dylib
0x00000010ac48f000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libnio.dylib
0x00000010ac795000 /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libgkl_compression2638078165374820023.dylib
0x00000010ac45a000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libosx.dylib
0x00000010ac876000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/./libosxapp.dylib
0x00000010ac7d2000 /System/Library/Frameworks/JavaVM.framework/Versions/A/Frameworks/JavaNativeFoundation.framework/Versions/A/JavaNativeFoundation
0x00000010ac7e7000 /System/Library/Frameworks/JavaVM.framework/Versions/A/Frameworks/JavaRuntimeSupport.framework/Versions/A/JavaRuntimeSupport
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
0x00000000391b0000 /System/Library/Frameworks/ExceptionHandling.framework/Versions/A/ExceptionHandling
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/CommonPanels.framework/Versions/A/CommonPanels
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Help.framework/Versions/A/Help
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/ImageCapture.framework/Versions/A/ImageCapture
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/OpenScripting.framework/Versions/A/OpenScripting
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/Print.framework/Versions/A/Print
0x00000000391b0000 /System/Library/Frameworks/Carbon.framework/Versions/A/Frameworks/SecurityHI.framework/Versions/A/SecurityHI
0x00000010ac892000 /System/Library/Frameworks/JavaVM.framework/Versions/A/JavaVM
0x00000010ac468000 /System/Library/PrivateFrameworks/JavaLaunching.framework/Versions/A/JavaLaunching
0x000000010aeb3000 /Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/jre/lib/libjaas_unix.dylib
0x00000010ac994000 /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/libbwa.4140429772138536592.jnilib
VM Arguments:
jvm_args: -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx60G
java_command: /Users/jude/gatk-4.1.8.0/gatk-package-4.1.8.0-local.jar PathSeqPipelineSpark -I SRR1839791.raw.bam --filter-bwa-image //Users/jude/GRCh38.primary_assembly.genome.fa --kmer-file //Users/jude/GRCh38.bfi --min-clipped-read-length 60 --min-score-identity 0.90 --identity-margin 0.02 --microbe-dict //Users/jude//gatk-4.1.8.0/pathseq_microbe.dict --microbe-bwa-image //Users/jude//gatk-4.1.8.0/pathseq_microbe.fa.img --taxonomy-file //Users/jude//gatk-4.1.8.0/pathseq_taxonomy.db --output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.bam --scores-output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_scores.txt --filter-metrics //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_filter_metrics.txt --score-metrics //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_score_metrics.txt
java_class_path (initial): /Users/jude/gatk-4.1.8.0/gatk-package-4.1.8.0-local.jar
Launcher Type: SUN_STANDARD
Environment Variables:
PATH=/Users/jude/.pyenv/versions/3.7.3/bin:/usr/local/Cellar/pyenv/1.2.18/libexec:/Users/jude/.pyenv/shims:/Library/Frameworks/Python.framework/Versions/3.7/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin
SHELL=/bin/bash
DISPLAY=/private/tmp/com.apple.launchd.2j2S8w567Z/org.macosforge.xquartz:0
Signal Handlers:
SIGSEGV: [libjvm.dylib+0x5b5439], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_ONSTACK|SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.dylib+0x5b5439], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.dylib+0x48ae24], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.dylib+0x48ae24], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.dylib+0x48ae24], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.dylib+0x48ae24], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR1: SIG_DFL, sa_mask[0]=11011111011111100000000111000110, sa_flags=none
SIGUSR2: [libjvm.dylib+0x48a942], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: [libjvm.dylib+0x488f0d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGINT: [libjvm.dylib+0x488f0d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGTERM: [libjvm.dylib+0x488f0d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
SIGQUIT: [libjvm.dylib+0x488f0d], sa_mask[0]=11111111011111110111111111111111, sa_flags=SA_RESTART|SA_SIGINFO
--------------- S Y S T E M ---------------
OS:Bsduname:Darwin 16.7.0 Darwin Kernel Version 16.7.0: Sun Jun 2 20:26:31 PDT 2019; root:xnu-3789.73.50~1/RELEASE_X86_64 x86_64
rlimit: STACK 8192k, CORE infinity, NPROC 709, NOFILE 10240, AS infinity
load average:23.34 15.19 8.52
CPU:total 24 (initial active 24) (6 cores per cpu, 2 threads per core) family 6 model 44 stepping 2, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, aes, clmul, ht, tsc, tscinvbit, tscinv
Memory: 4k page, physical 83886080k(1115316k free)
/proc/meminfo:
vm_info: Java HotSpot(TM) 64-Bit Server VM (25.161-b12) for bsd-amd64 JRE (1.8.0_161-b12), built on Dec 19 2017 16:22:20 by "java_re" with gcc 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)
time: Sat Jul 18 14:07:10 2020
elapsed time: 615 seconds (0d 0h 10m 15s) -
Hi Jude Taylor, thanks for the update. Are you sure you gave the correct file path for --filter-bwa-image? //Users/jude/GRCh38.primary_assembly.genome.fa looks like it is a .fa file and not an image.
-
Sorry, I should have picked that up, missed the '.img' when i copied the file name into the command line. It has now run but appears to have stalled after 5 hours. It is now 21.15 AEST and nothing has changed in terminal since 13.15 while the hard drive is continually active. I have enclosed excerpts from terminal and hopefully it will resolve overnight. If it doesn't do you mind to have a look at the full terminal log for the run if I post it? thanks again J.
jude$ ~/gatk-4.1.8.0/gatk PathSeqPipelineSpark -I SRR1839791.raw.bam --filter-bwa-image //Users/jude/GRCh38.primary_assembly.genome.fa.img --kmer-file //Users/jude/GRCh38.bfi --min-clipped-read-length 60 --min-score-identity 0.90 --identity-margin 0.02 --microbe-dict //Users/jude//gatk-4.1.8.0/pathseq_microbe.dict --microbe-bwa-image //Users/jude//gatk-4.1.8.0/pathseq_microbe.fa.img --taxonomy-file //Users/jude//gatk-4.1.8.0/pathseq_taxonomy.db --output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.bam --scores-output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_scores.txt --filter-metrics //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_filter_metrics.txt --score-metrics //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_score_metrics.txt --java-options "-Xmx60G"
Using GATK jar /Users/jude/gatk-4.1.8.0/gatk-package-4.1.8.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx60G -jar /Users/jude/gatk-4.1.8.0/gatk-package-4.1.8.0-local.jar PathSeqPipelineSpark -I SRR1839791.raw.bam --filter-bwa-image //Users/jude/GRCh38.primary_assembly.genome.fa.img --kmer-file //Users/jude/GRCh38.bfi --min-clipped-read-length 60 --min-score-identity 0.90 --identity-margin 0.02 --microbe-dict //Users/jude//gatk-4.1.8.0/pathseq_microbe.dict --microbe-bwa-image //Users/jude//gatk-4.1.8.0/pathseq_microbe.fa.img --taxonomy-file //Users/jude//gatk-4.1.8.0/pathseq_taxonomy.db --output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.bam --scores-output //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_scores.txt --filter-metrics //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_filter_metrics.txt --score-metrics //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq_score_metrics.txt
08:14:42.906 INFO NativeLibraryLoader - Loading libgkl_compression.dylib from jar:file:/Users/jude/gatk-4.1.8.0/gatk-package-4.1.8.0-local.jar!/com/intel/gkl/native/libgkl_compression.dylib
Jul 21, 2020 8:14:43 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
08:14:43.155 INFO PathSeqPipelineSpark - ------------------------------------------------------------
08:14:43.155 INFO PathSeqPipelineSpark - The Genome Analysis Toolkit (GATK) v4.1.8.0
08:14:43.155 INFO PathSeqPipelineSpark - For support and documentation go to https://software.broadinstitute.org/gatk/
08:14:43.156 INFO PathSeqPipelineSpark - Executing as jude@192-168-1-3.tpgi.com.au on Mac OS X v10.12.6 x86_64
08:14:43.156 INFO PathSeqPipelineSpark - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_161-b12
08:14:43.156 INFO PathSeqPipelineSpark - Start Date/Time: 21 July 2020 8:14:42 AM
08:14:43.156 INFO PathSeqPipelineSpark - ------------------------------------------------------------
08:14:43.156 INFO PathSeqPipelineSpark - ------------------------------------------------------------
08:14:43.156 INFO PathSeqPipelineSpark - HTSJDK Version: 2.22.0
08:14:43.156 INFO PathSeqPipelineSpark - Picard Version: 2.22.8
08:14:43.156 INFO PathSeqPipelineSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2
08:14:43.156 INFO PathSeqPipelineSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
08:14:43.157 INFO PathSeqPipelineSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
08:14:43.157 INFO PathSeqPipelineSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
08:14:43.157 INFO PathSeqPipelineSpark - Deflater: IntelDeflater
08:14:43.157 INFO PathSeqPipelineSpark - Inflater: IntelInflater
08:14:43.157 INFO PathSeqPipelineSpark - GCS max retries/reopens: 20
08:14:43.157 INFO PathSeqPipelineSpark - Requester pays: disabled
08:14:43.157 INFO PathSeqPipelineSpark - Initializing engine
08:14:43.157 INFO PathSeqPipelineSpark - Done initializing engine
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
20/07/21 08:14:43 INFO SparkContext: Running Spark version 2.4.5
08:14:43.729 WARN NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
20/07/21 08:14:43 INFO SparkContext: Submitted application: PathSeqPipelineSpark
20/07/21 08:14:43 INFO SecurityManager: Changing view acls to: jude
20/07/21 08:14:43 INFO SecurityManager: Changing modify acls to: jude
20/07/21 08:14:43 INFO SecurityManager: Changing view acls groups to:
20/07/21 08:14:43 INFO SecurityManager: Changing modify acls groups to:
20/07/21 08:14:43 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jude); groups with view permissions: Set(); users with modify permissions: Set(jude); groups with modify permissions: Set()
20/07/21 08:14:44 INFO Utils: Successfully started service 'sparkDriver' on port 57619.
20/07/21 08:14:44 INFO SparkEnv: Registering MapOutputTracker
20/07/21 08:14:44 INFO SparkEnv: Registering BlockManagerMaster
20/07/21 08:14:44 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
20/07/21 08:14:44 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
20/07/21 08:14:44 INFO DiskBlockManager: Created local directory at /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/blockmgr-bede01d7-5590-49a7-845c-604f83ef03b0
20/07/21 08:14:44 INFO MemoryStore: MemoryStore started with capacity 31.8 GB
20/07/21 08:14:44 INFO SparkEnv: Registering OutputCommitCoordinator
20/07/21 08:14:44 INFO Utils: Successfully started service 'SparkUI' on port 4040.
20/07/21 08:14:44 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://192-168-1-3.tpgi.com.au:4040
20/07/21 08:14:44 INFO Executor: Starting executor ID driver on host localhost
20/07/21 08:14:44 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 57620.
20/07/21 08:14:44 INFO NettyBlockTransferService: Server created on 192-168-1-3.tpgi.com.au:57620
20/07/21 08:14:44 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
20/07/21 08:14:44 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 192-168-1-3.tpgi.com.au, 57620, None)
20/07/21 08:14:44 INFO BlockManagerMasterEndpoint: Registering block manager 192-168-1-3.tpgi.com.au:57620 with 31.8 GB RAM, BlockManagerId(driver, 192-168-1-3.tpgi.com.au, 57620, None)
20/07/21 08:14:44 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 192-168-1-3.tpgi.com.au, 57620, None)
20/07/21 08:14:44 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, 192-168-1-3.tpgi.com.au, 57620, None)
08:14:44.650 INFO PathSeqPipelineSpark - Spark verbosity set to INFO (see --spark-verbosity argument)
20/07/21 08:14:44 INFO GoogleHadoopFileSystemBase: GHFS version: 1.6.3-hadoop2
20/07/21 08:14:45 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 306.2 KB, free 31.8 GB)
20/07/21 08:14:45 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 25.4 KB, free 31.8 GB)
20/07/21 08:14:45 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 25.4 KB, free: 31.8 GB)
20/07/21 08:14:45 INFO SparkContext: Created broadcast 0 from newAPIHadoopFile at PathSplitSource.java:96
20/07/21 08:14:46 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 306.2 KB, free 31.8 GB)
20/07/21 08:14:46 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 25.4 KB, free 31.8 GB)
20/07/21 08:14:46 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 25.4 KB, free: 31.8 GB)
20/07/21 08:14:46 INFO SparkContext: Created broadcast 1 from newAPIHadoopFile at PathSplitSource.java:96
08:14:46.398 INFO FileInputFormat - Total input files to process : 1
20/07/21 08:14:46 INFO SparkContext: Starting job: first at SparkUtils.java:185
20/07/21 08:14:46 INFO DAGScheduler: Got job 0 (first at SparkUtils.java:185) with 1 output partitions
20/07/21 08:14:46 INFO DAGScheduler: Final stage: ResultStage 0 (first at SparkUtils.java:185)
20/07/21 08:14:46 INFO DAGScheduler: Parents of final stage: List()
20/07/21 08:14:46 INFO DAGScheduler: Missing parents: List()
20/07/21 08:14:46 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[11] at filter at ReadsSparkSource.java:100), which has no missing parents
20/07/21 08:14:46 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 183.8 KB, free 31.8 GB)
20/07/21 08:14:46 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 80.0 KB, free 31.8 GB)
20/07/21 08:14:46 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 80.0 KB, free: 31.8 GB)
20/07/21 08:14:46 INFO SparkContext: Created broadcast 2 from broadcast at DAGScheduler.scala:1163
20/07/21 08:14:46 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[11] at filter at ReadsSparkSource.java:100) (first 15 tasks are for partitions Vector(0))
20/07/21 08:14:46 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks
20/07/21 08:14:46 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, executor driver, partition 0, PROCESS_LOCAL, 7954 bytes)
20/07/21 08:14:46 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
20/07/21 08:14:46 INFO NewHadoopRDD: Input split: file:/Users/jude/RNAseq/RawRNAseq/SRR1839791.raw.bam:0+33554432
20/07/21 08:14:46 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1010 bytes result sent to driver
20/07/21 08:14:46 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 252 ms on localhost (executor driver) (1/1)
20/07/21 08:14:46 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
20/07/21 08:14:46 INFO DAGScheduler: ResultStage 0 (first at SparkUtils.java:185) finished in 0.393 s
20/07/21 08:14:46 INFO DAGScheduler: Job 0 finished: first at SparkUtils.java:185, took 0.446829 s
20/07/21 08:14:47 INFO SparkContext: Starting job: collect at SparkUtils.java:197
20/07/21 08:14:47 INFO DAGScheduler: Got job 1 (collect at SparkUtils.java:197) with 311 output partitions
20/07/21 08:14:47 INFO DAGScheduler: Final stage: ResultStage 1 (collect at SparkUtils.java:197)
20/07/21 08:14:47 INFO DAGScheduler: Parents of final stage: List()
20/07/21 08:14:47 INFO DAGScheduler: Missing parents: List()
20/07/21 08:14:47 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[12] at mapPartitions at SparkUtils.java:188), which has no missing parents
20/07/21 08:14:47 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 184.1 KB, free 31.8 GB)
20/07/21 08:14:47 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 80.1 KB, free 31.8 GB)
20/07/21 08:14:47 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 80.1 KB, free: 31.8 GB)
20/07/21 08:14:47 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1163
20/07/21 08:14:47 INFO DAGScheduler: Submitting 311 missing tasks from ResultStage 1 (MapPartitionsRDD[12] at mapPartitions at SparkUtils.java:188) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14))
20/07/21 08:14:47 INFO TaskSchedulerImpl: Adding task set 1.0 with 311 tasks
20/07/21 08:14:47 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1, localhost, executor driver, partition 0, PROCESS_LOCAL, 7954 bytes)
……….many hours later……
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 150
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 139
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 144
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 141
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 130
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 128
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 145
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 169
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 146
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 154
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 129
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 125
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 170
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 135
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 165
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 161
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 160
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 152
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 142
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 155
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 163
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 143
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 174
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 131
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 164
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 126
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 151
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 147
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 166
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 171
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 140
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 134
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 162
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 168
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 133
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 138
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 172
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 156
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 173
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 127
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 137
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 148
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 149
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 157
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 159
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 132
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 136
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 167
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 158
20/07/21 13:14:48 INFO BlockManagerInfo: Removed broadcast_8_piece0 on 192-168-1-3.tpgi.com.au:57620 in memory (size: 3.1 KB, free: 31.7 GB)
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 153 -
Jude Taylor Great, I don't see any problems so far, please let me know if it works.
-
hmmmm....well nothing has changed in terminal during the last 32 hours,still 'stuck' at the following step. Any suggestions? Only one of the output files has appeared, the pathseq_filter_metrics file at 13.14 yesterday. Thanks again
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 158
20/07/21 13:14:48 INFO BlockManagerInfo: Removed broadcast_8_piece0 on 192-168-1-3.tpgi.com.au:57620 in memory (size: 3.1 KB, free: 31.7 GB)
20/07/21 13:14:48 INFO ContextCleaner: Cleaned accumulator 153 -
Hi Jude Taylor, I spoke to one of our developers about this issue, and it is common for this to happen if the BAM is large or there is a large portion of microbial reads. We presented some options for helping with this issue at our old forum linked here. The information is from a year ago, our forum has since moved to the current website and we preserved the old posts on that google sites link. Hope that helps!
-
Thanks a lot for the guidance, ultimately it was unable to write the output bam file (after 3 days). I hope I haven't done something stupid again. Would it help if I generated an img, index and db just for my microbial genome of interest? I have enclosed the remaining terminal log continued from the previous 2 code posts above. Thanks J
20/07/23 15:12:06 INFO MemoryStore: Block rdd_65_1 stored as values in memory (estimated size 2.1 MB, free 29.2 GB)
20/07/23 15:12:06 INFO BlockManagerInfo: Added rdd_65_1 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 2.1 MB, free: 31.7 GB)
20/07/23 15:22:14 INFO Executor: Finished task 5.0 in stage 38.0 (TID 5030). 1315 bytes result sent to driver
20/07/23 15:22:14 INFO TaskSetManager: Finished task 5.0 in stage 38.0 (TID 5030) in 180494023 ms on localhost (executor driver) (1/6)
20/07/23 15:44:28 INFO MemoryStore: Block rdd_65_0 stored as values in memory (estimated size 1933.3 KB, free 29.2 GB)
20/07/23 15:44:28 INFO BlockManagerInfo: Added rdd_65_0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 1933.3 KB, free: 31.7 GB)
20/07/23 15:52:12 INFO Executor: Finished task 4.0 in stage 38.0 (TID 5029). 1315 bytes result sent to driver
20/07/23 15:52:12 INFO TaskSetManager: Finished task 4.0 in stage 38.0 (TID 5029) in 182291668 ms on localhost (executor driver) (2/6)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (122, 154, 205)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 371)
[M::mem_pestat] mean and std.dev: (165.74, 72.72)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 457)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (130, 159, 217)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 391)
[M::mem_pestat] mean and std.dev: (171.79, 70.98)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 478)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (136, 180, 247)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 469)
[M::mem_pestat] mean and std.dev: (190.00, 84.90)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 580)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
20/07/24 07:16:05 INFO MemoryStore: Block rdd_64_3 stored as values in memory (estimated size 3.3 MB, free 29.2 GB)
20/07/24 07:16:05 INFO BlockManagerInfo: Added rdd_64_3 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 3.3 MB, free: 31.7 GB)
20/07/24 07:16:10 INFO Executor: Finished task 3.0 in stage 38.0 (TID 5028). 1315 bytes result sent to driver
20/07/24 07:16:11 INFO TaskSetManager: Finished task 3.0 in stage 38.0 (TID 5028) in 237730682 ms on localhost (executor driver) (3/6)
20/07/24 07:24:20 INFO MemoryStore: Block rdd_64_1 stored as values in memory (estimated size 3.3 MB, free 29.2 GB)
20/07/24 07:24:21 INFO BlockManagerInfo: Added rdd_64_1 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 3.3 MB, free: 31.7 GB)
20/07/24 07:24:25 INFO Executor: Finished task 1.0 in stage 38.0 (TID 5026). 1358 bytes result sent to driver
20/07/24 07:24:25 INFO TaskSetManager: Finished task 1.0 in stage 38.0 (TID 5026) in 238225457 ms on localhost (executor driver) (4/6)
20/07/24 07:25:02 INFO MemoryStore: Block rdd_64_0 stored as values in memory (estimated size 3.0 MB, free 29.2 GB)
20/07/24 07:25:02 INFO BlockManagerInfo: Added rdd_64_0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 3.0 MB, free: 31.7 GB)
20/07/24 07:29:34 INFO Executor: Finished task 0.0 in stage 38.0 (TID 5025). 1315 bytes result sent to driver
20/07/24 07:29:34 INFO TaskSetManager: Finished task 0.0 in stage 38.0 (TID 5025) in 238534115 ms on localhost (executor driver) (5/6)
[M::mem_pestat] skip orientation FF as there are not enough pairs
[M::mem_pestat] analyzing insert size distribution for orientation FR...
[M::mem_pestat] (25, 50, 75) percentile: (127, 172, 242)
[M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 472)
[M::mem_pestat] mean and std.dev: (187.54, 89.17)
[M::mem_pestat] low and high boundaries for proper pairs: (1, 587)
[M::mem_pestat] skip orientation RF as there are not enough pairs
[M::mem_pestat] skip orientation RR as there are not enough pairs
20/07/24 07:33:49 INFO MemoryStore: Block rdd_64_2 stored as values in memory (estimated size 3.3 MB, free 29.2 GB)
20/07/24 07:33:49 INFO BlockManagerInfo: Added rdd_64_2 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 3.3 MB, free: 31.7 GB)
20/07/24 07:34:22 INFO Executor: Finished task 2.0 in stage 38.0 (TID 5027). 1315 bytes result sent to driver
20/07/24 07:34:22 INFO TaskSetManager: Finished task 2.0 in stage 38.0 (TID 5027) in 238821749 ms on localhost (executor driver) (6/6)
20/07/24 07:34:22 INFO TaskSchedulerImpl: Removed TaskSet 38.0, whose tasks have all completed, from pool
20/07/24 07:34:22 INFO DAGScheduler: ShuffleMapStage 38 (mapPartitionsToPair at PSScorer.java:68) finished in 238822.303 s
20/07/24 07:34:22 INFO DAGScheduler: looking for newly runnable stages
20/07/24 07:34:22 INFO DAGScheduler: running: Set()
20/07/24 07:34:22 INFO DAGScheduler: waiting: Set(ResultStage 39)
20/07/24 07:34:22 INFO DAGScheduler: failed: Set()
20/07/24 07:34:22 INFO DAGScheduler: Submitting ResultStage 39 (ShuffledRDD[73] at reduceByKey at PSScorer.java:71), which has no missing parents
20/07/24 07:34:22 INFO MemoryStore: Block broadcast_24 stored as values in memory (estimated size 4.7 KB, free 29.2 GB)
20/07/24 07:34:22 INFO MemoryStore: Block broadcast_24_piece0 stored as bytes in memory (estimated size 2.6 KB, free 29.2 GB)
20/07/24 07:34:22 INFO BlockManagerInfo: Added broadcast_24_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 2.6 KB, free: 31.7 GB)
20/07/24 07:34:22 INFO SparkContext: Created broadcast 24 from broadcast at DAGScheduler.scala:1163
20/07/24 07:34:22 INFO DAGScheduler: Submitting 6 missing tasks from ResultStage 39 (ShuffledRDD[73] at reduceByKey at PSScorer.java:71) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5))
20/07/24 07:34:22 INFO TaskSchedulerImpl: Adding task set 39.0 with 6 tasks
20/07/24 07:34:23 INFO TaskSetManager: Starting task 0.0 in stage 39.0 (TID 5031, localhost, executor driver, partition 0, ANY, 7662 bytes)
20/07/24 07:34:23 INFO TaskSetManager: Starting task 1.0 in stage 39.0 (TID 5032, localhost, executor driver, partition 1, ANY, 7662 bytes)
20/07/24 07:34:23 INFO TaskSetManager: Starting task 2.0 in stage 39.0 (TID 5033, localhost, executor driver, partition 2, ANY, 7662 bytes)
20/07/24 07:34:23 INFO TaskSetManager: Starting task 3.0 in stage 39.0 (TID 5034, localhost, executor driver, partition 3, ANY, 7662 bytes)
20/07/24 07:34:23 INFO TaskSetManager: Starting task 4.0 in stage 39.0 (TID 5035, localhost, executor driver, partition 4, ANY, 7662 bytes)
20/07/24 07:34:23 INFO TaskSetManager: Starting task 5.0 in stage 39.0 (TID 5036, localhost, executor driver, partition 5, ANY, 7662 bytes)
20/07/24 07:34:23 INFO Executor: Running task 0.0 in stage 39.0 (TID 5031)
20/07/24 07:34:23 INFO Executor: Running task 1.0 in stage 39.0 (TID 5032)
20/07/24 07:34:23 INFO Executor: Running task 2.0 in stage 39.0 (TID 5033)
20/07/24 07:34:23 INFO Executor: Running task 3.0 in stage 39.0 (TID 5034)
20/07/24 07:34:23 INFO Executor: Running task 4.0 in stage 39.0 (TID 5035)
20/07/24 07:34:23 INFO Executor: Running task 5.0 in stage 39.0 (TID 5036)
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:23 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:24 INFO Executor: Finished task 2.0 in stage 39.0 (TID 5033). 43493 bytes result sent to driver
20/07/24 07:34:24 INFO Executor: Finished task 1.0 in stage 39.0 (TID 5032). 46939 bytes result sent to driver
20/07/24 07:34:24 INFO Executor: Finished task 4.0 in stage 39.0 (TID 5035). 47558 bytes result sent to driver
20/07/24 07:34:24 INFO Executor: Finished task 0.0 in stage 39.0 (TID 5031). 46204 bytes result sent to driver
20/07/24 07:34:24 INFO Executor: Finished task 5.0 in stage 39.0 (TID 5036). 45667 bytes result sent to driver
20/07/24 07:34:24 INFO Executor: Finished task 3.0 in stage 39.0 (TID 5034). 48311 bytes result sent to driver
20/07/24 07:34:24 INFO TaskSetManager: Finished task 1.0 in stage 39.0 (TID 5032) in 1140 ms on localhost (executor driver) (1/6)
20/07/24 07:34:24 INFO TaskSetManager: Finished task 2.0 in stage 39.0 (TID 5033) in 1141 ms on localhost (executor driver) (2/6)
20/07/24 07:34:24 INFO TaskSetManager: Finished task 5.0 in stage 39.0 (TID 5036) in 1165 ms on localhost (executor driver) (3/6)
20/07/24 07:34:24 INFO TaskSetManager: Finished task 0.0 in stage 39.0 (TID 5031) in 1233 ms on localhost (executor driver) (4/6)
20/07/24 07:34:24 INFO TaskSetManager: Finished task 4.0 in stage 39.0 (TID 5035) in 1167 ms on localhost (executor driver) (5/6)
20/07/24 07:34:24 INFO TaskSetManager: Finished task 3.0 in stage 39.0 (TID 5034) in 1167 ms on localhost (executor driver) (6/6)
20/07/24 07:34:24 INFO TaskSchedulerImpl: Removed TaskSet 39.0, whose tasks have all completed, from pool
20/07/24 07:34:24 INFO DAGScheduler: ResultStage 39 (collectAsMap at PSScorer.java:71) finished in 1.646 s
20/07/24 07:34:24 INFO DAGScheduler: Job 12 finished: collectAsMap at PSScorer.java:71, took 236587.303260 s
20/07/24 07:34:40 INFO SparkContext: Starting job: collect at PSBwaUtils.java:59
20/07/24 07:34:40 INFO DAGScheduler: Registering RDD 75 (distinct at PSBwaUtils.java:59) as input to shuffle 8
20/07/24 07:34:40 INFO DAGScheduler: Got job 13 (collect at PSBwaUtils.java:59) with 6 output partitions
20/07/24 07:34:40 INFO DAGScheduler: Final stage: ResultStage 48 (collect at PSBwaUtils.java:59)
20/07/24 07:34:40 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 47)
20/07/24 07:34:40 INFO DAGScheduler: Missing parents: List(ShuffleMapStage 47)
20/07/24 07:34:40 INFO DAGScheduler: Submitting ShuffleMapStage 47 (MapPartitionsRDD[75] at distinct at PSBwaUtils.java:59), which has no missing parents
20/07/24 07:34:40 INFO MemoryStore: Block broadcast_25 stored as values in memory (estimated size 11.8 KB, free 29.2 GB)
20/07/24 07:34:40 INFO MemoryStore: Block broadcast_25_piece0 stored as bytes in memory (estimated size 6.2 KB, free 29.2 GB)
20/07/24 07:34:40 INFO BlockManagerInfo: Added broadcast_25_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 6.2 KB, free: 31.7 GB)
20/07/24 07:34:40 INFO SparkContext: Created broadcast 25 from broadcast at DAGScheduler.scala:1163
20/07/24 07:34:40 INFO DAGScheduler: Submitting 6 missing tasks from ShuffleMapStage 47 (MapPartitionsRDD[75] at distinct at PSBwaUtils.java:59) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5))
20/07/24 07:34:40 INFO TaskSchedulerImpl: Adding task set 47.0 with 6 tasks
20/07/24 07:34:40 INFO TaskSetManager: Starting task 0.0 in stage 47.0 (TID 5037, localhost, executor driver, partition 0, PROCESS_LOCAL, 8036 bytes)
20/07/24 07:34:40 INFO TaskSetManager: Starting task 1.0 in stage 47.0 (TID 5038, localhost, executor driver, partition 1, PROCESS_LOCAL, 8036 bytes)
20/07/24 07:34:40 INFO TaskSetManager: Starting task 2.0 in stage 47.0 (TID 5039, localhost, executor driver, partition 2, PROCESS_LOCAL, 8036 bytes)
20/07/24 07:34:40 INFO TaskSetManager: Starting task 3.0 in stage 47.0 (TID 5040, localhost, executor driver, partition 3, PROCESS_LOCAL, 8036 bytes)
20/07/24 07:34:40 INFO TaskSetManager: Starting task 4.0 in stage 47.0 (TID 5041, localhost, executor driver, partition 4, PROCESS_LOCAL, 8036 bytes)
20/07/24 07:34:40 INFO TaskSetManager: Starting task 5.0 in stage 47.0 (TID 5042, localhost, executor driver, partition 5, PROCESS_LOCAL, 8036 bytes)
20/07/24 07:34:40 INFO Executor: Running task 0.0 in stage 47.0 (TID 5037)
20/07/24 07:34:40 INFO Executor: Running task 5.0 in stage 47.0 (TID 5042)
20/07/24 07:34:40 INFO Executor: Running task 4.0 in stage 47.0 (TID 5041)
20/07/24 07:34:40 INFO Executor: Running task 3.0 in stage 47.0 (TID 5040)
20/07/24 07:34:40 INFO Executor: Running task 2.0 in stage 47.0 (TID 5039)
20/07/24 07:34:40 INFO Executor: Running task 1.0 in stage 47.0 (TID 5038)
20/07/24 07:34:40 INFO BlockManager: Found block rdd_64_0 locally
20/07/24 07:34:40 INFO BlockManager: Found block rdd_64_2 locally
20/07/24 07:34:40 INFO BlockManager: Found block rdd_64_3 locally
20/07/24 07:34:40 INFO BlockManager: Found block rdd_65_0 locally
20/07/24 07:34:40 INFO BlockManager: Found block rdd_65_1 locally
20/07/24 07:34:40 INFO BlockManager: Found block rdd_64_1 locally
20/07/24 07:34:41 INFO Executor: Finished task 0.0 in stage 47.0 (TID 5037). 1057 bytes result sent to driver
20/07/24 07:34:41 INFO Executor: Finished task 2.0 in stage 47.0 (TID 5039). 1057 bytes result sent to driver
20/07/24 07:34:41 INFO Executor: Finished task 5.0 in stage 47.0 (TID 5042). 1057 bytes result sent to driver
20/07/24 07:34:41 INFO TaskSetManager: Finished task 2.0 in stage 47.0 (TID 5039) in 958 ms on localhost (executor driver) (1/6)
20/07/24 07:34:41 INFO TaskSetManager: Finished task 0.0 in stage 47.0 (TID 5037) in 1142 ms on localhost (executor driver) (2/6)
20/07/24 07:34:41 INFO TaskSetManager: Finished task 5.0 in stage 47.0 (TID 5042) in 911 ms on localhost (executor driver) (3/6)
20/07/24 07:34:41 INFO Executor: Finished task 4.0 in stage 47.0 (TID 5041). 1057 bytes result sent to driver
20/07/24 07:34:41 INFO Executor: Finished task 1.0 in stage 47.0 (TID 5038). 1057 bytes result sent to driver
20/07/24 07:34:41 INFO Executor: Finished task 3.0 in stage 47.0 (TID 5040). 1057 bytes result sent to driver
20/07/24 07:34:41 INFO TaskSetManager: Finished task 4.0 in stage 47.0 (TID 5041) in 1175 ms on localhost (executor driver) (4/6)
20/07/24 07:34:41 INFO TaskSetManager: Finished task 3.0 in stage 47.0 (TID 5040) in 1185 ms on localhost (executor driver) (5/6)
20/07/24 07:34:41 INFO TaskSetManager: Finished task 1.0 in stage 47.0 (TID 5038) in 1185 ms on localhost (executor driver) (6/6)
20/07/24 07:34:41 INFO TaskSchedulerImpl: Removed TaskSet 47.0, whose tasks have all completed, from pool
20/07/24 07:34:41 INFO DAGScheduler: ShuffleMapStage 47 (distinct at PSBwaUtils.java:59) finished in 1.551 s
20/07/24 07:34:41 INFO DAGScheduler: looking for newly runnable stages
20/07/24 07:34:41 INFO DAGScheduler: running: Set()
20/07/24 07:34:41 INFO DAGScheduler: waiting: Set(ResultStage 48)
20/07/24 07:34:41 INFO DAGScheduler: failed: Set()
20/07/24 07:34:41 INFO DAGScheduler: Submitting ResultStage 48 (MapPartitionsRDD[77] at distinct at PSBwaUtils.java:59), which has no missing parents
20/07/24 07:34:41 INFO MemoryStore: Block broadcast_26 stored as values in memory (estimated size 4.3 KB, free 29.2 GB)
20/07/24 07:34:41 INFO MemoryStore: Block broadcast_26_piece0 stored as bytes in memory (estimated size 2.5 KB, free 29.2 GB)
20/07/24 07:34:41 INFO BlockManagerInfo: Added broadcast_26_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 2.5 KB, free: 31.7 GB)
20/07/24 07:34:41 INFO SparkContext: Created broadcast 26 from broadcast at DAGScheduler.scala:1163
20/07/24 07:34:41 INFO DAGScheduler: Submitting 6 missing tasks from ResultStage 48 (MapPartitionsRDD[77] at distinct at PSBwaUtils.java:59) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5))
20/07/24 07:34:41 INFO TaskSchedulerImpl: Adding task set 48.0 with 6 tasks
20/07/24 07:34:41 INFO TaskSetManager: Starting task 0.0 in stage 48.0 (TID 5043, localhost, executor driver, partition 0, ANY, 7662 bytes)
20/07/24 07:34:41 INFO TaskSetManager: Starting task 1.0 in stage 48.0 (TID 5044, localhost, executor driver, partition 1, ANY, 7662 bytes)
20/07/24 07:34:41 INFO TaskSetManager: Starting task 2.0 in stage 48.0 (TID 5045, localhost, executor driver, partition 2, ANY, 7662 bytes)
20/07/24 07:34:41 INFO TaskSetManager: Starting task 3.0 in stage 48.0 (TID 5046, localhost, executor driver, partition 3, ANY, 7662 bytes)
20/07/24 07:34:41 INFO TaskSetManager: Starting task 4.0 in stage 48.0 (TID 5047, localhost, executor driver, partition 4, ANY, 7662 bytes)
20/07/24 07:34:41 INFO TaskSetManager: Starting task 5.0 in stage 48.0 (TID 5048, localhost, executor driver, partition 5, ANY, 7662 bytes)
20/07/24 07:34:41 INFO Executor: Running task 0.0 in stage 48.0 (TID 5043)
20/07/24 07:34:41 INFO Executor: Running task 2.0 in stage 48.0 (TID 5045)
20/07/24 07:34:41 INFO Executor: Running task 4.0 in stage 48.0 (TID 5047)
20/07/24 07:34:41 INFO Executor: Running task 1.0 in stage 48.0 (TID 5044)
20/07/24 07:34:41 INFO Executor: Running task 5.0 in stage 48.0 (TID 5048)
20/07/24 07:34:41 INFO Executor: Running task 3.0 in stage 48.0 (TID 5046)
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Getting 6 non-empty blocks including 6 local blocks and 0 remote blocks
20/07/24 07:34:42 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms
20/07/24 07:34:42 INFO Executor: Finished task 4.0 in stage 48.0 (TID 5047). 14995 bytes result sent to driver
20/07/24 07:34:42 INFO Executor: Finished task 5.0 in stage 48.0 (TID 5048). 14422 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 4.0 in stage 48.0 (TID 5047) in 160 ms on localhost (executor driver) (1/6)
20/07/24 07:34:42 INFO TaskSetManager: Finished task 5.0 in stage 48.0 (TID 5048) in 160 ms on localhost (executor driver) (2/6)
20/07/24 07:34:42 INFO Executor: Finished task 2.0 in stage 48.0 (TID 5045). 14621 bytes result sent to driver
20/07/24 07:34:42 INFO Executor: Finished task 0.0 in stage 48.0 (TID 5043). 14797 bytes result sent to driver
20/07/24 07:34:42 INFO Executor: Finished task 1.0 in stage 48.0 (TID 5044). 15138 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 2.0 in stage 48.0 (TID 5045) in 170 ms on localhost (executor driver) (3/6)
20/07/24 07:34:42 INFO TaskSetManager: Finished task 1.0 in stage 48.0 (TID 5044) in 189 ms on localhost (executor driver) (4/6)
20/07/24 07:34:42 INFO TaskSetManager: Finished task 0.0 in stage 48.0 (TID 5043) in 191 ms on localhost (executor driver) (5/6)
20/07/24 07:34:42 INFO Executor: Finished task 3.0 in stage 48.0 (TID 5046). 14865 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 3.0 in stage 48.0 (TID 5046) in 225 ms on localhost (executor driver) (6/6)
20/07/24 07:34:42 INFO TaskSchedulerImpl: Removed TaskSet 48.0, whose tasks have all completed, from pool
20/07/24 07:34:42 INFO DAGScheduler: ResultStage 48 (collect at PSBwaUtils.java:59) finished in 0.321 s
20/07/24 07:34:42 INFO DAGScheduler: Job 13 finished: collect at PSBwaUtils.java:59, took 1.939053 s
20/07/24 07:34:42 INFO SparkContext: Starting job: count at PSScoreFileLogger.java:30
20/07/24 07:34:42 INFO DAGScheduler: Got job 14 (count at PSScoreFileLogger.java:30) with 6 output partitions
20/07/24 07:34:42 INFO DAGScheduler: Final stage: ResultStage 56 (count at PSScoreFileLogger.java:30)
20/07/24 07:34:42 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 54, ShuffleMapStage 55)
20/07/24 07:34:42 INFO DAGScheduler: Missing parents: List()
20/07/24 07:34:42 INFO DAGScheduler: Submitting ResultStage 56 (MapPartitionsRDD[70] at flatMap at PSScorer.java:91), which has no missing parents
20/07/24 07:34:42 INFO MemoryStore: Block broadcast_27 stored as values in memory (estimated size 11.0 KB, free 29.2 GB)
20/07/24 07:34:42 INFO MemoryStore: Block broadcast_27_piece0 stored as bytes in memory (estimated size 6.0 KB, free 29.2 GB)
20/07/24 07:34:42 INFO BlockManagerInfo: Added broadcast_27_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 6.0 KB, free: 31.7 GB)
20/07/24 07:34:42 INFO SparkContext: Created broadcast 27 from broadcast at DAGScheduler.scala:1163
20/07/24 07:34:42 INFO DAGScheduler: Submitting 6 missing tasks from ResultStage 56 (MapPartitionsRDD[70] at flatMap at PSScorer.java:91) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5))
20/07/24 07:34:42 INFO TaskSchedulerImpl: Adding task set 56.0 with 6 tasks
20/07/24 07:34:42 INFO TaskSetManager: Starting task 0.0 in stage 56.0 (TID 5049, localhost, executor driver, partition 0, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 1.0 in stage 56.0 (TID 5050, localhost, executor driver, partition 1, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 2.0 in stage 56.0 (TID 5051, localhost, executor driver, partition 2, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 3.0 in stage 56.0 (TID 5052, localhost, executor driver, partition 3, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 4.0 in stage 56.0 (TID 5053, localhost, executor driver, partition 4, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 5.0 in stage 56.0 (TID 5054, localhost, executor driver, partition 5, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO Executor: Running task 3.0 in stage 56.0 (TID 5052)
20/07/24 07:34:42 INFO Executor: Running task 0.0 in stage 56.0 (TID 5049)
20/07/24 07:34:42 INFO Executor: Running task 4.0 in stage 56.0 (TID 5053)
20/07/24 07:34:42 INFO Executor: Running task 5.0 in stage 56.0 (TID 5054)
20/07/24 07:34:42 INFO Executor: Running task 2.0 in stage 56.0 (TID 5051)
20/07/24 07:34:42 INFO Executor: Running task 1.0 in stage 56.0 (TID 5050)
20/07/24 07:34:42 INFO BlockManager: Found block rdd_64_3 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_65_1 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_64_0 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_64_2 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_64_1 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_65_0 locally
20/07/24 07:34:42 INFO Executor: Finished task 5.0 in stage 56.0 (TID 5054). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 5.0 in stage 56.0 (TID 5054) in 60 ms on localhost (executor driver) (1/6)
20/07/24 07:34:42 INFO Executor: Finished task 2.0 in stage 56.0 (TID 5051). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 2.0 in stage 56.0 (TID 5051) in 89 ms on localhost (executor driver) (2/6)
20/07/24 07:34:42 INFO Executor: Finished task 3.0 in stage 56.0 (TID 5052). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 3.0 in stage 56.0 (TID 5052) in 90 ms on localhost (executor driver) (3/6)
20/07/24 07:34:42 INFO Executor: Finished task 4.0 in stage 56.0 (TID 5053). 753 bytes result sent to driver
20/07/24 07:34:42 INFO Executor: Finished task 1.0 in stage 56.0 (TID 5050). 753 bytes result sent to driver
20/07/24 07:34:42 INFO Executor: Finished task 0.0 in stage 56.0 (TID 5049). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 4.0 in stage 56.0 (TID 5053) in 126 ms on localhost (executor driver) (4/6)
20/07/24 07:34:42 INFO TaskSetManager: Finished task 1.0 in stage 56.0 (TID 5050) in 126 ms on localhost (executor driver) (5/6)
20/07/24 07:34:42 INFO TaskSetManager: Finished task 0.0 in stage 56.0 (TID 5049) in 147 ms on localhost (executor driver) (6/6)
20/07/24 07:34:42 INFO TaskSchedulerImpl: Removed TaskSet 56.0, whose tasks have all completed, from pool
20/07/24 07:34:42 INFO DAGScheduler: ResultStage 56 (count at PSScoreFileLogger.java:30) finished in 0.229 s
20/07/24 07:34:42 INFO DAGScheduler: Job 14 finished: count at PSScoreFileLogger.java:30, took 0.231548 s
20/07/24 07:34:42 INFO SparkContext: Starting job: count at PSScoreFileLogger.java:31
20/07/24 07:34:42 INFO DAGScheduler: Got job 15 (count at PSScoreFileLogger.java:31) with 6 output partitions
20/07/24 07:34:42 INFO DAGScheduler: Final stage: ResultStage 64 (count at PSScoreFileLogger.java:31)
20/07/24 07:34:42 INFO DAGScheduler: Parents of final stage: List(ShuffleMapStage 63, ShuffleMapStage 62)
20/07/24 07:34:42 INFO DAGScheduler: Missing parents: List()
20/07/24 07:34:42 INFO DAGScheduler: Submitting ResultStage 64 (MapPartitionsRDD[78] at filter at PSScoreFileLogger.java:31), which has no missing parents
20/07/24 07:34:42 INFO MemoryStore: Block broadcast_28 stored as values in memory (estimated size 11.5 KB, free 29.2 GB)
20/07/24 07:34:42 INFO MemoryStore: Block broadcast_28_piece0 stored as bytes in memory (estimated size 6.2 KB, free 29.2 GB)
20/07/24 07:34:42 INFO BlockManagerInfo: Added broadcast_28_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 6.2 KB, free: 31.7 GB)
20/07/24 07:34:42 INFO SparkContext: Created broadcast 28 from broadcast at DAGScheduler.scala:1163
20/07/24 07:34:42 INFO DAGScheduler: Submitting 6 missing tasks from ResultStage 64 (MapPartitionsRDD[78] at filter at PSScoreFileLogger.java:31) (first 15 tasks are for partitions Vector(0, 1, 2, 3, 4, 5))
20/07/24 07:34:42 INFO TaskSchedulerImpl: Adding task set 64.0 with 6 tasks
20/07/24 07:34:42 INFO TaskSetManager: Starting task 0.0 in stage 64.0 (TID 5055, localhost, executor driver, partition 0, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 1.0 in stage 64.0 (TID 5056, localhost, executor driver, partition 1, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 2.0 in stage 64.0 (TID 5057, localhost, executor driver, partition 2, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 3.0 in stage 64.0 (TID 5058, localhost, executor driver, partition 3, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 4.0 in stage 64.0 (TID 5059, localhost, executor driver, partition 4, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO TaskSetManager: Starting task 5.0 in stage 64.0 (TID 5060, localhost, executor driver, partition 5, PROCESS_LOCAL, 8047 bytes)
20/07/24 07:34:42 INFO Executor: Running task 1.0 in stage 64.0 (TID 5056)
20/07/24 07:34:42 INFO Executor: Running task 2.0 in stage 64.0 (TID 5057)
20/07/24 07:34:42 INFO Executor: Running task 5.0 in stage 64.0 (TID 5060)
20/07/24 07:34:42 INFO Executor: Running task 0.0 in stage 64.0 (TID 5055)
20/07/24 07:34:42 INFO Executor: Running task 3.0 in stage 64.0 (TID 5058)
20/07/24 07:34:42 INFO Executor: Running task 4.0 in stage 64.0 (TID 5059)
20/07/24 07:34:42 INFO BlockManager: Found block rdd_65_0 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_64_2 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_64_1 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_64_3 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_65_1 locally
20/07/24 07:34:42 INFO BlockManager: Found block rdd_64_0 locally
20/07/24 07:34:42 INFO Executor: Finished task 5.0 in stage 64.0 (TID 5060). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 5.0 in stage 64.0 (TID 5060) in 62 ms on localhost (executor driver) (1/6)
20/07/24 07:34:42 INFO Executor: Finished task 3.0 in stage 64.0 (TID 5058). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 3.0 in stage 64.0 (TID 5058) in 90 ms on localhost (executor driver) (2/6)
20/07/24 07:34:42 INFO Executor: Finished task 4.0 in stage 64.0 (TID 5059). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 4.0 in stage 64.0 (TID 5059) in 147 ms on localhost (executor driver) (3/6)
20/07/24 07:34:42 INFO Executor: Finished task 0.0 in stage 64.0 (TID 5055). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 0.0 in stage 64.0 (TID 5055) in 160 ms on localhost (executor driver) (4/6)
20/07/24 07:34:42 INFO Executor: Finished task 1.0 in stage 64.0 (TID 5056). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 1.0 in stage 64.0 (TID 5056) in 174 ms on localhost (executor driver) (5/6)
20/07/24 07:34:42 INFO Executor: Finished task 2.0 in stage 64.0 (TID 5057). 753 bytes result sent to driver
20/07/24 07:34:42 INFO TaskSetManager: Finished task 2.0 in stage 64.0 (TID 5057) in 309 ms on localhost (executor driver) (6/6)
20/07/24 07:34:42 INFO TaskSchedulerImpl: Removed TaskSet 64.0, whose tasks have all completed, from pool
20/07/24 07:34:42 INFO DAGScheduler: ResultStage 64 (count at PSScoreFileLogger.java:31) finished in 0.374 s
20/07/24 07:34:42 INFO DAGScheduler: Job 15 finished: count at PSScoreFileLogger.java:31, took 0.376476 s
20/07/24 07:34:43 INFO MemoryStore: Block broadcast_29 stored as values in memory (estimated size 7.6 MB, free 29.2 GB)
20/07/24 07:34:43 INFO MemoryStore: Block broadcast_29_piece0 stored as bytes in memory (estimated size 268.6 KB, free 29.2 GB)
20/07/24 07:34:43 INFO BlockManagerInfo: Added broadcast_29_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 268.6 KB, free: 31.7 GB)
20/07/24 07:34:43 INFO SparkContext: Created broadcast 29 from broadcast at ReadsSparkSink.java:146
20/07/24 07:34:43 INFO MemoryStore: Block broadcast_30 stored as values in memory (estimated size 7.6 MB, free 29.2 GB)
20/07/24 07:34:43 INFO MemoryStore: Block broadcast_30_piece0 stored as bytes in memory (estimated size 268.6 KB, free 29.2 GB)
20/07/24 07:34:43 INFO BlockManagerInfo: Added broadcast_30_piece0 in memory on 192-168-1-3.tpgi.com.au:57620 (size: 268.6 KB, free: 31.7 GB)
20/07/24 07:34:43 INFO SparkContext: Created broadcast 30 from broadcast at BamSink.java:76
20/07/24 07:34:44 INFO SparkUI: Stopped Spark web UI at http://192-168-1-3.tpgi.com.au:4040
20/07/24 07:34:44 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
20/07/24 07:34:46 INFO MemoryStore: MemoryStore cleared
20/07/24 07:34:46 INFO BlockManager: BlockManager stopped
20/07/24 07:34:47 INFO BlockManagerMaster: BlockManagerMaster stopped
20/07/24 07:34:47 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
20/07/24 07:34:47 INFO SparkContext: Successfully stopped SparkContext
07:34:47.252 INFO PathSeqPipelineSpark - Shutting down engine
[24 July 2020 7:34:47 AM] org.broadinstitute.hellbender.tools.spark.pathseq.PathSeqPipelineSpark done. Elapsed time: 4,280.07 minutes.
Runtime.totalMemory()=36463706112
***********************************************************************
A USER ERROR has occurred: Couldn't write file //Users/jude/RNAseq/SplitNCigarReads/SRR1839791.pathseq.bam because writing failed with exception No FileSystem for scheme: null
***********************************************************************
Set the system property GATK_STACKTRACE_ON_USER_EXCEPTION (--java-options '-DGATK_STACKTRACE_ON_USER_EXCEPTION=true') to print the stack trace.
20/07/24 07:34:47 INFO ShutdownHookManager: Shutdown hook called
20/07/24 07:34:47 INFO ShutdownHookManager: Deleting directory /private/var/folders/cc/smsmsly13wl6ll1z5wyb9mxc0000gn/T/spark-53783e6f-c4ae-4e3f-ae23-24be8925adf2 -
Hi Jude Taylor, it looks like it not being able to write to the file was a problem with your filesystem. I would look into that error more closely to see if the way you are running it is leading to the problem.
-
OK will do. I am relatively new to Unix commands so I will check the syntax, however the other pathseq output files were written so it seems to have mostly worked. Would there be any saving in process time if I generated an img, index and db just for my microbial genome of interest? thanks a lot Jude
-
Hi Jude Taylor, I am not sure the computational time to create those files just for the genome you are interested in. Was it able to finish since you commented?
-
Yes, it did actually work in the first run using the correct command line. I have since created the img, index and db just for my microbial genome of interest and it was only 30 minutes per run. However sequencing depth is an issue which wasn't sufficient for the samples I have. Thanks a lot anyway it is a great tool. J
-
Glad to hear this has been resolved!
Please sign in to leave a comment.
15 comments