error for gatk -- PathSeqPipelineSpark "java: bwa.c:329: bwa_mem2idx: Assertion `k == l_mem' failed."
REQUIRED for all errors and issues:
a) GATK version used:4.1.3.0
b) Exact command used: gatk --java-options "-Xmx150g" PathSeqPipelineSpark --input endometriosis.bam --filter-bwa-image pathseq_host.fa.img --kmer-file pathseq_host.bfi --min-clipped-read-length 60 --microbe-fasta pathseq_microbe.fa --microbe-bwa-image pathseq_microbe.fa.img --taxonomy-file pathseq_taxonomy.db --output endometriosis.pathseq.complete.bam --scores-output endometriosis.pathseq.complete.csv --is-host-aligned false --filter-duplicates false --min-score-identity .7
c) Entire program log:23/11/12 13:43:32 INFO TaskSchedulerImpl: Removed TaskSet 16.0, whose tasks have all completed, from pool
23/11/12 13:43:32 INFO DAGScheduler: ShuffleMapStage 16 (repartition at PathSeqPipelineSpark.java:198) finished in 7.562 s
23/11/12 13:43:32 INFO DAGScheduler: looking for newly runnable stages
23/11/12 13:43:32 INFO DAGScheduler: running: Set()
23/11/12 13:43:32 INFO DAGScheduler: waiting: Set(ShuffleMapStage 17, ResultStage 18)
23/11/12 13:43:32 INFO DAGScheduler: failed: Set()
23/11/12 13:43:32 INFO BlockManagerInfo: Removed broadcast_8_piece0 on 172.16.13.191:43771 in memory (size: 3.1 KB, free: 79.7 GB)
23/11/12 13:43:32 INFO DAGScheduler: Submitting ShuffleMapStage 17 (MapPartitionsRDD[63] at mapPartitionsToPair at PSScorer.java:68), which has no missing parents
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 160
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 150
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 157
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 202
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 165
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 173
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 210
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 158
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 168
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 154
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 172
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 211
23/11/12 13:43:32 INFO ContextCleaner: Cleaned accumulator 152
23/11/12 13:43:32 INFO MemoryStore: Block broadcast_15 stored as values in memory (estimated size 12.0 KB, free 77.2 GB)
23/11/12 13:43:32 INFO MemoryStore: Block broadcast_15_piece0 stored as bytes in memory (estimated size 6.2 KB, free 77.2 GB)
23/11/12 13:43:32 INFO BlockManagerInfo: Added broadcast_15_piece0 in memory on 172.16.13.191:43771 (size: 6.2 KB, free: 79.7 GB)
23/11/12 13:43:32 INFO SparkContext: Created broadcast 15 from broadcast at DAGScheduler.scala:1161
23/11/12 13:43:32 INFO DAGScheduler: Submitting 4 missing tasks from ShuffleMapStage 17 (MapPartitionsRDD[63] at mapPartitionsToPair at PSScorer.java:68) (first 15 tasks are for partitions Vector(0, 1, 2, 3))
23/11/12 13:43:32 INFO TaskSchedulerImpl: Adding task set 17.0 with 4 tasks
23/11/12 13:43:32 INFO TaskSetManager: Starting task 0.0 in stage 17.0 (TID 1401, localhost, executor driver, partition 0, PROCESS_LOCAL, 8036 bytes)
23/11/12 13:43:32 INFO TaskSetManager: Starting task 1.0 in stage 17.0 (TID 1402, localhost, executor driver, partition 1, ANY, 8036 bytes)
23/11/12 13:43:32 INFO TaskSetManager: Starting task 2.0 in stage 17.0 (TID 1403, localhost, executor driver, partition 2, ANY, 8036 bytes)
23/11/12 13:43:32 INFO TaskSetManager: Starting task 3.0 in stage 17.0 (TID 1404, localhost, executor driver, partition 3, ANY, 8036 bytes)
23/11/12 13:43:32 INFO Executor: Running task 0.0 in stage 17.0 (TID 1401)
23/11/12 13:43:32 INFO Executor: Running task 1.0 in stage 17.0 (TID 1402)
23/11/12 13:43:32 INFO Executor: Running task 3.0 in stage 17.0 (TID 1404)
23/11/12 13:43:32 INFO Executor: Running task 2.0 in stage 17.0 (TID 1403)
java: bwa.c:329: bwa_mem2idx: Assertion `k == l_mem' failed.
Due to the log is too long, I can't psate all.
I want to explore the reads of bacteria in each spot of spatial transcriptome data. But after converting the spatial transcriptome raw data fasta into a bam file, I encountered a bug using the above code.
-
Hi haha xixi
Did you use fastqs to align and produce bams or did you use fasta files to convert to bam? Do you observe this error with all your samples or only in a single sample?
Before we even proceed further can you try running this with the latest GATK version 4.4? There may be fixes along the way which could have solved this problem already.
If none of them works as this error is a low level one in the jni library we may need a sample file to reproduce this error. Can you send us a sample file?
Regards.
Please sign in to leave a comment.
1 comment