Recalibration ApplyBQSR crash
REQUIRED for all errors and issues:
a) GATK version used:
b) Exact command used:
c) Entire program log:
I got this error message when running gatk ApplyBQSR on GATK v4.5.0.0. Could you help me with this?
Thank you,
My code was:
parallel -j 32 "java -XX:ParallelGCThreads=1 \
-DGATK_STACKTRACE_ON_USER_EXCEPTION=true \
-jar gatk-package-4.5.0.0-local.jar BaseRecalibrator \
--input {}.sort.dedup.grp.bam \
--known-sites ${vcf} \
--reference ${reference_fasta} \
--output {}.recal.table \
2>${log_dir}/{}.recal1.log" ::: $(ls -1 *.sort.dedup.grp.bam | sed 's/.sort.dedup.grp.bam//')
The error message is below.
08:43:36.201 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/user1/Programs/gatk-4.5.0.0/gatk-package-4.5.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
08:43:36.410 INFO ApplyBQSR - ------------------------------------------------------------
08:43:36.414 INFO ApplyBQSR - The Genome Analysis Toolkit (GATK) v4.5.0.0
08:43:36.414 INFO ApplyBQSR - For support and documentation go to https://software.broadinstitute.org/gatk/
08:43:36.414 INFO ApplyBQSR - Executing as user1@agriserver.ca on Linux v5.4.0-150-generic amd64
08:43:36.415 INFO ApplyBQSR - Java runtime: OpenJDK 64-Bit Server VM v17.0.3-internal+0-adhoc..src
08:43:36.415 INFO ApplyBQSR - Start Date/Time: March 26, 2024 at 8:43:36 a.m. CST
08:43:36.416 INFO ApplyBQSR - ------------------------------------------------------------
08:43:36.416 INFO ApplyBQSR - ------------------------------------------------------------
08:43:36.417 INFO ApplyBQSR - HTSJDK Version: 4.1.0
08:43:36.418 INFO ApplyBQSR - Picard Version: 3.1.1
08:43:36.418 INFO ApplyBQSR - Built for Spark Version: 3.5.0
08:43:36.418 INFO ApplyBQSR - HTSJDK Defaults.COMPRESSION_LEVEL : 2
08:43:36.418 INFO ApplyBQSR - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
08:43:36.419 INFO ApplyBQSR - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
08:43:36.419 INFO ApplyBQSR - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
08:43:36.419 INFO ApplyBQSR - Deflater: IntelDeflater
08:43:36.420 INFO ApplyBQSR - Inflater: IntelInflater
08:43:36.420 INFO ApplyBQSR - GCS max retries/reopens: 20
08:43:36.420 INFO ApplyBQSR - Requester pays: disabled
08:43:36.421 INFO ApplyBQSR - Initializing engine
08:43:36.612 INFO ApplyBQSR - Done initializing engine
08:43:36.696 INFO ProgressMeter - Starting traversal
08:43:36.697 INFO ProgressMeter - Current Locus Elapsed Minutes Reads Processed Reads/Minute
08:43:46.706 INFO ProgressMeter - Chr1A:23226847 0.2 517000 3100139.9
08:43:56.714 INFO ProgressMeter - Chr1A:87627473 0.3 1171000 3510191.8
08:44:06.717 INFO ProgressMeter - Chr1A:234195235 0.5 1891000 3779606.2
08:44:16.728 INFO ProgressMeter - Chr1A:337380910 0.7 2597000 3892483.3
08:44:26.737 INFO ProgressMeter - Chr1A:400141378 0.8 3185000 3818944.8
08:44:36.741 INFO ProgressMeter - Chr1A:473780646 1.0 3844000 3841247.1
08:44:46.746 INFO ProgressMeter - Chr1A:519760911 1.2 4452000 3813385.1
08:44:51.207 INFO ApplyBQSR - Shutting down engine
[March 26, 2024 at 8:44:51 a.m. CST] org.broadinstitute.hellbender.tools.walkers.bqsr.ApplyBQSR done. Elapsed time: 1.25 minutes.
Runtime.totalMemory()=1493172224
htsjdk.samtools.SAMException: Exception when processing alignment for BAM index LH00315:84:22HCFMLT3:7:2291:5577:13834 2/2 156b aligned to Chr1A:536879422-536879577.
at htsjdk.samtools.BAMFileWriter.writeAlignment(BAMFileWriter.java:141)
at htsjdk.samtools.SAMFileWriterImpl.addAlignment(SAMFileWriterImpl.java:200)
at htsjdk.samtools.AsyncSAMFileWriter.synchronouslyWrite(AsyncSAMFileWriter.java:36)
at htsjdk.samtools.AsyncSAMFileWriter.synchronouslyWrite(AsyncSAMFileWriter.java:16)
at htsjdk.samtools.util.AbstractAsyncWriter$WriterRunnable.run(AbstractAsyncWriter.java:123)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: htsjdk.samtools.SAMException: Exception creating BAM index for record LH00315:84:22HCFMLT3:7:2291:5577:13834 2/2 156b aligned to Chr1A:536879422-536879577.
at htsjdk.samtools.BAMIndexer.processAlignment(BAMIndexer.java:132)
at htsjdk.samtools.BAMFileWriter.writeAlignment(BAMFileWriter.java:138)
... 5 more
Caused by: java.lang.IllegalStateException: Read position too high for BAI bin indexing.
at htsjdk.samtools.SAMRecord.computeIndexingBin(SAMRecord.java:1705)
at htsjdk.samtools.BAMIndexer$BAMIndexBuilder$1.getIndexingBin(BAMIndexer.java:250)
at htsjdk.samtools.BinningIndexBuilder.processFeature(BinningIndexBuilder.java:96)
at htsjdk.samtools.BAMIndexer$BAMIndexBuilder.processAlignment(BAMIndexer.java:238)
at htsjdk.samtools.BAMIndexer.processAlignment(BAMIndexer.java:130)
-
Hi E Ra
Looks like you are using a reference genome that exceeds the 2^29-1 limit of the bai index therefore you need a CSI index for your bam output. Unfortunately neither GATK nor its base library HTSJDK does support writing CSI indexes so you need to disable bam index creation by using the parameter
--CREATE_INDEX false
You can index your bam file later on using samtools index command with the csi index option on.
I hope this helps.
-
Sorry, the code I previously posted is not the right one. The code I used to get this error message is the following one.
parallel -j 32 "java -XX:ParallelGCThreads=1 \
-DGATK_STACKTRACE_ON_USER_EXCEPTION=true \
-jar gatk-package-4.5.0.0-local.jar ApplyBQSR \
--reference ${reference_fasta} \
--input {}.sort.dedup.grp.bam \
--bqsr-recal-file {}.recal.table \
--output {}.sort.dedup.grp.realigned.BQSR.bam \
2>${log_dir}/{}.realign.pass.log" ::: $(ls -1 *.sort.dedup.grp.bam | sed 's/.sort.dedup.grp.bam//') -
The issue seems to be solved when I used the parameter:
--create-output-bam-index false
Thank you!
Please sign in to leave a comment.
3 comments