munmap_chunk(): invalid pointer [FilterAlignmentArtifacts task in mutetct2 workflow]
Hello GATK Team,
I'm running somatic variant calling (mutect2) on 108 Whole Exome Tumor-Normal samples in Terra. However, one of the samples keeps failing at `munmap_chunk(): invalid pointer` at FilterAlignmentArtifacts task.
could you please shed some lights on how i can what is happening and i'll be able to resolve this.
gatk version used; 4.1.8.0
workflow used; https://app.terra.bio/#workspaces/help-gatk/Somatic-SNVs-Indels-GATK4/workflows/help-gatk/2-Mutect2-GATK4
Entire log file for the failed task is below
many thanks
`
2020/07/24 00:17:40 Starting container setup. 2020/07/24 00:17:43 Done container setup. 2020/07/24 00:17:44 Starting localization. 2020/07/24 00:17:50 Localization script execution started... 2020/07/24 00:17:50 Localizing input gs://gatk-test-data/mutect2/Homo_sapiens_assembly38.index_bundle -> /cromwell_root/gatk-test-data/mutect2/Homo_sapiens_assembly38.index_bundle 2020/07/24 00:18:29 Localizing input gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-FilterAlignmentArtifacts/attempt-3/script -> /cromwell_root/script 2020/07/24 00:18:33 Localization script execution complete. 2020/07/24 00:18:46 Done localization. 2020/07/24 00:18:47 Running user action: docker run -v /mnt/local-disk:/cromwell_root --entrypoint= us.gcr.io/broad-gatk/gatk@sha256:18146e79d06787483310e5de666502090a480e10ac0fad06a36a5e7a5c9bb1dc /bin/bash /cromwell_root/script Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.4be06a03 00:18:51.090 INFO NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/gatk/gatk-package-4.1.8.0-local.jar!/com/intel/gkl/native/libgkl_utils.so 00:18:51.108 INFO NativeLibraryLoader - Loading libgkl_smithwaterman.so from jar:file:/gatk/gatk-package-4.1.8.0-local.jar!/com/intel/gkl/native/libgkl_smithwaterman.so 00:18:51.110 INFO SmithWatermanAligner - Using AVX accelerated SmithWaterman implementation 00:18:51.328 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.1.8.0-local.jar!/com/intel/gkl/native/libgkl_compression.so 00:18:51.578 INFO FilterAlignmentArtifacts - ------------------------------------------------------------ 00:18:51.578 INFO FilterAlignmentArtifacts - The Genome Analysis Toolkit (GATK) v4.1.8.0 00:18:51.578 INFO FilterAlignmentArtifacts - For support and documentation go to https://software.broadinstitute.org/gatk/ 00:18:51.579 INFO FilterAlignmentArtifacts - Executing as root@3c67022d2450 on Linux v4.19.112+ amd64 00:18:51.579 INFO FilterAlignmentArtifacts - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_242-8u242-b08-0ubuntu3~18.04-b08 00:18:51.579 INFO FilterAlignmentArtifacts - Start Date/Time: July 24, 2020 12:18:51 AM GMT 00:18:51.579 INFO FilterAlignmentArtifacts - ------------------------------------------------------------ 00:18:51.579 INFO FilterAlignmentArtifacts - ------------------------------------------------------------ 00:18:51.580 INFO FilterAlignmentArtifacts - HTSJDK Version: 2.22.0 00:18:51.580 INFO FilterAlignmentArtifacts - Picard Version: 2.22.8 00:18:51.580 INFO FilterAlignmentArtifacts - HTSJDK Defaults.COMPRESSION_LEVEL : 2 00:18:51.580 INFO FilterAlignmentArtifacts - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false 00:18:51.580 INFO FilterAlignmentArtifacts - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true 00:18:51.580 INFO FilterAlignmentArtifacts - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false 00:18:51.581 INFO FilterAlignmentArtifacts - Deflater: IntelDeflater 00:18:51.581 INFO FilterAlignmentArtifacts - Inflater: IntelInflater 00:18:51.581 INFO FilterAlignmentArtifacts - GCS max retries/reopens: 20 00:18:51.581 INFO FilterAlignmentArtifacts - Requester pays: disabled 00:18:51.581 WARN FilterAlignmentArtifacts - [1m[31m !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Warning: FilterAlignmentArtifacts is an EXPERIMENTAL tool and should not be used for production !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!![0m 00:18:51.581 INFO FilterAlignmentArtifacts - Initializing engine 00:18:55.898 INFO FeatureManager - Using codec VCFCodec to read file gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-Filter/22.hg38-filtered.vcf 00:18:56.977 INFO FilterAlignmentArtifacts - Done initializing engine 00:18:57.854 INFO NativeLibraryLoader - Loading libgkl_pairhmm_omp.so from jar:file:/gatk/gatk-package-4.1.8.0-local.jar!/com/intel/gkl/native/libgkl_pairhmm_omp.so 00:18:57.903 INFO IntelPairHmm - Flush-to-zero (FTZ) is enabled when running PairHMM 00:18:57.904 INFO IntelPairHmm - Available threads: 4 00:18:57.904 INFO IntelPairHmm - Requested threads: 4 00:18:57.905 INFO PairHMM - Using the OpenMP multi-threaded AVX-accelerated native PairHMM implementation 00:18:57.905 INFO ProgressMeter - Starting traversal 00:18:57.905 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute munmap_chunk(): invalid pointer Using GATK jar /root/gatk.jar defined in environment variable GATK_LOCAL_JAR Running: java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx11500m -jar /root/gatk.jar FilterAlignmentArtifacts -R gs://gcp-public-data--broad-references/hg38/v0/Homo_sapiens_assembly38.fasta -V gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-Filter/22.hg38-filtered.vcf -I gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/209d1183-ed9a-4755-a4b3-d595797640ea/PreProcessingForVariantDiscovery_GATK4/9f7c0ab6-b61b-4797-92f1-7929bbf677d8/call-GatherBamFiles/22.hg38.bam --bwa-mem-index-image /cromwell_root/gatk-test-data/mutect2/Homo_sapiens_assembly38.index_bundle -O 22.hg38-filtered.vcf 2020/07/24 00:27:25 Starting delocalization. 2020/07/24 00:27:26 Delocalization script execution started... 2020/07/24 00:27:26 Delocalizing output /cromwell_root/memory_retry_rc -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-FilterAlignmentArtifacts/attempt-3/memory_retry_rc 2020/07/24 00:27:27 Delocalizing output /cromwell_root/rc -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-FilterAlignmentArtifacts/attempt-3/rc 2020/07/24 00:27:28 Delocalizing output /cromwell_root/stdout -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-FilterAlignmentArtifacts/attempt-3/stdout 2020/07/24 00:27:29 Delocalizing output /cromwell_root/stderr -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-FilterAlignmentArtifacts/attempt-3/stderr 2020/07/24 00:27:31 Delocalizing output /cromwell_root/22.hg38-filtered.vcf -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-FilterAlignmentArtifacts/attempt-3/22.hg38-filtered.vcf 2020/07/24 00:27:32 Delocalizing output /cromwell_root/22.hg38-filtered.vcf.idx -> gs://fc-ac4624cb-a8fc-49a2-b071-d3a0ae799418/5219ad90-3cbc-4ce6-9f1a-a6ed9791996f/Mutect2/ceafe1f6-0671-4bd9-bbf1-a24cc38033b8/call-FilterAlignmentArtifacts/attempt-3/22.hg38-filtered.vcf.idx Required file output '/cromwell_root/22.hg38-filtered.vcf.idx' does not exist.
`
-
Hey Sam,
I'll be happy to take a closer look at this. This is in the https://app.terra.bio/#workspaces/pazlabtest/Columbia_University_Breast_Cancer_07082020 workspace
submission ID: 5219ad90-3cbc-4ce6-9f1a-a6ed9791996f
workflow ID: ceafe1f6-0671-4bd9-bbf1-a24cc38033b8
correct?
Kind regards,
Jason
-
Hi Jason Cerrato . yes please, that's the right workspace and sub id.
many thanks
S
-
Hi sahuno,
This looks like it may be an example of a known issue: https://github.com/broadinstitute/gatk/issues/5690
Would you be able to add your own experience and log data to this thread to alert the engineers?
Kind regards,
Jason
-
i just did.
thanks
Sam
-
Hi Sam,
Perfect! Hopefully we'll get an answer from them about a fix soon.
Kind regards,
Jason
Please sign in to leave a comment.
5 comments