Running GATK mutect2 in slurm cluster has a memory error. My objective is to generate a VCF file from a given bam file
REQUIRED for all errors and issues:
a) GATK version used: Genome Analysis Toolkit (GATK) v4.1.8.1
b) Exact command used:
gatk Mutect2 \
-R $REFERENCE_GENOME \
-I $INPUT_BAM \
-O ${OUTPUT_VCF} \
--java-options -Xmx64G
c) Entire program log:
01:11:38.452 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gpfs42/robbyfs/homes/aplic/noarch/software/GATK/4.1.8.1-GCCcore-9.3.0-Java-1.8/gat
Apr 05, 2024 1:11:38 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
01:11:38.635 INFO Mutect2 - ------------------------------------------------------------
01:11:38.635 INFO Mutect2 - The Genome Analysis Toolkit (GATK) v4.1.8.1
01:11:38.635 INFO Mutect2 - For support and documentation go to https://software.broadinstitute.org/gatk/
01:11:38.635 INFO Mutect2 - Executing as jreig@mr-00-02.cls.upf.edu on Linux v3.10.0-1127.18.2.el7.x86_64 amd64
01:11:38.635 INFO Mutect2 - Java runtime: Java HotSpot(TM) 64-Bit Server VM v1.8.0_241-b07
01:11:38.635 INFO Mutect2 - Start Date/Time: April 5, 2024 1:11:38 AM CEST
01:11:38.635 INFO Mutect2 - ------------------------------------------------------------
01:11:38.635 INFO Mutect2 - ------------------------------------------------------------
01:11:38.636 INFO Mutect2 - HTSJDK Version: 2.23.0
01:11:38.636 INFO Mutect2 - Picard Version: 2.22.8
01:11:38.636 INFO Mutect2 - HTSJDK Defaults.COMPRESSION_LEVEL : 2
01:11:38.636 INFO Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
01:11:38.636 INFO Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
01:11:38.636 INFO Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
01:11:38.636 INFO Mutect2 - Deflater: IntelDeflater
01:11:38.636 INFO Mutect2 - Inflater: IntelInflater
01:11:38.636 INFO Mutect2 - GCS max retries/reopens: 20
01:11:38.636 INFO Mutect2 - Requester pays: disabled
01:11:38.636 INFO Mutect2 - Initializing engine
01:11:39.169 INFO Mutect2 - Done initializing engine
01:11:39.190 INFO NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/gpfs42/robbyfs/homes/aplic/noarch/software/GATK/4.1.8.1-GCCcore-9.3.0-Java-1.8/gatk-pack
01:11:39.191 INFO NativeLibraryLoader - Loading libgkl_pairhmm_omp.so from jar:file:/gpfs42/robbyfs/homes/aplic/noarch/software/GATK/4.1.8.1-GCCcore-9.3.0-Java-1.8/gat
01:11:39.240 INFO IntelPairHmm - Flush-to-zero (FTZ) is enabled when running PairHMM
01:11:39.240 INFO IntelPairHmm - Available threads: 4
01:11:39.240 INFO IntelPairHmm - Requested threads: 4
01:11:39.240 INFO PairHMM - Using the OpenMP multi-threaded AVX-accelerated native PairHMM implementation
01:11:39.286 INFO ProgressMeter - Starting traversal
01:11:39.287 INFO ProgressMeter - Current Locus Elapsed Minutes Regions Processed Regions/Minute
01:11:50.471 INFO ProgressMeter - chr1:139708 0.2 570 3057.9
Using GATK jar /gpfs42/robbyfs/homes/aplic/noarch/software/GATK/4.1.8.1-GCCcore-9.3.0-Java-1.8/gatk-package-4.1.8.1-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx64G -jar /gpfs42/robbyfs/homes/aplic/noarch/software/GATK/4.1.8.1-GCCcore-9.3.0-Java-1.8/gatk-package-4.1.8.1-local.jar Mutect2 -R /gpfs42/projects/lab_genresearch/shared_data/Jaume/somatic_positive_con/hg/hg38.fa -I /gpfs42/projects/lab_genresearch/shared_data/Jaume/somatic_positive_con/BAMs/BQSR_S1.bam -O /gpfs42/robbyfs/scratch/lab_genresearch/jreig/output.vcf
slurmstepd: error: Detected 1 oom-kill event(s) in step 24271678.batch cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler.
-
Hi Jaume
I see that you set your heapsize as 64G which is quite a bit more that what regular runs of Mutect2 really needs. Can you reduce that amount to 8 or 12G to see if that works?
-
Hi, thanks for answering. I have previously tried lower amounts as you suggest and it didn't work. Any other possible solutions?
Thanks!
-
Hi again. Actually the error message is coming directly from the cluster management
slurmstepd: error: Detected 1 oom-kill event(s) in step 24271678.batch cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler.
It is possible that there is a limit to the amount of memory allowed for users on the cluster. Can you check with your IT?
Please sign in to leave a comment.
3 comments