GATK4 Somatic CNV - proportional coverage calculation blunted
Hello GATK team,
I am experiencing an issue while trying to run the following GATK4 somatic cnv workflow: https://portal.firecloud.org/?return=terra#methods/kmeli/GATK4_Somatic_CNV_pipeline_KMelicopy/2
This first task of this workflow, gatk4CNVproportionalCoverageForCapture, uses the CalculateTargetCoverage tool to calculate target interval proportional coverages. However, the issue I am experiencing is that despite my input target interval containing target intervals across chromosomes 1-22, the output proportional coverage calculations are blunted at a seemingly arbitrary location on chromosome 1.
The exact command used in the task is:
java -jar /gatk/gatk.jar CalculateTargetCoverage \
--output ${sampleName}.pcov \
--groupBy SAMPLE \
--transform PCOV \
--targets ${paddedTargetTsv} \
--targetInformationColumns FULL \
--input ${inputBam} \
--reference ${refFasta} \
--cohortName "<ALL>"
The docker image used for this task is:
docker: "broadinstitute/gatk:4.beta.6"
The entire task log:
2022/08/12 22:00:06 Starting container setup.
2022/08/12 22:00:08 Done container setup.
2022/08/12 22:00:10 Starting localization.
2022/08/12 22:00:21 Localization script execution started...
2022/08/12 22:00:21 Localizing input gs://fc-a3480058-c708-4e6a-91d9-68e0500a0b83/5f752b6e-1c5d-4e1c-a9d5-10845d434b09/picardRealignment/afa5be59-bc6e-4bf9-9eb4-1bcb5bfa9a23/call-printReads/TCGA-3Z-A93Z-01A.realigned.hg19.bam -> /cromwell_root/fc-a3480058-c708-4e6a-91d9-68e0500a0b83/5f752b6e-1c5d-4e1c-a9d5-10845d434b09/picardRealignment/afa5be59-bc6e-4bf9-9eb4-1bcb5bfa9a23/call-printReads/TCGA-3Z-A93Z-01A.realigned.hg19.bam
2022/08/12 22:04:17 Localizing input gs://fc-a3480058-c708-4e6a-91d9-68e0500a0b83/VCRome_2_1_hg19_primary_targets_nochr_testpadded250bp.tsv -> /cromwell_root/fc-a3480058-c708-4e6a-91d9-68e0500a0b83/VCRome_2_1_hg19_primary_targets_nochr_testpadded250bp.tsv
2022/08/12 22:04:19 Localizing input gs://fc-a3480058-c708-4e6a-91d9-68e0500a0b83/submissions/00890f90-2e97-4abc-8a3b-2a24af6e64c9/GATK4SomaticCnvToolchainCaptureForSamples/83c80c50-d614-4b9a-bcff-8e1d5500378d/call-gatk4CNVproportionalCoverageForCapture/script -> /cromwell_root/script
2022/08/12 22:04:20 Localizing input gs://fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.fasta.fai -> /cromwell_root/fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.fasta.fai
2022/08/12 22:04:22 Localizing input gs://fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.dict -> /cromwell_root/fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.dict
2022/08/12 22:04:22 Localizing input gs://fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.fasta -> /cromwell_root/fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.fasta
Copying gs://fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.dict...
/ [0 files][ 0.0 B/ 14.5 KiB] / [1 files][ 14.5 KiB/ 14.5 KiB] Copying gs://fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.fasta...
/ [1 files][ 14.5 KiB/ 2.9 GiB] - - [1 files][ 56.7 MiB/ 2.9 GiB] \ | | [1 files][123.5 MiB/ 2.9 GiB] / - - [1 files][190.0 MiB/ 2.9 GiB] \ \ [1 files][258.9 MiB/ 2.9 GiB] | / / [1 files][327.2 MiB/ 2.9 GiB] - \ \ [1 files][394.0 MiB/ 2.9 GiB] | | [1 files][461.2 MiB/ 2.9 GiB] / - - [1 files][529.8 MiB/ 2.9 GiB] \ | | [1 files][597.6 MiB/ 2.9 GiB] / / [1 files][665.9 MiB/ 2.9 GiB] - \ \ [1 files][733.0 MiB/ 2.9 GiB] 67.7 MiB/s | / / [1 files][787.5 MiB/ 2.9 GiB] 65.1 MiB/s - - [1 files][851.2 MiB/ 2.9 GiB] 64.1 MiB/s \ | | [1 files][909.7 MiB/ 2.9 GiB] 62.2 MiB/s / - - [1 files][972.4 MiB/ 2.9 GiB] 61.1 MiB/s \ \ [1 files][ 1.0 GiB/ 2.9 GiB] 59.9 MiB/s | / / [1 files][ 1.1 GiB/ 2.9 GiB] 60.9 MiB/s - \ \ [1 files][ 1.1 GiB/ 2.9 GiB] 60.0 MiB/s | | [1 files][ 1.2 GiB/ 2.9 GiB] 60.5 MiB/s / - - [1 files][ 1.2 GiB/ 2.9 GiB] 60.2 MiB/s \ | | [1 files][ 1.3 GiB/ 2.9 GiB] 59.8 MiB/s / / [1 files][ 1.4 GiB/ 2.9 GiB] 58.7 MiB/s - \ \ [1 files][ 1.4 GiB/ 2.9 GiB] 58.9 MiB/s | | [1 files][ 1.5 GiB/ 2.9 GiB] 56.1 MiB/s / - - [1 files][ 1.5 GiB/ 2.9 GiB] 56.0 MiB/s \ | | [1 files][ 1.6 GiB/ 2.9 GiB] 53.4 MiB/s / / [1 files][ 1.6 GiB/ 2.9 GiB] 54.1 MiB/s - \ \ [1 files][ 1.7 GiB/ 2.9 GiB] 53.7 MiB/s | / / [1 files][ 1.7 GiB/ 2.9 GiB] 56.4 MiB/s - - [1 files][ 1.8 GiB/ 2.9 GiB] 56.4 MiB/s \ | | [1 files][ 1.9 GiB/ 2.9 GiB] 60.1 MiB/s / - - [1 files][ 1.9 GiB/ 2.9 GiB] 60.6 MiB/s \ \ [1 files][ 2.0 GiB/ 2.9 GiB] 61.6 MiB/s | / / [1 files][ 2.0 GiB/ 2.9 GiB] 61.9 MiB/s - \ \ [1 files][ 2.1 GiB/ 2.9 GiB] 62.3 MiB/s | | [1 files][ 2.2 GiB/ 2.9 GiB] 62.7 MiB/s / - - [1 files][ 2.2 GiB/ 2.9 GiB] 59.7 MiB/s \ | | [1 files][ 2.3 GiB/ 2.9 GiB] 58.9 MiB/s / / [1 files][ 2.3 GiB/ 2.9 GiB] 58.8 MiB/s - \ \ [1 files][ 2.4 GiB/ 2.9 GiB] 58.3 MiB/s | / / [1 files][ 2.4 GiB/ 2.9 GiB] 57.9 MiB/s - - [1 files][ 2.5 GiB/ 2.9 GiB] 60.5 MiB/s \ | | [1 files][ 2.6 GiB/ 2.9 GiB] 61.1 MiB/s / - - [1 files][ 2.6 GiB/ 2.9 GiB] 60.4 MiB/s \ \ [1 files][ 2.7 GiB/ 2.9 GiB] 60.8 MiB/s | / / [1 files][ 2.7 GiB/ 2.9 GiB] 60.8 MiB/s - \ \ [1 files][ 2.8 GiB/ 2.9 GiB] 60.6 MiB/s | | [1 files][ 2.9 GiB/ 2.9 GiB] 59.9 MiB/s / - - [1 files][ 2.9 GiB/ 2.9 GiB] 60.8 MiB/s
Operation completed over 2 objects/2.9 GiB.
- [2 files][ 2.9 GiB/ 2.9 GiB] 60.8 MiB/s 2022/08/12 22:05:13 Localization script execution complete.
2022/08/12 22:05:20 Done localization.
2022/08/12 22:05:21 Running user action: docker run -v /mnt/local-disk:/cromwell_root --entrypoint=/bin/bash broadinstitute/gatk@sha256:9512c7dcee1f6d1af094f5fbf242954d0a65895973ec28d93ff4b7a4c08e118c /cromwell_root/script
Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.2867cdd3
22:05:26.677 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/build/libs/gatk-package-4.beta.6-SNAPSHOT-local.jar!/com/intel/gkl/native/libgkl_compression.so
[August 12, 2022 10:05:26 PM UTC] CalculateTargetCoverage --output TCGA-3Z-A93Z-01A.pcov --groupBy SAMPLE --cohortName <ALL> --transform PCOV --targets /cromwell_root/fc-a3480058-c708-4e6a-91d9-68e0500a0b83/VCRome_2_1_hg19_primary_targets_nochr_testpadded250bp.tsv --targetInformationColumns FULL --input /cromwell_root/fc-a3480058-c708-4e6a-91d9-68e0500a0b83/5f752b6e-1c5d-4e1c-a9d5-10845d434b09/picardRealignment/afa5be59-bc6e-4bf9-9eb4-1bcb5bfa9a23/call-printReads/TCGA-3Z-A93Z-01A.realigned.hg19.bam --reference /cromwell_root/fc-09bfde39-82f5-4e45-b480-6682ffc1b6d4/reference/hg19/Homo_sapiens_assembly19.fasta --interval_set_rule UNION --interval_padding 0 --interval_exclusion_padding 0 --interval_merging_rule ALL --readValidationStringency SILENT --secondsBetweenProgressUpdates 10.0 --disableSequenceDictionaryValidation false --createOutputBamIndex true --createOutputBamMD5 false --createOutputVariantIndex true --createOutputVariantMD5 false --lenient false --addOutputSAMProgramRecord true --addOutputVCFCommandLine true --cloudPrefetchBuffer 40 --cloudIndexPrefetchBuffer -1 --disableBamIndexCaching false --help false --version false --showHidden false --verbosity INFO --QUIET false --use_jdk_deflater false --use_jdk_inflater false --gcs_max_retries 20 --disableToolDefaultReadFilters false22:05:26.892 INFO CalculateTargetCoverage - HTSJDK Defaults.COMPRESSION_LEVEL : 5
22:05:26.892 INFO CalculateTargetCoverage - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
22:05:26.892 INFO CalculateTargetCoverage - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : false
22:05:26.892 INFO CalculateTargetCoverage - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
22:05:26.892 INFO CalculateTargetCoverage - Deflater: IntelDeflater
22:05:26.892 INFO CalculateTargetCoverage - Inflater: IntelInflater
22:05:26.892 INFO CalculateTargetCoverage - GCS max retries/reopens: 20
22:05:26.893 INFO CalculateTargetCoverage - Using google-cloud-java patch c035098b5e62cb4fe9155eff07ce88449a361f5d from https://github.com/droazen/google-cloud-java/tree/dr_all_nio_fixes
22:05:26.893 INFO CalculateTargetCoverage - Initializing engine
[August 12, 2022 10:05:26 PM UTC] Executing as root@674e9314c98b on Linux 5.10.133+ amd64; OpenJDK 64-Bit Server VM 1.8.0_131-8u131-b11-0ubuntu1.16.04.2-b11; Version: 4.beta.6-SNAPSHOT
22:05:27.425 INFO CalculateTargetCoverage - Done initializing engine
22:05:27.439 INFO CalculateTargetCoverage - Reading targets locations from intervals...
22:05:27.440 INFO CalculateTargetCoverage - Reading target intervals from targets file '/cromwell_root/fc-a3480058-c708-4e6a-91d9-68e0500a0b83/VCRome_2_1_hg19_primary_targets_nochr_testpadded250bp.tsv' ...
22:05:27.441 INFO TargetTableReader - Reading targets from '/cromwell_root/fc-a3480058-c708-4e6a-91d9-68e0500a0b83/VCRome_2_1_hg19_primary_targets_nochr_testpadded250bp.tsv' ...
22:05:27.626 INFO CalculateTargetCoverage - Collecting read counts ...
22:05:27.627 INFO ProgressMeter - Starting traversal
22:05:27.627 INFO ProgressMeter - Current Locus Elapsed Minutes Reads Processed Reads/Minute
22:05:37.632 INFO ProgressMeter - 1:38435294 0.2 2237000 13419316.1
22:05:47.631 INFO ProgressMeter - 1:101383734 0.3 4767000 14298140.4
22:05:57.634 INFO ProgressMeter - 1:158669700 0.5 7236000 14468624.0
22:06:07.636 INFO ProgressMeter - 1:217184303 0.7 9718000 14573720.9
22:06:17.638 INFO ProgressMeter - 2:27916553 0.8 12237000 14681463.7
22:06:27.641 INFO ProgressMeter - 2:87234287 1.0 14710000 14706568.5
22:06:37.643 INFO ProgressMeter - 2:131949272 1.2 17235000 14769692.2
22:06:47.644 INFO ProgressMeter - 2:183902779 1.3 19792000 14840846.3
22:06:57.646 INFO ProgressMeter - 2:238006548 1.5 22302000 14864861.9
22:07:07.647 INFO ProgressMeter - 3:83885668 1.7 24763000 14854829.0
22:07:17.650 INFO ProgressMeter - 3:180334529 1.8 27252000 14861754.9
22:07:27.650 INFO ProgressMeter - 4:69519954 2.0 29727000 14860651.7
22:07:37.651 INFO ProgressMeter - 4:153864038 2.2 32115000 14819571.8
22:07:47.652 INFO ProgressMeter - 5:59268529 2.3 34629000 14838456.3
22:07:57.651 INFO ProgressMeter - 5:129070794 2.5 37114000 14843225.1
22:08:07.652 INFO ProgressMeter - 5:179025880 2.7 39591000 14844305.6
22:08:17.654 INFO ProgressMeter - 6:66012585 2.8 42033000 14832820.7
22:08:27.657 INFO ProgressMeter - 6:160517448 3.0 44528000 14840193.3
22:08:37.660 INFO ProgressMeter - 7:72984754 3.2 46860000 14795402.9
22:08:47.663 INFO ProgressMeter - 7:139168083 3.3 49332000 14797010.5
22:08:57.663 INFO ProgressMeter - 8:63942658 3.5 51799000 14797177.6
22:09:07.663 INFO ProgressMeter - 9:27949929 3.7 54290000 14803941.2
22:09:17.666 INFO ProgressMeter - 9:125877588 3.8 56778000 14809205.4
22:09:27.665 INFO ProgressMeter - 10:51087740 4.0 59241000 14807905.4
22:09:37.667 INFO ProgressMeter - 10:124389983 4.2 61644000 14792252.4
22:09:47.668 INFO ProgressMeter - 11:61547274 4.3 64124000 14795513.0
22:09:57.671 INFO ProgressMeter - 11:125351448 4.5 66630000 14804254.1
22:10:07.674 INFO ProgressMeter - 12:54978026 4.7 69118000 14808567.2
22:10:17.674 INFO ProgressMeter - 12:124408374 4.8 71658000 14823390.7
22:10:27.674 INFO ProgressMeter - 14:20019656 5.0 74160000 14829676.7
22:10:37.679 INFO ProgressMeter - 14:94155016 5.2 76570000 14817514.5
22:10:47.680 INFO ProgressMeter - 15:62283969 5.3 79086000 14826215.7
22:10:57.683 INFO ProgressMeter - 16:18538926 5.5 81555000 14825710.9
22:11:07.685 INFO ProgressMeter - 16:88791424 5.7 84060000 14831587.6
22:11:17.685 INFO ProgressMeter - 17:36375068 5.8 86551000 14834855.9
22:11:27.688 INFO ProgressMeter - 17:78325456 6.0 89016000 14833486.5
22:11:37.691 INFO ProgressMeter - 19:7049541 6.2 91420000 14822341.1
22:11:47.691 INFO ProgressMeter - 19:48830803 6.3 93871000 14819280.0
22:11:57.694 INFO ProgressMeter - 20:40111867 6.5 96365000 14822876.1
22:12:07.696 INFO ProgressMeter - 22:22869797 6.7 98850000 14824979.8
22:12:17.698 INFO ProgressMeter - X:48676618 6.8 101344000 14828261.4
22:12:27.699 INFO ProgressMeter - X:148664433 7.0 103802000 14826315.5
22:12:32.905 INFO CalculateTargetCoverage - 7713960 read(s) filtered by: (((WellformedReadFilter AND MappedReadFilter) AND NonZeroReferenceLengthAlignmentReadFilter) AND NotDuplicateReadFilter)
1257326 read(s) filtered by: ((WellformedReadFilter AND MappedReadFilter) AND NonZeroReferenceLengthAlignmentReadFilter)
1257307 read(s) filtered by: (WellformedReadFilter AND MappedReadFilter)
1257307 read(s) filtered by: MappedReadFilter
19 read(s) filtered by: NonZeroReferenceLengthAlignmentReadFilter
6456634 read(s) filtered by: NotDuplicateReadFilter
22:12:32.906 INFO ProgressMeter - GL000192.1:513772 7.1 104608174 14758524.3
22:12:32.906 INFO ProgressMeter - Traversal complete. Processed 104608174 total reads in 7.1 minutes.
22:12:32.906 INFO CalculateTargetCoverage - Collecting read counts done.
22:12:32.906 INFO CalculateTargetCoverage - Writing counts ...
22:12:33.065 INFO CalculateTargetCoverage - Writing counts done.
22:12:33.065 INFO CalculateTargetCoverage - Shutting down engine
[August 12, 2022 10:12:33 PM UTC] org.broadinstitute.hellbender.tools.exome.CalculateTargetCoverage done. Elapsed time: 7.11 minutes.Tool returned:
SUCCESS
Runtime.totalMemory()=294125568
2022/08/12 22:12:35 Starting delocalization.
2022/08/12 22:12:36 Delocalization script execution started...
2022/08/12 22:12:36 Delocalizing output /cromwell_root/memory_retry_rc -> gs://fc-a3480058-c708-4e6a-91d9-68e0500a0b83/submissions/00890f90-2e97-4abc-8a3b-2a24af6e64c9/GATK4SomaticCnvToolchainCaptureForSamples/83c80c50-d614-4b9a-bcff-8e1d5500378d/call-gatk4CNVproportionalCoverageForCapture/memory_retry_rc
2022/08/12 22:12:39 Delocalizing output /cromwell_root/rc -> gs://fc-a3480058-c708-4e6a-91d9-68e0500a0b83/submissions/00890f90-2e97-4abc-8a3b-2a24af6e64c9/GATK4SomaticCnvToolchainCaptureForSamples/83c80c50-d614-4b9a-bcff-8e1d5500378d/call-gatk4CNVproportionalCoverageForCapture/rc
2022/08/12 22:12:41 Delocalizing output /cromwell_root/stdout -> gs://fc-a3480058-c708-4e6a-91d9-68e0500a0b83/submissions/00890f90-2e97-4abc-8a3b-2a24af6e64c9/GATK4SomaticCnvToolchainCaptureForSamples/83c80c50-d614-4b9a-bcff-8e1d5500378d/call-gatk4CNVproportionalCoverageForCapture/stdout
2022/08/12 22:12:42 Delocalizing output /cromwell_root/stderr -> gs://fc-a3480058-c708-4e6a-91d9-68e0500a0b83/submissions/00890f90-2e97-4abc-8a3b-2a24af6e64c9/GATK4SomaticCnvToolchainCaptureForSamples/83c80c50-d614-4b9a-bcff-8e1d5500378d/call-gatk4CNVproportionalCoverageForCapture/stderr
2022/08/12 22:12:44 Delocalizing output /cromwell_root/TCGA-3Z-A93Z-01A.pcov -> gs://fc-a3480058-c708-4e6a-91d9-68e0500a0b83/submissions/00890f90-2e97-4abc-8a3b-2a24af6e64c9/GATK4SomaticCnvToolchainCaptureForSamples/83c80c50-d614-4b9a-bcff-8e1d5500378d/call-gatk4CNVproportionalCoverageForCapture/TCGA-3Z-A93Z-01A.pcov
2022/08/12 22:12:46 Delocalization script execution complete.
2022/08/12 22:12:47 Done delocalization.
I am also attaching two screenshots:
1) the end of the target input intervals - to show that the input intervals cover chromosomes 1-22 (first screenshot).
2) the end of the output target intervals from the proportional coverage calculation task - to show how it's blunted on chromosome 1 (second screenshot).
Lastly, as a result of this blunting, the next task in the workflow, gatk4CNVtangentNormalizationForCapture, fails and the entire workflow also fails.
I would appreciate any suggestion and/or help to understand what may be happening and how I can address this issue! Thank you so much in advance!
-
Hi Kevin Meli,
Thanks for reaching out. It looks like you've cloned a much older version of the GATK Somatic CNV workflow. The tools in the workflow have since been replaced, which is probably why you are seeing issues. The most up-to-date version of the Somatic CNV workflow is here: https://github.com/broadinstitute/gatk/tree/master/scripts/cnv_wdl/somatic.
If you are working on Terra, you can import the WDL to your workspace from Dockstore: https://dockstore.org/organizations/BroadInstitute/collections/GATKWorkflows.
Best,
Samantha
Please sign in to leave a comment.
1 comment