Funcotator data source not correct
AnsweredFuncotator is annotating all the variant regions as IGR, when that is not the case. How can I resolve this warning about the data source not being correct for the data? We are using the data sources folder listed here: "gs://broad-public-datasets/funcotator/funcotator_dataSources.v1.6.20190124s.tar.gz"
a) GATK version used: v 4.1.7.0
b) Exact command used:
We are running Funcotator on Terra, using the workflow from here:
https://dockstore.org/workflows/github.com/broadinstitute/gatk/mutect2:4.1.7.0
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx7500m -jar /root/gatk.jar Funcotator --data-sources-path /cromwell_root/datasources_dir --ref-version hg38 --output-file-format MAF -R gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/GRCh38.primary_assembly.genome.fa -V gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Filter/cacheCopy/RESPOND_60100256_T1_Malignant-filtered.vcf -O RESPOND_60100256_T1_Malignant-filtered.annotated.maf --annotation-default normal_barcode:KRESPONDX_0009_01_WBLCONTTD_R00001S8A1M0001P0000_C1_ASXE6S_A00033 --annotation-default tumor_barcode:KRESPONDX_0009_01_PROPRI1TD_R00001S8A1M0001P0000_T1_ASXE6S_A00009 --annotation-default Center:Unknown --annotation-default source:Unknown --remove-filtered-variants
c) Entire error log:
2021/03/29 19:24:48 Starting container setup. 2021/03/29 19:24:50 Done container setup. 2021/03/29 19:24:54 Starting localization. 2021/03/29 19:25:02 Localization script execution started... 2021/03/29 19:25:02 Localizing input gs://broad-public-datasets/funcotator/funcotator_dataSources.v1.6.20190124s.tar.gz -> /cromwell_root/broad-public-datasets/funcotator/funcotator_dataSources.v1.6.20190124s.tar.gz 2021/03/29 19:27:42 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Funcotate/script -> /cromwell_root/script 2021/03/29 19:27:43 Localization script execution complete. 2021/03/29 19:27:51 Done localization. 2021/03/29 19:27:52 Running user action: docker run -v /mnt/local-disk:/cromwell_root --entrypoint=/bin/bash broadinstitute/gatk@sha256:192fedf9b9d65809da4a2954030269e3f311d296e6b5e4c6c7deec12c7fe84b2 /cromwell_root/script Extracting data sources zip file... funcotator_dataSources.v1.6.20190124s/cosmic_fusion/ funcotator_dataSources.v1.6.20190124s/cosmic_fusion/hg38/ funcotator_dataSources.v1.6.20190124s/cosmic_fusion/hg38/cosmic_fusion.config funcotator_dataSources.v1.6.20190124s/cosmic_fusion/hg38/cosmic_fusion.tsv funcotator_dataSources.v1.6.20190124s/cosmic_fusion/hg19/ funcotator_dataSources.v1.6.20190124s/cosmic_fusion/hg19/cosmic_fusion.config funcotator_dataSources.v1.6.20190124s/cosmic_fusion/hg19/cosmic_fusion.tsv funcotator_dataSources.v1.6.20190124s/cosmic_fusion/metadata.txt funcotator_dataSources.v1.6.20190124s/gencode/ funcotator_dataSources.v1.6.20190124s/gencode/hg38/ funcotator_dataSources.v1.6.20190124s/gencode/hg38/gencode.v28.pc_transcripts.fa.fai funcotator_dataSources.v1.6.20190124s/gencode/hg38/gencode.v28.pc_transcripts.fa funcotator_dataSources.v1.6.20190124s/gencode/hg38/gencode.v28.pc_transcripts.dict funcotator_dataSources.v1.6.20190124s/gencode/hg38/gencode.v28.annotation.REORDERED.gtf.idx funcotator_dataSources.v1.6.20190124s/gencode/hg38/gencode.v28.annotation.REORDERED.gtf funcotator_dataSources.v1.6.20190124s/gencode/hg38/gencode.config funcotator_dataSources.v1.6.20190124s/gencode/hg19/ funcotator_dataSources.v1.6.20190124s/gencode/hg19/gencode.v19.annotation.REORDERED.gtf.idx funcotator_dataSources.v1.6.20190124s/gencode/hg19/gencode.v19.pc_transcripts.dict funcotator_dataSources.v1.6.20190124s/gencode/hg19/gencode.v19.pc_transcripts.fa funcotator_dataSources.v1.6.20190124s/gencode/hg19/gencode.v19.pc_transcripts.fa.fai funcotator_dataSources.v1.6.20190124s/gencode/hg19/gencode.v19.annotation.REORDERED.gtf funcotator_dataSources.v1.6.20190124s/gencode/hg19/gencode.config funcotator_dataSources.v1.6.20190124s/gencode_xrefseq/ funcotator_dataSources.v1.6.20190124s/gencode_xrefseq/hg38/ funcotator_dataSources.v1.6.20190124s/gencode_xrefseq/hg38/gencode_xrefseq_v90_38.tsv funcotator_dataSources.v1.6.20190124s/gencode_xrefseq/hg38/gencode_xrefseq.config funcotator_dataSources.v1.6.20190124s/gencode_xrefseq/hg19/ funcotator_dataSources.v1.6.20190124s/gencode_xrefseq/hg19/gencode_xrefseq_v75_37.tsv funcotator_dataSources.v1.6.20190124s/gencode_xrefseq/hg19/gencode_xrefseq.config funcotator_dataSources.v1.6.20190124s/clinvar/ funcotator_dataSources.v1.6.20190124s/clinvar/hg19/ funcotator_dataSources.v1.6.20190124s/clinvar/hg19/clinvar_hgmd.tsv funcotator_dataSources.v1.6.20190124s/clinvar/hg19/clinvar_hgmd.tsv.idx funcotator_dataSources.v1.6.20190124s/clinvar/hg19/clinvar_hgmd.config funcotator_dataSources.v1.6.20190124s/oreganno/ funcotator_dataSources.v1.6.20190124s/oreganno/hg38/ funcotator_dataSources.v1.6.20190124s/oreganno/hg38/oreganno.config funcotator_dataSources.v1.6.20190124s/oreganno/hg38/oreganno.tsv.idx funcotator_dataSources.v1.6.20190124s/oreganno/hg38/oreganno.tsv funcotator_dataSources.v1.6.20190124s/oreganno/hg19/ funcotator_dataSources.v1.6.20190124s/oreganno/hg19/oreganno.config funcotator_dataSources.v1.6.20190124s/oreganno/hg19/oreganno.tsv.idx funcotator_dataSources.v1.6.20190124s/oreganno/hg19/oreganno.tsv funcotator_dataSources.v1.6.20190124s/achilles/ funcotator_dataSources.v1.6.20190124s/achilles/hg38 funcotator_dataSources.v1.6.20190124s/achilles/hg19/ funcotator_dataSources.v1.6.20190124s/achilles/hg19/achilles.config funcotator_dataSources.v1.6.20190124s/achilles/hg19/achilles_lineage_results.import.txt funcotator_dataSources.v1.6.20190124s/cancer_gene_census/ funcotator_dataSources.v1.6.20190124s/cancer_gene_census/hg38 funcotator_dataSources.v1.6.20190124s/cancer_gene_census/hg19/ funcotator_dataSources.v1.6.20190124s/cancer_gene_census/hg19/CancerGeneCensus_Table_1_full_2012-03-15.txt funcotator_dataSources.v1.6.20190124s/cancer_gene_census/hg19/cancer_gene_census.config funcotator_dataSources.v1.6.20190124s/chr1_a_bed/ funcotator_dataSources.v1.6.20190124s/chr1_a_bed/hg38/ funcotator_dataSources.v1.6.20190124s/chr1_a_bed/hg38/chr1_a_bed.tsv funcotator_dataSources.v1.6.20190124s/chr1_a_bed/hg38/chr1_a_bed.tsv.idx funcotator_dataSources.v1.6.20190124s/chr1_a_bed/hg38/chr1_a_bed.config funcotator_dataSources.v1.6.20190124s/chr1_a_bed/hg19/ funcotator_dataSources.v1.6.20190124s/chr1_a_bed/hg19/chr1_a_bed.tsv funcotator_dataSources.v1.6.20190124s/chr1_a_bed/hg19/chr1_a_bed.tsv.idx funcotator_dataSources.v1.6.20190124s/chr1_a_bed/hg19/chr1_a_bed.config funcotator_dataSources.v1.6.20190124s/simple_uniprot/ funcotator_dataSources.v1.6.20190124s/simple_uniprot/hg38 funcotator_dataSources.v1.6.20190124s/simple_uniprot/hg19/ funcotator_dataSources.v1.6.20190124s/simple_uniprot/hg19/.simple_uniprot_Dec012014.tsv.swp funcotator_dataSources.v1.6.20190124s/simple_uniprot/hg19/simple_uniprot_Dec012014.tsv funcotator_dataSources.v1.6.20190124s/simple_uniprot/hg19/simple_uniprot.config funcotator_dataSources.v1.6.20190124s/.DS_Store funcotator_dataSources.v1.6.20190124s/dna_repair_genes/ funcotator_dataSources.v1.6.20190124s/dna_repair_genes/hg38 funcotator_dataSources.v1.6.20190124s/dna_repair_genes/hg19/ funcotator_dataSources.v1.6.20190124s/dna_repair_genes/hg19/dnaRepairGenes.config funcotator_dataSources.v1.6.20190124s/dna_repair_genes/hg19/dnaRepairGenes.20180524T145835.csv funcotator_dataSources.v1.6.20190124s/dbsnp/ funcotator_dataSources.v1.6.20190124s/dbsnp/hg38/ funcotator_dataSources.v1.6.20190124s/dbsnp/hg38/hg38_All_20170710.vcf.gz funcotator_dataSources.v1.6.20190124s/dbsnp/hg38/hg38_All_20170710.vcf.gz.tbi funcotator_dataSources.v1.6.20190124s/dbsnp/hg38/dbSNP.config funcotator_dataSources.v1.6.20190124s/dbsnp/hg19/ funcotator_dataSources.v1.6.20190124s/dbsnp/hg19/hg19_All_20170710.vcf.gz.tbi funcotator_dataSources.v1.6.20190124s/dbsnp/hg19/dbSNP.config funcotator_dataSources.v1.6.20190124s/dbsnp/hg19/hg19_All_20170710.vcf.gz funcotator_dataSources.v1.6.20190124s/chr1_b_bed/ funcotator_dataSources.v1.6.20190124s/chr1_b_bed/hg38/ funcotator_dataSources.v1.6.20190124s/chr1_b_bed/hg38/chr1_b_bed.config funcotator_dataSources.v1.6.20190124s/chr1_b_bed/hg38/chr1_b_bed.tsv funcotator_dataSources.v1.6.20190124s/chr1_b_bed/hg38/chr1_b_bed.tsv.idx funcotator_dataSources.v1.6.20190124s/chr1_b_bed/hg19/ funcotator_dataSources.v1.6.20190124s/chr1_b_bed/hg19/chr1_b_bed.config funcotator_dataSources.v1.6.20190124s/chr1_b_bed/hg19/chr1_b_bed.tsv funcotator_dataSources.v1.6.20190124s/chr1_b_bed/hg19/chr1_b_bed.tsv.idx funcotator_dataSources.v1.6.20190124s/template.config funcotator_dataSources.v1.6.20190124s/README.txt funcotator_dataSources.v1.6.20190124s/hgnc/ funcotator_dataSources.v1.6.20190124s/hgnc/hg38 funcotator_dataSources.v1.6.20190124s/hgnc/hg19/ funcotator_dataSources.v1.6.20190124s/hgnc/hg19/hgnc.config funcotator_dataSources.v1.6.20190124s/hgnc/hg19/hgnc_download_Nov302017.tsv funcotator_dataSources.v1.6.20190124s/cosmic_tissue/ funcotator_dataSources.v1.6.20190124s/cosmic_tissue/hg38/ funcotator_dataSources.v1.6.20190124s/cosmic_tissue/hg38/cosmic_tissue.tsv funcotator_dataSources.v1.6.20190124s/cosmic_tissue/hg38/cosmic_tissue.config funcotator_dataSources.v1.6.20190124s/cosmic_tissue/hg19/ funcotator_dataSources.v1.6.20190124s/cosmic_tissue/hg19/cosmic_tissue.tsv funcotator_dataSources.v1.6.20190124s/cosmic_tissue/hg19/cosmic_tissue.config funcotator_dataSources.v1.6.20190124s/familial/ funcotator_dataSources.v1.6.20190124s/familial/hg38 funcotator_dataSources.v1.6.20190124s/familial/hg19/ funcotator_dataSources.v1.6.20190124s/familial/hg19/familial.config funcotator_dataSources.v1.6.20190124s/familial/hg19/Familial_Cancer_Genes.no_dupes.tsv funcotator_dataSources.v1.6.20190124s/.idea/ funcotator_dataSources.v1.6.20190124s/.idea/funcotator_dataSources.iml funcotator_dataSources.v1.6.20190124s/.idea/misc.xml funcotator_dataSources.v1.6.20190124s/.idea/workspace.xml funcotator_dataSources.v1.6.20190124s/.idea/modules.xml funcotator_dataSources.v1.6.20190124s/MANIFEST.txt funcotator_dataSources.v1.6.20190124s/cosmic/ funcotator_dataSources.v1.6.20190124s/cosmic/hg38/ funcotator_dataSources.v1.6.20190124s/cosmic/hg38/Cosmic.db funcotator_dataSources.v1.6.20190124s/cosmic/hg38/cosmic.config funcotator_dataSources.v1.6.20190124s/cosmic/hg19/ funcotator_dataSources.v1.6.20190124s/cosmic/hg19/Cosmic.db funcotator_dataSources.v1.6.20190124s/cosmic/hg19/cosmic.config funcotator_dataSources.v1.6.20190124s/cosmic/metadata.txt funcotator_dataSources.v1.6.20190124s/gnomAD_exome.tar.gz funcotator_dataSources.v1.6.20190124s/gnomAD_genome.tar.gz funcotator_dataSources.v1.6.20190124s/gencode_xhgnc/ funcotator_dataSources.v1.6.20190124s/gencode_xhgnc/hg38/ funcotator_dataSources.v1.6.20190124s/gencode_xhgnc/hg38/gencode_xhgnc_v90_38.hg38.tsv funcotator_dataSources.v1.6.20190124s/gencode_xhgnc/hg38/gencode_xhgnc.config funcotator_dataSources.v1.6.20190124s/gencode_xhgnc/hg19/ funcotator_dataSources.v1.6.20190124s/gencode_xhgnc/hg19/gencode_xhgnc_v75_37.hg19.tsv funcotator_dataSources.v1.6.20190124s/gencode_xhgnc/hg19/gencode_xhgnc.config Enabling gnomAD... gnomAD_exome/ gnomAD_exome/hg38/ gnomAD_exome/hg38/gnomAD_exome.config gnomAD_exome/hg19/ gnomAD_exome/hg19/gnomAD_exome.config /cromwell_root gnomAD_genome/ gnomAD_genome/hg38/ gnomAD_genome/hg38/gnomAD_genome.config gnomAD_genome/hg19/ gnomAD_genome/hg19/gnomAD_genome.config /cromwell_root Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.68b540e3 19:58:48.196 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.1.7.0-local.jar!/com/intel/gkl/native/libgkl_compression.so 19:58:49.653 INFO Funcotator - ------------------------------------------------------------ 19:58:49.654 INFO Funcotator - The Genome Analysis Toolkit (GATK) v4.1.7.0 19:58:49.655 INFO Funcotator - For support and documentation go to https://software.broadinstitute.org/gatk/ 19:58:49.655 INFO Funcotator - Executing as root@3af3b90ba22a on Linux v5.4.89+ amd64 19:58:49.656 INFO Funcotator - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_212-8u212-b03-0ubuntu1.16.04.1-b03 19:58:49.656 INFO Funcotator - Start Date/Time: March 29, 2021 7:58:47 PM UTC 19:58:49.657 INFO Funcotator - ------------------------------------------------------------ 19:58:49.657 INFO Funcotator - ------------------------------------------------------------ 19:58:49.658 INFO Funcotator - HTSJDK Version: 2.21.2 19:58:49.658 INFO Funcotator - Picard Version: 2.21.9 19:58:49.658 INFO Funcotator - HTSJDK Defaults.COMPRESSION_LEVEL : 2 19:58:49.658 INFO Funcotator - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false 19:58:49.658 INFO Funcotator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true 19:58:49.658 INFO Funcotator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false 19:58:49.658 INFO Funcotator - Deflater: IntelDeflater 19:58:49.658 INFO Funcotator - Inflater: IntelInflater 19:58:49.659 INFO Funcotator - GCS max retries/reopens: 20 19:58:49.659 INFO Funcotator - Requester pays: disabled 19:58:49.659 INFO Funcotator - Initializing engine 19:58:53.373 INFO FeatureManager - Using codec VCFCodec to read file gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Filter/cacheCopy/RESPOND_60100256_T1_Malignant-filtered.vcf 19:58:59.364 INFO Funcotator - Done initializing engine 19:58:59.367 INFO Funcotator - Validating Sequence Dictionaries... 19:58:59.371 INFO Funcotator - Processing user transcripts/defaults/overrides... 19:58:59.372 INFO Funcotator - Initializing data sources... 19:58:59.377 INFO DataSourceUtils - Initializing data sources from directory: /cromwell_root/datasources_dir 19:58:59.457 INFO DataSourceUtils - Data sources version: 1.6.2019124s 19:58:59.457 INFO DataSourceUtils - Data sources source: ftp://gsapubftp-anonymous@ftp.broadinstitute.org/bundle/funcotator/funcotator_dataSources.v1.6.20190124s.tar.gz 19:58:59.458 INFO DataSourceUtils - Data sources alternate source: gs://broad-public-datasets/funcotator/funcotator_dataSources.v1.6.20190124s.tar.gz 19:58:59.627 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/gencode.v28.annotation.REORDERED.gtf -> file:///cromwell_root/datasources_dir/gencode/hg38/gencode.v28.annotation.REORDERED.gtf 19:58:59.628 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/gencode.v28.pc_transcripts.fa -> file:///cromwell_root/datasources_dir/gencode/hg38/gencode.v28.pc_transcripts.fa 19:58:59.658 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/achilles_lineage_results.import.txt -> file:///cromwell_root/datasources_dir/achilles/hg38/achilles_lineage_results.import.txt 19:58:59.660 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/chr1_b_bed.tsv -> file:///cromwell_root/datasources_dir/chr1_b_bed/hg38/chr1_b_bed.tsv 19:58:59.670 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/simple_uniprot_Dec012014.tsv -> file:///cromwell_root/datasources_dir/simple_uniprot/hg38/simple_uniprot_Dec012014.tsv 19:58:59.718 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/cosmic_fusion.tsv -> file:///cromwell_root/datasources_dir/cosmic_fusion/hg38/cosmic_fusion.tsv 19:58:59.719 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/Familial_Cancer_Genes.no_dupes.tsv -> file:///cromwell_root/datasources_dir/familial/hg38/Familial_Cancer_Genes.no_dupes.tsv 19:58:59.747 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/hg38_All_20170710.vcf.gz -> file:///cromwell_root/datasources_dir/dbsnp/hg38/hg38_All_20170710.vcf.gz 19:58:59.749 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/chr1_a_bed.tsv -> file:///cromwell_root/datasources_dir/chr1_a_bed/hg38/chr1_a_bed.tsv 19:58:59.760 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/gencode_xrefseq_v90_38.tsv -> file:///cromwell_root/datasources_dir/gencode_xrefseq/hg38/gencode_xrefseq_v90_38.tsv 19:58:59.761 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/Cosmic.db -> file:///cromwell_root/datasources_dir/cosmic/hg38/Cosmic.db 19:58:59.802 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/oreganno.tsv -> file:///cromwell_root/datasources_dir/oreganno/hg38/oreganno.tsv 19:58:59.803 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/gencode_xhgnc_v90_38.hg38.tsv -> file:///cromwell_root/datasources_dir/gencode_xhgnc/hg38/gencode_xhgnc_v90_38.hg38.tsv 19:58:59.807 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/dnaRepairGenes.20180524T145835.csv -> file:///cromwell_root/datasources_dir/dna_repair_genes/hg38/dnaRepairGenes.20180524T145835.csv 19:58:59.808 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/cosmic_tissue.tsv -> file:///cromwell_root/datasources_dir/cosmic_tissue/hg38/cosmic_tissue.tsv 19:59:00.039 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/CancerGeneCensus_Table_1_full_2012-03-15.txt -> file:///cromwell_root/datasources_dir/cancer_gene_census/hg38/CancerGeneCensus_Table_1_full_2012-03-15.txt 19:59:00.048 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/hgnc_download_Nov302017.tsv -> file:///cromwell_root/datasources_dir/hgnc/hg38/hgnc_download_Nov302017.tsv 19:59:00.049 INFO Funcotator - Finalizing data sources (this step can be long if data sources are cloud-based)... 19:59:00.050 INFO DataSourceUtils - Setting lookahead cache for data source: gnomAD_exome : 100000 19:59:01.283 INFO FeatureManager - Using codec VCFCodec to read file gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.exomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz 19:59:06.219 INFO FeatureManager - Using codec VCFCodec to read file gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.exomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz 19:59:06.806 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/gencode.v28.annotation.REORDERED.gtf -> file:///cromwell_root/datasources_dir/gencode/hg38/gencode.v28.annotation.REORDERED.gtf 19:59:06.806 INFO DataSourceUtils - Setting lookahead cache for data source: Gencode : 100000 19:59:06.822 INFO FeatureManager - Using codec GencodeGtfCodec to read file file:///cromwell_root/datasources_dir/gencode/hg38/gencode.v28.annotation.REORDERED.gtf 19:59:06.940 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/gencode.v28.pc_transcripts.fa -> file:///cromwell_root/datasources_dir/gencode/hg38/gencode.v28.pc_transcripts.fa 19:59:16.991 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/achilles_lineage_results.import.txt -> file:///cromwell_root/datasources_dir/achilles/hg38/achilles_lineage_results.import.txt 19:59:17.045 INFO DataSourceUtils - Setting lookahead cache for data source: chr1_b_bed : 100000 19:59:17.053 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/chr1_b_bed.tsv -> file:///cromwell_root/datasources_dir/chr1_b_bed/hg38/chr1_b_bed.tsv 19:59:17.054 INFO FeatureManager - Using codec XsvLocatableTableCodec to read file file:///cromwell_root/datasources_dir/chr1_b_bed/hg38/chr1_b_bed.config 19:59:17.059 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/chr1_b_bed.tsv -> file:///cromwell_root/datasources_dir/chr1_b_bed/hg38/chr1_b_bed.tsv 19:59:17.060 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/chr1_b_bed.tsv -> file:///cromwell_root/datasources_dir/chr1_b_bed/hg38/chr1_b_bed.tsv WARNING 2021-03-29 19:59:17 AsciiLineReader Creating an indexable source for an AsciiFeatureCodec using a stream that is neither a PositionalBufferedStream nor a BlockCompressedInputStream 19:59:17.066 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/simple_uniprot_Dec012014.tsv -> file:///cromwell_root/datasources_dir/simple_uniprot/hg38/simple_uniprot_Dec012014.tsv 19:59:17.349 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/cosmic_fusion.tsv -> file:///cromwell_root/datasources_dir/cosmic_fusion/hg38/cosmic_fusion.tsv 19:59:17.400 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/Familial_Cancer_Genes.no_dupes.tsv -> file:///cromwell_root/datasources_dir/familial/hg38/Familial_Cancer_Genes.no_dupes.tsv 19:59:17.401 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/hg38_All_20170710.vcf.gz -> file:///cromwell_root/datasources_dir/dbsnp/hg38/hg38_All_20170710.vcf.gz 19:59:17.401 INFO DataSourceUtils - Setting lookahead cache for data source: dbSNP : 100000 19:59:17.409 INFO FeatureManager - Using codec VCFCodec to read file file:///cromwell_root/datasources_dir/dbsnp/hg38/hg38_All_20170710.vcf.gz 19:59:17.508 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/hg38_All_20170710.vcf.gz -> file:///cromwell_root/datasources_dir/dbsnp/hg38/hg38_All_20170710.vcf.gz 19:59:17.564 INFO FeatureManager - Using codec VCFCodec to read file file:///cromwell_root/datasources_dir/dbsnp/hg38/hg38_All_20170710.vcf.gz 19:59:17.619 INFO DataSourceUtils - Setting lookahead cache for data source: chr1_a_bed : 100000 19:59:17.622 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/chr1_a_bed.tsv -> file:///cromwell_root/datasources_dir/chr1_a_bed/hg38/chr1_a_bed.tsv 19:59:17.660 INFO FeatureManager - Using codec XsvLocatableTableCodec to read file file:///cromwell_root/datasources_dir/chr1_a_bed/hg38/chr1_a_bed.config 19:59:17.663 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/chr1_a_bed.tsv -> file:///cromwell_root/datasources_dir/chr1_a_bed/hg38/chr1_a_bed.tsv 19:59:17.667 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/chr1_a_bed.tsv -> file:///cromwell_root/datasources_dir/chr1_a_bed/hg38/chr1_a_bed.tsv WARNING 2021-03-29 19:59:17 AsciiLineReader Creating an indexable source for an AsciiFeatureCodec using a stream that is neither a PositionalBufferedStream nor a BlockCompressedInputStream 19:59:17.670 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/gencode_xrefseq_v90_38.tsv -> file:///cromwell_root/datasources_dir/gencode_xrefseq/hg38/gencode_xrefseq_v90_38.tsv 19:59:18.000 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/Cosmic.db -> file:///cromwell_root/datasources_dir/cosmic/hg38/Cosmic.db 19:59:18.441 INFO DataSourceUtils - Setting lookahead cache for data source: Oreganno : 100000 19:59:18.446 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/oreganno.tsv -> file:///cromwell_root/datasources_dir/oreganno/hg38/oreganno.tsv 19:59:18.454 INFO FeatureManager - Using codec XsvLocatableTableCodec to read file file:///cromwell_root/datasources_dir/oreganno/hg38/oreganno.config 19:59:18.684 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/oreganno.tsv -> file:///cromwell_root/datasources_dir/oreganno/hg38/oreganno.tsv 19:59:18.685 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/oreganno.tsv -> file:///cromwell_root/datasources_dir/oreganno/hg38/oreganno.tsv WARNING 2021-03-29 19:59:18 AsciiLineReader Creating an indexable source for an AsciiFeatureCodec using a stream that is neither a PositionalBufferedStream nor a BlockCompressedInputStream 19:59:18.687 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/gencode_xhgnc_v90_38.hg38.tsv -> file:///cromwell_root/datasources_dir/gencode_xhgnc/hg38/gencode_xhgnc_v90_38.hg38.tsv 19:59:22.825 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/dnaRepairGenes.20180524T145835.csv -> file:///cromwell_root/datasources_dir/dna_repair_genes/hg38/dnaRepairGenes.20180524T145835.csv 19:59:22.830 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/cosmic_tissue.tsv -> file:///cromwell_root/datasources_dir/cosmic_tissue/hg38/cosmic_tissue.tsv 19:59:22.858 INFO DataSourceUtils - Setting lookahead cache for data source: gnomAD_genome : 100000 19:59:24.356 INFO FeatureManager - Using codec VCFCodec to read file gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.genomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz 19:59:29.240 INFO FeatureManager - Using codec VCFCodec to read file gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.genomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz 19:59:29.842 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/CancerGeneCensus_Table_1_full_2012-03-15.txt -> file:///cromwell_root/datasources_dir/cancer_gene_census/hg38/CancerGeneCensus_Table_1_full_2012-03-15.txt 19:59:29.885 INFO DataSourceUtils - Resolved data source file path: file:///cromwell_root/hgnc_download_Nov302017.tsv -> file:///cromwell_root/datasources_dir/hgnc/hg38/hgnc_download_Nov302017.tsv 19:59:30.167 INFO Funcotator - Initializing Funcotator Engine... 19:59:30.170 INFO Funcotator - Creating a MAF file for output: file:/cromwell_root/RESPOND_60100256_T1_Malignant-filtered.annotated.maf 19:59:30.266 INFO ProgressMeter - Starting traversal 19:59:30.267 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute 19:59:54.621 INFO ProgressMeter - unmapped 0.4 0 0.0 19:59:54.622 INFO ProgressMeter - Traversal complete. Processed 0 total variants in 0.4 minutes. 19:59:54.622 WARN Funcotator - ================================================================================ 19:59:54.622 WARN Funcotator - [43m _ _ _ __ __ _ _ _ _ 19:59:54.622 WARN Funcotator - | || || | \ \ / /_ _ _ __ _ __ (_)_ __ __ _ | || || | 19:59:54.622 WARN Funcotator - | || || | \ \ /\ / / _` | '__| '_ \| | '_ \ / _` | | || || | 19:59:54.623 WARN Funcotator - |_||_||_| \ \V V / (_| | | | | | | | | | | (_| | |_||_||_| 19:59:54.623 WARN Funcotator - (_)(_)(_) \_/\_/ \__,_|_| |_| |_|_|_| |_|\__, | (_)(_)(_) 19:59:54.623 WARN Funcotator - |___/ [0;0m 19:59:54.623 WARN Funcotator - -------------------------------------------------------------------------------- 19:59:54.623 WARN Funcotator - Only IGRs were produced for this dataset. This STRONGLY indicates that this 19:59:54.623 WARN Funcotator - run was misconfigured. 19:59:54.624 WARN Funcotator - You MUST check your data sources to make sure they are correct for these data. 19:59:54.624 WARN Funcotator - ================================================================================ 19:59:54.624 INFO VcfFuncotationFactory - dbSNP 9606_b150 cache hits/total: 0/0 19:59:54.625 INFO VcfFuncotationFactory - gnomAD_exome 2.1 cache hits/total: 0/0 19:59:54.625 INFO VcfFuncotationFactory - gnomAD_genome 2.1 cache hits/total: 0/0 19:59:54.701 INFO Funcotator - Shutting down engine [March 29, 2021 7:59:54 PM UTC] org.broadinstitute.hellbender.tools.funcotator.Funcotator done. Elapsed time: 1.11 minutes. Runtime.totalMemory()=2595749888 Tool returned: true Using GATK jar /root/gatk.jar defined in environment variable GATK_LOCAL_JAR Running: java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx7500m -jar /root/gatk.jar Funcotator --data-sources-path /cromwell_root/datasources_dir --ref-version hg38 --output-file-format MAF -R gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/GRCh38.primary_assembly.genome.fa -V gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Filter/cacheCopy/RESPOND_60100256_T1_Malignant-filtered.vcf -O RESPOND_60100256_T1_Malignant-filtered.annotated.maf --annotation-default normal_barcode:KRESPONDX_0009_01_WBLCONTTD_R00001S8A1M0001P0000_C1_ASXE6S_A00033 --annotation-default tumor_barcode:KRESPONDX_0009_01_PROPRI1TD_R00001S8A1M0001P0000_T1_ASXE6S_A00009 --annotation-default Center:Unknown --annotation-default source:Unknown --remove-filtered-variants 2021/03/29 19:59:59 Starting delocalization. 2021/03/29 20:00:00 Delocalization script execution started... 2021/03/29 20:00:00 Delocalizing output /cromwell_root/memory_retry_rc -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Funcotate/memory_retry_rc 2021/03/29 20:00:00 Delocalizing output /cromwell_root/rc -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Funcotate/rc 2021/03/29 20:00:02 Delocalizing output /cromwell_root/stdout -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Funcotate/stdout 2021/03/29 20:00:03 Delocalizing output /cromwell_root/stderr -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Funcotate/stderr 2021/03/29 20:00:05 Delocalizing output /cromwell_root/RESPOND_60100256_T1_Malignant-filtered.annotated.maf -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Funcotate/RESPOND_60100256_T1_Malignant-filtered.annotated.maf 2021/03/29 20:00:06 Delocalizing output /cromwell_root/RESPOND_60100256_T1_Malignant-filtered.annotated.maf.idx -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/59bd808d-ee06-4114-a560-ff93f1db28cf/Mutect2/304e09df-9483-408a-a62b-4605c5b63734/call-Funcotate/RESPOND_60100256_T1_Malignant-filtered.annotated.maf.idx 2021/03/29 20:00:08 Delocalization script execution complete. 2021/03/29 20:00:09 Done delocalization.
-
Hi Jamie Xie, there was an update to the Funcotator data sources in GATK 4.1.9.0 and 4.2.0.0, you can find those release notes here: https://github.com/broadinstitute/gatk/releases.
Could you try updating to 4.2.0.0 and see if this issue is resolved?
-
Hi Genevieve,
Thanks for the response! I can't seem to find version 4.2.0.0 of the funcotator data sources or the workflow on Terra. The workflow listed here is still 4.1.7.0 and https://app.terra.bio/#workspaces/help-gatk/Variant-Functional-Annotation-With-Funcotator/workflows and this bucket gs://broad-public-datasets/funcotator only contains up to 4.1.7.0 as well.
Where can I find the latest versions of these?
Thanks,
Jamie
-
Jamie,
The version 4.2.0.0 is the GATK version, not the Funcotator data sources version, sorry about that!
Genevieve
-
Hi Genevieve,
I tried re-running Funcotator workflow (version 4.1.7.0) with gatk docker version 4.2.0.0 but unfortunately I got the same warning. Is there something else I might be doing incorrectly?
Thanks,
Jamie
-
Hi Jamie,
Thank you for checking that. Let's see if there is a different issue going on. I noticed that Funcotator did not process any variants:
19:59:54.622 INFO ProgressMeter - Traversal complete. Processed 0 total variants in 0.4 minutes.
Do you know why this might be the case? How many variants are in your input file?
Best,
Genevieve
-
Hi Genevieve,
Yeah I don't know why it said it processed 0 variants. When running this in conjunction with Mutect2 (using the workflow I posted above), FilterMutectCalls processed around 4 million variants. The vcf files from Mutect2 are not empty.
13:36:31.507 INFO ProgressMeter - Traversal complete. Processed 4532912 total variants in 12.6 minutes.
So even though there are clearly a lot of variants, the MAF output from Funcotator is empty even though the run was successful.
Thanks,
Jamie
-
Hi Jamie,
Funcotator is usually run after FilterMutectCalls, could you check the output from FilterMutectCalls and verify there are variants in that file?
Genevieve
-
Hi Genevieve,
Yes, in the Mutect2 workflow I'm referring to (https://dockstore.org/workflows/github.com/broadinstitute/gatk/mutect2:4.1.7.0) Funcotator runs after Mutect2 and FilterMutectCalls finishes. The filtered output from FilterMutectCalls does contain variants (it's a large file). I took a screenshot of the file size and the FilterMutectCalls log file.
2021/03/26 13:23:18 Starting container setup. 2021/03/26 13:23:20 Done container setup. 2021/03/26 13:23:24 Starting localization. 2021/03/26 13:23:31 Localization script execution started... 2021/03/26 13:23:31 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeVCFs/RESPOND_60100256_T1_Malignant-unfiltered.vcf -> /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeVCFs/RESPOND_60100256_T1_Malignant-unfiltered.vcf 2021/03/26 13:23:36 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeVCFs/RESPOND_60100256_T1_Malignant-unfiltered.vcf.idx -> /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeVCFs/RESPOND_60100256_T1_Malignant-unfiltered.vcf.idx Copying gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeVCFs/RESPOND_60100256_T1_Malignant-unfiltered.vcf.idx... / [0/1 files][ 0.0 B/ 3.0 MiB] 0% Done / [1/1 files][ 3.0 MiB/ 3.0 MiB] 100% Done Operation completed over 1 objects/3.0 MiB. 2021/03/26 13:23:37 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-Filter/script -> /cromwell_root/script 2021/03/26 13:23:39 Localizing input gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeStats/merged.stats -> /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeStats/merged.stats 2021/03/26 13:23:41 Localization script execution complete. 2021/03/26 13:23:46 Done localization. 2021/03/26 13:23:47 Running user action: docker run -v /mnt/local-disk:/cromwell_root --entrypoint=/bin/bash broadinstitute/gatk@sha256:192fedf9b9d65809da4a2954030269e3f311d296e6b5e4c6c7deec12c7fe84b2 /cromwell_root/script Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.98cb9fe5 13:23:52.091 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.1.7.0-local.jar!/com/intel/gkl/native/libgkl_compression.so 13:23:52.377 INFO FilterMutectCalls - ------------------------------------------------------------ 13:23:52.378 INFO FilterMutectCalls - The Genome Analysis Toolkit (GATK) v4.1.7.0 13:23:52.379 INFO FilterMutectCalls - For support and documentation go to https://software.broadinstitute.org/gatk/ 13:23:52.380 INFO FilterMutectCalls - Executing as root@d0dd8d871fcb on Linux v5.4.89+ amd64 13:23:52.380 INFO FilterMutectCalls - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_212-8u212-b03-0ubuntu1.16.04.1-b03 13:23:52.381 INFO FilterMutectCalls - Start Date/Time: March 26, 2021 1:23:52 PM UTC 13:23:52.381 INFO FilterMutectCalls - ------------------------------------------------------------ 13:23:52.381 INFO FilterMutectCalls - ------------------------------------------------------------ 13:23:52.383 INFO FilterMutectCalls - HTSJDK Version: 2.21.2 13:23:52.383 INFO FilterMutectCalls - Picard Version: 2.21.9 13:23:52.384 INFO FilterMutectCalls - HTSJDK Defaults.COMPRESSION_LEVEL : 2 13:23:52.384 INFO FilterMutectCalls - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false 13:23:52.385 INFO FilterMutectCalls - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true 13:23:52.385 INFO FilterMutectCalls - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false 13:23:52.386 INFO FilterMutectCalls - Deflater: IntelDeflater 13:23:52.386 INFO FilterMutectCalls - Inflater: IntelInflater 13:23:52.386 INFO FilterMutectCalls - GCS max retries/reopens: 20 13:23:52.386 INFO FilterMutectCalls - Requester pays: disabled 13:23:52.387 INFO FilterMutectCalls - Initializing engine 13:23:54.186 INFO FeatureManager - Using codec VCFCodec to read file file:///cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeVCFs/RESPOND_60100256_T1_Malignant-unfiltered.vcf 13:23:54.318 INFO FilterMutectCalls - Done initializing engine 13:23:54.436 INFO ProgressMeter - Starting traversal 13:23:54.436 INFO ProgressMeter - Current Locus Elapsed Minutes Variants Processed Variants/Minute 13:23:54.438 INFO FilterMutectCalls - Starting pass 0 through the variants 13:24:04.449 INFO ProgressMeter - chr1:42986564 0.2 34000 203775.8 13:24:14.536 INFO ProgressMeter - chr1:245222266 0.3 101000 301507.5 13:24:24.540 INFO ProgressMeter - chr2:197693615 0.5 167000 332857.2 13:24:34.541 INFO ProgressMeter - chr3:136061521 0.7 230000 344105.3 13:24:44.567 INFO ProgressMeter - chr4:129088897 0.8 288000 344703.8 13:24:54.601 INFO ProgressMeter - chr5:176891354 1.0 356000 355029.6 13:25:04.660 INFO ProgressMeter - chr6:169434913 1.2 418000 357153.0 13:25:14.800 INFO ProgressMeter - chr8:244125 1.3 486000 362853.6 13:25:24.833 INFO ProgressMeter - chr9:89334980 1.5 554000 367719.5 13:25:34.835 INFO ProgressMeter - chr10:94027559 1.7 616000 368134.8 13:25:44.924 INFO ProgressMeter - chr11:114909723 1.8 677000 367648.4 13:25:54.959 INFO ProgressMeter - chr12:116395048 2.0 733000 364912.6 13:26:05.101 INFO ProgressMeter - chr14:90836185 2.2 800000 367354.4 13:26:15.194 INFO ProgressMeter - chr16:24786480 2.3 865000 368720.6 13:26:25.339 INFO ProgressMeter - chr17:47754984 2.5 934000 371366.8 13:26:35.460 INFO ProgressMeter - chr19:13815219 2.7 1003000 373735.4 13:26:45.566 INFO ProgressMeter - chr20:46583790 2.9 1068000 374454.4 13:26:56.955 INFO ProgressMeter - chrX:154352329 3.0 1128000 370812.7 13:26:57.715 INFO FilterMutectCalls - Finished pass 0 through the variants 13:26:58.029 INFO FilterMutectCalls - Starting pass 1 through the variants 13:27:06.960 INFO ProgressMeter - chr1:112422914 3.2 1190000 370864.8 13:27:16.962 INFO ProgressMeter - chr2:40865689 3.4 1251000 370620.9 13:27:26.964 INFO ProgressMeter - chr2:233846383 3.5 1315000 371248.7 13:27:36.994 INFO ProgressMeter - chr3:193608779 3.7 1381000 372309.1 13:27:47.104 INFO ProgressMeter - chr5:31564663 3.9 1444000 372377.7 13:27:57.104 INFO ProgressMeter - chr6:20171837 4.0 1501000 371125.9 13:28:07.217 INFO ProgressMeter - chr7:42288088 4.2 1568000 372182.8 13:28:17.358 INFO ProgressMeter - chr8:34388828 4.4 1631000 372203.1 13:28:27.362 INFO ProgressMeter - chr9:114321725 4.5 1698000 373289.4 13:28:37.472 INFO ProgressMeter - chr10:129746007 4.7 1765000 374160.0 13:28:47.605 INFO ProgressMeter - chr12:26882636 4.9 1829000 374325.9 13:28:57.695 INFO ProgressMeter - chr13:32368867 5.1 1886000 373147.6 13:29:07.724 INFO ProgressMeter - chr15:43334929 5.2 1952000 373842.5 13:29:17.750 INFO ProgressMeter - chr16:68616162 5.4 2015000 373941.0 13:29:27.821 INFO ProgressMeter - chr17:69384288 5.6 2081000 374523.1 13:29:37.833 INFO ProgressMeter - chr19:23087299 5.7 2148000 375310.1 13:29:47.913 INFO ProgressMeter - chr21:10601785 5.9 2212000 375471.0 13:29:57.982 INFO ProgressMeter - chrX:71902901 6.1 2257000 372498.6 13:29:59.421 INFO FilterMutectCalls - Finished pass 1 through the variants 13:29:59.550 INFO FilterMutectCalls - Starting pass 2 through the variants 13:30:08.039 INFO ProgressMeter - chr1:111548156 6.2 2323000 373070.8 13:30:18.058 INFO ProgressMeter - chr2:55726342 6.4 2391000 373962.8 13:30:28.080 INFO ProgressMeter - chr3:11521735 6.6 2455000 374196.9 13:30:38.112 INFO ProgressMeter - chr4:9721881 6.7 2523000 375005.6 13:30:48.157 INFO ProgressMeter - chr5:76922639 6.9 2591000 375761.4 13:30:58.224 INFO ProgressMeter - chr6:42852460 7.1 2647000 374763.7 13:31:08.251 INFO ProgressMeter - chr7:71819459 7.2 2714000 375368.2 13:31:18.361 INFO ProgressMeter - chr8:87223541 7.4 2781000 375875.1 13:31:28.391 INFO ProgressMeter - chr9:135675692 7.6 2844000 375897.9 13:31:38.472 INFO ProgressMeter - chr11:35850732 7.7 2912000 376523.3 13:31:48.584 INFO ProgressMeter - chr12:65921536 7.9 2980000 377098.2 13:31:58.668 INFO ProgressMeter - chr14:31090188 8.1 3044000 377175.4 13:32:08.778 INFO ProgressMeter - chr15:86908424 8.2 3106000 376986.7 13:32:18.798 INFO ProgressMeter - chr17:12787192 8.4 3175000 377705.7 13:32:28.872 INFO ProgressMeter - chr18:53124794 8.6 3241000 378006.9 13:32:38.961 INFO ProgressMeter - chr19:55293165 8.7 3311000 378744.1 13:32:48.968 INFO ProgressMeter - chr22:41984997 8.9 3380000 379398.7 13:32:51.849 INFO FilterMutectCalls - Finished pass 2 through the variants 13:32:51.891 INFO FilterMutectCalls - Starting pass 3 through the variants 13:32:59.173 INFO ProgressMeter - chr1:23328688 9.1 3418000 376475.9 13:33:09.220 INFO ProgressMeter - chr1:150571067 9.2 3462000 374416.7 13:33:19.311 INFO ProgressMeter - chr2:39023194 9.4 3517000 373570.0 13:33:29.565 INFO ProgressMeter - chr2:206978301 9.6 3571000 372543.8 13:33:39.714 INFO ProgressMeter - chr3:94994352 9.8 3617000 370798.8 13:33:49.723 INFO ProgressMeter - chr4:75557406 9.9 3673000 370208.6 13:33:59.754 INFO ProgressMeter - chr5:98918337 10.1 3730000 369723.6 13:34:09.885 INFO ProgressMeter - chr6:36573890 10.3 3777000 368219.6 13:34:20.036 INFO ProgressMeter - chr7:43800161 10.4 3835000 367807.5 13:34:30.209 INFO ProgressMeter - chr8:23757197 10.6 3893000 367395.9 13:34:40.270 INFO ProgressMeter - chr9:71408835 10.8 3947000 366689.2 13:34:50.293 INFO ProgressMeter - chr10:62241897 10.9 4002000 366117.0 13:35:00.295 INFO ProgressMeter - chr11:65020634 11.1 4057000 365573.4 13:35:10.381 INFO ProgressMeter - chr12:48698210 11.3 4102000 364113.0 13:35:20.475 INFO ProgressMeter - chr13:49721590 11.4 4159000 363740.8 13:35:30.606 INFO ProgressMeter - chr15:39780250 11.6 4215000 363273.9 13:35:40.738 INFO ProgressMeter - chr16:28479566 11.8 4267000 362480.0 13:35:50.782 INFO ProgressMeter - chr17:36582310 11.9 4322000 362004.8 13:36:00.808 INFO ProgressMeter - chr18:62347714 12.1 4377000 361550.8 13:36:10.925 INFO ProgressMeter - chr19:36554365 12.3 4421000 360168.8 13:36:21.035 INFO ProgressMeter - chr20:62852996 12.4 4476000 359712.1 13:36:31.086 INFO ProgressMeter - KI270442.1:239384 12.6 4531000 359294.7 13:36:31.433 INFO FilterMutectCalls - Finished pass 3 through the variants 13:36:31.506 INFO FilterMutectCalls - No variants filtered by: AllowAllVariantsVariantFilter 13:36:31.507 INFO FilterMutectCalls - 0 read(s) filtered by: AllowAllReadsReadFilter 13:36:31.507 INFO ProgressMeter - KI270728.1:1585634 12.6 4532912 359246.5 13:36:31.507 INFO ProgressMeter - Traversal complete. Processed 4532912 total variants in 12.6 minutes. 13:36:31.724 INFO FilterMutectCalls - Shutting down engine [March 26, 2021 1:36:31 PM UTC] org.broadinstitute.hellbender.tools.walkers.mutect.filtering.FilterMutectCalls done. Elapsed time: 12.66 minutes. Runtime.totalMemory()=3106406400 Using GATK jar /root/gatk.jar defined in environment variable GATK_LOCAL_JAR Running: java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xmx7500m -jar /root/gatk.jar FilterMutectCalls -V /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeVCFs/RESPOND_60100256_T1_Malignant-unfiltered.vcf -R gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/GRCh38.primary_assembly.genome.fa -O RESPOND_60100256_T1_Malignant-filtered.vcf -stats /cromwell_root/fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-MergeStats/merged.stats --filtering-stats filtering.stats --min-reads-per-strand 5 --min-allele-fraction 5.0 2021/03/26 13:36:34 Starting delocalization. 2021/03/26 13:36:36 Delocalization script execution started... 2021/03/26 13:36:36 Delocalizing output /cromwell_root/memory_retry_rc -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-Filter/memory_retry_rc 2021/03/26 13:36:36 Delocalizing output /cromwell_root/rc -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-Filter/rc 2021/03/26 13:36:37 Delocalizing output /cromwell_root/stdout -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-Filter/stdout 2021/03/26 13:36:38 Delocalizing output /cromwell_root/stderr -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-Filter/stderr 2021/03/26 13:36:40 Delocalizing output /cromwell_root/filtering.stats -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-Filter/filtering.stats 2021/03/26 13:36:41 Delocalizing output /cromwell_root/RESPOND_60100256_T1_Malignant-filtered.vcf -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-Filter/RESPOND_60100256_T1_Malignant-filtered.vcf 2021/03/26 13:36:46 Delocalizing output /cromwell_root/RESPOND_60100256_T1_Malignant-filtered.vcf.idx -> gs://fc-9828bda3-33d6-40b9-ad27-544fb296a8e0/37a63de4-948f-45a9-bd5a-75b8ee8fb165/Mutect2/09c6ec7f-abc9-40b0-9435-6e9270471089/call-Filter/RESPOND_60100256_T1_Malignant-filtered.vcf.idx 2021/03/26 13:36:48 Delocalization script execution complete. 2021/03/26 13:36:49 Done delocalization.
Best,
Jamie
-
Jamie,
Okay great! Thank you for posting the details. Could make sure that there are variants that passed the filtering (marked as PASS in the FILTER column)? It's still so strange to me that there were no variants processed in the Funcotator progress meter. You can run SelectVariants with --exclude-filtered to get only variants that have passed the filtering steps.
Let me know the output that you get from that.
Best,
Genevieve
-
Hi Genevieve,
I think you found the problem! I looked through the vcf file and none of the variants were marked with PASS, and because I set this filter argument to true, none of the variants were annotated.
As a follow up question, are we calling FilterMutectCalls correctly in the workflow? I couldn't find any documentation as to how this field should be formatted.
Thanks for the help!
Jamie
-
Hi Jamie,
For the funco_filter_funcotations, I think it would be fine to leave that as true. It is best to only annotate the PASS variants because the filtered variants don't need to be analyzed with this step. You should be able to get variants that PASS your FilterMutectCalls command by removing your FilterMutectCalls options, they are too stringent.
For the --min-reads-per-strand, here is a forum discussion related to that option. It is an advanced filter and we don't recommend it for most users.
The --min-allele-fraction can be useful if you are looking for certain variants above a certain allele fraction (filters out variants with low allele fraction). Generally AF values are between 0 and 1 and so with 5.0 you would be filtering out all of your variants.
FilterMutectCalls generally does quite well with the default filters, and then if you are seeing strange results you can go back and change the filtering options depending on what you find.
This should fix your issue!
Genevieve
-
Hi Genevieve,
Thanks for the clarification, that makes a lot of sense now! We thought the AF was a percentage rather than a fraction.
Regarding the default arguments for FilterMutectCalls, how would I input those? Or is FilterMutectCalls always called after Mutect2 with the default arguments?
Best,
Jamie
-
I believe if you just leave the FilterMutectCalls step of the Mutect2 WDL as-is, it should be using the recommended options for most use cases. You probably won't need to change the m2_extra_filtering_args parameter from what it was originally.
Please sign in to leave a comment.
13 comments