Genome Analysis Toolkit

Variant Discovery in High-Throughput Sequencing Data

GATK process banner

Need Help?

Search our documentation

Community Forum

Hi, How can we help?

Developed in the Data Sciences Platform at the Broad Institute, the toolkit offers a wide variety of tools with a primary focus on variant discovery and genotyping. Its powerful processing engine and high-performance computing features make it capable of taking on projects of any size. Learn more

Funcotator problem

Follow
Avatar
Robert Bremel

I started two Docker containers of Funcotator this morning and they both crashed in about the same place.   I had started them earlier but they seemed to have finished quicker than they should have and the output files seems a bit too small so I ran them a second time and they crashed after a few minutes again.

Seems as though it is something at google?

I can also sent the .vcf and .maf.  The input files were Mutect2 files that had been run through FilterMutect.

Docker Desktop GATK run from Powershell on Windows 10 Pro

REQUIRED for all errors and issues:
a) GATK version used:

"latest" Docker version of GATK
b) Exact command used:

Docker command and log below

c) Entire program log:

 
docker exec GBM_00177a2_BashfulGar_funky_STAR_gatk   ./gatk Funcotator   -R                  mydata/refs/Homo_sapiens_assembly38.fasta -V                  mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_filtered_hg38.vcf -O                  mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_funcotator_hg38.maf --output-file-format MAF  --disable-sequence-dictionary-validation true   --data-sources-path mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/ --verbosity DEBUG  --ref-version hg38 
14:18:40.804 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.4.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
14:18:40.903 DEBUG NativeLibraryLoader - Extracting libgkl_compression.so to /tmp/libgkl_compression17641255252170407632.so
14:18:40.915 INFO  Funcotator - ------------------------------------------------------------
14:18:40.924 INFO  Funcotator - The Genome Analysis Toolkit (GATK) v4.4.0.0
14:18:40.927 INFO  Funcotator - For support and documentation go to https://software.broadinstitute.org/gatk/
14:18:40.927 INFO  Funcotator - Executing as root@2bc6c797f8f9 on Linux v5.15.153.1-microsoft-standard-WSL2 amd64
14:18:40.928 INFO  Funcotator - Java runtime: OpenJDK 64-Bit Server VM v17.0.6+10-Ubuntu-0ubuntu118.04.1
14:18:40.928 INFO  Funcotator - Start Date/Time: August 12, 2024 at 2:18:40 PM GMT
14:18:40.928 INFO  Funcotator - ------------------------------------------------------------
14:18:40.929 INFO  Funcotator - ------------------------------------------------------------
14:18:40.933 INFO  Funcotator - HTSJDK Version: 3.0.5
14:18:40.936 INFO  Funcotator - Picard Version: 3.0.0
14:18:40.938 INFO  Funcotator - Built for Spark Version: 3.3.1
14:18:40.941 INFO  Funcotator - HTSJDK Defaults.BUFFER_SIZE : 131072
14:18:40.942 INFO  Funcotator - HTSJDK Defaults.COMPRESSION_LEVEL : 2
14:18:40.942 INFO  Funcotator - HTSJDK Defaults.CREATE_INDEX : false
14:18:40.942 INFO  Funcotator - HTSJDK Defaults.CREATE_MD5 : false
14:18:40.942 INFO  Funcotator - HTSJDK Defaults.CUSTOM_READER_FACTORY : 
14:18:40.942 INFO  Funcotator - HTSJDK Defaults.DISABLE_SNAPPY_COMPRESSOR : false
14:18:40.942 INFO  Funcotator - HTSJDK Defaults.EBI_REFERENCE_SERVICE_URL_MASK : https://www.ebi.ac.uk/ena/cram/md5/%s
14:18:40.942 INFO  Funcotator - HTSJDK Defaults.NON_ZERO_BUFFER_SIZE : 131072
14:18:40.942 INFO  Funcotator - HTSJDK Defaults.REFERENCE_FASTA : null
14:18:40.943 INFO  Funcotator - HTSJDK Defaults.SAM_FLAG_FIELD_FORMAT : DECIMAL
14:18:40.943 INFO  Funcotator - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
14:18:40.943 INFO  Funcotator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
14:18:40.943 INFO  Funcotator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
14:18:40.943 INFO  Funcotator - HTSJDK Defaults.USE_CRAM_REF_DOWNLOAD : false
14:18:40.943 DEBUG ConfigFactory - Configuration file values: 
14:18:40.948 DEBUG ConfigFactory - gcsMaxRetries = 20
14:18:40.948 DEBUG ConfigFactory - gcsProjectForRequesterPays = 
14:18:40.948 DEBUG ConfigFactory - gatk_stacktrace_on_user_exception = false
14:18:40.948 DEBUG ConfigFactory - samjdk.use_async_io_read_samtools = false
14:18:40.948 DEBUG ConfigFactory - samjdk.use_async_io_write_samtools = true
14:18:40.948 DEBUG ConfigFactory - samjdk.use_async_io_write_tribble = false
14:18:40.948 DEBUG ConfigFactory - samjdk.compression_level = 2
14:18:40.949 DEBUG ConfigFactory - spark.kryoserializer.buffer.max = 512m
14:18:40.949 DEBUG ConfigFactory - spark.driver.maxResultSize = 0
14:18:40.949 DEBUG ConfigFactory - spark.driver.userClassPathFirst = true
14:18:40.949 DEBUG ConfigFactory - spark.io.compression.codec = lzf
14:18:40.949 DEBUG ConfigFactory - spark.executor.memoryOverhead = 600
14:18:40.949 DEBUG ConfigFactory - spark.driver.extraJavaOptions = 
14:18:40.949 DEBUG ConfigFactory - spark.executor.extraJavaOptions = 
14:18:40.950 DEBUG ConfigFactory - codec_packages = [htsjdk.variant, htsjdk.tribble, org.broadinstitute.hellbender.utils.codecs]
14:18:40.950 DEBUG ConfigFactory - read_filter_packages = [org.broadinstitute.hellbender.engine.filters]
14:18:40.950 DEBUG ConfigFactory - annotation_packages = [org.broadinstitute.hellbender.tools.walkers.annotator]
14:18:40.950 DEBUG ConfigFactory - cloudPrefetchBuffer = 40
14:18:40.950 DEBUG ConfigFactory - cloudIndexPrefetchBuffer = -1
14:18:40.955 DEBUG ConfigFactory - createOutputBamIndex = true
14:18:40.957 INFO  Funcotator - Deflater: IntelDeflater
14:18:40.959 INFO  Funcotator - Inflater: IntelInflater
14:18:40.960 INFO  Funcotator - GCS max retries/reopens: 20
14:18:40.960 INFO  Funcotator - Requester pays: disabled
14:18:40.961 INFO  Funcotator - Initializing engine
14:18:42.739 INFO  FeatureManager - Using codec VCFCodec to read file file:///gatk/mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_filtered_hg38.vcf
14:18:43.391 INFO  Funcotator - Done initializing engine
14:18:43.393 INFO  Funcotator - Skipping sequence dictionary validation.
14:18:43.397 INFO  Funcotator - Processing user transcripts/defaults/overrides...
14:18:43.403 INFO  Funcotator - Initializing data sources...
14:18:43.407 INFO  DataSourceUtils - Initializing data sources from directory: mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/
14:18:43.449 INFO  DataSourceUtils - Data sources version: 1.7.2020429s
14:18:43.450 INFO  DataSourceUtils - Data sources source: ftp://gsapubftp-anonymous@ftp.broadinstitute.org/bundle/funcotator/funcotator_dataSources.v1.7.20200429s.tar.gz
14:18:43.450 INFO  DataSourceUtils - Data sources alternate source: gs://broad-public-datasets/funcotator/funcotator_dataSources.v1.7.20200429s.tar.gz
14:18:43.525 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/gencode_xhgnc_v90_38.hg38.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode_xhgnc/hg38/gencode_xhgnc_v90_38.hg38.tsv
14:18:43.543 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/oreganno.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.tsv
14:18:43.558 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/simple_uniprot_Dec012014.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/simple_uniprot/hg38/simple_uniprot_Dec012014.tsv
14:18:43.572 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/Cosmic.db -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cosmic/hg38/Cosmic.db
14:18:44.833 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/cosmic_tissue.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cosmic_tissue/hg38/cosmic_tissue.tsv
14:18:44.845 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/Familial_Cancer_Genes.no_dupes.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/familial/hg38/Familial_Cancer_Genes.no_dupes.tsv
14:18:44.857 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/gencode.v34.annotation.REORDERED.gtf -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.annotation.REORDERED.gtf
14:18:44.858 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/gencode.v34.pc_transcripts.fa -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.pc_transcripts.fa
14:18:44.870 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/cosmic_fusion.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cosmic_fusion/hg38/cosmic_fusion.tsv
14:18:44.884 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/gencode_xrefseq_v90_38.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode_xrefseq/hg38/gencode_xrefseq_v90_38.tsv
14:18:44.898 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/hgnc_download_Nov302017.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/hgnc/hg38/hgnc_download_Nov302017.tsv
14:18:44.920 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/CancerGeneCensus_Table_1_full_2012-03-15.txt -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cancer_gene_census/hg38/CancerGeneCensus_Table_1_full_2012-03-15.txt
14:18:44.936 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/hg38_All_20180418.vcf.gz -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz
14:18:44.947 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/clinvar_20180429_hg38.vcf -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf
14:18:44.958 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/achilles_lineage_results.import.txt -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/achilles/hg38/achilles_lineage_results.import.txt
14:18:44.983 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/dnaRepairGenes.20180524T145835.csv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dna_repair_genes/hg38/dnaRepairGenes.20180524T145835.csv
14:18:45.245 INFO  Funcotator - Finalizing data sources (this step can be long if data sources are cloud-based)...
14:18:45.246 DEBUG DataSourceUtils - Creating Funcotation Factory for Gencode_XHGNC ...
14:18:45.248 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/gencode_xhgnc_v90_38.hg38.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode_xhgnc/hg38/gencode_xhgnc_v90_38.hg38.tsv
14:18:48.658 DEBUG DataSourceUtils - Creating Funcotation Factory for Oreganno ...
14:18:48.672 INFO  DataSourceUtils - Setting lookahead cache for data source: Oreganno : 100000
14:18:48.792 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/oreganno.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.tsv
14:18:48.805 INFO  FeatureManager - Using codec XsvLocatableTableCodec to read file file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.config
14:18:49.013 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/oreganno.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.tsv
14:18:49.021 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/oreganno.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.tsv
WARNING 2024-08-12 14:18:49 AsciiLineReader Creating an indexable source for an AsciiFeatureCodec using a stream that is neither a PositionalBufferedStream nor a BlockCompressedInputStream
14:18:49.042 DEBUG DataSourceUtils - Creating Funcotation Factory for Simple_Uniprot ...
14:18:49.052 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/simple_uniprot_Dec012014.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/simple_uniprot/hg38/simple_uniprot_Dec012014.tsv
14:18:49.429 DEBUG DataSourceUtils - Creating Funcotation Factory for Cosmic ...
14:18:49.432 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/Cosmic.db -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cosmic/hg38/Cosmic.db
14:18:49.457 DEBUG CosmicFuncotationFactory - Connecting to SQLite database at: file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cosmic/hg38/Cosmic.db
14:18:49.599 DEBUG CosmicFuncotationFactory - Connected to SQLite database!
14:18:49.600 DEBUG DataSourceUtils - Creating Funcotation Factory for gnomAD_exome ...
14:18:49.600 INFO  DataSourceUtils - Setting lookahead cache for data source: gnomAD_exome : 100000
14:18:53.183 INFO  FeatureManager - Using codec VCFCodec to read file gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.exomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz
14:19:07.795 INFO  FeatureManager - Using codec VCFCodec to read file gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.exomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz
14:19:09.009 DEBUG FeatureDataSource - Cache statistics for FeatureInput gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.exomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz:gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.exomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz:
14:19:09.012 DEBUG FeatureCache - Cache hit rate  was 0.00% (0 out of 0 total queries)
14:19:09.019 DEBUG DataSourceUtils - Creating Funcotation Factory for CosmicTissue ...
14:19:09.024 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/cosmic_tissue.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cosmic_tissue/hg38/cosmic_tissue.tsv
14:19:09.332 DEBUG DataSourceUtils - Creating Funcotation Factory for Familial_Cancer_Genes ...
14:19:09.342 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/Familial_Cancer_Genes.no_dupes.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/familial/hg38/Familial_Cancer_Genes.no_dupes.tsv
14:19:09.351 DEBUG DataSourceUtils - Creating Funcotation Factory for Gencode ...
14:19:09.352 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/gencode.v34.annotation.REORDERED.gtf -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.annotation.REORDERED.gtf
14:19:09.353 INFO  DataSourceUtils - Setting lookahead cache for data source: Gencode : 100000
14:19:09.393 INFO  FeatureManager - Using codec GencodeGtfCodec to read file file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.annotation.REORDERED.gtf
14:19:09.612 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/gencode.v34.pc_transcripts.fa -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.pc_transcripts.fa
14:19:28.389 DEBUG DataSourceUtils - Creating Funcotation Factory for CosmicFusion ...
14:19:28.397 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/cosmic_fusion.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cosmic_fusion/hg38/cosmic_fusion.tsv
14:19:28.437 DEBUG DataSourceUtils - Creating Funcotation Factory for Gencode_XRefSeq ...
14:19:28.441 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/gencode_xrefseq_v90_38.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode_xrefseq/hg38/gencode_xrefseq_v90_38.tsv
14:19:30.164 DEBUG DataSourceUtils - Creating Funcotation Factory for HGNC ...
14:19:30.166 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/hgnc_download_Nov302017.tsv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/hgnc/hg38/hgnc_download_Nov302017.tsv
14:19:33.431 DEBUG DataSourceUtils - Creating Funcotation Factory for CGC ...
14:19:33.436 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/CancerGeneCensus_Table_1_full_2012-03-15.txt -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/cancer_gene_census/hg38/CancerGeneCensus_Table_1_full_2012-03-15.txt
14:19:33.483 DEBUG DataSourceUtils - Creating Funcotation Factory for dbSNP ...
14:19:33.485 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/hg38_All_20180418.vcf.gz -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz
14:19:33.487 INFO  DataSourceUtils - Setting lookahead cache for data source: dbSNP : 100000
14:19:33.610 INFO  FeatureManager - Using codec VCFCodec to read file file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz
14:19:33.866 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/hg38_All_20180418.vcf.gz -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz
14:19:34.146 INFO  FeatureManager - Using codec VCFCodec to read file file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz
14:19:34.450 DEBUG FeatureDataSource - Cache statistics for FeatureInput file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz:/gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz:
14:19:34.451 DEBUG FeatureCache - Cache hit rate  was 0.00% (0 out of 0 total queries)
14:19:34.451 DEBUG DataSourceUtils - Creating Funcotation Factory for ClinVar_VCF ...
14:19:34.452 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/clinvar_20180429_hg38.vcf -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf
14:19:34.453 INFO  DataSourceUtils - Setting lookahead cache for data source: ClinVar_VCF : 100000
14:19:34.476 INFO  FeatureManager - Using codec VCFCodec to read file file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf
14:19:35.028 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/clinvar_20180429_hg38.vcf -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf
14:19:35.713 INFO  FeatureManager - Using codec VCFCodec to read file file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf
14:19:36.288 DEBUG FeatureDataSource - Cache statistics for FeatureInput file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf:/gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf:
14:19:36.288 DEBUG FeatureCache - Cache hit rate  was 0.00% (0 out of 0 total queries)
14:19:36.288 DEBUG DataSourceUtils - Creating Funcotation Factory for Achilles ...
14:19:36.301 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/achilles_lineage_results.import.txt -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/achilles/hg38/achilles_lineage_results.import.txt
14:19:36.321 DEBUG DataSourceUtils - Creating Funcotation Factory for DNARepairGenes ...
14:19:36.326 INFO  DataSourceUtils - Resolved data source file path: file:///gatk/dnaRepairGenes.20180524T145835.csv -> file:///gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dna_repair_genes/hg38/dnaRepairGenes.20180524T145835.csv
14:19:36.336 DEBUG DataSourceUtils - Creating Funcotation Factory for gnomAD_genome ...
14:19:36.337 INFO  DataSourceUtils - Setting lookahead cache for data source: gnomAD_genome : 100000
14:19:41.905 INFO  FeatureManager - Using codec VCFCodec to read file gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.genomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz
14:20:10.008 INFO  FeatureManager - Using codec VCFCodec to read file gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.genomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz
14:20:12.446 DEBUG FeatureDataSource - Cache statistics for FeatureInput gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.genomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz:gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.genomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz:
14:20:12.457 DEBUG FeatureCache - Cache hit rate  was 0.00% (0 out of 0 total queries)
14:20:12.473 DEBUG DataSourceUtils - All Data Sources have been created.
14:20:12.490 INFO  Funcotator - Initializing Funcotator Engine...
14:20:12.541 INFO  FuncotatorEngine - Using given VCF and Reference.  No conversion required.
14:20:12.570 INFO  Funcotator - Creating a MAF file for output: file:/gatk/mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_funcotator_hg38.maf
14:20:12.654 INFO  ProgressMeter - Starting traversal
14:20:12.655 INFO  ProgressMeter -        Current Locus  Elapsed Minutes    Variants Processed  Variants/Minute
14:21:54.777 INFO  ProgressMeter -        chr1:11054839              1.7                  1000            587.5
14:22:57.769 INFO  ProgressMeter -        chr1:20090571              2.8                  2000            726.8
14:24:04.757 INFO  ProgressMeter -        chr1:27334493              3.9                  3000            775.5
14:24:21.685 INFO  VcfFuncotationFactory - ClinVar_VCF 20180429_hg38 cache hits/total: 0/29
14:24:21.687 INFO  VcfFuncotationFactory - dbSNP 9606_b151 cache hits/total: 0/1514
14:24:21.688 INFO  VcfFuncotationFactory - gnomAD_exome 2.1 cache hits/total: 0/1097
14:24:21.688 INFO  VcfFuncotationFactory - gnomAD_genome 2.1 cache hits/total: 0/491
14:24:21.689 INFO  Funcotator - Shutting down engine
14:24:21.691 DEBUG FeatureDataSource - Cache statistics for FeatureInput drivingVariantFile:/gatk/mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_filtered_hg38.vcf:
14:24:21.691 DEBUG FeatureCache - Cache hit rate  was 0.00% (0 out of 0 total queries)
14:24:21.691 DEBUG FeatureDataSource - Cache statistics for FeatureInput Oreganno:/gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.config:
14:24:21.691 DEBUG FeatureCache - Cache hit rate  was 95.01% (3329 out of 3504 total queries)
14:24:21.692 DEBUG FeatureDataSource - Cache statistics for FeatureInput gnomAD_exome:gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.exomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz:
14:24:21.692 DEBUG FeatureCache - Cache hit rate  was 95.01% (3329 out of 3504 total queries)
14:24:24.512 DEBUG FeatureDataSource - Cache statistics for FeatureInput Gencode:/gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.annotation.REORDERED.gtf:
14:24:24.512 DEBUG FeatureCache - Cache hit rate  was 95.01% (3329 out of 3504 total queries)
14:24:24.513 DEBUG FeatureDataSource - Cache statistics for FeatureInput dbSNP:/gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz:
14:24:24.513 DEBUG FeatureCache - Cache hit rate  was 95.01% (3329 out of 3504 total queries)
14:24:24.514 DEBUG FeatureDataSource - Cache statistics for FeatureInput ClinVar_VCF:/gatk/mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf:
14:24:24.514 DEBUG FeatureCache - Cache hit rate  was 95.01% (3329 out of 3504 total queries)
14:24:24.515 DEBUG FeatureDataSource - Cache statistics for FeatureInput gnomAD_genome:gs://broad-public-datasets/funcotator/gnomAD_2.1_VCF_INFO_AF_Only/hg38/gnomad.genomes.r2.1.sites.liftoverToHg38.INFO_ANNOTATIONS_FIXED.vcf.gz:
14:24:24.515 DEBUG FeatureCache - Cache hit rate  was 95.01% (3329 out of 3504 total queries)
14:24:24.516 DEBUG FeatureDataSource - Cache statistics for FeatureInput drivingVariantFile:/gatk/mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_filtered_hg38.vcf:
14:24:24.516 DEBUG FeatureCache - Cache hit rate  was 0.00% (0 out of 0 total queries)
[August 12, 2024 at 2:24:24 PM GMT] org.broadinstitute.hellbender.tools.funcotator.Funcotator done. Elapsed time: 5.73 minutes.
Runtime.totalMemory()=5167382528
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.io.IOException: com.google.cloud.RetryHelper$RetryHelperException: com.google.cloud.storage.StorageException: storage.googleapis.com
at com.google.cloud.storage.contrib.nio.SeekableByteChannelPrefetcher.read(SeekableByteChannelPrefetcher.java:401)
at htsjdk.samtools.seekablestream.SeekablePathStream.read(SeekablePathStream.java:86)
at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:244)
at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:343)
at htsjdk.samtools.seekablestream.SeekableBufferedStream.read(SeekableBufferedStream.java:133)
at htsjdk.samtools.util.BlockCompressedInputStream.readBytes(BlockCompressedInputStream.java:571)
at htsjdk.samtools.util.BlockCompressedInputStream.readBytes(BlockCompressedInputStream.java:560)
at htsjdk.samtools.util.BlockCompressedInputStream.processNextBlock(BlockCompressedInputStream.java:525)
at htsjdk.samtools.util.BlockCompressedInputStream.nextBlock(BlockCompressedInputStream.java:468)
at htsjdk.samtools.util.BlockCompressedInputStream.readBlock(BlockCompressedInputStream.java:458)
at htsjdk.samtools.util.BlockCompressedInputStream.available(BlockCompressedInputStream.java:196)
at htsjdk.samtools.util.BlockCompressedInputStream.read(BlockCompressedInputStream.java:241)
at htsjdk.tribble.readers.TabixReader.readLine(TabixReader.java:215)
at htsjdk.tribble.readers.TabixReader.access$300(TabixReader.java:48)
at htsjdk.tribble.readers.TabixReader$IteratorImpl.next(TabixReader.java:434)
at htsjdk.tribble.readers.TabixIteratorLineReader.readLine(TabixIteratorLineReader.java:46)
at htsjdk.tribble.TabixFeatureReader$FeatureIterator.readNextRecord(TabixFeatureReader.java:170)
at htsjdk.tribble.TabixFeatureReader$FeatureIterator.next(TabixFeatureReader.java:205)
at htsjdk.tribble.TabixFeatureReader$FeatureIterator.next(TabixFeatureReader.java:149)
at org.broadinstitute.hellbender.engine.FeatureCache.fill(FeatureCache.java:142)
at org.broadinstitute.hellbender.engine.FeatureDataSource.refillQueryCache(FeatureDataSource.java:623)
at org.broadinstitute.hellbender.engine.FeatureDataSource.queryAndPrefetch(FeatureDataSource.java:591)
at org.broadinstitute.hellbender.engine.FeatureManager.getFeatures(FeatureManager.java:363)
at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:173)
at org.broadinstitute.hellbender.engine.FeatureContext.getValues(FeatureContext.java:125)
at org.broadinstitute.hellbender.tools.funcotator.DataSourceFuncotationFactory.queryFeaturesFromFeatureContext(DataSourceFuncotationFactory.java:310)
at org.broadinstitute.hellbender.tools.funcotator.DataSourceFuncotationFactory.getFeaturesFromFeatureContext(DataSourceFuncotationFactory.java:229)
at org.broadinstitute.hellbender.tools.funcotator.DataSourceFuncotationFactory.createFuncotations(DataSourceFuncotationFactory.java:207)
at org.broadinstitute.hellbender.tools.funcotator.FuncotatorEngine.createFuncotationMapForVariant(FuncotatorEngine.java:178)
at org.broadinstitute.hellbender.tools.funcotator.Funcotator.enqueueAndHandleVariant(Funcotator.java:924)
at org.broadinstitute.hellbender.tools.funcotator.Funcotator.apply(Funcotator.java:878)
at org.broadinstitute.hellbender.engine.VariantWalker.lambda$traverse$0(VariantWalker.java:104)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1845)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
at org.broadinstitute.hellbender.engine.VariantWalker.traverse(VariantWalker.java:102)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1098)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:149)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:198)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:217)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
at org.broadinstitute.hellbender.Main.main(Main.java:289)
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: com.google.cloud.RetryHelper$RetryHelperException: com.google.cloud.storage.StorageException: storage.googleapis.com
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at com.google.cloud.storage.contrib.nio.SeekableByteChannelPrefetcher$WorkUnit.getBuf(SeekableByteChannelPrefetcher.java:230)
at com.google.cloud.storage.contrib.nio.SeekableByteChannelPrefetcher.fetch(SeekableByteChannelPrefetcher.java:347)
at com.google.cloud.storage.contrib.nio.SeekableByteChannelPrefetcher.read(SeekableByteChannelPrefetcher.java:392)
... 52 more
Caused by: java.io.IOException: com.google.cloud.RetryHelper$RetryHelperException: com.google.cloud.storage.StorageException: storage.googleapis.com
at com.google.cloud.storage.BlobReadChannel.read(BlobReadChannel.java:140)
at com.google.cloud.storage.contrib.nio.CloudStorageReadChannel.read(CloudStorageReadChannel.java:157)
at com.google.cloud.storage.contrib.nio.SeekableByteChannelPrefetcher$WorkUnit.call(SeekableByteChannelPrefetcher.java:225)
at com.google.cloud.storage.contrib.nio.SeekableByteChannelPrefetcher$WorkUnit.call(SeekableByteChannelPrefetcher.java:193)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.google.cloud.RetryHelper$RetryHelperException: com.google.cloud.storage.StorageException: storage.googleapis.com
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:54)
at com.google.cloud.storage.BlobReadChannel.read(BlobReadChannel.java:127)
... 7 more
Caused by: com.google.cloud.storage.StorageException: storage.googleapis.com
at com.google.cloud.storage.StorageException.translate(StorageException.java:116)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.read(HttpStorageRpc.java:741)
at com.google.cloud.storage.BlobReadChannel.lambda$read$0(BlobReadChannel.java:128)
at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
... 8 more
Caused by: java.net.UnknownHostException: storage.googleapis.com
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:567)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
at java.base/java.net.Socket.connect(Socket.java:633)
at java.base/sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:304)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:178)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:151)
at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:84)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeMedia(AbstractGoogleClientRequest.java:479)
at com.google.api.services.storage.Storage$Objects$Get.executeMedia(Storage.java:7222)
at com.google.cloud.storage.spi.v1.HttpStorageRpc.read(HttpStorageRpc.java:736)
... 12 more
Using GATK jar /gatk/gatk-package-4.4.0.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /gatk/gatk-package-4.4.0.0-local.jar Funcotator -R mydata/refs/Homo_sapiens_assembly38.fasta -V mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_filtered_hg38.vcf -O mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_funcotator_hg38.maf --output-file-format MAF --disable-sequence-dictionary-validation true --data-sources-path mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/ --verbosity DEBUG --ref-version hg38
 
{pid_box = "GBM_00177a2_BashfulGar", rb_box1 = 1, tb_start = Empty(), tb_elapsed =
Empty(), command_box =
"docker exec GBM_00177a2_BashfulGar_funky_STAR_gatk   ./gatk Funcotator   -R                  mydata/refs/Homo_sapiens_assembly38.fasta -V                  mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_filtered_hg38.vcf -O                  mydata/GBM_00177a2_BashfulGar/analysis/GBM_00177a2-4e329_RT_BashfulGar_mutect2_STAR-cigarsplit-transcriptome_funcotator_hg38.maf --output-file-format MAF  --disable-sequence-dictionary-validation true   --data-sources-path mydata/dataSourcesFolder/funcotator_dataSources.v1.7.20200521s/ --verbosity DEBUG  --ref-version hg38 
",
Button( 1 )}

 

0

1 comment

Sort by Date Votes
  • Avatar
    Gökalp Çelik

    Hi Robert Bremel

    Can you try running this task with the latest GATK 4.6.0.0 and latest resource for Funcotator? 

    Do you still get the same error?

    0
    Comment actions Permalink

Please sign in to leave a comment.

Welcome

Forum Guidelines

GATK Announcements

Become a GATK Guru

Read Before You Post

Get Email Notifications

Didn't find what you were looking for?

New post

Quick Links

GATK Latest Releases

Known Issues & Troubleshooting

Submitting a Bug Report

How to Track a Github issue

GATK on the Cloud with Terra

Recent GATK News

GATK Blog Posts

GATK Twitter Page

About the GATK community

  • 133,119 Total Users
  • 0 Comments
  • 0 Posts
  • 0 Votes
Powered by Zendesk