Write variant tensors for training a CNN to filter variants
Category Variant Filtering
OverviewWrite variant tensors for training a Convolutional Neural Network (CNN) for filtering variants. After running this tool, a model can be trained with the CNNVariantTrain tool.
- The input variants to make into tensors. These variant calls must be annotated with the standard best practices annotations.
- The truth VCF has validated variant calls, like those in the genomes in a bottle, platinum genomes, or CHM VCFs. Variants in both the input VCF and the truth VCF will be used as positive training data.
- The truth BED is a bed file define the confident region for the validated calls. Variants from the input VCF inside this region, but not included in the truth VCF will be used as negative training data.
- The --tensor-type argument determines what types of tensors will be written. Set it to "reference" to write 1D tensors or "read_tensor" to write 2D tensors.
- The bam-file argument is necessary to write 2D tensors which incorporate read data.
- data-dir This directory is created and populated with variant tensors. it will be divided into training, validation and test sets and each set will be further divided into positive and negative SNPs and INDELs.
Write Reference Tensors
gatk CNNVariantWriteTensors \ -R reference.fasta \ -V input.vcf.gz \ -truth-vcf platinum-genomes.vcf \ -truth-bed platinum-confident-region.bed \ -tensor-type reference \ -output-tensor-dir my-tensor-folder
Write Read Tensors
gatk CNNVariantWriteTensors \ -R reference.fasta \ -V input.vcf.gz \ -truth-vcf platinum-genomes.vcf \ -truth-bed platinum-confident-region.bed \ -tensor-type read_tensor \ -bam-file input.bam \ -output-tensor-dir my-tensor-folder
CNNVariantWriteTensors specific arguments
This table summarizes the command-line arguments that are specific to this tool. For more details on each argument, see the list further down below the table or click on an argument name to jump directly to that entry in the list.
|Argument name(s)||Default value||Summary|
||Directory of training tensors. Subdivided into train, valid and test sets.|
|Reference fasta file.|
||Confident region of the validated VCF file.|
||Validated VCF file.|
|Input VCF file|
|Optional Tool Arguments|
||read one or more arguments files and add them to the command line|
||BAM or BAMout file to use for read data when generating 2D tensors.|
||0.5||Fraction of INDELs to write tensors for.|
||0.05||Fraction of SNPs to write tensors for.|
|20||If the GCS bucket channel errors out, how many times it will attempt to re-initiate the connection|
||Project to bill when accessing "requester pays" buckets. If unset, these buckets cannot be accessed. User must have storage.buckets.get permission on the bucket being accessed.|
|false||display the help message|
||1000000||Maximum number of tensors to write.|
||reference||Name of the tensors to generate.|
||false||display the version number for this tool|
|Optional Common Arguments|
||A configuration file to use with the GATK.|
||false||Whether to suppress job-summary info on System.err.|
||Temp directory to use.|
|false||Whether to use the JdkDeflater (as opposed to IntelDeflater)|
|false||Whether to use the JdkInflater (as opposed to IntelInflater)|
||INFO||Control verbosity of logging.|
||best_practices||Which set of annotations to use.|
||true||Store the channels in the last axis of tensors, tensorflow->true, theano->false|
||false||display hidden arguments|
Arguments in this list are specific to this tool. Keep in mind that other arguments are available that are shared with other tools (e.g. command-line GATK arguments); see Inherited arguments above.
--annotation-set / -annotation-set
Which set of annotations to use.
read one or more arguments files and add them to the command line
--bam-file / -bam-file
BAM or BAMout file to use for read data when generating 2D tensors.
--channels-last / -channels-last
Store the channels in the last axis of tensors, tensorflow->true, theano->false
--downsample-indels / -downsample-indels
Fraction of INDELs to write tensors for.
float 0.5 [ [ -∞ ∞ ] ]
--downsample-snps / -downsample-snps
Fraction of SNPs to write tensors for.
float 0.05 [ [ -∞ ∞ ] ]
A configuration file to use with the GATK.
--gcs-max-retries / -gcs-retries
If the GCS bucket channel errors out, how many times it will attempt to re-initiate the connection
int 20 [ [ -∞ ∞ ] ]
Project to bill when accessing "requester pays" buckets. If unset, these buckets cannot be accessed. User must have storage.buckets.get permission on the bucket being accessed.
--help / -h
display the help message
--max-tensors / -max-tensors
Maximum number of tensors to write.
int 1000000 [ [ 0 ∞ ] ]
--output-tensor-dir / -output-tensor-dir
Directory of training tensors. Subdivided into train, valid and test sets.
R String null
Whether to suppress job-summary info on System.err.
--reference / -R
Reference fasta file.
R String null
--showHidden / -showHidden
display hidden arguments
--tensor-type / -tensor-type
Name of the tensors to generate.
The --tensor-type argument is an enumerated type (TensorType), which can have one of the following values:
- 1 Hot encoding of a reference sequence.
- Read tensor are 3D tensors spanning aligned reads, sites and channels. The maximum number of reads is a hyper-parameter typically set to 128. There are 15 channels in the read tensor. They correspond to the reference sequence data (4), read sequence data (4), insertions and deletions (2) read flags (4) and mapping quality (1).
Temp directory to use.
--truth-bed / -truth-bed
Confident region of the validated VCF file.
R String null
--truth-vcf / -truth-vcf
Validated VCF file.
R String null
--use-jdk-deflater / -jdk-deflater
Whether to use the JdkDeflater (as opposed to IntelDeflater)
--use-jdk-inflater / -jdk-inflater
Whether to use the JdkInflater (as opposed to IntelInflater)
--variant / -V
Input VCF file
R String null
--verbosity / -verbosity
Control verbosity of logging.
The --verbosity argument is an enumerated type (LogLevel), which can have one of the following values:
display the version number for this tool
GATK version 22.214.171.124-1-gf548ccd-SNAPSHOT built at Mon, 2 Aug 2021 14:22:45 -0700.