Downsample a SAM or BAM file.This tool applies a downsampling algorithm to a SAM or BAM file to retain only a (deterministically random) subset of the reads. Reads from the same template (e.g. read-pairs, secondary and supplementary reads) are all either kept or discarded as a unit, with the goal of retaining readsfrom PROBABILITY * input templates. The results will contain approximately PROBABILITY * input reads, however for very small PROBABILITIES this may not be the case. A number of different downsampling strategies are supported using the STRATEGY option: ConstantMemory: Downsamples a stream or file of SAMRecords using a hash-projection strategy such that it can run in constant memory. The downsampling is stochastic, and therefore the actual retained proportion will vary around the requested proportion. Due to working in fixed memory this strategy is good for large inputs, and due to the stochastic nature the accuracy of this strategy is highest with a high number of output records, and diminishes at low output volumes. HighAccuracy: Attempts (but does not guarantee) to provide accuracy up to a specified limit. Accuracy is defined as emitting a proportion of reads as close to the requested proportion as possible. In order to do so this strategy requires memory that is proportional to the number of template names in the incoming stream of reads, and will thus require large amounts of memory when running on large input files. Chained: Attempts to provide a compromise strategy that offers some of the advantages of both the ConstantMemory and HighAccuracy strategies. Uses a ConstantMemory strategy to downsample the incoming stream to approximately the desired proportion, and then a HighAccuracy strategy to finish. Works in a single pass, and will provide accuracy close to (but often not as good as) HighAccuracy while requiring memory proportional to the set of reads emitted from the ConstantMemory strategy to the HighAccuracy strategy. Works well when downsampling large inputs to small proportions (e.g. downsampling hundreds of millions of reads and retaining only 2%. Should be accurate 99.9% of the time when the input contains more than 50,000 templates (read names). For smaller inputs, HighAccuracy is recommended instead.
Usage examples:
Downsample file, keeping about 10% of the reads
java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ P=0.2Downsample file, keeping about 2% of the reads
java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ STRATEGY=Chained \ P=0.02 \ ACCURACY=0.0001Downsample file, keeping about 0.001% of the reads (may require more memory)
java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ STRATEGY=HighAccuracy \ P=0.00001 \ ACCURACY=0.0000001Category Read Data Manipulation
Overview
Summary
This tool applies a downsampling algorithm to a SAM or BAM file to retain only a (deterministically random) subset of the reads. Reads from the same template (e.g. read-pairs, secondary and supplementary reads) are all either kept or discarded as a unit, with the goal of retaining reads fromPROBABILITY * (input templates)
. The results
will contain approximately PROBABILITY * (input reads)
, however for very small
probabilities this may not be the case.
A number of different downsampling strategies are supported using the #STRATEGY option:
- ConstantMemory
- Downsamples a stream or file of SAMRecords using a hash-projection strategy such that it can run in constant memory. The downsampling is stochastic, and therefore the actual retained proportion will vary around the requested proportion. Due to working in fixed memory this strategy is good for large inputs, and due to the stochastic nature the accuracy of this strategy is highest with a high number of output records, and diminishes at low output volumes.
- HighAccuracy
- Attempts (but does not guarantee) to provide accuracy up to a specified limit. Accuracy is defined as emitting a proportion of reads as close to the requested proportion as possible. In order to do so this strategy requires memory that is proportional to the number of template names in the incoming stream of reads, and will thus require large amounts of memory when running on large input files.
- Chained
- Attempts to provide a compromise strategy that offers some of the advantages of both the ConstantMemory and HighAccuracy strategies. Uses a ConstantMemory strategy to downsample the incoming stream to approximately the desired proportion, and then a HighAccuracy strategy to finish. Works in a single pass, and will provide accuracy close to (but often not as good as) HighAccuracy while requiring memory proportional to the set of reads emitted from the ConstantMemory strategy to the HighAccuracy strategy. Works well when downsampling large inputs to small proportions (e.g. downsampling hundreds of millions of reads and retaining only 2%. Should be accurate 99.9% of the time when the input contains more than 50,000 templates (read names). For smaller inputs, HighAccuracy is recommended instead.
Usage examples:
Downsample file, keeping about 10% of the reads
java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ P=0.1
Downsample file, keeping 2% of the reads
java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ STRATEGY=Chained \ P=0.02 \ ACCURACY=0.0001
Downsample file, keeping 0.001% of the reads (may require more memory)
java -jar picard.jar DownsampleSam \ I=input.bam \ O=downsampled.bam \ STRATEGY=HighAccuracy \ P=0.00001 \ ACCURACY=0.0000001
DownsampleSam (Picard) specific arguments
This table summarizes the command-line arguments that are specific to this tool. For more details on each argument, see the list further down below the table or click on an argument name to jump directly to that entry in the list.
Argument name(s) | Default value | Summary | |
---|---|---|---|
Required Arguments | |||
--INPUT -I |
The input SAM or BAM file to downsample. | ||
--OUTPUT -O |
The output, downsampled, SAM or BAM file to write. | ||
Optional Tool Arguments | |||
--ACCURACY -A |
1.0E-4 | The accuracy that the downsampler should try to achieve if the selected strategy supports it. Note that accuracy is never guaranteed, but some strategies will attempt to provide accuracy within the requested bounds.Higher accuracy will generally require more memory. | |
--arguments_file |
read one or more arguments files and add them to the command line | ||
--help -h |
false | display the help message | |
--METRICS_FILE -M |
The metrics file (of type QualityYieldMetrics) which will contain information about the downsampled file. | ||
--PROBABILITY -P |
1.0 | The probability of keeping any individual read, between 0 and 1. | |
--RANDOM_SEED -R |
1 | Random seed used for deterministic results. Setting to null will cause multiple invocations to produce different results. The header if the file will be checked for any previous runs of DownsampleSam. If DownsampleSam has been run before on this data with the same seed, the seed will be updated in a deterministic fashion so the DownsampleSam will perform correctly, and still deterministically. | |
--REFERENCE_SEQUENCE |
The reference sequence file. | ||
--STRATEGY -S |
ConstantMemory | The downsampling strategy to use. See usage for discussion. | |
--version |
false | display the version number for this tool | |
Optional Common Arguments | |||
--COMPRESSION_LEVEL |
5 | Compression level for all compressed files created (e.g. BAM and VCF). | |
--CREATE_INDEX |
false | Whether to create an index when writing VCF or coordinate sorted BAM output. | |
--CREATE_MD5_FILE |
false | Whether to create an MD5 digest for any BAM or FASTQ files created. | |
--GA4GH_CLIENT_SECRETS |
client_secrets.json | Google Genomics API client_secrets.json file path. | |
--MAX_RECORDS_IN_RAM |
500000 | When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed. | |
--QUIET |
false | Whether to suppress job-summary info on System.err. | |
--TMP_DIR |
One or more directories with space available to be used by this program for temporary storage of working files | ||
--USE_JDK_DEFLATER -use_jdk_deflater |
false | Use the JDK Deflater instead of the Intel Deflater for writing compressed output | |
--USE_JDK_INFLATER -use_jdk_inflater |
false | Use the JDK Inflater instead of the Intel Inflater for reading compressed input | |
--VALIDATION_STRINGENCY |
STRICT | Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded. | |
--VERBOSITY |
INFO | Control verbosity of logging. | |
Advanced Arguments | |||
--showHidden |
false | display hidden arguments |
Argument details
Arguments in this list are specific to this tool. Keep in mind that other arguments are available that are shared with other tools (e.g. command-line GATK arguments); see Inherited arguments above.
--ACCURACY / -A
The accuracy that the downsampler should try to achieve if the selected strategy supports it. Note that accuracy is never guaranteed, but some strategies will attempt to provide accuracy within the requested bounds.Higher accuracy will generally require more memory.
double 1.0E-4 [ [ -∞ ∞ ] ]
--arguments_file
read one or more arguments files and add them to the command line
List[File] []
--COMPRESSION_LEVEL
Compression level for all compressed files created (e.g. BAM and VCF).
int 5 [ [ -∞ ∞ ] ]
--CREATE_INDEX
Whether to create an index when writing VCF or coordinate sorted BAM output.
Boolean false
--CREATE_MD5_FILE
Whether to create an MD5 digest for any BAM or FASTQ files created.
boolean false
--GA4GH_CLIENT_SECRETS
Google Genomics API client_secrets.json file path.
String client_secrets.json
--help / -h
display the help message
boolean false
--INPUT / -I
The input SAM or BAM file to downsample.
R File null
--MAX_RECORDS_IN_RAM
When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.
Integer 500000 [ [ -∞ ∞ ] ]
--METRICS_FILE / -M
The metrics file (of type QualityYieldMetrics) which will contain information about the downsampled file.
File null
--OUTPUT / -O
The output, downsampled, SAM or BAM file to write.
R File null
--PROBABILITY / -P
The probability of keeping any individual read, between 0 and 1.
double 1.0 [ [ -∞ ∞ ] ]
--QUIET
Whether to suppress job-summary info on System.err.
Boolean false
--RANDOM_SEED / -R
Random seed used for deterministic results. Setting to null will cause multiple invocations to produce different results. The header if the file will be checked for any previous runs of DownsampleSam. If DownsampleSam has been run before on this data with the same seed, the seed will be updated in a deterministic fashion so the DownsampleSam will perform correctly, and still deterministically.
Integer 1 [ [ -∞ ∞ ] ]
--REFERENCE_SEQUENCE
The reference sequence file.
File null
--showHidden / -showHidden
display hidden arguments
boolean false
--STRATEGY / -S
The downsampling strategy to use. See usage for discussion.
The --STRATEGY argument is an enumerated type (Strategy), which can have one of the following values:
- HighAccuracy
- ConstantMemory
- Chained
Strategy ConstantMemory
--TMP_DIR
One or more directories with space available to be used by this program for temporary storage of working files
List[File] []
--USE_JDK_DEFLATER / -use_jdk_deflater
Use the JDK Deflater instead of the Intel Deflater for writing compressed output
Boolean false
--USE_JDK_INFLATER / -use_jdk_inflater
Use the JDK Inflater instead of the Intel Inflater for reading compressed input
Boolean false
--VALIDATION_STRINGENCY
Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.
The --VALIDATION_STRINGENCY argument is an enumerated type (ValidationStringency), which can have one of the following values:
- STRICT
- LENIENT
- SILENT
ValidationStringency STRICT
--VERBOSITY
Control verbosity of logging.
The --VERBOSITY argument is an enumerated type (LogLevel), which can have one of the following values:
- ERROR
- WARNING
- INFO
- DEBUG
LogLevel INFO
--version
display the version number for this tool
boolean false
GATK version 4.2.4.0-SNAPSHOT built at Thu, 16 Dec 2021 11:57:48 -0800.
0 comments
Please sign in to leave a comment.