Summary
Class to downsample a SAM/BAM file based on the position of the read in a flowcell. As with DownsampleSam, all the reads with the same queryname are either kept or dropped as a unit.Details
The downsampling is _not_ random (and there is no random seed). It is deterministically determined by the position of each read within its tile. Specifically, it draws an ellipse that covers a FRACTION of the total tile's area and of all the edges of the tile. It uses this area to determine whether to keep or drop the record. Since reads with the same name have the same position (mates, secondary and supplemental alignments), the decision will be the same for all of them. The main concern of this downsampling method is that due to "optical duplicates" downsampling randomly can create a result that has a different optical duplicate rate, and therefore a different estimated library size (when running MarkDuplicates). This method keeps (physically) close read together, so that (except for reads near the boundary of the circle) optical duplicates are kept or dropped as a group. By default the program expects the read names to have 5 or 7 fields separated by colons (:) and it takes the last two to indicate the x and y coordinates of the reads within the tile whence it was sequenced. See DEFAULT_READ_NAME_REGEX for more detail. The program traverses the INPUT twice: first to find out the size of each of the tiles, and next to perform the downsampling. Downsampling invalidates the duplicate flag because duplicate reads before downsampling may not all remain duplicated after downsampling. Thus, the default setting also removes the duplicate information. Example java -jar picard.jar PositionBasedDownsampleSam \ I=input.bam \ O=downsampled.bam \ FRACTION=0.1 Caveats Note 1: This method is technology and read-name dependent. If the read-names do not have coordinate information embedded in them, or if your BAM contains reads from multiple technologies (flowcell versions, sequencing machines) this will not work properly. It has been designed to work with Illumina technology and reads-names. Consider modifying {@link #READ_NAME_REGEX} in other cases. Note 2: The code has been designed to simulate, as accurately as possible, sequencing less, not for getting an exact downsampled fraction (Use {@link DownsampleSam} for that.) In particular, since the reads may be distributed non-evenly within the lanes/tiles, the resulting downsampling percentage will not be accurately determined by the input argument FRACTION. Note 3:Consider running {@link MarkDuplicates} after downsampling in order to "expose" the duplicates whose representative has been downsampled away. Note 4:The downsampling assumes a uniform distribution of reads in the flowcell. Input already downsampled with PositionBasedDownsampleSam violates this assumption. To guard against such input, PositionBasedDownsampleSam always places a PG record in the header of its output, and aborts whenever it finds such a PG record in its input.Category Read Data Manipulation
Overview
Summary
Class to downsample a SAM/BAM file based on the position of the read in a flowcell. As with DownsampleSam, all the reads with the same queryname are either kept or dropped as a unit.Details
The downsampling is not random (and there is no random seed). It is deterministically determined by the position of each read within its tile. Specifically, it draws an ellipse that covers a #FRACTION of the total tile's area and of all the edges of the tile. It uses this area to determine whether to keep or drop the record. Since reads with the same name have the same position (mates, secondary and supplemental alignments), the decision will be the same for all of them. The main concern of this downsampling method is that due to "optical duplicates" downsampling randomly can create a result that has a different optical duplicate rate, and therefore a different estimated library size (when running MarkDuplicates). This method keeps (physically) close read together, so that (except for reads near the boundary of the circle) optical duplicates are kept or dropped as a group. By default the program expects the read names to have 5 or 7 fields separated by colons (:), and it takes the last two to indicate the x and y coordinates of the reads within the tile whence it was sequenced. See ReadNameParser#DEFAULT_READ_NAME_REGEX for more detail. The program traverses the #INPUT twice: first to find out the size of each of the tiles, and next to perform the downsampling. Downsampling invalidates the duplicate flag because duplicate reads before downsampling may not all remain duplicated after downsampling. Thus, the default setting also removes the duplicate information.Example
java -jar picard.jar PositionBasedDownsampleSam \ I=input.bam \ O=downsampled.bam \ FRACTION=0.1
Caveats
- This method is technology and read-name dependent. If the read-names do not have coordinate information embedded in them, or if your BAM contains reads from multiple technologies (flowcell versions, sequencing machines). this will not work properly. It has been designed to work with Illumina technology and reads-names. Consider modifying #READ_NAME_REGEX in other cases.
- The code has been designed to simulate, as accurately as possible, sequencing less, not for getting an exact downsampled fraction (Use DownsampleSam for that.) In particular, since the reads may be distributed non-evenly within the lanes/tiles, the resulting downsampling percentage will not be accurately determined by the input argument #FRACTION.
- Consider running MarkDuplicates after downsampling in order to "expose" the duplicates whose representative has been downsampled away.
- The downsampling assumes a uniform distribution of reads in the flowcell. Input already downsampled with PositionBasedDownsampleSam violates this assumption. To guard against such input, PositionBasedDownsampleSam always places a PG record in the header of its output, and aborts whenever it finds such a PG record in its input.
PositionBasedDownsampleSam (Picard) specific arguments
This table summarizes the command-line arguments that are specific to this tool. For more details on each argument, see the list further down below the table or click on an argument name to jump directly to that entry in the list.
Argument name(s) | Default value | Summary | |
---|---|---|---|
Required Arguments | |||
--FRACTION -F |
The (approximate) fraction of reads to be kept, between 0 and 1. | ||
--INPUT -I |
The input SAM or BAM file to downsample. | ||
--OUTPUT -O |
The output, downsampled, SAM or BAM file. | ||
Optional Tool Arguments | |||
--ALLOW_MULTIPLE_DOWNSAMPLING_DESPITE_WARNINGS |
false | Allow downsampling again despite this being a bad idea with possibly unexpected results. | |
--arguments_file |
read one or more arguments files and add them to the command line | ||
--help -h |
false | display the help message | |
--READ_NAME_REGEX |
Use these regular expressions to parse read names in the input SAM file. Read names are parsed to extract three variables: tile/region, x coordinate and y coordinate. The x and y coordinates are used to determine the downsample decision. Set this option to null to disable optical duplicate detection, e.g. for RNA-seq The regular expression should contain three capture groups for the three variables, in order. It must match the entire read name. Note that if the default regex is specified, a regex match is not actually done, but instead the read name is split on colons (:). For 5 element names, the 3rd, 4th and 5th elements are assumed to be tile, x and y values. For 7 element names (CASAVA 1.8), the 5th, 6th, and 7th elements are assumed to be tile, x and y values. | ||
--REMOVE_DUPLICATE_INFORMATION |
true | Determines whether the duplicate tag should be reset since the downsampling requires re-marking duplicates. | |
--STOP_AFTER |
Stop after processing N reads, mainly for debugging. | ||
--version |
false | display the version number for this tool | |
Optional Common Arguments | |||
--COMPRESSION_LEVEL |
5 | Compression level for all compressed files created (e.g. BAM and VCF). | |
--CREATE_INDEX |
false | Whether to create an index when writing VCF or coordinate sorted BAM output. | |
--CREATE_MD5_FILE |
false | Whether to create an MD5 digest for any BAM or FASTQ files created. | |
--GA4GH_CLIENT_SECRETS |
client_secrets.json | Google Genomics API client_secrets.json file path. | |
--MAX_RECORDS_IN_RAM |
500000 | When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed. | |
--QUIET |
false | Whether to suppress job-summary info on System.err. | |
--REFERENCE_SEQUENCE -R |
Reference sequence file. | ||
--TMP_DIR |
One or more directories with space available to be used by this program for temporary storage of working files | ||
--USE_JDK_DEFLATER -use_jdk_deflater |
false | Use the JDK Deflater instead of the Intel Deflater for writing compressed output | |
--USE_JDK_INFLATER -use_jdk_inflater |
false | Use the JDK Inflater instead of the Intel Inflater for reading compressed input | |
--VALIDATION_STRINGENCY |
STRICT | Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded. | |
--VERBOSITY |
INFO | Control verbosity of logging. | |
Advanced Arguments | |||
--showHidden |
false | display hidden arguments |
Argument details
Arguments in this list are specific to this tool. Keep in mind that other arguments are available that are shared with other tools (e.g. command-line GATK arguments); see Inherited arguments above.
--ALLOW_MULTIPLE_DOWNSAMPLING_DESPITE_WARNINGS
Allow downsampling again despite this being a bad idea with possibly unexpected results.
boolean false
--arguments_file
read one or more arguments files and add them to the command line
List[File] []
--COMPRESSION_LEVEL
Compression level for all compressed files created (e.g. BAM and VCF).
int 5 [ [ -∞ ∞ ] ]
--CREATE_INDEX
Whether to create an index when writing VCF or coordinate sorted BAM output.
Boolean false
--CREATE_MD5_FILE
Whether to create an MD5 digest for any BAM or FASTQ files created.
boolean false
--FRACTION / -F
The (approximate) fraction of reads to be kept, between 0 and 1.
R Double null
--GA4GH_CLIENT_SECRETS
Google Genomics API client_secrets.json file path.
String client_secrets.json
--help / -h
display the help message
boolean false
--INPUT / -I
The input SAM or BAM file to downsample.
R File null
--MAX_RECORDS_IN_RAM
When writing files that need to be sorted, this will specify the number of records stored in RAM before spilling to disk. Increasing this number reduces the number of file handles needed to sort the file, and increases the amount of RAM needed.
Integer 500000 [ [ -∞ ∞ ] ]
--OUTPUT / -O
The output, downsampled, SAM or BAM file.
R File null
--QUIET
Whether to suppress job-summary info on System.err.
Boolean false
--READ_NAME_REGEX
Use these regular expressions to parse read names in the input SAM file. Read names are parsed to extract three variables: tile/region, x coordinate and y coordinate. The x and y coordinates are used to determine the downsample decision. Set this option to null to disable optical duplicate detection, e.g. for RNA-seq The regular expression should contain three capture groups for the three variables, in order. It must match the entire read name. Note that if the default regex is specified, a regex match is not actually done, but instead the read name is split on colons (:). For 5 element names, the 3rd, 4th and 5th elements are assumed to be tile, x and y values. For 7 element names (CASAVA 1.8), the 5th, 6th, and 7th elements are assumed to be tile, x and y values.
String
--REFERENCE_SEQUENCE / -R
Reference sequence file.
File null
--REMOVE_DUPLICATE_INFORMATION
Determines whether the duplicate tag should be reset since the downsampling requires re-marking duplicates.
boolean true
--showHidden / -showHidden
display hidden arguments
boolean false
--STOP_AFTER
Stop after processing N reads, mainly for debugging.
Long null
--TMP_DIR
One or more directories with space available to be used by this program for temporary storage of working files
List[File] []
--USE_JDK_DEFLATER / -use_jdk_deflater
Use the JDK Deflater instead of the Intel Deflater for writing compressed output
Boolean false
--USE_JDK_INFLATER / -use_jdk_inflater
Use the JDK Inflater instead of the Intel Inflater for reading compressed input
Boolean false
--VALIDATION_STRINGENCY
Validation stringency for all SAM files read by this program. Setting stringency to SILENT can improve performance when processing a BAM file in which variable-length data (read, qualities, tags) do not otherwise need to be decoded.
The --VALIDATION_STRINGENCY argument is an enumerated type (ValidationStringency), which can have one of the following values:
- STRICT
- LENIENT
- SILENT
ValidationStringency STRICT
--VERBOSITY
Control verbosity of logging.
The --VERBOSITY argument is an enumerated type (LogLevel), which can have one of the following values:
- ERROR
- WARNING
- INFO
- DEBUG
LogLevel INFO
--version
display the version number for this tool
boolean false
GATK version 4.2.3.0-SNAPSHOT built at Mon, 8 Nov 2021 14:59:32 -0800.
0 comments
Please sign in to leave a comment.