Genome Analysis Toolkit

Variant Discovery in High-Throughput Sequencing Data

GATK process banner

Need Help?

Search our documentation

Community Forum

Hi, How can we help?

Developed in the Data Sciences Platform at the Broad Institute, the toolkit offers a wide variety of tools with a primary focus on variant discovery and genotyping. Its powerful processing engine and high-performance computing features make it capable of taking on projects of any size. Learn more

question about Picard AddOrReplaceReadGroups

Answered
0

3 comments

  • Avatar
    Genevieve Brandt (she/her)

    Hi Davy Deng,

    Could you include your AddOrReplaceReadGroups command line and entire program log? I would recommend keeping the validation stringency as STRICT so that you can fix the errors that come up. 

    Best,

    Genevieve

    0
    Comment actions Permalink
  • Avatar
    Davy Deng

    Hello Genevieve-Brandt-she-her,

    Thank you so much for your response. I ran this on terras so here is the command part of the script and the docker information:

    command <<<
    		java -jar /usr/gitc/picard.jar AddOrReplaceReadGroups \
    		I=${inputBam} \
    		O=${sampleName}.readgroupadded.bam \
    		RGID=1 \
    		RGLB=lib1 \
    		RGPL=illumina \
    		RGPU=unit1 \
            RGSM=${sampleName}
            VALIDATION_STRINGENCY=SILENT
    	>>>
    
    runtime {
    		docker: "broadinstitute/genomes-in-the-cloud:2.3.1-1500064817"
    		memory: "${memoryGb} GB"
    		cpu: "2"
    		disks: "local-disk ${diskSpaceGb} HDD"
    	}

    And here is the log file, i set the stringency to silent but the log file still recorded this error with CIGAR string:

    Picked up _JAVA_OPTIONS: -Djava.io.tmpdir=/cromwell_root/tmp.c04e388b
    [Sun Sep 26 21:53:53 UTC 2021] picard.sam.AddOrReplaceReadGroups INPUT=/cromwell_root/fc-b323d821-07e9-4d64-bde5-81d71367bcda/30X_70Purity_sorted.bam OUTPUT=1_30X_70purity_tumor.readgroupadded.bam RGID=1 RGLB=lib1 RGPL=illumina RGPU=unit1 RGSM=1_30X_70purity_tumor    VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json
    [Sun Sep 26 21:53:53 UTC 2021] Executing as root@7202468349a3 on Linux 5.4.120+ amd64; OpenJDK 64-Bit Server VM 1.8.0_111-8u111-b14-2~bpo8+1-b14; Picard version: 2.8.3-SNAPSHOT
    INFO	2021-09-26 21:53:53	AddOrReplaceReadGroups	Created read group ID=1 PL=illumina LB=lib1 SM=1_30X_70purity_tumor
    
    INFO	2021-09-26 21:54:06	AddOrReplaceReadGroups	Processed     1,000,000 records.  Elapsed time: 00:00:12s.  Time for last 1,000,000:   12s.  Last read position: chr1:3,375,986
    INFO	2021-09-26 21:54:17	AddOrReplaceReadGroups	Processed     2,000,000 records.  Elapsed time: 00:00:24s.  Time for last 1,000,000:   11s.  Last read position: chr1:6,741,944
    INFO	2021-09-26 21:54:30	AddOrReplaceReadGroups	Processed     3,000,000 records.  Elapsed time: 00:00:37s.  Time for last 1,000,000:   12s.  Last read position: chr1:10,101,832
    INFO	2021-09-26 21:54:43	AddOrReplaceReadGroups	Processed     4,000,000 records.  Elapsed time: 00:00:49s.  Time for last 1,000,000:   12s.  Last read position: chr1:13,469,279
    INFO	2021-09-26 21:54:55	AddOrReplaceReadGroups	Processed     5,000,000 records.  Elapsed time: 00:01:01s.  Time for last 1,000,000:   11s.  Last read position: chr1:16,830,201
    INFO	2021-09-26 21:55:08	AddOrReplaceReadGroups	Processed     6,000,000 records.  Elapsed time: 00:01:15s.  Time for last 1,000,000:   13s.  Last read position: chr1:20,198,305
    INFO	2021-09-26 21:55:20	AddOrReplaceReadGroups	Processed     7,000,000 records.  Elapsed time: 00:01:27s.  Time for last 1,000,000:   12s.  Last read position: chr1:23,558,761
    INFO	2021-09-26 21:55:32	AddOrReplaceReadGroups	Processed     8,000,000 records.  Elapsed time: 00:01:39s.  Time for last 1,000,000:   12s.  Last read position: chr1:26,909,108
    INFO	2021-09-26 21:55:45	AddOrReplaceReadGroups	Processed     9,000,000 records.  Elapsed time: 00:01:52s.  Time for last 1,000,000:   13s.  Last read position: chr1:30,274,430
    INFO	2021-09-26 21:55:58	AddOrReplaceReadGroups	Processed    10,000,000 records.  Elapsed time: 00:02:04s.  Time for last 1,000,000:   12s.  Last read position: chr1:33,637,168
    INFO	2021-09-26 21:56:10	AddOrReplaceReadGroups	Processed    11,000,000 records.  Elapsed time: 00:02:17s.  Time for last 1,000,000:   12s.  Last read position: chr1:37,007,616
    INFO	2021-09-26 21:56:22	AddOrReplaceReadGroups	Processed    12,000,000 records.  Elapsed time: 00:02:29s.  Time for last 1,000,000:   12s.  Last read position: chr1:40,377,876
    INFO	2021-09-26 21:56:34	AddOrReplaceReadGroups	Processed    13,000,000 records.  Elapsed time: 00:02:41s.  Time for last 1,000,000:   12s.  Last read position: chr1:43,747,264
    INFO	2021-09-26 21:56:48	AddOrReplaceReadGroups	Processed    14,000,000 records.  Elapsed time: 00:02:55s.  Time for last 1,000,000:   13s.  Last read position: chr1:47,111,040
    INFO	2021-09-26 21:57:00	AddOrReplaceReadGroups	Processed    15,000,000 records.  Elapsed time: 00:03:07s.  Time for last 1,000,000:   11s.  Last read position: chr1:50,484,571
    INFO	2021-09-26 21:57:13	AddOrReplaceReadGroups	Processed    16,000,000 records.  Elapsed time: 00:03:19s.  Time for last 1,000,000:   12s.  Last read position: chr1:53,849,699
    INFO	2021-09-26 21:57:25	AddOrReplaceReadGroups	Processed    17,000,000 records.  Elapsed time: 00:03:31s.  Time for last 1,000,000:   12s.  Last read position: chr1:57,211,176
    INFO	2021-09-26 21:57:37	AddOrReplaceReadGroups	Processed    18,000,000 records.  Elapsed time: 00:03:43s.  Time for last 1,000,000:   12s.  Last read position: chr1:60,576,562
    INFO	2021-09-26 21:57:51	AddOrReplaceReadGroups	Processed    19,000,000 records.  Elapsed time: 00:03:57s.  Time for last 1,000,000:   13s.  Last read position: chr1:63,939,497
    INFO	2021-09-26 21:58:03	AddOrReplaceReadGroups	Processed    20,000,000 records.  Elapsed time: 00:04:09s.  Time for last 1,000,000:   11s.  Last read position: chr1:67,303,887
    INFO	2021-09-26 21:58:16	AddOrReplaceReadGroups	Processed    21,000,000 records.  Elapsed time: 00:04:22s.  Time for last 1,000,000:   13s.  Last read position: chr1:70,670,289
    INFO	2021-09-26 21:58:28	AddOrReplaceReadGroups	Processed    22,000,000 records.  Elapsed time: 00:04:34s.  Time for last 1,000,000:   11s.  Last read position: chr1:74,039,518
    INFO	2021-09-26 21:58:40	AddOrReplaceReadGroups	Processed    23,000,000 records.  Elapsed time: 00:04:46s.  Time for last 1,000,000:   12s.  Last read position: chr1:77,408,879
    INFO	2021-09-26 21:58:54	AddOrReplaceReadGroups	Processed    24,000,000 records.  Elapsed time: 00:05:00s.  Time for last 1,000,000:   13s.  Last read position: chr1:80,772,595
    INFO	2021-09-26 21:59:06	AddOrReplaceReadGroups	Processed    25,000,000 records.  Elapsed time: 00:05:12s.  Time for last 1,000,000:   11s.  Last read position: chr1:84,143,101
    INFO	2021-09-26 21:59:18	AddOrReplaceReadGroups	Processed    26,000,000 records.  Elapsed time: 00:05:25s.  Time for last 1,000,000:   12s.  Last read position: chr1:87,513,324
    INFO	2021-09-26 21:59:31	AddOrReplaceReadGroups	Processed    27,000,000 records.  Elapsed time: 00:05:37s.  Time for last 1,000,000:   12s.  Last read position: chr1:90,882,531
    INFO	2021-09-26 21:59:43	AddOrReplaceReadGroups	Processed    28,000,000 records.  Elapsed time: 00:05:49s.  Time for last 1,000,000:   12s.  Last read position: chr1:94,241,211
    INFO	2021-09-26 21:59:57	AddOrReplaceReadGroups	Processed    29,000,000 records.  Elapsed time: 00:06:03s.  Time for last 1,000,000:   13s.  Last read position: chr1:97,611,281
    INFO	2021-09-26 22:00:09	AddOrReplaceReadGroups	Processed    30,000,000 records.  Elapsed time: 00:06:15s.  Time for last 1,000,000:   12s.  Last read position: chr1:100,979,853
    INFO	2021-09-26 22:00:21	AddOrReplaceReadGroups	Processed    31,000,000 records.  Elapsed time: 00:06:28s.  Time for last 1,000,000:   12s.  Last read position: chr1:104,339,804
    INFO	2021-09-26 22:00:34	AddOrReplaceReadGroups	Processed    32,000,000 records.  Elapsed time: 00:06:40s.  Time for last 1,000,000:   12s.  Last read position: chr1:107,711,743
    INFO	2021-09-26 22:00:47	AddOrReplaceReadGroups	Processed    33,000,000 records.  Elapsed time: 00:06:53s.  Time for last 1,000,000:   12s.  Last read position: chr1:111,072,736
    INFO	2021-09-26 22:01:00	AddOrReplaceReadGroups	Processed    34,000,000 records.  Elapsed time: 00:07:07s.  Time for last 1,000,000:   13s.  Last read position: chr1:114,441,526
    INFO	2021-09-26 22:01:12	AddOrReplaceReadGroups	Processed    35,000,000 records.  Elapsed time: 00:07:19s.  Time for last 1,000,000:   12s.  Last read position: chr1:117,807,947
    INFO	2021-09-26 22:01:24	AddOrReplaceReadGroups	Processed    36,000,000 records.  Elapsed time: 00:07:31s.  Time for last 1,000,000:   12s.  Last read position: chr1:121,175,572
    INFO	2021-09-26 22:01:30	AddOrReplaceReadGroups	Processed    37,000,000 records.  Elapsed time: 00:07:36s.  Time for last 1,000,000:    5s.  Last read position: chr1:124,542,412
    INFO	2021-09-26 22:01:34	AddOrReplaceReadGroups	Processed    38,000,000 records.  Elapsed time: 00:07:41s.  Time for last 1,000,000:    4s.  Last read position: chr1:127,904,941
    INFO	2021-09-26 22:01:39	AddOrReplaceReadGroups	Processed    39,000,000 records.  Elapsed time: 00:07:45s.  Time for last 1,000,000:    4s.  Last read position: chr1:131,272,359
    INFO	2021-09-26 22:01:43	AddOrReplaceReadGroups	Processed    40,000,000 records.  Elapsed time: 00:07:50s.  Time for last 1,000,000:    4s.  Last read position: chr1:134,643,267
    INFO	2021-09-26 22:01:49	AddOrReplaceReadGroups	Processed    41,000,000 records.  Elapsed time: 00:07:56s.  Time for last 1,000,000:    5s.  Last read position: chr1:135,791,721
    INFO	2021-09-26 22:01:54	AddOrReplaceReadGroups	Processed    42,000,000 records.  Elapsed time: 00:08:01s.  Time for last 1,000,000:    5s.  Last read position: chr1:136,676,946
    INFO	2021-09-26 22:01:59	AddOrReplaceReadGroups	Processed    43,000,000 records.  Elapsed time: 00:08:05s.  Time for last 1,000,000:    4s.  Last read position: chr1:137,563,738
    INFO	2021-09-26 22:02:03	AddOrReplaceReadGroups	Processed    44,000,000 records.  Elapsed time: 00:08:10s.  Time for last 1,000,000:    4s.  Last read position: chr1:138,447,751
    INFO	2021-09-26 22:02:08	AddOrReplaceReadGroups	Processed    45,000,000 records.  Elapsed time: 00:08:14s.  Time for last 1,000,000:    4s.  Last read position: chr1:139,332,695
    INFO	2021-09-26 22:02:12	AddOrReplaceReadGroups	Processed    46,000,000 records.  Elapsed time: 00:08:19s.  Time for last 1,000,000:    4s.  Last read position: chr1:140,217,781
    INFO	2021-09-26 22:02:17	AddOrReplaceReadGroups	Processed    47,000,000 records.  Elapsed time: 00:08:23s.  Time for last 1,000,000:    4s.  Last read position: chr1:141,103,509
    INFO	2021-09-26 22:02:21	AddOrReplaceReadGroups	Processed    48,000,000 records.  Elapsed time: 00:08:28s.  Time for last 1,000,000:    4s.  Last read position: chr1:141,989,409
    INFO	2021-09-26 22:02:29	AddOrReplaceReadGroups	Processed    49,000,000 records.  Elapsed time: 00:08:35s.  Time for last 1,000,000:    7s.  Last read position: chr1:142,874,702
    INFO	2021-09-26 22:02:38	AddOrReplaceReadGroups	Processed    50,000,000 records.  Elapsed time: 00:08:45s.  Time for last 1,000,000:    9s.  Last read position: chr1:143,759,544
    INFO	2021-09-26 22:02:48	AddOrReplaceReadGroups	Processed    51,000,000 records.  Elapsed time: 00:08:54s.  Time for last 1,000,000:    9s.  Last read position: chr1:144,645,536
    INFO	2021-09-26 22:03:00	AddOrReplaceReadGroups	Processed    52,000,000 records.  Elapsed time: 00:09:06s.  Time for last 1,000,000:   12s.  Last read position: chr1:145,531,534
    INFO	2021-09-26 22:03:11	AddOrReplaceReadGroups	Processed    53,000,000 records.  Elapsed time: 00:09:17s.  Time for last 1,000,000:   10s.  Last read position: chr1:146,417,530
    INFO	2021-09-26 22:03:22	AddOrReplaceReadGroups	Processed    54,000,000 records.  Elapsed time: 00:09:29s.  Time for last 1,000,000:   11s.  Last read position: chr1:147,302,684
    INFO	2021-09-26 22:03:33	AddOrReplaceReadGroups	Processed    55,000,000 records.  Elapsed time: 00:09:40s.  Time for last 1,000,000:   10s.  Last read position: chr1:148,190,248
    INFO	2021-09-26 22:03:43	AddOrReplaceReadGroups	Processed    56,000,000 records.  Elapsed time: 00:09:50s.  Time for last 1,000,000:   10s.  Last read position: chr1:149,078,571
    INFO	2021-09-26 22:03:56	AddOrReplaceReadGroups	Processed    57,000,000 records.  Elapsed time: 00:10:02s.  Time for last 1,000,000:   12s.  Last read position: chr1:149,965,098
    INFO	2021-09-26 22:04:08	AddOrReplaceReadGroups	Processed    58,000,000 records.  Elapsed time: 00:10:14s.  Time for last 1,000,000:   12s.  Last read position: chr1:150,851,055
    INFO	2021-09-26 22:04:20	AddOrReplaceReadGroups	Processed    59,000,000 records.  Elapsed time: 00:10:26s.  Time for last 1,000,000:   12s.  Last read position: chr1:151,735,301
    INFO	2021-09-26 22:04:32	AddOrReplaceReadGroups	Processed    60,000,000 records.  Elapsed time: 00:10:39s.  Time for last 1,000,000:   12s.  Last read position: chr1:152,622,819
    INFO	2021-09-26 22:04:44	AddOrReplaceReadGroups	Processed    61,000,000 records.  Elapsed time: 00:10:51s.  Time for last 1,000,000:   12s.  Last read position: chr1:153,509,263
    INFO	2021-09-26 22:04:57	AddOrReplaceReadGroups	Processed    62,000,000 records.  Elapsed time: 00:11:04s.  Time for last 1,000,000:   12s.  Last read position: chr1:154,395,518
    [Sun Sep 26 22:04:59 UTC 2021] picard.sam.AddOrReplaceReadGroups done. Elapsed time: 11.10 minutes.
    Runtime.totalMemory()=113246208
    To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
    Exception in thread "main" htsjdk.samtools.SAMFormatException: SAM validation error: WARNING: Read name FC:0:0:368:54711, No M or N operator between pair of D operators in CIGAR
    	at htsjdk.samtools.SAMUtils.processValidationErrors(SAMUtils.java:448)
    	at htsjdk.samtools.BAMRecord.getCigar(BAMRecord.java:252)
    	at htsjdk.samtools.SAMRecord.getAlignmentEnd(SAMRecord.java:600)
    	at htsjdk.samtools.SAMRecord.computeIndexingBin(SAMRecord.java:1531)
    	at htsjdk.samtools.SAMRecord.isValid(SAMRecord.java:2038)
    	at htsjdk.samtools.BAMFileReader$BAMFileIterator.advance(BAMFileReader.java:664)
    	at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:650)
    	at htsjdk.samtools.BAMFileReader$BAMFileIterator.next(BAMFileReader.java:620)
    	at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:569)
    	at htsjdk.samtools.SamReader$AssertingIterator.next(SamReader.java:543)
    	at picard.sam.AddOrReplaceReadGroups.doWork(AddOrReplaceReadGroups.java:140)
    	at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:205)
    	at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:94)
    	at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:104)

    Thank you very much! 

    0
    Comment actions Permalink
  • Avatar
    Genevieve Brandt (she/her)

    Thanks for sharing that! The AddOrReplaceReadGroups command was not successful, you have an error message you need to address:

    Exception in thread "main" htsjdk.samtools.SAMFormatException: SAM validation error: WARNING: Read name FC:0:0:368:54711, No M or N operator between pair of D operators in CIGAR
    	at htsjdk.samtools.SAMUtils.processValidationErrors(SAMUtils.java:448)

    There is an error in the read named FC:0:0:368:54711, an issue with the CIGAR string. You'll want to resolve that issue before continuing. We have an article that covers the details of CIGAR strings here: https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats

    This indicates a problem when you were creating this BAM file.

    Hope this helps!

    Genevieve

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk