Append GenomicsDBimport

Answered

Follow

wbsimey

• November 18, 2021 18:26
• Edited

my version and error info:
a) GATK version 4.2.0.0
b) Exact command used:

gatk --java-options "-Xmx200g -Xms200g" \
GenomicsDBImport \
--genomicsdb-workspace-path Neo_96_GATK_dbase \
--batch-size 50 \
-L Chr2_nm_RagTag \
--sample-name-map Neo_96sample_map.txt \
--tmp-dir /tmp \
--reader-threads 16

c) Entire error log: A USER ERROR has occurred: Error creating GenomicsDB workspace: Neo_96_GATK_dbase already exists

How do I append and existing GenomcsDB?

I ran GenomicsDBImport for 96 genome samples and 29 chromosomes, but the run died after 2 months and completed 23 chromosomes and started on two others, but those two were terminated when the server went down. I deleted those  two DB directories and tried to run GenomicsDBImport on one chromosome at a time (in parallel), but I am getting the above "workspace already exists". Is there a way to append the database?

Thank you,

0

• 

  Genevieve Brandt (she/her)

  • November 19, 2021 19:05

  Hi wbsimey,

  I'm so sorry to hear about your issue when the server went down, that's such a bummer! There are two different ways you can use GenomicsDB without uploading the entire workspace at once.

  1. You can add samples incrementally to GenomicsDBImport with the command --genomicsdb-update-workspace-path. All samples must have the same intervals. Samples cannot be added at to the workspace the same time, so it is not possible to parallelize this method. We recommend always backing up the workspace before import in case there are issues like a server going down.
  2. You can make multiple GenomicsDB workspaces - one per interval, or any other way to break them up. Each workspace should contain all your samples. You can then run multiple GenomicsDBImport commands at once to parallelize the analysis. Following GenotypeGVCFs, the VCFs can be combined with MergeVCFs. (This is the method the Broad runs for our production pipelines).

  Hope this helps!

  Best,

  Genevieve

  0

  Comment actions Permalink
• 

  wbsimey

  • November 20, 2021 17:28

  Thank you for the quick response Genevieve Brandt. I will create separate databases for the remaining chromosomes.

  0

  Comment actions Permalink
• 

  Genevieve Brandt (she/her)

  • November 22, 2021 16:28

  You're welcome!

  0

  Comment actions Permalink

Please sign in to leave a comment.