MarkDuplicates using more cpus than allowed

Follow

ThyagoLC

• September 16, 2021 11:30
• Edited

Hello,

I'm running the GATK RNAseq Best Practices workflow locally on our server. I'm only allowed to use 60 threads. However, during Picard MarkDuplicates, the tool uses all available CPU resources, as issued here. 

Following this page, I already use the argument to constrain the JavaGC thread usage:

-XX:ConcGCThreads=60

Is there a way to limit the overall workflow or specifically the offending tool to use at most that many cpus?  I'm willing to even run a resource management program upstream java/cromwell call, i.e., cpulimit.

Does anyone have suggestions?

What about creating a nesting Docker? An external container and run the commands inside of it, which will call downstream containers as needed. 

I also already used renice to give the process a less priority over the process scheduling sys.

Thank you.

1

• 

  ThyagoLC

  • September 17, 2021 11:30

  Just to let others know that writing a script to run cpulimit on java worked. It controlled the cpu usage of all GATK tools invoked downstream by the workflow without problems.

  Thanks

  0

  Comment actions Permalink
• 

  Pamela Bretscher

  • September 17, 2021 13:10

  Hi ThyagoLC,

  I'm glad you were able to solve your problem. Thank you for posting your solution here for other users!

  Kind regards,

  Pamela

  0

  Comment actions Permalink

Please sign in to leave a comment.