MarkDuplicates using more cpus than allowed
I'm running the GATK RNAseq Best Practices workflow locally on our server. I'm only allowed to use 60 threads. However, during Picard MarkDuplicates, the tool uses all available CPU resources, as issued here.
Following this page, I already use the argument to constrain the JavaGC thread usage:
Is there a way to limit the overall workflow or specifically the offending tool to use at most that many cpus? I'm willing to even run a resource management program upstream java/cromwell call, i.e., cpulimit.
Does anyone have suggestions?
What about creating a nesting Docker? An external container and run the commands inside of it, which will call downstream containers as needed.
I also already used renice to give the process a less priority over the process scheduling sys.
Just to let others know that writing a script to run cpulimit on java worked. It controlled the cpu usage of all GATK tools invoked downstream by the workflow without problems.
I'm glad you were able to solve your problem. Thank you for posting your solution here for other users!
Please sign in to leave a comment.