PathSeqPipelineSpark multiple cpus

Follow

Caitlin Newman

• September 12, 2023 16:39

Hi!

I am trying to use PathSeqPipelineSpark but when I run htop in terminal in the cluster it appears to only be using one cpu? Is there a way to tell the code to look for multiple cpus? 

0

• 

  Gökalp Çelik

  • September 13, 2023 10:22

  Hi Caitlin Newman

  Can you share your command line? Depending on the spark-master certain rules may apply on your cluster. Local execution requires

  --spark-master local[number of threads] 

  to ensure parallel execution. 

  1

  Comment actions Permalink

Please sign in to leave a comment.