Cromwell: can't find file ID
I am using Cromwell to test a CWL implementation of the GATK somatic CNV pipeline. I am currently getting the following error (full log at the end):
`` Cannot find a tool or workflow with ID file:///var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl in file /var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl ``
The ID's are the same except for the first starts with "file://". I am not very familiar with Cromwell so I'm not sure how to proceed in resolving this error. Any help would be appreciated.
Thank you
Stephanie Hoyt
Full log:
``
[2020-12-03 15:20:24,82] [info] Running with database db.url = jdbc:hsqldb:mem:28b7f7a3-d261-42ee-b941-e329cdcf46ba;shutdown=false;hsqldb.tx=mvcc
[2020-12-03 15:20:29,97] [info] Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000
[2020-12-03 15:20:29,98] [info] [RenameWorkflowOptionsInMetadata] 100%
[2020-12-03 15:20:30,03] [info] Running with database db.url = jdbc:hsqldb:mem:ae8f4f13-d319-4439-8681-b684e13ca6ac;shutdown=false;hsqldb.tx=mvcc
[2020-12-03 15:20:30,23] [info] Slf4jLogger started
[2020-12-03 15:20:30,25] [info] Pre Processing Workflow...
[2020-12-03 15:20:30,27] [info] Pre Processing Inputs...
[2020-12-03 15:20:30,46] [info] Workflow heartbeat configuration:
{
"cromwellId" : "cromid-d8084bc",
"heartbeatInterval" : "2 minutes",
"ttl" : "10 minutes",
"writeBatchSize" : 10000,
"writeThreshold" : 10000
}
[2020-12-03 15:20:30,48] [info] Metadata summary refreshing every 2 seconds.
[2020-12-03 15:20:30,50] [info] KvWriteActor configured to flush with batch size 200 and process rate 5 seconds.
[2020-12-03 15:20:30,50] [info] WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds.
[2020-12-03 15:20:30,50] [info] CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds.
[2020-12-03 15:20:30,78] [info] JobExecutionTokenDispenser - Distribution rate: 50 per 1 seconds.
[2020-12-03 15:20:30,80] [info] SingleWorkflowRunnerActor: Submitting workflow
[2020-12-03 15:20:30,82] [info] cwl (Unspecified version) workflow 837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a submitted
[2020-12-03 15:20:30,82] [info] SingleWorkflowRunnerActor: Workflow submitted [38;5;2m837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a[0m
[2020-12-03 15:20:30,82] [info] 1 new workflows fetched
[2020-12-03 15:20:30,82] [info] WorkflowManagerActor Starting workflow [38;5;2m837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a[0m
[2020-12-03 15:20:30,83] [[38;5;220mwarn[0m] SingleWorkflowRunnerActor: received unexpected message: Done in state RunningSwraData
[2020-12-03 15:20:30,84] [[38;5;220mwarn[0m] Couldn't find a suitable DSN, defaulting to a Noop one.
[2020-12-03 15:20:30,84] [info] Using noop to send events.
[2020-12-03 15:20:30,85] [info] WorkflowManagerActor Successfully started WorkflowActor-837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a
[2020-12-03 15:20:30,85] [info] Retrieved 1 workflows from the WorkflowStoreActor
[2020-12-03 15:20:30,85] [info] WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes.
[2020-12-03 15:20:30,86] [info] MaterializeWorkflowDescriptorActor [[38;5;2m837d7e7d[0m]: Parsing workflow as CWL v1.0
[2020-12-03 15:20:30,91] [info] Pre-Processing /var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl
[2020-12-03 15:20:32,24] [[38;5;1merror[0m] WorkflowManagerActor Workflow 837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a failed (during MaterializingWorkflowDescriptorState): cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor$$anon$1: Workflow input processing failed:
Cannot find a tool or workflow with ID file:///var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl in file /var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl
at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.cromwell$engine$workflow$lifecycle$materialization$MaterializeWorkflowDescriptorActor$$workflowInitializationFailed(MaterializeWorkflowDescriptorActor.scala:200)
at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor$$anonfun$2.applyOrElse(MaterializeWorkflowDescriptorActor.scala:170)
at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor$$anonfun$2.applyOrElse(MaterializeWorkflowDescriptorActor.scala:165)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
at akka.actor.FSM.processEvent(FSM.scala:665)
at akka.actor.FSM.processEvent$(FSM.scala:662)
at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.akka$actor$LoggingFSM$$super$processEvent(MaterializeWorkflowDescriptorActor.scala:123)
at akka.actor.LoggingFSM.processEvent(FSM.scala:801)
at akka.actor.LoggingFSM.processEvent$(FSM.scala:783)
at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.processEvent(MaterializeWorkflowDescriptorActor.scala:123)
at akka.actor.FSM.akka$actor$FSM$$processMsg(FSM.scala:659)
at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:653)
at akka.actor.Actor.aroundReceive(Actor.scala:514)
at akka.actor.Actor.aroundReceive$(Actor.scala:512)
at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.aroundReceive(MaterializeWorkflowDescriptorActor.scala:123)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
at akka.actor.ActorCell.invoke(ActorCell.scala:496)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
[2020-12-03 15:20:32,24] [info] WorkflowManagerActor WorkflowActor-837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a is in a terminal state: WorkflowFailedState
[2020-12-03 15:20:37,26] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.
[2020-12-03 15:20:40,51] [info] Workflow polling stopped
[2020-12-03 15:20:40,52] [info] Shutting down WorkflowStoreActor - Timeout = 5 seconds
[2020-12-03 15:20:40,52] [info] Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds
[2020-12-03 15:20:40,52] [info] Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds
[2020-12-03 15:20:40,52] [info] Aborting all running workflows.
[2020-12-03 15:20:40,52] [info] JobExecutionTokenDispenser stopped
[2020-12-03 15:20:40,52] [info] WorkflowStoreActor stopped
[2020-12-03 15:20:40,53] [info] WorkflowLogCopyRouter stopped
[2020-12-03 15:20:40,53] [info] Shutting down WorkflowManagerActor - Timeout = 3600 seconds
[2020-12-03 15:20:40,53] [info] WorkflowManagerActor All workflows finished
[2020-12-03 15:20:40,53] [info] WorkflowManagerActor stopped
[2020-12-03 15:20:40,53] [info] Connection pools shut down
[2020-12-03 15:20:40,53] [info] Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds
[2020-12-03 15:20:40,53] [info] Shutting down JobStoreActor - Timeout = 1800 seconds
[2020-12-03 15:20:40,53] [info] Shutting down CallCacheWriteActor - Timeout = 1800 seconds
[2020-12-03 15:20:40,53] [info] SubWorkflowStoreActor stopped
[2020-12-03 15:20:40,53] [info] Shutting down ServiceRegistryActor - Timeout = 1800 seconds
[2020-12-03 15:20:40,53] [info] Shutting down DockerHashActor - Timeout = 1800 seconds
[2020-12-03 15:20:40,53] [info] JobStoreActor stopped
[2020-12-03 15:20:40,53] [info] Shutting down IoProxy - Timeout = 1800 seconds
[2020-12-03 15:20:40,53] [info] CallCacheWriteActor Shutting down: 0 queued messages to process
[2020-12-03 15:20:40,53] [info] WriteMetadataActor Shutting down: 0 queued messages to process
[2020-12-03 15:20:40,53] [info] KvWriteActor Shutting down: 0 queued messages to process
[2020-12-03 15:20:40,53] [info] CallCacheWriteActor stopped
[2020-12-03 15:20:40,53] [info] DockerHashActor stopped
[2020-12-03 15:20:40,53] [info] IoProxy stopped
[2020-12-03 15:20:40,53] [info] ServiceRegistryActor stopped
[2020-12-03 15:20:40,54] [info] Database closed
[2020-12-03 15:20:40,54] [info] Stream materializer shut down
Workflow 837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a transitioned to state Failed
[2020-12-03 15:20:40,57] [info] Automatic shutdown of the async connection
[2020-12-03 15:20:40,57] [info] Gracefully shutdown sentry threads.
[2020-12-03 15:20:40,57] [info] Shutdown finished.
``
-
Hi Stephanie,
We mainly write our workflows in WDL, if you need help running a CWL workflow with cromwell Bioinformatics StackExchange and cromwellhq.slack.com are the best resources for those questions.
Please sign in to leave a comment.
1 comment