CNNScoreVariants/ValueError/version 4.1.9.0 and 4.2.0.0
Planned#1) version info.
a-1) GATK version used: 4.1.9.0. and 4.2.0.0. (I have run both 4.1.9.0. and 4.2.0.0. version.
a-2) Python version used: 3.6.10
a-3) java version used: 1.8.0_282
#2) code
b) Exact command used: java -Xmx8g -jar gatk-package-4.1.9.0-local.jar CNNScoreVariants -V ${1}.Haplotype.vcf -R Homo_sapiens_assembly38.fasta -O ${1}_CNN_1D_trained_scored.vcf --architecture ${1}_1D_CNN_model.json --weights ${1}_1D_CNN_model.hd5
#3) eroor message
c) Entire error log:
09:22:24.194 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/BiO/Install/gatk-4.1.9.0/gatk-package-4.1.9.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
Mar 30, 2021 9:22:24 AM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
09:22:24.523 INFO CNNScoreVariants - ------------------------------------------------------------
09:22:24.524 INFO CNNScoreVariants - The Genome Analysis Toolkit (GATK) v4.1.9.0
09:22:24.524 INFO CNNScoreVariants - For support and documentation go to https://software.broadinstitute.org/gatk/
09:22:24.524 INFO CNNScoreVariants - Executing as kmj1019@ajoubig on Linux v4.15.0-88-generic amd64
09:22:24.524 INFO CNNScoreVariants - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_282-8u282-b08-0ubuntu1~16.04-b08
09:22:24.524 INFO CNNScoreVariants - Start Date/Time: March 30, 2021 9:22:23 AM KST
09:22:24.524 INFO CNNScoreVariants - ------------------------------------------------------------
09:22:24.524 INFO CNNScoreVariants - ------------------------------------------------------------
09:22:24.525 INFO CNNScoreVariants - HTSJDK Version: 2.23.0
09:22:24.525 INFO CNNScoreVariants - Picard Version: 2.23.3
09:22:24.525 INFO CNNScoreVariants - HTSJDK Defaults.COMPRESSION_LEVEL : 2
09:22:24.525 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
09:22:24.525 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
09:22:24.525 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
09:22:24.525 INFO CNNScoreVariants - Deflater: IntelDeflater
09:22:24.525 INFO CNNScoreVariants - Inflater: IntelInflater
09:22:24.525 INFO CNNScoreVariants - GCS max retries/reopens: 20
09:22:24.525 INFO CNNScoreVariants - Requester pays: disabled
09:22:24.525 INFO CNNScoreVariants - Initializing engine
09:22:25.090 INFO FeatureManager - Using codec VCFCodec to read file file:///BiO2/research/collab2/Korea/ctDNA/WES_ming/germline_GATK/HaplotypeCaller/CRC/CRC_002_C/CRC_002_C.markdup.Haplotype.vcf
09:22:25.479 INFO CNNScoreVariants - Done initializing engine
09:22:25.480 INFO NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/BiO/Install/gatk-4.1.9.0/gatk-package-4.1.9.0-local.jar!/com/intel/gkl/native/libgkl_utils.so
09:22:27.488 INFO CNNScoreVariants - Using key:CNN_1D for CNN architecture:/BiO2/research/collab2/Korea/ctDNA/WES_ming/germline_GATK/CNN/tensor/CRC/CRC_002_C/CRC_002_C_1D/CRC_002_C_1D_CNN_model.json and weights:/BiO2/research/collab2/Korea/ctDNA/WES_ming/germline_GATK/CNN/tensor/CRC/CRC_002_C/CRC_002_C_1D/CRC_002_C_1D_CNN_model.hd5
09:22:28.365 INFO CNNScoreVariants - Done scoring variants with CNN.
09:22:28.366 INFO CNNScoreVariants - Shutting down engine
[March 30, 2021 9:22:28 AM KST] org.broadinstitute.hellbender.tools.walkers.vqsr.CNNScoreVariants done. Elapsed time: 0.07 minutes.
Runtime.totalMemory()=1982332928
org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException: A nack was received from the Python process (most likely caused by a raised exception caused by): nkm received
: Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/kmj1019/anaconda3/envs/gatk/lib/python3.6/site-packages/vqsr_cnn/vqsr_cnn/models.py", line 26, in start_session_get_args_and_model
return args_and_model_from_semantics(semantics_json, weights_hd5, tensor_type)
File "/home/kmj1019/anaconda3/envs/gatk/lib/python3.6/site-packages/vqsr_cnn/vqsr_cnn/models.py", line 33, in args_and_model_from_semantics
model = set_args_and_get_model_from_semantics(args, semantics_json, weights_hd5)
File "/home/kmj1019/anaconda3/envs/gatk/lib/python3.6/site-packages/vqsr_cnn/vqsr_cnn/models.py", line 90, in set_args_and_get_model_from_semantics
model = load_model(weights_hd5, custom_objects=get_metric_dict(args.labels))
File "/home/kmj1019/anaconda3/envs/gatk/lib/python3.6/site-packages/keras/engine/saving.py", line 419, in load_model
model = _deserialize_model(f, custom_objects, compile)
File "/home/kmj1019/anaconda3/envs/gatk/lib/python3.6/site-packages/keras/engine/saving.py", line 251, in _deserialize_model
if weight_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
at org.broadinstitute.hellbender.utils.python.StreamingPythonScriptExecutor.waitForAck(StreamingPythonScriptExecutor.java:222)
at org.broadinstitute.hellbender.utils.python.StreamingPythonScriptExecutor.sendSynchronousCommand(StreamingPythonScriptExecutor.java:183)
at org.broadinstitute.hellbender.tools.walkers.vqsr.CNNScoreVariants.initializePythonArgsAndModel(CNNScoreVariants.java:561)
at org.broadinstitute.hellbender.tools.walkers.vqsr.CNNScoreVariants.onTraversalStart(CNNScoreVariants.java:321)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1047)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
at org.broadinstitute.hellbender.Main.main(Main.java:289)
#other)
But, it is well working in pre-trained mode.
code) java -Xmx8g -jar gatk-package-4.1.9.0-local.jar CNNScoreVariants -V ${1}.Haplotype.vcf -R Homo_sapiens_assembly38.fasta -O ${1}_CNN_1D_pre_trained_scored.vcf --tensor-type reference
I don't know the cause of this issue.
Please, fix this issue.
-
Thank you for the detailed post! We are looking into this on our end.
-
Hello,
There is a chance that this could be caused by your Python configuration. When you updated to GATK 4.2.0.0, did you update your gatk conda environment as well (according to this how-to)?
Another possibility is an issue with how you generated your CNN model. Could you share your command lines that created the 1D_CNN_model.json and 1D_CNN_model.hd5 files?
Thank you,
Genevieve
Please sign in to leave a comment.
2 comments