Keras error in CNNScoreVariants
I am getting an error when running CNNScoreVariants on a custom model. Before I had the issue described here but solved it with running 'pip install tensorflow "h5py==2.10.0"'. Not sure if that has anything to do with it.
If you are seeing an error, please provide(REQUIRED) :
a) GATK version used: 4.1.9.0
b) Exact command used: ~/software/gatk-4.1.9.0/gatk CNNScoreVariants -V test.vcf.gz -R ~/refgenome/MTB-h37rv_asm19595v2-eg18.fa -O annotated.vcf.gz -architecture my_1d_model.json -weights my_1d_model.hd5
c) Entire error log:
~/software/gatk-4.1.9.0/gatk CNNScoreVariants -V test.vcf.gz -R ~/refgenome/MTB-h37rv_asm19595v2-eg18.fa -O annotated.vcf.gz -architecture my_1d_model.json -weights my_1d_model.hd5
Using GATK jar /mnt/storageG1/jody/software/gatk-4.1.9.0/gatk-package-4.1.9.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /mnt/storageG1/jody/software/gatk-4.1.9.0/gatk-package-4.1.9.0-local.jar CNNScoreVariants -V test.vcf.gz -R /mnt/storageG1//jody/refgenome/MTB-h37rv_asm19595v2-eg18.fa -O annotated.vcf.gz -architecture my_1d_model.json -weights my_1d_model.hd5
15:14:32.302 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/mnt/storageG1/jody/software/gatk-4.1.9.0/gatk-package-4.1.9.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
Feb 11, 2021 3:14:32 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
INFO: Failed to detect whether we are running on Google Compute Engine.
15:14:32.474 INFO CNNScoreVariants - ------------------------------------------------------------
15:14:32.474 INFO CNNScoreVariants - The Genome Analysis Toolkit (GATK) v4.1.9.0
15:14:32.474 INFO CNNScoreVariants - For support and documentation go to https://software.broadinstitute.org/gatk/
15:14:32.475 INFO CNNScoreVariants - Executing as jody@plum-g1 on Linux v4.15.0-124-generic amd64
15:14:32.475 INFO CNNScoreVariants - Java runtime: OpenJDK 64-Bit Server VM v11.0.8-internal+0-adhoc..src
15:14:32.475 INFO CNNScoreVariants - Start Date/Time: February 11, 2021 at 3:14:32 PM UTC
15:14:32.475 INFO CNNScoreVariants - ------------------------------------------------------------
15:14:32.475 INFO CNNScoreVariants - ------------------------------------------------------------
15:14:32.476 INFO CNNScoreVariants - HTSJDK Version: 2.23.0
15:14:32.476 INFO CNNScoreVariants - Picard Version: 2.23.3
15:14:32.476 INFO CNNScoreVariants - HTSJDK Defaults.COMPRESSION_LEVEL : 2
15:14:32.476 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
15:14:32.476 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
15:14:32.476 INFO CNNScoreVariants - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
15:14:32.477 INFO CNNScoreVariants - Deflater: IntelDeflater
15:14:32.477 INFO CNNScoreVariants - Inflater: IntelInflater
15:14:32.477 INFO CNNScoreVariants - GCS max retries/reopens: 20
15:14:32.477 INFO CNNScoreVariants - Requester pays: disabled
15:14:32.477 INFO CNNScoreVariants - Initializing engine
15:14:32.748 INFO FeatureManager - Using codec VCFCodec to read file file:///mnt/storageG1/jody/temp/gatk/test.vcf.gz
15:14:32.776 INFO CNNScoreVariants - Done initializing engine
15:14:32.777 INFO NativeLibraryLoader - Loading libgkl_utils.so from jar:file:/mnt/storageG1/jody/software/gatk-4.1.9.0/gatk-package-4.1.9.0-local.jar!/com/intel/gkl/native/libgkl_utils.so
15:14:34.864 INFO CNNScoreVariants - Using key:CNN_1D for CNN architecture:my_1d_model.json and weights:my_1d_model.hd5
15:14:35.501 INFO CNNScoreVariants - Done scoring variants with CNN.
15:14:35.502 INFO CNNScoreVariants - Shutting down engine
[February 11, 2021 at 3:14:35 PM UTC] org.broadinstitute.hellbender.tools.walkers.vqsr.CNNScoreVariants done. Elapsed time: 0.05 minutes.
Runtime.totalMemory()=2147483648
org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException: A nack was received from the Python process (most likely caused by a raised exception caused by): nkm received
: Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/mnt/storageG1/jody/miniconda3/envs/gatk/lib/python3.6/site-packages/vqsr_cnn/vqsr_cnn/models.py", line 26, in start_session_get_args_and_model
return args_and_model_from_semantics(semantics_json, weights_hd5, tensor_type)
File "/mnt/storageG1/jody/miniconda3/envs/gatk/lib/python3.6/site-packages/vqsr_cnn/vqsr_cnn/models.py", line 33, in args_and_model_from_semantics
model = set_args_and_get_model_from_semantics(args, semantics_json, weights_hd5)
File "/mnt/storageG1/jody/miniconda3/envs/gatk/lib/python3.6/site-packages/vqsr_cnn/vqsr_cnn/models.py", line 90, in set_args_and_get_model_from_semantics
model = load_model(weights_hd5, custom_objects=get_metric_dict(args.labels))
File "/mnt/storageG1/jody/miniconda3/envs/gatk/lib/python3.6/site-packages/keras/engine/saving.py", line 419, in load_model
model = _deserialize_model(f, custom_objects, compile)
File "/mnt/storageG1/jody/miniconda3/envs/gatk/lib/python3.6/site-packages/keras/engine/saving.py", line 251, in _deserialize_model
if weight_names:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
at org.broadinstitute.hellbender.utils.python.StreamingPythonScriptExecutor.waitForAck(StreamingPythonScriptExecutor.java:222)
at org.broadinstitute.hellbender.utils.python.StreamingPythonScriptExecutor.sendSynchronousCommand(StreamingPythonScriptExecutor.java:183)
at org.broadinstitute.hellbender.tools.walkers.vqsr.CNNScoreVariants.initializePythonArgsAndModel(CNNScoreVariants.java:561)
at org.broadinstitute.hellbender.tools.walkers.vqsr.CNNScoreVariants.onTraversalStart(CNNScoreVariants.java:321)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1047)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
at org.broadinstitute.hellbender.Main.main(Main.java:289)
-
Hi jodybk,
This is a known issue with CNNScoreVariants in GATK 4.1.9.0. You can read more about how to solve this here:
Genevieve
-
I want to read https://gatk.broadinstitute.org/hc/en-us/articles/360052489832-Known-Issue-with-CNNScoreVariants-version-4-1-9-0 because of the same error.
But, I can't read !!!!!!!!!!!!!!!! OTL website message:
You're not authorized to access this page
-
The page said that there was a issue with one of the python library versions. If you update to the latest version of gatk it should be fixed.
-
Thank you, jodybk.
I also have done CNNScoreVariants using GATK 4.2.0.0 (th lastest version).
but, the same error message :
"ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all():
*python version: 3.6.10
-
Please, tell me more for "a issue with one of the python library versions".
-
Hello,
I have made that article public, thanks for pointing that out. There is a chance you are dealing with a different issue. Please create a new post and include your complete command and error message.
Best,
Genevieve
-
Please sign in to leave a comment.
7 comments