java.lang.RuntimeException: A required Python package ("gcnvkernel") could not be imported into the Python environment.

Follow

Beetle

• November 04, 2024 18:55

Hi 

Can someone please help with the following error. I am aware there are several solutions mentioned in the gatk forum but non of them seems to work for me. 

Thanks a lot. 

a) GATK version used:
The Genome Analysis Toolkit (GATK) v4.6.0.0
HTSJDK Version: 4.1.1
Picard Version: 3.2.0

$ gatk DetermineGermlineContigPloidy -L /GATK4/Homo_sapiens/GRCh38.preprocessed.interval_list --interval-merging-rule OVERLAPPING_ONLY --input ${wdir}/hdf5/SRR3333333.counts.hdf5 --input ${wdir}/hdf5/SRR2222222.counts.hdf5 --input ${wdir}/hdf5/SRR1111111.counts.hdf5 --contig-ploidy-priors a_valid_ploidy_priors_table.tsv --output ${wdir}/ploidy --output-prefix Normal_Blood_cohort
Using GATK jar /GATK4/gatk-4.6.0.0/gatk-package-4.6.0.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /GATK4/gatk-4.6.0.0/gatk-package-4.6.0.0-local.jar DetermineGermlineContigPloidy -L /GATK4/Homo_sapiens/GRCh38.preprocessed.interval_list --interval-merging-rule OVERLAPPING_ONLY --input /hdf5/SRR3333333.counts.hdf5 --input /hdf5/SRR2222222.counts.hdf5 --input /hdf5/SRR1111111.counts.hdf5 --contig-ploidy-priors a_valid_ploidy_priors_table.tsv --output /ploidy --output-prefix Normal_Blood_cohort
12:49:46.793 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/GATK4/gatk-4.6.0.0/gatk-package-4.6.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
12:49:46.937 INFO  DetermineGermlineContigPloidy - ------------------------------------------------------------
12:49:46.940 INFO  DetermineGermlineContigPloidy - The Genome Analysis Toolkit (GATK) v4.6.0.0
12:49:46.940 INFO  DetermineGermlineContigPloidy - For support and documentation go to https://software.broadinstitute.org/gatk/
12:49:46.940 INFO  DetermineGermlineContigPloidy - Executing as beetle@hpc on Linux v4.18.0-425.3.1.el8.x86_64 amd64
12:49:46.940 INFO  DetermineGermlineContigPloidy - Java runtime: OpenJDK 64-Bit Server VM v21.0.2+13-LTS
12:49:46.941 INFO  DetermineGermlineContigPloidy - Start Date/Time: November 4, 2024, 12:49:46 PM CST
12:49:46.941 INFO  DetermineGermlineContigPloidy - ------------------------------------------------------------
12:49:46.941 INFO  DetermineGermlineContigPloidy - ------------------------------------------------------------
12:49:46.942 INFO  DetermineGermlineContigPloidy - HTSJDK Version: 4.1.1
12:49:46.942 INFO  DetermineGermlineContigPloidy - Picard Version: 3.2.0
12:49:46.942 INFO  DetermineGermlineContigPloidy - Built for Spark Version: 3.5.0
12:49:46.942 INFO  DetermineGermlineContigPloidy - HTSJDK Defaults.COMPRESSION_LEVEL : 2
12:49:46.942 INFO  DetermineGermlineContigPloidy - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false
12:49:46.942 INFO  DetermineGermlineContigPloidy - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true
12:49:46.942 INFO  DetermineGermlineContigPloidy - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false
12:49:46.942 INFO  DetermineGermlineContigPloidy - Deflater: IntelDeflater
12:49:46.942 INFO  DetermineGermlineContigPloidy - Inflater: IntelInflater
12:49:46.943 INFO  DetermineGermlineContigPloidy - GCS max retries/reopens: 20
12:49:46.943 INFO  DetermineGermlineContigPloidy - Requester pays: disabled
12:49:46.943 INFO  DetermineGermlineContigPloidy - Initializing engine
12:49:46.970 INFO  DetermineGermlineContigPloidy - Shutting down engine
[November 4, 2024, 12:49:46 PM CST] org.broadinstitute.hellbender.tools.copynumber.DetermineGermlineContigPloidy done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=142606336
java.lang.RuntimeException: A required Python package ("gcnvkernel") could not be imported into the Python environment. This tool requires that the GATK Python environment is properly established and activated. Please refer to GATK README.md file for instructions on setting up the GATK Python environment.
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.checkPythonEnvironmentForPackage(PythonScriptExecutor.java:228)
        at org.broadinstitute.hellbender.tools.copynumber.DetermineGermlineContigPloidy.onStartup(DetermineGermlineContigPloidy.java:299)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:147)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:198)
        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:217)
        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:166)
        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:209)
        at org.broadinstitute.hellbender.Main.main(Main.java:306)
Caused by: org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException:
python exited with 1
Command Line: python -c import gcnvkernel

Stdout:
Stderr: Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'gcnvkernel'

        at org.broadinstitute.hellbender.utils.python.PythonExecutorBase.getScriptException(PythonExecutorBase.java:75)
        at org.broadinstitute.hellbender.utils.runtime.ScriptExecutor.executeCuratedArgs(ScriptExecutor.java:112)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeArgs(PythonScriptExecutor.java:193)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.executeCommand(PythonScriptExecutor.java:78)
        at org.broadinstitute.hellbender.utils.python.PythonScriptExecutor.checkPythonEnvironmentForPackage(PythonScriptExecutor.java:221)
        ... 7 more

0

• 

  Gökalp Çelik

  • November 04, 2024 19:11

  Hi Beetle

  Have you tried installing miniconda3 and later the gatk environment using the command and the yml file?

  conda create -f gatkcondaenv.yml

  This command installs the gatk python environment for GermlineCNVCaller tools and NVScoreVariants(new for 4.6.1.0)/CNN(deprecated now. only for 4.6.0.0 and earlier). Without this python environment gcnvkernel and its dependencies cannot be resolved and used properly. Our docker image contains this environment by default and works out of the box. If you are unable to install this environment we suggest you to try the docker image. 

  I hope this helps.

  Regards. 

  0

  Comment actions Permalink
• 

  Beetle

  • November 04, 2024 19:56
  • Edited

  Hi Gökalp Çelik

  thanks for the quick reply. 

   

  I found this file : https://github.com/broadinstitute/gatk/blob/master/scripts/gatkcondaenv.yml.template

   

  Is this the same file you were referring ?

  I typed the following command: 

  conda create -n "gatk"
  conda activate gatk
  conda create -f gatkcondaenv.yml -n gatk

  Channels:
   - conda-forge
  Platform: linux-64
  Collecting package metadata (repodata.json): done
  Solving environment: failed

  PackagesNotFoundError: The following packages are not available from current channels:

    - gatkcondaenv.yml

  Current channels:

    - https://conda.anaconda.org/conda-forge

  To search for alternate channels that may provide the conda package you're
  looking for, navigate to

      https://anaconda.org

  and use the search bar at the top of the page.

  $

   

  I had also tried this: 

  conda install bioconda::gatk

  but it didn't work. 

  I had tried with udocker which didn't work. But I don't know if they are the same . I can try with docker. 

  0

  Comment actions Permalink
• 

  Beetle

  • November 04, 2024 20:45

  conda install bioconda::gcnvkernel

  Channels:
   - conda-forge
   - bioconda
  Platform: linux-64
  Collecting package metadata (repodata.json): done
  Solving environment: \ warning  libmamba Added empty dependency for problem type SOLVER_RULE_UPDATE
  failed

  LibMambaUnsatisfiableError: Encountered problems while solving:
    - nothing provides toolchain_c_linux-64 needed by theano-1.0.3-py27hfc679d8_1

  Could not solve for environment specs
  The following packages are incompatible
  ├─ gcnvkernel is installable with the potential options
  │  ├─ gcnvkernel 0.7 would require
  │  │  └─ theano with the potential options
  │  │     ├─ theano [0.8.2|0.9.0|...|1.0.4] would require
  │  │     │  └─ python [2.7* |>=2.7,<2.8.0a0 ], which can be installed;
  │  │     ├─ theano 0.8.2 would require
  │  │     │  └─ python 3.4* , which can be installed;
  │  │     ├─ theano [0.8.2|0.9.0|1.0.1|1.0.2|1.0.3] would require
  │  │     │  └─ python [3.5* |>=3.5,<3.6.0a0 ], which can be installed;
  │  │     ├─ theano [0.8.2|0.9.0|1.0.1] would require
  │  │     │  └─ python 3.6* , which can be installed;
  │  │     ├─ theano [1.0.2|1.0.3|1.0.4|1.0.5] would require
  │  │     │  └─ python >=3.6,<3.7.0a0 , which can be installed;
  │  │     ├─ theano 1.0.3 would require
  │  │     │  └─ toolchain_c_linux-64, which does not exist (perhaps a missing channel);
  │  │     ├─ theano [1.0.3|1.0.4|1.0.5] would require
  │  │     │  └─ python >=3.7,<3.8.0a0 , which can be installed;
  │  │     ├─ theano [1.0.4|1.0.5] would require
  │  │     │  └─ python >=3.8,<3.9.0a0 , which can be installed;
  │  │     ├─ theano 1.0.5 would require
  │  │     │  └─ python >=3.10,<3.11.0a0 , which can be installed;
  │  │     └─ theano 1.0.5 would require
  │  │        └─ python >=3.9,<3.10.0a0 , which can be installed;
  │  ├─ gcnvkernel 0.8 would require
  │  │  └─ intel-openmp 2019.4.* , which does not exist (perhaps a missing channel);
  │  ├─ gcnvkernel 0.8 would require
  │  │  └─ intel-openmp, which does not exist (perhaps a missing channel);
  │  └─ gcnvkernel 0.9 would require
  │     └─ python 3.10.13.* , which can be installed;
  └─ pin-1 is not installable because it requires
     └─ python 3.13.* , which conflicts with any installable versions previously reported.

   

  0

  Comment actions Permalink
• 

  Beetle

  • November 04, 2024 22:27

  thanks i was able to install after downgrading python3 version. 

  0

  Comment actions Permalink
• 

  Gökalp Çelik

  • November 05, 2024 08:13

  Hi again. We do not recommend manual installing of components since many of them have specific version dependencies and results produced by different versions of those dependencies may be different from what is expected. 

  We recommend a fresh miniconda3 install and generating the environment directly from the yml file not by creating an empty one before. GATK versions 4.6 and below require python 3.6 dependent environment with theano and specific numpy versions however 4.6.1.0 requries a more recent one with Pytorch instead. 

  I hope this helps. 

  Regards. 

  0

  Comment actions Permalink
• 

  Beetle

  • November 19, 2024 20:56

  thanks i was able to install 4.6.1.0. using conda environment. 

  0

  Comment actions Permalink

Please sign in to leave a comment.