Genome Analysis Toolkit

Variant Discovery in High-Throughput Sequencing Data

GATK process banner

Need Help?

Search our documentation

Community Forum

Hi, How can we help?

Developed in the Data Sciences Platform at the Broad Institute, the toolkit offers a wide variety of tools with a primary focus on variant discovery and genotyping. Its powerful processing engine and high-performance computing features make it capable of taking on projects of any size. Learn more

GermlineCNVCaller/denoising_config.json

Answered
0

11 comments

  • Avatar
    Bhanu Gandham

    Hi,

     

    Have you run in cohort mode to create the PON? PON is needed for case mode. The doc you shared above has more information on this.

     

    0
    Comment actions Permalink
  • Avatar
    Heba B abusamra

    I have the same error can you explain how to create PON in cohort mode 

    my commands

    /app/Genome/GATK-4.1.8.1/gatk DetermineGermlineContigPloidy -L target.preprocessed.interval_list --interval-merging-rule OVERLAPPING_ONLY -I PBG-1095-19_S2.tsv -I PBG-1077-19R_S8.tsv -I PBG-1076-19R_S7.tsv -I PBG-1096-19_S1.tsv --contig-ploidy-priors contig_ploidy_priors.tsv --output . --output-prefix ploidy --verbosity DEBUG

    /app/Genome/GATK-4.1.8.1/gatk DetermineGermlineContigPloidy --model ploidy-model -I POCG-490-20_S2.tsv -O . --output-prefix ploidy-case --verbosity DEBUG

    /app/Genome/GATK-4.1.8.1/gatk GermlineCNVCaller --run-mode CASE -I POCG-490-20_S2.tsv --contig-ploidy-calls ploidy-case-calls --model ploidy-model --output POCG-490-20 --output-prefix POCG-490-20 --verbosity DEBUG

    0
    Comment actions Permalink
  • Avatar
    Genevieve Brandt (she/her)

    Please see this document under 3. Call autosomal and allosomal contig ploidy with DetermineGermlineContigPloidy https://gatk.broadinstitute.org/hc/en-us/articles/360035531152--How-to-Call-common-and-rare-germline-copy-number-variants

    0
    Comment actions Permalink
  • Avatar
    Dr N Ch

    Genevieve Brandt

    Hi... Should we run "GermlineCNVCaller --run-mode COHORT" step first... to make work  GermlineCNVCaller --run-mode CASE?

    1. Initially i tried running case step only and it exited with "FileNotFoundError: [Errno 2] No such file or directory: 'CNV/ploidy-case-calls/denoising_config.json' error

     

    2. To resolve the above error... i ran it in cohort mode ---- and these specific errors pop up.

    a) "Sample has an anomalous ploidy (3) for contig 19. The presence of unmasked PAR regions and regions of low mappability in the coverage metadata can result in unreliable ploidy designations. It is recommended that the user verifies this designation by orthogonal methods."

    How to I make this list :"Optionally provide intervals to exclude from analysis with --exclude-intervals or -XL, e.g. pseudoautosomal (PAR) regions, "

    b) 06:09:58.115 INFO gcnvkernel.tasks.task_cohort_denoising_calling - Instantiating the denoising model (warm-up)...
    06:10:05.004 INFO GermlineCNVCaller - Shutting down engine
    [18 November, 2021 6:10:05 AM IST] org.broadinstitute.hellbender.tools.copynumber.GermlineCNVCaller done. Elapsed time: 0.38 minutes.
    Runtime.totalMemory()=1178075136
    org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException:
    python exited with 139
    Command Line: python /tmp/cohort_denoising_calling.8874612537117360904.py --ploidy_calls_path=CohortBuild/ploidy-calls --output_calls_path=cohort5cnv-calls --output_tracking_path=CohortBuild/cohortfivecnv/cohort5cnv-tracking --random_seed=1984 --modeling_interval_list=/tmp/intervals594573086710043408.tsv --output_model_path=CohortBuild/cohortfivecnv/cohort5cnv-model --enable_explicit_gc_bias_modeling=False --read_count_tsv_files /tmp/MMG0067808841026474173153131.tsv /tmp/MMG0067827618115401963325547.tsv /tmp/MMG0067922876341803010914191.tsv /tmp/MMG006877700705929029814375.tsv /tmp/MMG0068793470283993599927353.tsv --psi_s_scale=1.000000e-04 --mapping_error_rate=1.000000e-02 --depth_correction_tau=1.000000e+04 --q_c_expectation_mode=hybrid --max_bias_factors=5 --psi_t_scale=1.000000e-03 --log_mean_bias_std=1.000000e-01 --init_ard_rel_unexplained_variance=1.000000e-01 --num_gc_bins=20 --gc_curve_sd=1.000000e+00 --active_class_padding_hybrid_mode=50000 --enable_bias_factors=True --disable_bias_factors_in_active_class=False --p_alt=1.000000e-06 --cnv_coherence_length=1.000000e+04 --max_copy_number=5 --p_active=0.010000 --class_coherence_length=10000.000000 --learning_rate=1.000000e-02 --adamax_beta1=9.000000e-01 --adamax_beta2=9.900000e-01 --log_emission_samples_per_round=50 --log_emission_sampling_rounds=10 --log_emission_sampling_median_rel_error=5.000000e-03 --max_advi_iter_first_epoch=5000 --max_advi_iter_subsequent_epochs=200 --min_training_epochs=10 --max_training_epochs=50 --initial_temperature=1.500000e+00 --num_thermal_advi_iters=2500 --convergence_snr_averaging_window=500 --convergence_snr_trigger_threshold=1.000000e-01 --convergence_snr_countdown_window=10 --max_calling_iters=10 --caller_update_convergence_threshold=1.000000e-03 --caller_internal_admixing_rate=7.500000e-01 --caller_external_admixing_rate=1.000000e+00 --disable_caller=false --disable_sampler=false --disable_annealing=false
    at org.broadinstitute.hellbender.utils.python.PythonExecutorBase.getScriptException(PythonExecutorBase.java:75)
    at org.broadinstitute.hellbender.tools.copynumber.GermlineCNVCaller.doWork(GermlineCNVCaller.java:340)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)

    no clue what it says about!

    When I download the tutorial data I see that the ploidy model folders include a “denoising_config.json” file, but it doesn’t get created with any commands that I run.

     

    Any help would be appreciated

     

    1
    Comment actions Permalink
  • Avatar
    Genevieve Brandt (she/her)

    Hi Dr N Ch,

    In order to run case mode, you'll need to use a model already constructed with cohort mode. For cohort mode, the general recommendation is to use around 100 samples or more. Do you have this many samples?

    This information is all covered in the introduction section of our tutorial (How to) Call common and rare germline copy number variants: https://gatk.broadinstitute.org/hc/en-us/articles/360035531152--How-to-Call-common-and-rare-germline-copy-number-variants.

    Please let me know if you have further questions.

    Best,

    Genevieve

    0
    Comment actions Permalink
  • Avatar
    Dr N Ch

    Hi, Thank you for the response... I am currently following the protocol as in the link shared above.. I can get hold of 100 samples... but now i am trying with 5 samples... to setup the steps ..

    I am stuck up at the above error and i am unable to find "denoising_config.json" script... and thus the program exits...

    How can we resolve this issue?

    0
    Comment actions Permalink
  • Avatar
    Genevieve Brandt (she/her)

    Hi Dr N Ch,

    You'll need to do cohort mode first to create the model which is needed for case mode. Could you share the complete command and error message for where you are getting stuck in cohort mode? 

    Best,

    Genevieve

    0
    Comment actions Permalink
  • Avatar
    Dr N Ch

    The model has been created with "DetermineGermlineContigPloidy" and the command successfully worked...

    the next step is to call GermlineCNVCaller : when i tried to execute it... it is throwing up the error

     

    6:09:58.115 INFO gcnvkernel.tasks.task_cohort_denoising_calling - Instantiating the denoising model (warm-up)...
    06:10:05.004 INFO GermlineCNVCaller - Shutting down engine
    [18 November, 2021 6:10:05 AM IST] org.broadinstitute.hellbender.tools.copynumber.GermlineCNVCaller done. Elapsed time: 0.38 minutes.
    Runtime.totalMemory()=1178075136
    org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException:
    python exited with 139
    Command Line: python /tmp/cohort_denoising_calling.8874612537117360904.py --ploidy_calls_path=CohortBuild/ploidy-calls --output_calls_path=cohort5cnv-calls --output_tracking_path=CohortBuild/cohortfivecnv/cohort5cnv-tracking --random_seed=1984 --modeling_interval_list=/tmp/intervals594573086710043408.tsv --output_model_path=CohortBuild/cohortfivecnv/cohort5cnv-model --enable_explicit_gc_bias_modeling=False --read_count_tsv_files /tmp/MMG0067808841026474173153131.tsv /tmp/MMG0067827618115401963325547.tsv /tmp/MMG0067922876341803010914191.tsv /tmp/MMG006877700705929029814375.tsv /tmp/MMG0068793470283993599927353.tsv --psi_s_scale=1.000000e-04 --mapping_error_rate=1.000000e-02 --depth_correction_tau=1.000000e+04 --q_c_expectation_mode=hybrid --max_bias_factors=5 --psi_t_scale=1.000000e-03 --log_mean_bias_std=1.000000e-01 --init_ard_rel_unexplained_variance=1.000000e-01 --num_gc_bins=20 --gc_curve_sd=1.000000e+00 --active_class_padding_hybrid_mode=50000 --enable_bias_factors=True --disable_bias_factors_in_active_class=False --p_alt=1.000000e-06 --cnv_coherence_length=1.000000e+04 --max_copy_number=5 --p_active=0.010000 --class_coherence_length=10000.000000 --learning_rate=1.000000e-02 --adamax_beta1=9.000000e-01 --adamax_beta2=9.900000e-01 --log_emission_samples_per_round=50 --log_emission_sampling_rounds=10 --log_emission_sampling_median_rel_error=5.000000e-03 --max_advi_iter_first_epoch=5000 --max_advi_iter_subsequent_epochs=200 --min_training_epochs=10 --max_training_epochs=50 --initial_temperature=1.500000e+00 --num_thermal_advi_iters=2500 --convergence_snr_averaging_window=500 --convergence_snr_trigger_threshold=1.000000e-01 --convergence_snr_countdown_window=10 --max_calling_iters=10 --caller_update_convergence_threshold=1.000000e-03 --caller_internal_admixing_rate=7.500000e-01 --caller_external_admixing_rate=1.000000e+00 --disable_caller=false --disable_sampler=false --disable_annealing=false
    at org.broadinstitute.hellbender.utils.python.PythonExecutorBase.getScriptException(PythonExecutorBase.java:75)
    at org.broadinstitute.hellbender.tools.copynumber.GermlineCNVCaller.doWork(GermlineCNVCaller.java:340)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)

    no clue what it says about!

    When I download the tutorial data I see that the ploidy model folders include a “denoising_config.json” file, but it doesn’t get created with any commands that I run.

     

    Any help would be appreciated

    0
    Comment actions Permalink
  • Avatar
    Genevieve Brandt (she/her)

    Dr N Ch Could you please also share your command?

    Is this for cohort mode?

    0
    Comment actions Permalink
  • Avatar
    Dr N Ch
     

    command used:

    /home/Applications/gatk-4.2.0.0/gatk GermlineCNVCaller \
    --run-mode COHORT \
    -L /home/Projects/CNV/5.cohort.interval_list \
    -I /home/Projects/CNV/80.tsv -I /home/Projects/CNV/82.tsv -I /home/Projects/CNV/92.tsv -I /home/Projects/CNV/77.tsv -I /home/Projects/CNV/79.tsv --contig-ploidy-calls /home/Projects/CNV/ploidy-calls --interval-merging-rule OVERLAPPING_ONLY --output /home/Projects/CNV/cohortfivecnv --output-prefix cohort5cnv --verbosity DEBUG

    ERROR OBTAINED:

    [29 November, 2021 8:17:01 PM IST] org.broadinstitute.hellbender.tools.copynumber.GermlineCNVCaller done. Elapsed time: 1.12 minutes.
    Runtime.totalMemory()=1244659712
    org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException:
    python exited with 139
    Command Line: python /tmp/cohort_denoising_calling.2460559553786744739.py --ploidy_calls_path=/home/Projects/CNV/ploidy-calls --output_calls_path=/home/Projects/CNV/cohortfivecnv/cohort5cnv-calls --output_tracking_path=/home/Projects/CNV/cohortfivecnv/cohort5cnv-tracking --random_seed=1984 --modeling_interval_list=/tmp/intervals1605665010689778693.tsv --output_model_path=/home/Projects/CNV/cohortfivecnv/cohort5cnv-model --enable_explicit_gc_bias_modeling=False --read_count_tsv_files /tmp/804360048707576940571.tsv /tmp/82883171617448040778.tsv /tmp/921039727042770894278.tsv /tmp/777516631403810019298.tsv /tmp/798364616654579626422.tsv --psi_s_scale=1.000000e-04 --mapping_error_rate=1.000000e-02 --depth_correction_tau=1.000000e+04 --q_c_expectation_mode=hybrid --max_bias_factors=5 --psi_t_scale=1.000000e-03 --log_mean_bias_std=1.000000e-01 --init_ard_rel_unexplained_variance=1.000000e-01 --num_gc_bins=20 --gc_curve_sd=1.000000e+00 --active_class_padding_hybrid_mode=50000 --enable_bias_factors=True --disable_bias_factors_in_active_class=False --p_alt=1.000000e-06 --cnv_coherence_length=1.000000e+04 --max_copy_number=5 --p_active=0.010000 --class_coherence_length=10000.000000 --learning_rate=1.000000e-02 --adamax_beta1=9.000000e-01 --adamax_beta2=9.900000e-01 --log_emission_samples_per_round=50 --log_emission_sampling_rounds=10 --log_emission_sampling_median_rel_error=5.000000e-03 --max_advi_iter_first_epoch=5000 --max_advi_iter_subsequent_epochs=200 --min_training_epochs=10 --max_training_epochs=50 --initial_temperature=1.500000e+00 --num_thermal_advi_iters=2500 --convergence_snr_averaging_window=500 --convergence_snr_trigger_threshold=1.000000e-01 --convergence_snr_countdown_window=10 --max_calling_iters=10 --caller_update_convergence_threshold=1.000000e-03 --caller_internal_admixing_rate=7.500000e-01 --caller_external_admixing_rate=1.000000e+00 --disable_caller=false --disable_sampler=false --disable_annealing=false
    at org.broadinstitute.hellbender.utils.python.PythonExecutorBase.getScriptException(PythonExecutorBase.java:75)
    at org.broadinstitute.hellbender.tools.copynumber.GermlineCNVCaller.doWork(GermlineCNVCaller.java:340)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)
    at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)
    at org.broadinstitute.hellbender.Main.main(Main.java:289)

     

    STUCK UP HERE!

    0
    Comment actions Permalink
  • Avatar
    Genevieve Brandt (she/her)

    Hi Dr N Ch,

    Thank you! This is helpful for determining the problem. The error message I am seeing is this one:

    org.broadinstitute.hellbender.utils.python.PythonScriptExecutorException:
    python exited with 139

    Could you create a new post for this issue? I don't think that it is the same as the original post we are commenting on. My first thoughts regarding this issue is that it could be a problem with the environment/hardware relating to the python packages (related issue). In the new post, please also include your conda/docker environment details.

    Best,

    Genevieve

    0
    Comment actions Permalink

Please sign in to leave a comment.

Powered by Zendesk