Genome Analysis Toolkit

Variant Discovery in High-Throughput Sequencing Data

GATK process banner

Need Help?

Search our documentation

Community Forum

Hi, How can we help?

Developed in the Data Sciences Platform at the Broad Institute, the toolkit offers a wide variety of tools with a primary focus on variant discovery and genotyping. Its powerful processing engine and high-performance computing features make it capable of taking on projects of any size. Learn more

libgkl_compression.so not found + Failed to detect Google Compute Engine in a GATK workflow

Answered
Follow
Avatar
Marta Giovalli
  • Edited

I have got a problem with the workflow "GATK Broad Best Practice Germline snps and indels variant calling 4.1.0.0" available here. It is a workflow using GATK IntervalListTools, HaplotypeCaller + MergeVCF.

I've got the following warnings:

Using GATK jar /opt/gatk-4.1.0.0/gatk-package-4.1.0.0-local.jar
Running:
    java -Dsamjdk.use_async_io_read_samtools=false Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xms1g -jar /opt/gatk-4.1.0.0/gatk-package-4.1.0.0-local.jar IntervalListTools --OUTPUT out --INPUT /var/extern/geniosave/hg38_v0_wgs_calling_regions.hg38.interval_list --SCATTER_COUNT 20 --SORT --SUBDIVISION_MODE INTERVAL_SUBDIVISION --UNIQUE
13:21:11.860 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/opt/gatk-4.1.0.0/gatk-package-4.1.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so

// warning on missing lib

13:21:11.879 WARN  NativeLibraryLoader - Unable to load libgkl_compression.so from native/libgkl_compression.so (No such file or directory)
13:21:11.882 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/opt/gatk-4.1.0.0/gatk-package-4.1.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so
13:21:11.884 WARN  NativeLibraryLoader - Unable to load libgkl_compression.so from native/libgkl_compression.so (No such file or directory)
[Mon Sep 12 13:21:11 UTC 2022] IntervalListTools  --INPUT /var/extern/geniosave/hg38_v0_wgs_calling_regions.hg38.interval_list --OUTPUT out --UNIQUE true --SORT true --SCATTER_COUNT 20 --SUBDIVISION_MODE INTERVAL_SUBDIVISION  --PADDING 0 --ACTION CONCAT --INCLUDE_FILTERED false --BREAK_BANDS_AT_MULTIPLES_OF 0 --INVERT false --OUTPUT_VALUE NONE --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 2 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --GA4GH_CLIENT_SECRETS client_secrets.json --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false
Sep 12, 2022 1:21:12 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine

// warning on Google Compute EngineWARNING: Failed to detect whether we are running on Google Compute Engine.
shaded.cloud_nio.com.google.api.client.http.HttpResponseException: 404 Not Found
not found
    at shaded.cloud_nio.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1072)
    at shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials.runningOnComputeEngine(ComputeEngineCredentials.java:210)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.tryGetComputeCredentials(DefaultCredentialsProvider.java:290)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentialsUnsynchronized(DefaultCredentialsProvider.java:207)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentials(DefaultCredentialsProvider.java:124)
    at shaded.cloud_nio.com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:127)
    at shaded.cloud_nio.com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:100)
    at com.google.cloud.ServiceOptions.defaultCredentials(ServiceOptions.java:304)
    at com.google.cloud.ServiceOptions.<init>(ServiceOptions.java:278)
    at com.google.cloud.storage.StorageOptions.<init>(StorageOptions.java:83)
    at com.google.cloud.storage.StorageOptions.<init>(StorageOptions.java:31)
    at com.google.cloud.storage.StorageOptions$Builder.build(StorageOptions.java:78)
    at picard.nio.GoogleStorageUtils.initialize(GoogleStorageUtils.java:53)
    at picard.nio.PathProvider$1.initialize(PathProvider.java:14)
    at picard.nio.PathProvider.<init>(PathProvider.java:28)
    at picard.nio.PathProvider.<init>(PathProvider.java:9)
    at picard.nio.PathProvider$1.<init>(PathProvider.java:10)
    at picard.nio.PathProvider.<clinit>(PathProvider.java:10)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:272)
    at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgramExecutor.instanceMain(PicardCommandLineProgramExecutor.java:25)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
    at org.broadinstitute.hellbender.Main.main(Main.java:291)

Sep 12, 2022 1:21:12 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
WARNING: Failed to detect whether we are running on Google Compute Engine.
shaded.cloud_nio.com.google.api.client.http.HttpResponseException: 404 Not Found
not found
    at shaded.cloud_nio.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1072)
    at shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials.runningOnComputeEngine(ComputeEngineCredentials.java:210)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.tryGetComputeCredentials(DefaultCredentialsProvider.java:290)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentialsUnsynchronized(DefaultCredentialsProvider.java:207)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentials(DefaultCredentialsProvider.java:124)
    at shaded.cloud_nio.com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:127)
    at shaded.cloud_nio.com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:100)
    at com.google.cloud.ServiceOptions.defaultCredentials(ServiceOptions.java:304)
    at com.google.cloud.ServiceOptions.<init>(ServiceOptions.java:278)
    at com.google.cloud.storage.StorageOptions.<init>(StorageOptions.java:83)
    at com.google.cloud.storage.StorageOptions.<init>(StorageOptions.java:31)
    at com.google.cloud.storage.StorageOptions$Builder.build(StorageOptions.java:78)
    at picard.nio.GoogleStorageUtils.initialize(GoogleStorageUtils.java:53)
    at picard.nio.PathProvider$1.initialize(PathProvider.java:14)
    at picard.nio.PathProvider.<init>(PathProvider.java:28)
    at picard.nio.PathProvider.<init>(PathProvider.java:9)
    at picard.nio.PathProvider$1.<init>(PathProvider.java:10)
    at picard.nio.PathProvider.<clinit>(PathProvider.java:10)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:272)
    at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgramExecutor.instanceMain(PicardCommandLineProgramExecutor.java:25)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
    at org.broadinstitute.hellbender.Main.main(Main.java:291)

Sep 12, 2022 1:21:12 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine
WARNING: Failed to detect whether we are running on Google Compute Engine.
shaded.cloud_nio.com.google.api.client.http.HttpResponseException: 404 Not Found
not found
    at shaded.cloud_nio.com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1072)
    at shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials.runningOnComputeEngine(ComputeEngineCredentials.java:210)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.tryGetComputeCredentials(DefaultCredentialsProvider.java:290)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentialsUnsynchronized(DefaultCredentialsProvider.java:207)
    at shaded.cloud_nio.com.google.auth.oauth2.DefaultCredentialsProvider.getDefaultCredentials(DefaultCredentialsProvider.java:124)
    at shaded.cloud_nio.com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:127)
    at shaded.cloud_nio.com.google.auth.oauth2.GoogleCredentials.getApplicationDefault(GoogleCredentials.java:100)
    at com.google.cloud.ServiceOptions.defaultCredentials(ServiceOptions.java:304)
    at com.google.cloud.ServiceOptions.<init>(ServiceOptions.java:278)
    at com.google.cloud.storage.StorageOptions.<init>(StorageOptions.java:83)
    at com.google.cloud.storage.StorageOptions.<init>(StorageOptions.java:31)
    at com.google.cloud.storage.StorageOptions$Builder.build(StorageOptions.java:78)
    at picard.nio.GoogleStorageUtils.initialize(GoogleStorageUtils.java:53)
    at picard.nio.PathProvider$1.initialize(PathProvider.java:14)
    at picard.nio.PathProvider.<init>(PathProvider.java:28)
    at picard.nio.PathProvider.<init>(PathProvider.java:9)
    at picard.nio.PathProvider$1.<init>(PathProvider.java:10)
    at picard.nio.PathProvider.<clinit>(PathProvider.java:10)
    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:272)
    at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgramExecutor.instanceMain(PicardCommandLineProgramExecutor.java:25)
    at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:162)
    at org.broadinstitute.hellbender.Main.mainEntry(Main.java:205)
    at org.broadinstitute.hellbender.Main.main(Main.java:291)

[Mon Sep 12 13:21:12 UTC 2022] Executing as root@ce50e1f9ef21 on Linux 5.4.0-72-generic amd64; OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~16.04-b09; Deflater: Jdk; Inflater: Jdk; Provider GCS is available; Picard version: Version:4.1.0.0
INFO    2022-09-12 13:21:14 IntervalListScatterer   idealSplitWeight=146187273
INFO    2022-09-12 13:21:15 IntervalListTools   Wrote 20 scatter subdirectories to out.
INFO    2022-09-12 13:21:15 IntervalListTools   Produced 375 intervals totalling 2923745463 bases.
[Mon Sep 12 13:21:15 UTC 2022] picard.util.IntervalListTools done. Elapsed time: 0.06 minutes.
Runtime.totalMemory()=1037959168

Thank you in advance for your help

0

4 comments

Sort by Date Votes
  • Avatar
    Anthony DiCi

    Hi Marta Giovalli,

    Thank you for writing to the GATK forum! I hope that we can help you sort this out.

    When reviewing your inquiry, a couple of things stand out to me as potential problems. The version of GATK you are using (4.1.0.0) and the source of your GATK Best Practices Workflow seem to be the origin of the issue at first glance. 

    Here are some suggestions on things to try first:

    1. Update to the latest version of GATK (4.2.6.1)
    2. Follow the Germline short variant discovery (SNPs and Indels) Best Practices Workflow directly from our website.

    I hope this helps! Please let me know whether or not this leads you to success. If not, I can take this to our developers for a closer look. If any other questions arise in the meantime, please do not hesitate to reach out.

    Best,

    Anthony

    0
    Comment actions Permalink
  • Avatar
    Anthony DiCi

    Hi Marta Giovalli,

    We haven’t heard from you in a while so we will be closing out your ticket in our system. If you still require assistance, you need only respond to this thread, and we’ll create a follow-up ticket to pick up where we left off.

    Thank you again for contributing to our GATK forum!

    Best,

    Anthony

    0
    Comment actions Permalink
  • Avatar
    Marta Giovalli

    Updating to the latest version of GATK (4.2.6.1) did the trick.

    Thank you!

    0
    Comment actions Permalink
  • Avatar
    Genevieve Brandt (she/her)

    Thank you for posting the solution Marta Giovalli!

    0
    Comment actions Permalink

Please sign in to leave a comment.

Welcome

Forum Guidelines

GATK Announcements

Become a GATK Guru

Read Before You Post

Get Email Notifications

Didn't find what you were looking for?

New post

Quick Links

GATK Latest Releases

Known Issues & Troubleshooting

Submitting a Bug Report

How to Track a Github issue

GATK on the Cloud with Terra

Recent GATK News

GATK Blog Posts

GATK Twitter Page

About the GATK community

  • 133,119 Total Users
  • 0 Comments
  • 0 Posts
  • 0 Votes
Powered by Zendesk