FuncotatorDataSourceDownloader: Requester Pays
Hello, I was attempting to use FuncotatorDataSourceDownloader to acquire the germline datasource, but it looks like we now have to pay to acquire these resources. Is this correct? I wanted to ensure this wasn't a potential bug.
a) GATK version used: 4.6.1.0
b) Exact command used:
java -jar ${GATK} FuncotatorDataSourceDownloader \
--germline \
--hg38 \
--validate-integrity \
--extract-after-download
c) Entire program log:
[February 23, 2025 at 4:27:39 PM CST] org.broadinstitute.hellbender.tools.funcotator.FuncotatorDataSourceDownloader done. Elapsed time: 0.02 minutes. Runtime.totalMemory()=63291392 code: 400 message: Bucket is a requester pays bucket but no user project provided. reason: required location: null retryable: false com.google.cloud.storage.StorageException: Bucket is a requester pays bucket but no user project provided. at com.google.cloud.storage.StorageException.translate(StorageException.java:170) at com.google.cloud.storage.spi.v1.HttpStorageRpc.translate(HttpStorageRpc.java:329) at com.google.cloud.storage.spi.v1.HttpStorageRpc.get(HttpStorageRpc.java:554) at com.google.cloud.storage.StorageImpl.lambda$internalGetBlob$63(StorageImpl.java:1585) at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:103) at com.google.cloud.RetryHelper.run(RetryHelper.java:76) at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) at com.google.cloud.storage.Retrying.run(Retrying.java:65) at com.google.cloud.storage.StorageImpl.run(StorageImpl.java:1524) at com.google.cloud.storage.StorageImpl.internalGetBlob(StorageImpl.java:1583) at com.google.cloud.storage.StorageImpl.get(StorageImpl.java:328) at com.google.cloud.storage.contrib.nio.CloudStorageReadChannel.fetchSize(CloudStorageReadChannel.java:231) at com.google.cloud.storage.contrib.nio.CloudStorageReadChannel.<init>(CloudStorageReadChannel.java:110) at com.google.cloud.storage.contrib.nio.CloudStorageReadChannel.create(CloudStorageReadChannel.java:90) at com.google.cloud.storage.contrib.nio.CloudStorageFileSystemProvider.newReadChannel(CloudStorageFileSystemProvider.java:401) at com.google.cloud.storage.contrib.nio.CloudStorageFileSystemProvider.newByteChannel(CloudStorageFileSystemProvider.java:316) at java.base/java.nio.file.Files.newByteChannel(Files.java:380) at java.base/java.nio.file.Files.newByteChannel(Files.java:432) at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:422) at com.google.cloud.storage.contrib.nio.CloudStorageFileSystemProvider.newInputStream(CloudStorageFileSystemProvider.java:493) at java.base/java.nio.file.Files.newInputStream(Files.java:160) at java.base/java.nio.file.Files.newBufferedReader(Files.java:2922) at java.base/java.nio.file.Files.lines(Files.java:4141) at java.base/java.nio.file.Files.lines(Files.java:4227) at org.broadinstitute.hellbender.tools.funcotator.FuncotatorDataSourceDownloader.readSha256SumFromPath(FuncotatorDataSourceDownloader.java:322) at org.broadinstitute.hellbender.tools.funcotator.FuncotatorDataSourceDownloader.downloadDataSources(FuncotatorDataSourceDownloader.java:297) at org.broadinstitute.hellbender.tools.funcotator.FuncotatorDataSourceDownloader.downloadAndValidateDataSources(FuncotatorDataSourceDownloader.java:275) at org.broadinstitute.hellbender.tools.funcotator.FuncotatorDataSourceDownloader.doWork(FuncotatorDataSourceDownloader.java:248) at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:150) at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:203) at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:222) at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:166) at org.broadinstitute.hellbender.Main.mainEntry(Main.java:209) at org.broadinstitute.hellbender.Main.main(Main.java:306) Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request GET https://storage.googleapis.com/storage/v1/b/broad-public-datasets/o/funcotator%2Ffuncotator_dataSources.v1.8.hg38.20230908g.sha256?fields=bucket,name,generation,size&projection=full { "code" : 400, "errors" : [ { "domain" : "global", "message" : "Bucket is a requester pays bucket but no user project provided.", "reason" : "required" } ], "message" : "Bucket is a requester pays bucket but no user project provided." } at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146) at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118) at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37) at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$3.interceptResponse(AbstractGoogleClientRequest.java:466) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
-
Hi Kaina Millan
There is a problem with the bucket that we store these resources. We are working on a fix. We are sorry for any inconvenience.
Regards.
-
Thank you so much for your quick response! Would you be able to notify me once the issue is fixed?
-
Hi again.
Funcotator bucket is now fixed. You can download resource files without any issues.
-
Thank you again!
Please sign in to leave a comment.
4 comments