Picard CheckIlluminaDirectory - Issue with file/folder structure?
I am trying to get FASTQ files from .bcl files from a VeriSeq run on a NextSeq550Dx. When I try to use Picard for IlluminaBasecallsToFastq and CheckIlluminaDirectory, I get an error indicating that files for certain types of data (BaseCalls in this case, but have seen other such as Positions listed) are missing. When I run with LINK_LOCS=true Picard seems to be expecting a file called s.locs in the Intensities folder, but that is not how the folders are structured. Is my folder structure correct? I'm not familiar with Java, so I'm having difficulty troubleshooting this.
Picard version: 2.25.5
Folder Structure (Not all files shown)
input_folder/
├─ Data/
│ ├─ Intensities/
│ │ ├─ BaseCalls/
│ │ │ ├─ L001/
│ │ │ │ ├─ 0087.bcl.bgzf
│ │ │ │ ├─ 0087.bcl.bgzf.bci
│ │ │ ├─ L002/
│ │ │ ├─ L003/
│ │ │ ├─ L004/
│ │ ├─ L001/
│ │ │ ├─ s_1.locs
│ │ ├─ L002/
│ │ ├─ L003/
│ │ ├─ L004/
├─ SampleSheet.csv
Command Used
java -jar picard.jar CheckIlluminaDirectory BASECALLS_DIR=/home/username/input_folder/Data/Intensities/BaseCalls/ READ_STRUCTURE=36T8B8B36T LANES=4 DATA_TYPES=BaseCalls LINK_LOCS=false
Error Log
$ java -jar picard.jar CheckIlluminaDirectory BASECALLS_DIR=/home/username/input_folder/Data/Intensities/BaseCalls/ READ_STRUCTURE=36T8B8B36T LANES=4 DATA_TYPES=BaseCalls LINK_LOCS=false
INFO 2021-06-14 14:13:49 CheckIlluminaDirectory
********** NOTE: Picard's command line syntax is changing.
**********
********** For more information, please see:
********** https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)
**********
********** The command line looks like this in the new syntax:
**********
********** CheckIlluminaDirectory -BASECALLS_DIR /home/username/input_folder/Data/Intensities/BaseCalls/ -READ_STRUCTURE 36T8B8B36T -LANES 4 -DATA_TYPES BaseCalls -LINK_LOCS false
**********
14:13:50.008 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/username/picard.jar!/com/intel/gkl/native/libgkl_compression.so
[Mon Jun 14 14:13:50 GMT 2021] CheckIlluminaDirectory BASECALLS_DIR=/home/username/input_folder/Data/Intensities/BaseCalls DATA_TYPES=[BaseCalls] READ_STRUCTURE=36T8B8B36T LANES=[4] LINK_LOCS=false FAKE_FILES=false VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false
[Mon Jun 14 14:13:50 GMT 2021] Executing as username@MB-DPQG853-DT on Linux 4.4.0-17763-Microsoft amd64; OpenJDK 64-Bit Server VM 1.8.0_292-8u292-b10-0ubuntu1~20.04-b10; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: 2.25.5
INFO 2021-06-14 14:13:50 CheckIlluminaDirectory Checking lanes(4 in basecalls directory (/home/username/input_folder/Data/Intensities/BaseCalls)
INFO 2021-06-14 14:13:50 CheckIlluminaDirectory Expected cycles: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88
INFO 2021-06-14 14:13:50 CheckIlluminaDirectory Checking lane 4
INFO 2021-06-14 14:13:50 CheckIlluminaDirectory Expected tiles: 11401, 11402, 11403, 11404, 11405, 11406, 11407, 11408, 11409, 11410, 11411, 11412, 11501, 11502, 11503, 11504, 11505, 11506, 11507, 11508, 11509, 11510, 11511, 11512, 11601, 11602, 11603, 11604, 11605, 11606, 11607, 11608, 11609, 11610, 11611, 11612, 12401, 12402, 12403, 12404, 12405, 12406, 12407, 12408, 12409, 12410, 12411, 12412, 12501, 12502, 12503, 12504, 12505, 12506, 12507, 12508, 12509, 12510, 12511, 12512, 12601, 12602, 12603, 12604, 12605, 12606, 12607, 12608, 12609, 12610, 12611, 12612, 13401, 13402, 13403, 13404, 13405, 13406, 13407, 13408, 13409, 13410, 13411, 13412, 13501, 13502, 13503, 13504, 13505, 13506, 13507, 13508, 13509, 13510, 13511, 13512, 13601, 13602, 13603, 13604, 13605, 13606, 13607, 13608, 13609, 13610, 13611, 13612, 21401, 21402, 21403, 21404, 21405, 21406, 21407, 21408, 21409, 21410, 21411, 21412, 21501, 21502, 21503, 21504, 21505, 21506, 21507, 21508, 21509, 21510, 21511, 21512, 21601, 21602, 21603, 21604, 21605, 21606, 21607, 21608, 21609, 21610, 21611, 21612, 22401, 22402, 22403, 22404, 22405, 22406, 22407, 22408, 22409, 22410, 22411, 22412, 22501, 22502, 22503, 22504, 22505, 22506, 22507, 22508, 22509, 22510, 22511, 22512, 22601, 22602, 22603, 22604, 22605, 22606, 22607, 22608, 22609, 22610, 22611, 22612, 23401, 23402, 23403, 23404, 23405, 23406, 23407, 23408, 23409, 23410, 23411, 23412, 23501, 23502, 23503, 23504, 23505, 23506, 23507, 23508, 23509, 23510, 23511, 23512, 23601, 23602, 23603, 23604, 23605, 23606, 23607, 23608, 23609, 23610, 23611, 23612
INFO 2021-06-14 14:13:50 CheckIlluminaDirectory Could not find a format with available files for the following data types: BaseCalls
INFO 2021-06-14 14:13:50 CheckIlluminaDirectory Lane 4 FAILED Total Errors: 1
INFO 2021-06-14 14:13:50 CheckIlluminaDirectory FAILED! There were 1 in the following lanes: 4
[Mon Jun 14 14:13:50 GMT 2021] picard.illumina.CheckIlluminaDirectory done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=255328256
Error Output with LINK_LOCS=true
$ java -jar picard.jar CheckIlluminaDirectory BASECALLS_DIR=/home/username/input_folder/Data/Intensities/BaseCalls/ READ_STRUCTURE=36T8B8B36T LANES=4 DATA_TYPES=BaseCalls LINK_LOCS=true
INFO 2021-06-14 14:37:48 CheckIlluminaDirectory
********** NOTE: Picard's command line syntax is changing.
**********
********** For more information, please see:
********** https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)
**********
********** The command line looks like this in the new syntax:
**********
********** CheckIlluminaDirectory -BASECALLS_DIR /home/username/input_folder/Data/Intensities/BaseCalls/ -READ_STRUCTURE 36T8B8B36T -LANES 4 -DATA_TYPES BaseCalls -LINK_LOCS true
**********
14:37:48.567 INFO NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/username/picard.jar!/com/intel/gkl/native/libgkl_compression.so
[Mon Jun 14 14:37:48 GMT 2021] CheckIlluminaDirectory BASECALLS_DIR=/home/username/input_folder/Data/Intensities/BaseCalls DATA_TYPES=[BaseCalls] READ_STRUCTURE=36T8B8B36T LANES=[4] LINK_LOCS=true FAKE_FILES=false VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false
[Mon Jun 14 14:37:48 GMT 2021] Executing as username@MB-DPQG853-DT on Linux 4.4.0-17763-Microsoft amd64; OpenJDK 64-Bit Server VM 1.8.0_292-8u292-b10-0ubuntu1~20.04-b10; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: 2.25.5
INFO 2021-06-14 14:37:48 CheckIlluminaDirectory Checking lanes(4 in basecalls directory (/home/username/input/Data/Intensities/BaseCalls)
INFO 2021-06-14 14:37:48 CheckIlluminaDirectory Expected cycles: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88
[Mon Jun 14 14:37:48 GMT 2021] picard.illumina.CheckIlluminaDirectory done. Elapsed time: 0.00 minutes.
Runtime.totalMemory()=255328256
To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp
Exception in thread "main" picard.PicardException: Locations file /home/username/input_folder/Data/Intensities/s.locs does not exist.
at picard.illumina.CheckIlluminaDirectory.createLocFileSymlinks(CheckIlluminaDirectory.java:273)
at picard.illumina.CheckIlluminaDirectory.doWork(CheckIlluminaDirectory.java:212)
at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:308)
at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:103)
at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:113)
-
Hi Jacob,
Could you give a bit more information regarding what sequencer you used and how you generated this folder structure?
Best,
Genevieve
Please sign in to leave a comment.
1 comment