Сбой памяти Gapcloser
Я запускал Gapcloser (инструмент, выпущенный из Soapdenovo), и он продолжал выдавать сообщение и потерпел неудачу в середине процесса:
*** Error in `GapCloser': free(): invalid pointer: 0x0000000001eb4a80 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7f7658b0a7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7f7658b1337a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f7658b1753c]
GapCloser[0x405d41]
GapCloser[0x40983b]
GapCloser[0x405ad3]
GapCloser[0x403982]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f7658ab3830]
GapCloser[0x402b59]
======= Memory map: ========
00400000-0042a000 r-xp 00000000 fc:02 93192245 /opt/GapCloser-v1.12-r6/Debug/GapCloser
00629000-0062a000 r--p 00029000 fc:02 93192245 /opt/GapCloser-v1.12-r6/Debug/GapCloser
0062a000-0062b000 rw-p 0002a000 fc:02 93192245 /opt/GapCloser-v1.12-r6/Debug/GapCloser
01e9b000-01ed7000 rw-p 00000000 00:00 0 [heap]
7f6689a9b000-7f6aeab13000 rw-p 00000000 00:00 0
7f6f2ac77000-7f6f3c4b9000 rw-p 00000000 00:00 0
7f6f44000000-7f6f44021000 rw-p 00000000 00:00 0
7f6f44021000-7f6f48000000 ---p 00000000 00:00 0
7f6f4b9f4000-7f7658a93000 rw-p 00000000 00:00 0
7f7658a93000-7f7658c53000 r-xp 00000000 fc:00 66322933 /lib/x86_64-linux-gnu/libc-2.23.so
7f7658c53000-7f7658e53000 ---p 001c0000 fc:00 66322933 /lib/x86_64-linux-gnu/libc-2.23.so
7f7658e53000-7f7658e57000 r--p 001c0000 fc:00 66322933 /lib/x86_64-linux-gnu/libc-2.23.so
7f7658e57000-7f7658e59000 rw-p 001c4000 fc:00 66322933 /lib/x86_64-linux-gnu/libc-2.23.so
7f7658e59000-7f7658e5d000 rw-p 00000000 00:00 0
7f7658e5d000-7f7658e74000 r-xp 00000000 fc:00 66326925 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7658e74000-7f7659073000 ---p 00017000 fc:00 66326925 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7659073000-7f7659074000 r--p 00016000 fc:00 66326925 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7659074000-7f7659075000 rw-p 00017000 fc:00 66326925 /lib/x86_64-linux-gnu/libgcc_s.so.1
7f7659075000-7f765917d000 r-xp 00000000 fc:00 66322974 /lib/x86_64-linux-gnu/libm-2.23.so
7f765917d000-7f765937c000 ---p 00108000 fc:00 66322974 /lib/x86_64-linux-gnu/libm-2.23.so
7f765937c000-7f765937d000 r--p 00107000 fc:00 66322974 /lib/x86_64-linux-gnu/libm-2.23.so
7f765937d000-7f765937e000 rw-p 00108000 fc:00 66322974 /lib/x86_64-linux-gnu/libm-2.23.so
7f765937e000-7f76594fa000 r-xp 00000000 fc:00 51382039 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7f76594fa000-7f76596fa000 ---p 0017c000 fc:00 51382039 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7f76596fa000-7f7659704000 r--p 0017c000 fc:00 51382039 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7f7659704000-7f7659706000 rw-p 00186000 fc:00 51382039 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
7f7659706000-7f765970a000 rw-p 00000000 00:00 0
7f765970a000-7f7659722000 r-xp 00000000 fc:00 66323020 /lib/x86_64-linux-gnu/libpthread-2.23.so
7f7659722000-7f7659921000 ---p 00018000 fc:00 66323020 /lib/x86_64-linux-gnu/libpthread-2.23.so
7f7659921000-7f7659922000 r--p 00017000 fc:00 66323020 /lib/x86_64-linux-gnu/libpthread-2.23.so
7f7659922000-7f7659923000 rw-p 00018000 fc:00 66323020 /lib/x86_64-linux-gnu/libpthread-2.23.so
7f7659923000-7f7659927000 rw-p 00000000 00:00 0
7f7659927000-7f765994d000 r-xp 00000000 fc:00 66322909 /lib/x86_64-linux-gnu/ld-2.23.so
7f7659afc000-7f7659b3a000 rw-p 00000000 00:00 0
7f7659b4b000-7f7659b4c000 rw-p 00000000 00:00 0
7f7659b4c000-7f7659b4d000 r--p 00025000 fc:00 66322909 /lib/x86_64-linux-gnu/ld-2.23.so
7f7659b4d000-7f7659b4e000 rw-p 00026000 fc:00 66322909 /lib/x86_64-linux-gnu/ld-2.23.so
7f7659b4e000-7f7659b4f000 rw-p 00000000 00:00 0
7ffc5a9e7000-7ffc5aa08000 rw-p 00000000 00:00 0 [stack]
7ffc5abaa000-7ffc5abad000 r--p 00000000 00:00 0 [vvar]
7ffc5abad000-7ffc5abaf000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]
И это мой файл журнала:
cat gapcloser.log
[WARNING] Maximum supported read length is 155 bp! Program will use 155 instead of 251.
Program: GapCloser
Version: 1.12
Parameters:
-a (scaffold file): sunbird_platanus_raw_kraken_scaff_scaffold.fa
-b (config file): sunbird_raw.config
-o (output file): sunbird_platanus_raw_kraken_scaff_gapclosed
-l (max read len): 155
-p (overlap para): 25
-t (thread num): 40
>>>>>>>>>>hash initializing<<<<<<<<<<
counting reads
in lib:
total reads sum: 289413008
Paired-end reads sum: 289413008
in lib:
total reads sum: 21371568
Paired-end reads sum: 21371568
in lib:
total reads sum: 21944342
Paired-end reads sum: 21944342
in lib:
total reads sum: 20671814
Paired-end reads sum: 20671814
in lib:
total reads sum: 20949232
Paired-end reads sum: 20949232
in lib:
total reads sum: 22977390
Paired-end reads sum: 22977390
in lib:
total reads sum: 19285920
Paired-end reads sum: 19285920
in lib:
total reads sum: 23065494
Paired-end reads sum: 23065494
in lib:
total reads sum: 21131136
Paired-end reads sum: 21131136
in lib:
total reads sum: 20350118
Paired-end reads sum: 20350118
in lib:
total reads sum: 20579926
Paired-end reads sum: 20579926
in lib:
total reads sum: 18998014
Paired-end reads sum: 18998014
in lib:
total reads sum: 20039196
Paired-end reads sum: 20039196
reads sum: 540777158
spent time: 1658s
reads counted
allocating memorys
spent time: 19s
memorys allocated
initializing reads
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
in lib:
spent time: 1984s
reads initialized
sorting reads in reversed direction
spent time: 1357s
reads sorted
initializing hash on reversed reads
kmers sum: 413463339
spent time: 951s
hash initialized
reversing reads
spent time: 633s
reads reversed
sorting reads in forward direction
spent time: 1363s
reads sorted
initializing hash on forward reads
kmers sum: 783012674
spent time: 994s
hash initialized
marking unique reads
spent time: 41s
unique reads marked
sorting reads by serial number
spent time: 1011s
reads sorted
spent total time: 10011s
>>>>>>>>>>hash initialization finished<<<<<<<<<<
Фактически, я успешно запустил эту программу с тем же набором данных, однако наш сервер был переустановлен, поэтому Gapcloser был переустановлен снова, но запуск тех же данных больше не работает из-за этой проблемы...
Я думал, что это проблема с памятью, поэтому я очистил кеш страницы памяти этой командой, но это не помогло, это моя текущая память, которой определенно должно хватить для Gapcloser.
free -h
total used free shared buff/cache available
Mem: 755G 2.7G 539G 14M 214G 751G
Swap: 810G 106M 810G
Кстати, я использую версию исходного кода Gapcloser, а моя версия Ubuntu - 16.04.
Это мой файл конфигурации Soapdenovo/Gapcloser, который, я думаю, не является проблемой, потому что, кажется, он успешно прочитал все библиотеки?
#FOR MATEPAIR LIBRARIES
#maximal read length
max_rd_len=251
#average insert size for 1kb
[LIB]
#average insert size for gel free
avg_ins=1805
#reverse-forward, generated from circularizing libraries with typical insert size greater than 2Kb
reverse_seq=1
#In which parts the reads are used. 2 - only scaffold assembly
asm_flags=4
#assembler will cut the reads from the current library to this length
rd_len_cutof=150
#In which order the reads are used while scaffolding,1-170/200/250bp,2-350/500bp,3-800bp,4-2KB,5-5Kb,6-10Kb,7-20Kb
rank=4
#fastq files for 1kb1
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads1K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads1K1_R2.fastq
[LIB]
#average insert size for 3kb1
avg_ins=3918
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=5
#fastq files for 3kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads3K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads3K1_R2.fastq
[LIB]
#average insert size for 5kb1
avg_ins=6243
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=5
#fastq files for 5kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads5K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads5K1_R2.fastq
[LIB]
#average insert size for 8kb1
avg_ins=8930
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=6
#fastq files for 8kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads8K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads8K1_R2.fastq
[LIB]
#average insert size for 12kb1
avg_ins=12507
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=6
#fastq files for 12kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads12K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads12K1_R2.fastq
[LIB]
#average insert size for 20kb1
avg_ins=17407
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=7
#fastq files for 20kb
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads20K1_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads20K1_R2.fastq
[LIB]
#average insert size for 1kb2
avg_ins=9793
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=4
#fastq files for 1kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads1K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads1K2_R2.fastq
[LIB]
#average insert size for 3kb2
avg_ins=3944
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=5
#fastq files for 3kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads3K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads3K2_R2.fastq
[LIB]
#average insert size for 5kb2
avg_ins=6225
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=5
#fastq files for 5kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads5K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads5K2_R2.fastq
[LIB]
#average insert size for 8kb2
avg_ins=8913
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=6
#fastq files for 8kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads8K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads8K2_R2.fastq
[LIB]
#average insert size for 12kb2
avg_ins=12491
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=6
#fastq files for 12kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads12K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads12K2_R2.fastq
[LIB]
#average insert size for 20kb2
avg_ins=17370
reverse_seq=1
asm_flags=4
rd_len_cutoff=150
rank=7
#fastq files for 20kb2
q1=/home/cch/sunbird/sunbird_kraken/Pure_MPreads20K2_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_MPreads20K2_R2.fastq
### 2x250 run data
[LIB]
avg_ins=434
reverse_seq=0
#only contigs assembly
asm_flags=4
rank=2
#fastq files from LGC16_JM06_591bp
q1=/home/cch/sunbird/sunbird_kraken/Pure_LGC16_JM09_594bp_CTTGTA_L002_R1.fastq
q2=/home/cch/sunbird/sunbird_kraken/Pure_LGC16_JM09_594bp_CTTGTA_L002_R2.fastq
Я запустил Gapcloser через Valgrind и получил 2 ошибки, но я не знаю, как это исправить, у кого-нибудь есть идеи?
valgrind --leak-check=yes GapCloser
==77240== Memcheck, a memory error detector
==77240== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==77240== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==77240== Command: GapCloser
==77240==
==77240== Syscall param open(filename) points to unaddressable byte(s)
==77240== at 0x59FB040: __open_nocancel (syscall-template.S:84)
==77240== by 0x597DACD: _IO_file_open (fileops.c:221)
==77240== by 0x597DD34: _IO_file_fopen@@GLIBC_2.2.5 (fileops.c:328)
==77240== by 0x5971D33: __fopen_internal (iofopen.c:86)
==77240== by 0x510AD2F: std::__basic_file<char>::open(char const*, std::_Ios_Openmode, int) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==77240== by 0x514B259: std::basic_filebuf<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==77240== by 0x514B42F: std::basic_ifstream<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==77240== by 0x403450: main (Main.cpp:121)
==77240== Address 0x0 is not stack'd, malloc'd or (recently) free'd
==77240==
[Error] Can not open input file.
Version:
1.12
Contact:
soap@genomics.org.cn
Usage:
GapCloser [options]
-a <string> input scaffold file name, required.
-b <string> input library info file name, required.
-o <string> output file name, required.
-l <int> maximum read length (<=155), default=100.
-p <int> overlap param(<=31), default=25.
-t <int> thread number, default=1.
-h -? output help information.
==77240==
==77240== Process terminating with default action of signal 27 (SIGPROF)
==77240== at 0x5A0CE0F: write_gmon (gmon.c:354)
==77240== by 0x5A0D589: _mcleanup (gmon.c:422)
==77240== by 0x593DFF7: __run_exit_handlers (exit.c:82)
==77240== by 0x593E044: exit (exit.c:104)
==77240== by 0x40315F: usage() (Main.cpp:60)
==77240== by 0x403497: main (Main.cpp:124)
==77240==
==77240== HEAP SUMMARY:
==77240== in use at exit: 299,124 bytes in 2 blocks
==77240== total heap usage: 4 allocs, 2 frees, 300,700 bytes allocated
==77240==
==77240== LEAK SUMMARY:
==77240== definitely lost: 0 bytes in 0 blocks
==77240== indirectly lost: 0 bytes in 0 blocks
==77240== possibly lost: 0 bytes in 0 blocks
==77240== still reachable: 299,124 bytes in 2 blocks
==77240== suppressed: 0 bytes in 0 blocks
==77240== Reachable blocks (those to which a pointer was found) are not shown.
==77240== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==77240==
==77240== For counts of detected and suppressed errors, rerun with: -v
==77240== ERROR SUMMARY: 2 errors from 1 contexts (suppressed: 0 from 0)
Profiling timer expired