<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
The segmentation fault occurs after MAKER is finished, so everything is shutting down and OpenMPI is exiting. Why MPI can't shut down correctly can be for a number of reasons.</div>
<ol style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<li>You can try updating OpenMPi or making sure you use the most up to date version available on your system (not uncommon for clusters to have multiple versions installed). Note that switching versions will require that you reinstall MAKER since it is compiled
against the shared libraries of the version you are using.</li><li>Make sure you are not using a different version of mpiexec than the version of OpenMPi you installed MAKER with (check with 'which mpiexec'). This can happen on systems with multiple versions of MPI installed.</li><li>Segmentation faults can be caused by perl/C bindings when perl is exiting under the control of mpiexec (mpiexec actually affects how C code operates and can lead to some odd behavior when the MPI_Finalize shuts down communication). You can reinstall any
perl modules that are written in C if that is the case. One example is the 'forks' module. One previous user who had a segfault on exit was able to solve it by downgrading the Proc-ProcessTable module from 0.45 to 0.44.</li><li>Make sure you run this command before starting mpiexec or add it to your bash profile to fix issues with shared libraries on OpenMPI. --> <span style="font-size: medium;">export LD_PRELOAD=/<openmpi_location>/lib/libmpi.so</span></li></ol>
<div><font face="Calibri,sans-serif">Fortunately MAKER is already finished, so it won't affect your results, but it is a symptom that there is some incompatibility with a C based library and your MPI installation.</font></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
Also the -mca btl ^openib options disables the OpenFabrics libraries used by OpenMPI for infiiband communication since those libraries have a known bug where failures occur whenever programs perform a system calls (I.e. every time you open an external program
from within a program). An example of this would be MAKER calling BLAST or MAKER calling Augustus or SNAP. All of which immediately cause OpenMPI to blow up if you don't add the flag.</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
--Carson</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Justin Peyton <<a href="mailto:cognitiveshrapnel@gmail.com">cognitiveshrapnel@gmail.com</a>><br>
<span style="font-weight:bold">Date: </span>Wednesday, October 8, 2014 at 11:52 AM<br>
<span style="font-weight:bold">To: </span><<a href="mailto:maker-devel@yandell-lab.org">maker-devel@yandell-lab.org</a>><br>
<span style="font-weight:bold">Subject: </span>Segmentation fault on exit<br>
</div>
<div><br>
</div>
<div dir="ltr">
<div>Hello,</div>
<div>I am getting a segmentation fault on exit (below). I have read about something similar to this involving hydra. I do no think that is the situation here. I am running Maker 2.31.6 with openMPI and the -mca btl ^openib option to work around an infiniband
installation (even though I have no idea what it does). My understanding is that this error has no effect on output because it happens after maker is finished. Can you confirm this? Any ideas what may be causing it? Thank you in advance</div>
<div><br>
</div>
<div>Justin Peyton</div>
<div>The Ohio State University</div>
<div><br>
</div>
<div>Maker is now finished!!!</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Start_time: 1412784571</div>
<div>End_time: 1412784747</div>
<div>Elapsed: 176</div>
<div>[n0617:27212] *** Process received signal ***</div>
<div>[n0617:27212] Signal: Segmentation fault (11)</div>
<div>[n0617:27212] Signal code: Address not mapped (1)</div>
<div>[n0617:27212] Failing at address: (nil)</div>
<div>[n0617:27212] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27212] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27212] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27212] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27212] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27212] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27212] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27212] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27212] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27212] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27212] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27212] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27212] *** End of error message ***</div>
<div>[n0617:27207] *** Process received signal ***</div>
<div>[n0617:27207] Signal: Segmentation fault (11)</div>
<div>[n0617:27207] Signal code: Address not mapped (1)</div>
<div>[n0617:27207] Failing at address: (nil)</div>
<div>[n0617:27207] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27207] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27207] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27207] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27207] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27207] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27207] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27207] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27207] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27207] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27207] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27207] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27207] *** End of error message ***</div>
<div>[n0617:27205] *** Process received signal ***</div>
<div>[n0617:27205] Signal: Segmentation fault (11)</div>
<div>[n0617:27205] Signal code: Address not mapped (1)</div>
<div>[n0617:27205] Failing at address: (nil)</div>
<div>[n0617:27205] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27205] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27205] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27205] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27205] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27205] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27205] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27205] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27205] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27205] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27205] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27205] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27205] *** End of error message ***</div>
<div>[n0617:27209] *** Process received signal ***</div>
<div>[n0617:27209] Signal: Segmentation fault (11)</div>
<div>[n0617:27209] Signal code: Address not mapped (1)</div>
<div>[n0617:27209] Failing at address: (nil)</div>
<div>[n0617:27209] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27209] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27209] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27209] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27209] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27209] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27209] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27209] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27209] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27209] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27209] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27209] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27209] *** End of error message ***</div>
<div>[n0617:27215] *** Process received signal ***</div>
<div>[n0617:27215] Signal: Segmentation fault (11)</div>
<div>[n0617:27215] Signal code: Address not mapped (1)</div>
<div>[n0617:27215] Failing at address: (nil)</div>
<div>[n0617:27215] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27215] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27215] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27215] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27215] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27215] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27215] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27215] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27215] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27215] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27215] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27215] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27215] *** End of error message ***</div>
<div>[n0617:27216] *** Process received signal ***</div>
<div>[n0617:27216] Signal: Segmentation fault (11)</div>
<div>[n0617:27216] Signal code: Address not mapped (1)</div>
<div>[n0617:27216] Failing at address: (nil)</div>
<div>[n0617:27216] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27216] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27216] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27216] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27216] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27216] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27216] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27216] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27216] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27216] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27216] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27216] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27216] *** End of error message ***</div>
<div>[n0617:27210] *** Process received signal ***</div>
<div>[n0617:27210] Signal: Segmentation fault (11)</div>
<div>[n0617:27210] Signal code: Address not mapped (1)</div>
<div>[n0617:27210] Failing at address: (nil)</div>
<div>[n0617:27210] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27210] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27210] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27210] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27210] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27210] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27210] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27210] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27210] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27210] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27210] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27210] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27210] *** End of error message ***</div>
<div>[n0617:27217] *** Process received signal ***</div>
<div>[n0617:27217] Signal: Segmentation fault (11)</div>
<div>[n0617:27217] Signal code: Address not mapped (1)</div>
<div>[n0617:27217] Failing at address: (nil)</div>
<div>[n0617:27217] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27217] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27217] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27217] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27217] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27217] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27217] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27217] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27217] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27217] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27217] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27217] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27217] *** End of error message ***</div>
<div>[n0617:27213] *** Process received signal ***</div>
<div>[n0617:27213] Signal: Segmentation fault (11)</div>
<div>[n0617:27213] Signal code: Address not mapped (1)</div>
<div>[n0617:27213] Failing at address: (nil)</div>
<div>[n0617:27213] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27213] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27213] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27213] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27213] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27213] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27213] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27213] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27213] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27213] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27213] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27213] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27213] *** End of error message ***</div>
<div>[n0617:27208] *** Process received signal ***</div>
<div>[n0617:27208] Signal: Segmentation fault (11)</div>
<div>[n0617:27208] Signal code: Address not mapped (1)</div>
<div>[n0617:27208] Failing at address: (nil)</div>
<div>[n0617:27208] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27208] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27208] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27208] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27208] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27208] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27208] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27208] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27208] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27208] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27208] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27208] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27208] *** End of error message ***</div>
<div>[n0617:27211] *** Process received signal ***</div>
<div>[n0617:27211] Signal: Segmentation fault (11)</div>
<div>[n0617:27211] Signal code: Address not mapped (1)</div>
<div>[n0617:27211] Failing at address: (nil)</div>
<div>[n0617:27211] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0617:27211] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0617:27211] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0617:27211] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0617:27211] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0617:27211] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0617:27211] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0617:27211] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0617:27211] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0617:27211] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0617:27211] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0617:27211] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0617:27211] *** End of error message ***</div>
<div>[n0628:09116] *** Process received signal ***</div>
<div>[n0628:09116] Signal: Segmentation fault (11)</div>
<div>[n0628:09116] Signal code: Address not mapped (1)</div>
<div>[n0628:09116] Failing at address: (nil)</div>
<div>[n0628:09113] *** Process received signal ***</div>
<div>[n0628:09113] Signal: Segmentation fault (11)</div>
<div>[n0628:09113] Signal code: Address not mapped (1)</div>
<div>[n0628:09113] Failing at address: (nil)</div>
<div>[n0628:09113] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0628:09113] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0628:09113] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0628:09113] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0628:09113] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0628:09113] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0628:09113] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0628:09113] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0628:09113] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0628:09113] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0628:09113] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0628:09113] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0628:09113] *** End of error message ***</div>
<div>[n0628:09116] [ 0] /lib64/libpthread.so.0() [0x39cc40f710]</div>
<div>[n0628:09116] [ 1] /usr/lib64/perl5/CORE/libperl.so(Perl_pp_helem+0x3bd) [0x39d0ca9ded]</div>
<div>[n0628:09116] [ 2] /usr/lib64/perl5/CORE/libperl.so(Perl_runops_standard+0x16) [0x39d0ca4b06]</div>
<div>[n0628:09116] [ 3] /usr/lib64/perl5/CORE/libperl.so(Perl_call_sv+0x4cf) [0x39d0c4c5df]</div>
<div>[n0628:09116] [ 4] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clear+0xb6) [0x39d0cb8dd6]</div>
<div>[n0628:09116] [ 5] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_free2+0x52) [0x39d0cb95d2]</div>
<div>[n0628:09116] [ 6] /usr/lib64/perl5/CORE/libperl.so() [0x39d0cae5c1]</div>
<div>[n0628:09116] [ 7] /usr/lib64/perl5/CORE/libperl.so(Perl_sv_clean_objs+0x21) [0x39d0cae621]</div>
<div>[n0628:09116] [ 8] /usr/lib64/perl5/CORE/libperl.so(perl_destruct+0x11d1) [0x39d0c4e901]</div>
<div>[n0628:09116] [ 9] /usr/bin/perl(main+0xe1) [0x400e01]</div>
<div>[n0628:09116] [10] /lib64/libc.so.6(__libc_start_main+0xfd) [0x39cc01ed1d]</div>
<div>[n0628:09116] [11] /usr/bin/perl() [0x400c59]</div>
<div>[n0628:09116] *** End of error message ***</div>
<div>+ date</div>
<div>Wed Oct 8 12:12:28 EDT 2014</div>
</div>
</span>
</body>
</html>