[maker-devel] thread terminated, causing all processes to fail

[Ramón Fallon]

This is a standalone machine and no NFS at all. "df" gives a healthy amount
of disk space, so there should be no problem there.

Yes that file does exist although it has the nominal 12288 bytes size,
which appears to be the minimum for a DB_file tie.

As I mentioned the dpp_contig.fa example set does work so part of my
investigation is looking at how.

I can do some trivial unit tests on the Bioperl stat-before-tied-hashes
situation and see what comes up.

So I'll attempt to clear that up and then revert.

Many thanks! / Ramón.


On Thu, Mar 7, 2013 at 5:44 PM, Carson Holt <carsonhh at gmail.com> wrote:

> That is extremely odd.  It fails to even generate the indexes. Could you
> check the drive space of your working directory and your /tmp directory?
>
> It is odd because Bioperl uses the stat command to check on the file right
> before making a tied hash.  So it was there for the stat but not the tie,
> which is immediately following.
>
> If you check manually does it exist now? -->
>  /home/ramonf/makertrials/mgallocut7/sca29310_8.maker.output/mpi_blastdb/sca29310_8%2Efa.mpi.1/sca29310_8%2Efa.mpi.1.0.index
>
> Are you running in an NFS mounted directory?
>
> --Carson
>
>
> From: Ramón Fallon <ramonfallon at gmail.com>
> Date: Thursday, 7 March, 2013 9:40 AM
>
> To: Carson Holt <carson.holt at oicr.on.ca>
> Cc: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> Subject: Re: [maker-devel] thread terminated, causing all processes to
> fail
>
> Hi Carson,
>
> I send you a zip of the text file of my repeated maker session, this time
> having deleted the mpi_blastdb dir and with the -a flag added to "mpiexec
> -n 8 maker -debug". Command line.
>
> Cheers / Ramón.
>
>
> On Wed, Mar 6, 2013 at 7:49 PM, Ramón Fallon <ramonfallon at gmail.com>wrote:
>
>> OK, will do.
>>
>> Will get back to you tomorrow on it.
>>
>> Many thanks!
>>
>>
>> On Wed, Mar 6, 2013 at 7:22 PM, Carson Holt <Carson.Holt at oicr.on.ca>wrote:
>>
>>> Could you delete your ../*maker.output/mpi_blastdb directory, and then
>>> when rerunning maker, run with the –a flag.
>>>
>>> Thanks,
>>> Carson
>>>
>>>
>>> From: Ramón Fallon <ramonfallon at gmail.com>
>>> Date: Wednesday, 6 March, 2013 1:15 PM
>>> To: Carson Holt <carson.holt at oicr.on.ca>
>>> Cc: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
>>>
>>> Subject: Re: thread terminated, causing all processes to fail
>>>
>>> OK great, here goes .. many thanks!
>>>
>>>
>>>
>>> On Wed, Mar 6, 2013 at 7:04 PM, Carson Holt <Carson.Holt at oicr.on.ca>wrote:
>>>
>>>> If you do reply all to this message, I should get the attachment.  It
>>>> will be stripped from the one going to the list though.
>>>>
>>>> Thanks,
>>>> Carson
>>>>
>>>>
>>>>
>>>> From: Ramón Fallon <ramonfallon at gmail.com>
>>>> Date: Wednesday, 6 March, 2013 12:57 PM
>>>> To: <maker-devel at yandell-lab.org>
>>>> Subject: Re: thread terminated, causing all processes to fail
>>>>
>>>> Hi,
>>>>
>>>> Many thanks for your quick reply and hint.
>>>>
>>>> Yes, you're right .. further up there is indeed
>>>>
>>>> Calling FastaDB::new at /opt/src/maker_svn/bin/../lib/FastaSeq.pm line
>>>> 148 thread 1.
>>>> Thread 1 terminated abnormally: ERROR: Could not reestablish DB to thaw
>>>> FastaSeq for Storable
>>>> --> rank=5, hostname=fatnode, at /opt/src/maker_svn/bin/maker line 1457
>>>> thread 1.
>>>>
>>>> I run a "script" session and have maker on -debug so I have everything
>>>> in one file. Do you prefer to have it attached to a post to this mailing
>>>> list (if it accepts txt attachments)
>>>>
>>>> Cheers.
>>>>
>>>>
>>>> On Wed, Mar 6, 2013 at 6:34 PM, Ramón Fallon <ramonfallon at gmail.com>wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I'm using the maker_svn rev 995 version and hand-compiled MPICH2 on a
>>>>> single multicore machine.
>>>>>
>>>>> I've successfully run the dpp_contig.fasta (MPI/8 processes) example
>>>>> but am having trouble with larger contigs fasta files of my own, which are
>>>>> well formed.
>>>>>
>>>>> I've run into a problem whereby an mpiexec run of 8 processes will
>>>>> stop due to a perl-thread related problem which says
>>>>>
>>>>> FATAL: Thread terminated, causing all processes to fail
>>>>>
>>>>> this corresponds to line 924 in the maker executable (which is for the
>>>>> secondary/worker threads), and is the result of a test on !$thr OR'd with
>>>>> !$thr->is_running, so clearly one of these is failing.
>>>>>
>>>>> $thr itself is a threads->new(\&$node_thread, $gdbfile). Despite being
>>>>> a programmer, I've only recently started to look at the code and have not
>>>>> got the hang of the parallelisation setup here, though I gather the master
>>>>> must use threads to initially generate the parallel instances which then
>>>>> use the message passing. Of course threads don't have message passing
>>>>> ability, so I guess something clever is going on and will take some time
>>>>> for me to understand.
>>>>>
>>>>> Clearly however, it has worked before on dpp_contigs, so it may be is
>>>>> something wrong with my datafile or the way I am carrying out the analysis.
>>>>>
>>>>> Any clues that can be put my way are welcome.
>>>>>
>>>>> Thank you!
>>>>>
>>>>
>>>>
>>>
>>
> _______________________________________________ maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20130307/e759ca54/attachment-0003.html>