<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hello folks, <br>
</p>
<p>Can anyone inform me on the ability of MAKER to restart from a
checkpoint following annotation processing has compelted. <br>
</p>
<p>I had an MPI MAKER job running successfully for 6 weeks for a de
novo fly genome I am working on. It was running with
mpich3-3.1-icc on LSF batch system using 96 cpu's and 140Gb RAM.
MAKER had processed 91% of the overall assembly length of my
genome under MAKER_Finished contigs. Numbers of "Finished" contigs
hadn't changed for ~10 days when it died, as I assume MAKER was
collecting annotated gene stats, collecting contig statistics and
clustering of transcripts into fasta files etc: (As follows)<br>
</p>
<p><i>............</i></p>
<p><i>clustering transcripts into genes for annotations</i><i><br>
</i><i>Processing transcripts into genes</i><i><br>
</i><i>adding statistics to annotations</i><i><br>
</i><i>Calculating annotation quality statistics</i><i><br>
</i><i>choosing best annotation set</i><i><br>
</i><i>Choosing best annotations</i><i><br>
</i><i>processing chunk output</i><i><br>
</i><i>processing contig output</i></p>
<p>However, this job exited after processing 26,767 of the 42,207
"MAKER Finished" contigs. The job died with a 255 exit code, which
I suspect means somoene in our systems team may have killed the
job to maintain system stability or someting. <br>
</p>
<p><b><u>The following error output was captured</u></b><i><b><u>:</u></b><br>
</i></p>
<p><font color="#990000">Calculating annotation quality statistics<br>
choosing best annotation set<br>
Choosing best annotations<br>
processing chunk output<br>
processing contig output</font></p>
<p><font color="#990000">[<a class="moz-txt-link-abbreviated" href="mailto:proxy:0:0@loom7.ebi.ac.uk">proxy:0:0@loom7.ebi.ac.uk</a>]
HYD_pmcd_pmip_control_cmd_cb (./pm/pmiserv/pmip_cb.c:886):
assert (!closed) failed<br>
[<a class="moz-txt-link-abbreviated" href="mailto:proxy:0:0@loom7.ebi.ac.uk">proxy:0:0@loom7.ebi.ac.uk</a>] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status<br>
[<a class="moz-txt-link-abbreviated" href="mailto:proxy:0:0@loom7.ebi.ac.uk">proxy:0:0@loom7.ebi.ac.uk</a>] main (./pm/pmiserv/pmip.c:206):
demux engine error waiting for event<br>
[proxy:0:1@loom15] HYD_pmcd_pmip_control_cmd_cb
(./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed<br>
[proxy:0:5@loom14] HYD_pmcd_pmip_control_cmd_cb
(./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed<br>
[proxy:0:5@loom14] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status<br>
[<a class="moz-txt-link-abbreviated" href="mailto:proxy:0:4@loom6.ebi.ac.uk">proxy:0:4@loom6.ebi.ac.uk</a>] HYD_pmcd_pmip_control_cmd_cb
(./pm/pmiserv/pmip_cb.c:886): assert (!closed) failed<br>
[<a class="moz-txt-link-abbreviated" href="mailto:proxy:0:4@loom6.ebi.ac.uk">proxy:0:4@loom6.ebi.ac.uk</a>] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status<br>
[<a class="moz-txt-link-abbreviated" href="mailto:proxy:0:4@loom6.ebi.ac.uk">proxy:0:4@loom6.ebi.ac.uk</a>] main (./pm/pmiserv/pmip.c:206):
demux engine error waiting for event<br>
[proxy:0:5@loom14] main (./pm/pmiserv/pmip.c:206): demux engine
error waiting for event<br>
[proxy:0:1@loom15] HYDT_dmxu_poll_wait_for_event
(./tools/demux/demux_poll.c:77): callback returned error status<br>
[proxy:0:1@loom15] main (./pm/pmiserv/pmip.c:206): demux engine
error waiting for event<br>
[<a class="moz-txt-link-abbreviated" href="mailto:mpiexec@loom7.ebi.ac.uk">mpiexec@loom7.ebi.ac.uk</a>] HYDT_bscu_wait_for_completion
(./tools/bootstrap/utils/bscu_wait.c:76): one of the processes
terminated badly; aborting<br>
[<a class="moz-txt-link-abbreviated" href="mailto:mpiexec@loom7.ebi.ac.uk">mpiexec@loom7.ebi.ac.uk</a>] HYDT_bsci_wait_for_completion
(./tools/bootstrap/src/bsci_wait.c:23): launcher returned error
waiting for completion<br>
[<a class="moz-txt-link-abbreviated" href="mailto:mpiexec@loom7.ebi.ac.uk">mpiexec@loom7.ebi.ac.uk</a>] HYD_pmci_wait_for_completion
(./pm/pmiserv/pmiserv_pmci.c:217): launcher returned error
waiting for completion<br>
[<a class="moz-txt-link-abbreviated" href="mailto:mpiexec@loom7.ebi.ac.uk">mpiexec@loom7.ebi.ac.uk</a>] main (./ui/mpich/mpiexec.c:331):
process manager error waiting for completion</font></p>
<p><font color="#990000"><font color="#000000">Unfortunately since
this process died, I have been unable to get the job
reschduled again on our system due to resource limitations and
job queing. But can anyone tell me, will MAKER be able to
finish processing contig stats and information to completion,
following this early exit ? I really can't afford another 6
weeks of computation so Im worried as you might expect. Would
you recommend I submit this MAKER job again to finalize contig
information/produce fasta files etc with the same amount of
resources, or might I be able to request less resources
without too much of a penalty in terms of compute time. <br>
</font></font></p>
<p><font color="#990000"><font color="#000000">Any hints or insight
on this would be greatly appreciated. <br>
</font></font></p>
<p><font color="#990000"><font color="#000000">Thank you in advance,</font></font></p>
<p><font color="#990000"><font color="#000000">Lahcen <br>
</font></font></p>
<p><font color="#990000"><font color="#000000">EBI-Hinxton, UK. <br>
</font></font></p>
<p><br>
</p>
</body>
</html>