<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Tuesday I ran the same option files, this time with 480 cores, and the annotation completed in ~6 hours. Perhaps I’m trying too many simultaneous writes at
higher levels, or there is too much MPI communication as you mentioned… Thanks for the input on the RAM disk.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">-eric<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Carson Holt [mailto:carsonhh@gmail.com]
<br>
<b>Sent:</b> Thursday, July 09, 2015 3:01 PM<br>
<b>To:</b> Ganko Eric USRE<br>
<b>Cc:</b> maker-devel@yandell-lab.org<br>
<b>Subject:</b> Re: [maker-devel] MAKER processing time in a 2Gb genome<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Runtimes are the result of gene density, evidence dataset size, ans evidence dataset type. For example protein data takes ~10 times longer to process than EST data, and alt-EST data takes ~10 times longer than protein data. If you double
the size of input datasets, then you double runtime. Also the assembly size doesn’t seem to have a large effect on runtime. It tends to be gene density that has the most effect, so a 2Gb assembly runs only somewhat slower than a 300Mb assembly containing
the same number of genes.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For best MPI performance, you can submit multiple jobs with 200 CPUs or less. Over 200 CPUs per job tens to get limited throughput increases due to MPI communication overhead. I never use RAM disk. In general MAKER produces too many
temporary files to fit in RAM.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">—Carson<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Jul 6, 2015, at 9:37 AM, Ganko Eric USRE <<a href="mailto:eric.ganko@syngenta.com">eric.ganko@syngenta.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">I’m hoping for some advice on an unexpectedly long process time for a 2Gb genome. Currently I’m using an install of MAKER-P on the iForge system @ NCSA and I’ve successfully
run ~1Gb genomes in 2-3 hours across 20 nodes (24 Intel "Haswell" cores, 64 GB of RAM per node) via MPICH. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">I recently ran some tests on 50Mb of corn that took ~2 hours on 2 nodes (48 cores). Based on that I was surprised when the full 2Gb corn genome run timed out at >24h with
30 nodes (720 cores); in that time it hadn’t processed many sequences based on the master_datastore_index.log :<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">TOTAL: 25000 seqs<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">STARTED: 3594<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">FINISHED: 2979<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">FAILED: 10<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">RETRY: 9<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">DIED_SKIPPED_PERMANENT: 0<o:p></o:p></span></p>
</div>
<div style="margin-left:.5in">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">SKIPPED_SMALL: 7635<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">While I can set a longer wall clock, these results are several times longer than what was reported in the MAKER-P paper, i.e. running the corn B73 genome in less than 4 hours;
here it is not close to done after 24h. I don’t have an enormous amount of supporting data– this trial run has ~100k transcripts and another ~100k proteins. Corn has a very high repeat content, so my suspicion is Repeatmasker IO. In discussions with the iForge
admins I have discovered that the temp space is network attached (GPFS), and they’ve suggested using a RAM disk (i.e /dev/shm) as the temp directory. In tests on smaller sequence that ran a little slower so I’m not sure if MAKER is meant to run that way. I’d
appreciate input on experience with a RAM disk approach, or if anyone has alternative thoughts or suggestions?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Thanks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">Eric<o:p></o:p></span></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<class="MsoNormal"><span lang="EN-US"></span><font color="Gray" face="Arial" size="1"></font>
<hr>
<class="a86f9321-94be-422f-8440-19283f7e68c6"><i><span style="FONT-SIZE: 7.5pt; FONT-FAMILY: Arial"><font size="1">This message may contain confidential information. If you are not the designated recipient, please notify the sender immediately, and delete the
original and any copies. Any use of the message by you is prohibited.</font></span></i>
</body>
</html>