<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Runtimes are the result of gene density, evidence dataset size, ans evidence dataset type. For example protein data takes ~10 times longer to process than EST data, and alt-EST data takes ~10 times longer than protein data. If you double the size of input datasets, then you double runtime. Also the assembly size doesn’t seem to have a large effect on runtime. It tends to be gene density that has the most effect, so a 2Gb assembly runs only somewhat slower than a 300Mb assembly containing the same number of genes.<div class=""><br class=""></div><div class="">For best MPI performance, you can submit multiple jobs with 200 CPUs or less. Over 200 CPUs per job tens to get limited throughput increases due to MPI communication overhead. I never use RAM disk. In general MAKER produces too many temporary files to fit in RAM.</div><div class=""><br class=""></div><div class="">—Carson<br class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jul 6, 2015, at 9:37 AM, Ganko Eric USRE <<a href="mailto:eric.ganko@syngenta.com" class="">eric.ganko@syngenta.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">I’m hoping for some advice on an unexpectedly long process time for a 2Gb genome. Currently I’m using an install of MAKER-P on the iForge system @ NCSA and I’ve successfully run ~1Gb genomes in 2-3 hours across 20 nodes (24 Intel "Haswell" cores, 64 GB of RAM per node) via MPICH. <o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">I recently ran some tests on 50Mb of corn that took ~2 hours on 2 nodes (48 cores). Based on that I was surprised when the full 2Gb corn genome run timed out at >24h with 30 nodes (720 cores); in that time it hadn’t processed many sequences based on the master_datastore_index.log :<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">TOTAL: 25000 seqs<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">STARTED: 3594<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">FINISHED: 2979<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">FAILED: 10<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">RETRY: 9<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">DIED_SKIPPED_PERMANENT: 0<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt 0.5in; font-size: 11pt; font-family: Calibri, sans-serif;" class="">SKIPPED_SMALL: 7635<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">While I can set a longer wall clock, these results are several times longer than what was reported in the MAKER-P paper, i.e. running the corn B73 genome in less than 4 hours; here it is not close to done after 24h. I don’t have an enormous amount of supporting data– this trial run has ~100k transcripts and another ~100k proteins. Corn has a very high repeat content, so my suspicion is Repeatmasker IO. In discussions with the iForge admins I have discovered that the temp space is network attached (GPFS), and they’ve suggested using a RAM disk (i.e /dev/shm) as the temp directory. In tests on smaller sequence that ran a little slower so I’m not sure if MAKER is meant to run that way. I’d appreciate input on experience with a RAM disk approach, or if anyone has alternative thoughts or suggestions?<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Thanks,<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Eric<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><class="msonormal" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span lang="EN-US" class=""></span><font color="Gray" face="Arial" size="1" class=""></font><hr class=""><class="a86f9321-94be-422f-8440-19283f7e68c6" class=""><i class=""><span style="font-size: 7.5pt; font-family: Arial;" class=""><font size="1" class="">This message may contain confidential information. If you are not the designated recipient, please notify the sender immediately, and delete the original and any copies. Any use of the message by you is prohibited.</font></span></i><span class="Apple-converted-space"> </span>_______________________________________________<br class="">maker-devel mailing list<br class=""><a href="mailto:maker-devel@box290.bluehost.com" style="color: purple; text-decoration: underline;" class="">maker-devel@box290.bluehost.com</a><br class=""><a href="http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org" style="color: purple; text-decoration: underline;" class="">http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org</a></class="a86f9321-94be-422f-8440-19283f7e68c6"></class="msonormal"></div></blockquote></div><br class=""></div></div></body></html>