What about cdbfasta -- wonder if perl Api to this indexing is possible -- or could be NCBI blast index since that also is a dependency in maker. <br><div class="gmail_quote"><div dir="ltr">On Thu, Sep 17, 2015 at 7:05 PM Fields, Christopher J <<a href="mailto:cjfields@illinois.edu">cjfields@illinois.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<div>Carson, </div>
<div><br>
</div>
<div>Thanks! Will pass this on to the folks at NCSA, that should help quite a bit.</div>
<div><br>
</div>
Yeah, I kinda think it would be nice to come up with an alternative indexing scheme for fasta indexing, at least add some more flexibility (I’m guessing this is BioPerl still?).
<div><br>
</div>
<div>chris</div></div><div style="word-wrap:break-word">
<div> <br>
<div>
<blockquote type="cite">
<div>On Sep 16, 2015, at 12:22 PM, Carson Holt <<a href="mailto:carsonhh@gmail.com" target="_blank">carsonhh@gmail.com</a>> wrote:</div>
<br>
<div>
<div style="word-wrap:break-word">
Sorry for the slow reply. I’m out of the lab right now and will be for the next two weeks.
<div><br>
</div>
<div>MAKER uses MPI for parallelization. So it is optimized for distributed non-shared memory systems, but should still work fine on a shared memory system.</div>
<div><br>
</div>
<div>With MPI, you specify the number of processes to start using the -n flag for mpiexec. Each MAKER process will need about 2Gb. It could be more or less depending on the amount of evidence it has to hold in RAM (i.e. deep evidence alignments use
more memory). By default each MAKER process will use a single CPU (even though it will start 3 threads - two of the threads will use close to 0% CPU).</div>
<div><br>
</div>
<div>MAKER will use a lot of IO. Each process will write/read independently of the others, so the more processes you start, the more simultaneous IO you will have. I’ve tried to put most very heavy IO operations in /tmp or whatever temporary directory
you specify. It is important that you never specify an NFS location for your temporary directory. The rest of the IO will occur in the working directory.</div>
<div><br>
</div>
<div>Also the Berkley DB implementation that sits behind the fasta indexes for sequence access don’t always work well with in memory scratch. You should always try and set /tmp to a physical drive if possible. You will get several Gb of files in /tmp.<br>
<div><br>
</div>
<div>—Carson</div>
<div><br>
</div>
<div><br>
<div>
<blockquote type="cite">
<div>On Sep 15, 2015, at 10:39 AM, Fields, Christopher J <<a href="mailto:cjfields@illinois.edu" target="_blank">cjfields@illinois.edu</a>> wrote:</div>
<br>
<div>
<div style="word-wrap:break-word">
<font face="Calibri" style="font-size:14px">We have a group locally (at NCSA) who is interested in profiling MAKER with various performance analysis tools. They would like to know CPU, RAM, I/O patterns and usage. In particular, we’re seeing some
odd performance problems on a local system which uses a large shared memory cache for storing temp/scratch data (/dev/shm). </font>
<div>
<div style="margin:0in 0in 0.0001pt"><font face="Calibri" style="font-size:14px"> </font></div>
</div>
<div>
<div style="margin:0in 0in 0.0001pt"><font face="Calibri" style="font-size:14px">The question is: are there any particular pain points users and developers know of or could point us to that we can start focusing on? Any help would be
greatly appereciated.</font></div>
</div>
<div style="margin:0in 0in 0.0001pt"><font face="Calibri" style="font-size:14px"><br>
</font></div>
<div style="margin:0in 0in 0.0001pt"><font face="Calibri" style="font-size:14px">Thanks,</font></div>
<div style="margin:0in 0in 0.0001pt"><font face="Calibri" style="font-size:14px"><br>
</font></div>
<div style="margin:0in 0in 0.0001pt"><font face="Calibri" style="font-size:14px">chris</font></div>
<div style="margin:0in 0in 0.0001pt"><font face="Calibri" style="font-size:14px"><br>
</font></div>
<div style="margin:0in 0in 0.0001pt"><b style="font-size:10px">Chris Fields</b></div>
<div>
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word">
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<div style="letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-align:start;text-indent:0px;word-wrap:break-word;font-size:10px">
<b>Technical Lead in Genome Informatics</b></div>
<div style="letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-align:start;text-indent:0px;word-wrap:break-word;font-size:10px">
<b>High Performance Computing in Biology</b></div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<font size="1">University of Illinois at Urbana-Champaign</font><br>
<font size="1">Roy J. Carver Biotechnology Center / W.M. Keck Center</font><br>
<font size="1">Carl R. Woese Institute for Genomic Biology</font></div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word">
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
maker-devel mailing list<br>
<a href="mailto:maker-devel@box290.bluehost.com" target="_blank">maker-devel@box290.bluehost.com</a><br>
<a href="http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org" target="_blank">http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org</a><br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
_______________________________________________<br>
maker-devel mailing list<br>
<a href="mailto:maker-devel@box290.bluehost.com" target="_blank">maker-devel@box290.bluehost.com</a><br>
<a href="http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org" rel="noreferrer" target="_blank">http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org</a><br>
</blockquote></div>