<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Yes. Still BioPerl. You’re right, I probably need to switch indexing schemes. I’ve actually made a faidx implementation, but don’t particularly like it. An NCBI index API might be more ideal.<div class=""><br class=""></div><div class="">—Carson</div><div class=""><br class=""><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Sep 17, 2015, at 8:50 PM, Fields, Christopher J <<a href="mailto:cjfields@illinois.edu" class="">cjfields@illinois.edu</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div class="">Possibly. Might be also feasible to use faidx via samtools API (if we’re intent on that path, there is Bio::DB::Sam, where I added a branch with samtools 1.2 support so could possibly tap into faidx at the XS level).</div>
<div class=""><br class="">
</div>
<div class="">chris</div>
<br class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On Sep 17, 2015, at 9:25 PM, Jason Stajich <<a href="mailto:jason.stajich@gmail.com" class="">jason.stajich@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">What about cdbfasta -- wonder if perl Api to this indexing is possible -- or could be NCBI blast index since that also is a dependency in maker.
<br class="">
<div class="gmail_quote">
<div dir="ltr" class="">On Thu, Sep 17, 2015 at 7:05 PM Fields, Christopher J <<a href="mailto:cjfields@illinois.edu" class="">cjfields@illinois.edu</a>> wrote:<br class="">
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word" class="">
<div class="">Carson, </div>
<div class=""><br class="">
</div>
<div class="">Thanks! Will pass this on to the folks at NCSA, that should help quite a bit.</div>
<div class=""><br class="">
</div>
Yeah, I kinda think it would be nice to come up with an alternative indexing scheme for fasta indexing, at least add some more flexibility (I’m guessing this is BioPerl still?).
<div class=""><br class="">
</div>
<div class="">chris</div>
</div>
<div style="word-wrap:break-word" class="">
<div class=""> <br class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On Sep 16, 2015, at 12:22 PM, Carson Holt <<a href="mailto:carsonhh@gmail.com" target="_blank" class="">carsonhh@gmail.com</a>> wrote:</div>
<br class="">
<div class="">
<div style="word-wrap:break-word" class="">Sorry for the slow reply. I’m out of the lab right now and will be for the next two weeks.
<div class=""><br class="">
</div>
<div class="">MAKER uses MPI for parallelization. So it is optimized for distributed non-shared memory systems, but should still work fine on a shared memory system.</div>
<div class=""><br class="">
</div>
<div class="">With MPI, you specify the number of processes to start using the -n flag for mpiexec. Each MAKER process will need about 2Gb. It could be more or less depending on the amount of evidence it has to hold in RAM (i.e. deep evidence alignments use
more memory). By default each MAKER process will use a single CPU (even though it will start 3 threads - two of the threads will use close to 0% CPU).</div>
<div class=""><br class="">
</div>
<div class="">MAKER will use a lot of IO. Each process will write/read independently of the others, so the more processes you start, the more simultaneous IO you will have. I’ve tried to put most very heavy IO operations in /tmp or whatever temporary directory
you specify. It is important that you never specify an NFS location for your temporary directory. The rest of the IO will occur in the working directory.</div>
<div class=""><br class="">
</div>
<div class="">Also the Berkley DB implementation that sits behind the fasta indexes for sequence access don’t always work well with in memory scratch. You should always try and set /tmp to a physical drive if possible. You will get several Gb of files in /tmp.<br class="">
<div class=""><br class="">
</div>
<div class="">—Carson</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<div class="">
<blockquote type="cite" class="">
<div class="">On Sep 15, 2015, at 10:39 AM, Fields, Christopher J <<a href="mailto:cjfields@illinois.edu" target="_blank" class="">cjfields@illinois.edu</a>> wrote:</div>
<br class="">
<div class="">
<div style="word-wrap:break-word" class=""><font face="Calibri" style="font-size:14px" class="">We have a group locally (at NCSA) who is interested in profiling MAKER with various performance analysis tools. They would like to know CPU, RAM, I/O patterns and
usage. In particular, we’re seeing some odd performance problems on a local system which uses a large shared memory cache for storing temp/scratch data (/dev/shm). </font>
<div class="">
<div style="margin:0in 0in 0.0001pt" class=""><font face="Calibri" style="font-size:14px" class=""> </font></div>
</div>
<div class="">
<div style="margin:0in 0in 0.0001pt" class=""><font face="Calibri" style="font-size:14px" class="">The question is: are there any particular pain points users and developers know of or could point us to that we can start focusing on? Any help would be greatly
appereciated.</font></div>
</div>
<div style="margin:0in 0in 0.0001pt" class=""><font face="Calibri" style="font-size:14px" class=""><br class="">
</font></div>
<div style="margin:0in 0in 0.0001pt" class=""><font face="Calibri" style="font-size:14px" class="">Thanks,</font></div>
<div style="margin:0in 0in 0.0001pt" class=""><font face="Calibri" style="font-size:14px" class=""><br class="">
</font></div>
<div style="margin:0in 0in 0.0001pt" class=""><font face="Calibri" style="font-size:14px" class="">chris</font></div>
<div style="margin:0in 0in 0.0001pt" class=""><font face="Calibri" style="font-size:14px" class=""><br class="">
</font></div>
<div style="margin:0in 0in 0.0001pt" class=""><b style="font-size:10px" class="">Chris Fields</b></div>
<div class="">
<div style="letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wrap:break-word" class="">
<div style="text-align:start;text-indent:0px;word-wrap:break-word" class="">
<div style="text-align:start;text-indent:0px;word-wrap:break-word" class="">
<div style="text-align:start;text-indent:0px;word-wrap:break-word" class="">
<div style="text-align:start;text-indent:0px;word-wrap:break-word" class="">
<div style="text-align:start;text-indent:0px;word-wrap:break-word" class="">
<div style="letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-align:start;text-indent:0px;word-wrap:break-word;font-size:10px" class="">
<b class="">Technical Lead in Genome Informatics</b></div>
<div style="letter-spacing:normal;text-transform:none;white-space:normal;word-spacing:0px;text-align:start;text-indent:0px;word-wrap:break-word;font-size:10px" class="">
<b class="">High Performance Computing in Biology</b></div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word" class=""><font size="1" class="">University of Illinois at Urbana-Champaign</font><br class="">
<font size="1" class="">Roy J. Carver Biotechnology Center / W.M. Keck Center</font><br class="">
<font size="1" class="">Carl R. Woese Institute for Genomic Biology</font></div>
<div style="text-align:start;text-indent:0px;word-wrap:break-word" class=""><br class="">
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
_______________________________________________<br class="">
maker-devel mailing list<br class="">
<a href="mailto:maker-devel@box290.bluehost.com" target="_blank" class="">maker-devel@box290.bluehost.com</a><br class="">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=fbHa8Njtvh9VmSnzJxiEUTW9NWDwMMwQAzhgZDO41GQ&m=3kWOgPLUEWO5vIQ6wgmfto8L8wscvRDsXktQwovt8e0&s=Tos660YhJSdwQx-rJPlr5TCprK7416rThaZR-JdQwhE&e=" target="_blank" class="">http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
_______________________________________________<br class="">
maker-devel mailing list<br class="">
<a href="mailto:maker-devel@box290.bluehost.com" target="_blank" class="">maker-devel@box290.bluehost.com</a><br class="">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=AwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=fbHa8Njtvh9VmSnzJxiEUTW9NWDwMMwQAzhgZDO41GQ&m=3kWOgPLUEWO5vIQ6wgmfto8L8wscvRDsXktQwovt8e0&s=Tos660YhJSdwQx-rJPlr5TCprK7416rThaZR-JdQwhE&e=" rel="noreferrer" target="_blank" class="">http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org</a><br class="">
</blockquote>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</div></blockquote></div><br class=""></div></div></body></html>