<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div>Hi Hayley,</div><div><br class=""></div><div>Which flavor of MPI are you using? OpenMPI, MVAPICH2, MPICH, or Intel MPI? There is a known issue with infiniband network cards that causes them to freeze on certain system calls, and I bet that is what you are seeing. It is not a MAKER issue, but a limitation in order to allow performance benefits elsewhere with MPI. You need to add extra command line flags to tell MPI to communicate a bit differently to get around it. You can find multiple threads with more info in the maker-devel archives here —> <a href="https://groups.google.com/g/maker-devel" class="">https://groups.google.com/g/maker-devel</a></div><div><br class=""></div><div><br class=""></div><div>For OpenMPI you may need to add these flags to the mpiexec command line —> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 --mca btl_openib_want_fork_support 1 --mca mpi_warn_on_fork 0</div><div><br class=""></div><div>For intel MPI set the environmental variable I_MPI_FABRICS=shm:tcp before running mpiexec</div><div><br class=""></div><div>For MPICH, no modifications should be needed (it doesn’t experience the issue).</div><div><br class=""></div><div>For <font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">MVAPICH2, there is no work around and you just need to use one of the other MPI flavors.</span></font></div><div><br class=""></div><div><br class=""></div><div>For the second failure, if you can regenerate it, send me the STDERR and I will take a look.</div><div><br class=""></div><div>Thanks,</div><div>Carson</div><div><br class=""></div><div><br class=""></div><div><br class=""></div><div><blockquote type="cite" class=""><div style="position: relative;" class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class="">From:</b><span class="Apple-converted-space"> </span>Hayley Mangelson <<a href="mailto:hayley@phasegenomics.com" style="color: rgb(149, 79, 114); text-decoration: underline;" class="">hayley@phasegenomics.com</a>><span class="Apple-converted-space"> </span><br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>Thursday, March 4, 2021 12:03 PM<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>AARON A DUFFY <<a href="mailto:aaron.duffy@utah.edu" style="color: rgb(149, 79, 114); text-decoration: underline;" class="">aaron.duffy@utah.edu</a>><br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span>Mary Wood <<a href="mailto:mary@phasegenomics.com" style="color: rgb(149, 79, 114); text-decoration: underline;" class="">mary@phasegenomics.com</a>><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>MAKER Commercial User -- Question<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Hi Aaron, <o:p class=""></o:p></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Mary and I are bioinformaticians at Phase Genomics, and we are using the MAKER software (for which Phase purchased a licence) to annotate a couple of genomes. We are using the same workflow that I used for a previous project, but are having the same issues while working on separate projects. I am hoping you could help us out or direct us to someone who can help!<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Here is some background. I am trying to get it to run using<span class="Apple-converted-space"> </span><i class="">mpiexec</i><span class="Apple-converted-space"> </span>for parallelization. I let it run for about a week and all processes remained “active”, but no new output seemed to be generated. It was hung up on “prepare section files” for nearly the whole 1-week period. I canceled the run, then tried running again without parallelization (in the same folder as the failed run). I got an error about each contig failing, and something about Augustus. Unfortunately, I didn’t save the error message. I tried to regenerate the error, but it looks like I would probably need to completely rerun… this is the error message now (repeated for each contig), if I run in the same folder again.<br class=""><br class=""><i class="">The contig failed 2 times!!<br class="">Maker will not try again!!<br class="">The contig will be stored in a fasta file that you can use for debugging.<br class="">SeqID: PGA_scaffold4__8_contigs__length_31799528<br class="">Length: 31799528<br class="">FASTA: /home/ubuntu/PGScratch/chunk1/chunk1.maker.output/chunk1_datastore/10/91/PGA_scaffold4__8_contigs__length_31799528//PGA_scaffold4__8_contigs__length_31799528.died.fasta</i><br class=""><br class="">Any thoughts about why<span class="Apple-converted-space"> </span><i class="">mpiexec<span class="Apple-converted-space"> </span></i>seems unable to finish? I tried a run on the test data using parallelization, and it completed in under a minute.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Thanks!<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Hayley<br class="">--<span class="Apple-converted-space"> </span><o:p class=""></o:p></div><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="color: rgb(102, 102, 102);" class="">Hayley Mangelson</span><o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="color: rgb(102, 102, 102);" class="">Bioinformatics Scientist</span><o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="color: rgb(102, 102, 102);" class="">Phase Genomics, Inc</span></div></div></div></div></div></div></div></div></div></div></div></blockquote></div><br class=""></body></html>