[maker-devel] MAKER Commercial User -- Install and run issues

Fri Nov 10 11:29:52 MST 2023

Are you able to give me the specific errors for those contigs when run in isolation (i.e. redirect  STDERR to a file)?

--Carson

From: Hayley Mangelson <hayley at phasegenomics.com>
Date: Monday, November 6, 2023 at 9:03 AM
To: Carson Hinton Holt <carson.holt at genetics.utah.edu>
Cc: AARON A DUFFY <Aaron.Duffy at tvc.utah.edu>, Maker Mailing List <maker-devel at yandell-lab.org>
Subject: Re: [maker-devel] MAKER Commercial User -- Install and run issues
Hello again, Carson,

I was able to get most of these contigs to run... turns out the fasta headers were too long. Two of them still failed in augustus, though. I was able to run them with no ab-initio gene prediction, but not sure how to proceed at this point as the results for those two contigs are not treated as "genes" when generating the transcript files, etc. What do you recommend?

Thanks,

Hayley

On Tue, Sep 26, 2023 at 2:07 PM Carson Hinton Holt <carson.holt at genetics.utah.edu<mailto:carson.holt at genetics.utah.edu>> wrote:
What does the error say? One thing you can do is run on a single thread (i.e. no mpiexec, just call maker directly - increase try count before doing this). That will run up to where the error happens, and the last thing it prints out before the error is the Repeatmasker command line used. You can then cut and paste that command to test it outside of MAKER.

—Carson

On Sep 26, 2023, at 8:22 AM, Hayley Mangelson <hayley at phasegenomics.com<mailto:hayley at phasegenomics.com>> wrote:

Hi Carson,

Thought I should report back that this worked! Now I am just dealing with 4/32 of my contigs failing. It looks like they are failing on the repeat masking step. I am a bit confused as to why just a handful would fail here. Any suggestions on how to overcome this (hopefully final) obstacle?

Thanks,

Hayley

On Wed, Sep 13, 2023 at 11:20 AM Carson Hinton Holt <carson.holt at genetics.utah.edu<mailto:carson.holt at genetics.utah.edu>> wrote:
The seg fault is being thrown by MPI itself. That is why MAKER can run on its own.

There are several things you can check (or just do them all as a sort of nuclear option):
*First make sure you are not using conda or miniconda environments. This can create subtle conflicts between executables and libraries as condo/minicanda will try to override some paths.
*Next you may need to install OpenMPI or MPICH yourself (i.e. don’t use apt-get or other package install managers). https://www.mpich.org/downloads/.
*Lastly you can also install your own perl and override the system perl. Depends on your system configuration, i.e. if using CentOS 7, your perl is very out of date with other libraries that may have been updated on the system.

After compiling everything and installing, you will want to reinstall MAKER (the new MPI install must recompile it’s bindings to the application bering run).

Here is a step by step (comments marked with ‘#'):
#create a directory to prep install
mkdir $HOME/install_prep

#install MPICH
cd $HOME/install_prep
wget https://www.mpich.org/static/downloads/4.1.2/mpich-4.1.2.tar.gz
tar -zxvf mpich-4.1.2.tar.gz
cd mpich-4.1.2/
./configure --prefix=$HOME/opt/mpich
make
make install
echo 'export PATH=$HOME/opt/mpich/bin:$PATH' >> ~/.bash_profile
source ~/.bash_profile

#install a new Perl
cd $HOME/install_prep
wget https://www.cpan.org/src/5.0/perl-5.38.0.tar.gz --no-check-certificate
tar -zxvf perl-5.38.0.tar.gz
cd perl-5.38.0
./Configure -d -Dprefix=$HOME/opt/perl5
make
make install
echo 'export PATH=$HOME/opt/perl5/bin:$PATH' >> ~/.bash_profile
source ~/.bash_profile

#install MAKER with MPI support
cd $HOME/opt
wget http://weatherby.genetics.utah.edu/maker_downloads/AD6E/A1B0/C2B7/558563CFD35E343BA1B888148294/maker-3.01.04.tgz
tar -zxvf maker-3.01.04.tgz
rm -rf maker-3.01.04.tgz
cd maker/src
perl Build.PL ##say yes to MPI question
./Build installdeps ##just hit 'enter' to accept any defaults
### if you get errors mentioning '--no-check-certificate', run the command 'cpan' in your terminal.
### If it asks to set up everything automatically, say 'yes'.
### Next, whil einside cpan, run the command 'o conf init pushy_https'.
### Say 'no' to the question it asks, and then run 'o conf commit' to save changes and 'q' to exit.
### Then retry the './Build installdeps' command
./Build installexes
./Build install
echo 'export PATH=$HOME/opt/maker/bin:$PATH' >> ~/.bash_profile
source ~/.bash_profile

#test MAKER
cd $HOME/install_prep
mpiexec -n 10 maker --help ##should print a single help message
cp -R $HOME/opt/maker/data ./test
cd test
maker -CTL
mpiexec -n 10 maker --protein dpp_protein.fasta --genome dpp_contig.fasta --est dpp_est.fasta ##should run MAKER under MPI

—Carson

On Sep 12, 2023, at 2:22 PM, Hayley Mangelson <hayley at phasegenomics.com<mailto:hayley at phasegenomics.com>> wrote:

Hi Carson,

I have tried installing MPICH and OpenMPI various ways now and am still having issues. MAKER appears to be running when I am not using MPI, but I am attaching the maker output (with -debug flag) and here is the error I get when trying to multithread.
===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 35952 RUNNING AT ip-172-31-12-194
=   EXIT CODE: 9
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions

On Thu, Aug 24, 2023 at 6:26 PM Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
Once you install MPICH or OpenMPI you must launch MAKER through mpiexec and speciffy the CPU count to MPI. On a single instance you just give the CPU count off that instance, but you can also go across instances using Amazon’s Parallel Cluster.

Here is an example —> https://efish.integrativebiology.msu.edu/2015/02/10/annotate.html
That example uses StarCluster which is now deprecated, but the MPI install and MAKER run still apply.

Here is another example of using MPi —> https://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_WGS_Assembly_and_Annotation_Winter_School_2018#MPI_Support

Note that for OpenMPI you have to set the LD_PRELOAD environmental variable before launch.
export LD_PRELOAD=/usr/lib64/openmpi-1.10/lib/libmpi.so

—Carson

On Aug 18, 2023, at 2:53 PM, Hayley Mangelson <hayley at phasegenomics.com<mailto:hayley at phasegenomics.com>> wrote:

I am working on an AWS EC2 instance that runs Amazon Linux, and I have tried a couple of installs (MPICH and Open MPI), neither of which worked. I don't have any installed on my current instance.

On Tue, Aug 15, 2023 at 5:08 PM Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
Do you already have MPI installed on your system, and if so which MPI and what version (i.e. MPICH, MVAPICH2, Intel MPI, Open MPI, etc)?

—Carson

On Aug 15, 2023, at 9:34 AM, Hayley Mangelson <hayley at phasegenomics.com<mailto:hayley at phasegenomics.com>> wrote:

Hi Carson,

Excellent, thanks so much! I have that part working. The multithreading is still eluding me. Do you have a recommended method of installing a compatible version of MPI?

Thanks,

Hayley

On Fri, Aug 11, 2023 at 4:04 PM Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
The error is coming from Augustus. There is a problem with the species file you selected. Or you put it in the wrong place.

This is the error —>
Could not find the config file /home/ec2-user/MAKERTOOLS/maker/exe/augustus/config/species/BUSCO_evans_trout_hap1/BUSCO_evans_trout_hap1_parameters.cfg

The species info must exist under this base directory by default (it’s where you let MAKER install Augustus) —>  /home/ec2-user/MAKERTOOLS/maker/exe/augustus/config/species/

You can also place the species files somewhere else but you need to set the AUGUSTUS_CONFIG_PATH so Augustus knows where to look.

Example (in your ~/.bash_profile):
export AUGUSTUS_CONFIG_PATH=$HOME/muy_augustus_species

Thanks,
Carson

On Aug 11, 2023, at 7:20 AM, Hayley Mangelson <hayley at phasegenomics.com<mailto:hayley at phasegenomics.com>> wrote:

Hi Carson,

Sure thing! Sorry for the delay, I was on vacation. I reran with just a single scaffold. Here is the STDERR.

On Mon, Jul 31, 2023 at 12:06 PM Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
Can you send me the STDERR from the maker run? If you don’t have it, run maker again with the output from the screen redirected to a file. Example —> maker &> all_output.txt

--Carson

Sent from my iPhone

On Jul 31, 2023, at 10:04 AM, Hayley Mangelson <hayley at phasegenomics.com<mailto:hayley at phasegenomics.com>> wrote:

Hi!

I am trying to get a fresh install of MAKER to work for me and am having a few issues.

1. I got MAKER installed (I thought) and now the log is saying all contigs failed. See attached log.
2. Having tried several versions and methods of installation, I cannot get multithreading to work.

Could you help me work through these issues?
Thanks,

Hayley
--
Hayley Mangelson
Senior Bioinformatics Lead
Phase Genomics, Inc
www.phasegenomics.com<http://www.phasegenomics.com/>

<PGA_assembly_hap1_master_datastore_index.log>
_______________________________________________
maker-devel mailing list
maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>
http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org

--
Hayley Mangelson
Senior Bioinformatics Lead
Phase Genomics, Inc
www.phasegenomics.com<http://www.phasegenomics.com/>

<all_output.txt>

--
Hayley Mangelson
Senior Bioinformatics Lead
Phase Genomics, Inc
www.phasegenomics.com<http://www.phasegenomics.com/>

--
Hayley Mangelson
Senior Bioinformatics Lead
Phase Genomics, Inc
www.phasegenomics.com<http://www.phasegenomics.com/>

--
Hayley Mangelson
Senior Bioinformatics Lead
Phase Genomics, Inc
www.phasegenomics.com<http://www.phasegenomics.com/>

<all_output.txt>

--
Hayley Mangelson
Senior Bioinformatics Lead
Phase Genomics, Inc
www.phasegenomics.com<http://www.phasegenomics.com/>

--
Hayley Mangelson
Senior Bioinformatics Lead
Phase Genomics, Inc
www.phasegenomics.com<http://www.phasegenomics.com/>

[Image removed by sender.]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20231110/57c030f3/attachment-0002.html>