[maker-devel] MAKER Commercial User -- Install and run issues

Mon Nov 6 09:00:41 MST 2023

Hello again, Carson,

I was able to get most of these contigs to run... turns out the fasta
headers were too long. Two of them still failed in augustus, though. I was
able to run them with no ab-initio gene prediction, but not sure how to
proceed at this point as the results for those two contigs are not treated
as "genes" when generating the transcript files, etc. What do you recommend?

Thanks,

Hayley

On Tue, Sep 26, 2023 at 2:07 PM Carson Hinton Holt <
carson.holt at genetics.utah.edu> wrote:

> What does the error say? One thing you can do is run on a single thread
> (i.e. no mpiexec, just call maker directly - increase try count before
> doing this). That will run up to where the error happens, and the last
> thing it prints out before the error is the Repeatmasker command line used.
> You can then cut and paste that command to test it outside of MAKER.
>
> —Carson
>
> On Sep 26, 2023, at 8:22 AM, Hayley Mangelson <hayley at phasegenomics.com>
> wrote:
>
> Hi Carson,
>
> Thought I should report back that this worked! Now I am just dealing with
> 4/32 of my contigs failing. It looks like they are failing on the repeat
> masking step. I am a bit confused as to why just a handful would fail here.
> Any suggestions on how to overcome this (hopefully final) obstacle?
>
> Thanks,
>
> Hayley
>
> On Wed, Sep 13, 2023 at 11:20 AM Carson Hinton Holt <
> carson.holt at genetics.utah.edu> wrote:
>
>> The seg fault is being thrown by MPI itself. That is why MAKER can run on
>> its own.
>>
>> There are several things you can check (or just do them all as a sort of
>> nuclear option):
>> *First make sure you are not using conda or miniconda environments. This
>> can create subtle conflicts between executables and libraries as
>> condo/minicanda will try to override some paths.
>> *Next you may need to install OpenMPI or MPICH yourself (i.e. don’t use
>> apt-get or other package install managers).
>> https://www.mpich.org/downloads/.
>> *Lastly you can also install your own perl and override the system perl.
>> Depends on your system configuration, i.e. if using CentOS 7, your perl is
>> very out of date with other libraries that may have been updated on the
>> system.
>>
>> After compiling everything and installing, you will want to reinstall
>> MAKER (the new MPI install must recompile it’s bindings to the application
>> bering run).
>>
>>
>> Here is a step by step (comments marked with ‘#'):
>> #create a directory to prep install
>> mkdir $HOME/install_prep
>>
>> #install MPICH
>> cd $HOME/install_prep
>> wget https://www.mpich.org/static/downloads/4.1.2/mpich-4.1.2.tar.gz
>> tar -zxvf mpich-4.1.2.tar.gz
>> cd mpich-4.1.2/
>> ./configure --prefix=$HOME/opt/mpich
>> make
>> make install
>> echo 'export PATH=$HOME/opt/mpich/bin:$PATH' >> ~/.bash_profile
>> source ~/.bash_profile
>>
>> #install a new Perl
>> cd $HOME/install_prep
>> wget https://www.cpan.org/src/5.0/perl-5.38.0.tar.gz
>> --no-check-certificate
>> tar -zxvf perl-5.38.0.tar.gz
>> cd perl-5.38.0
>> ./Configure -d -Dprefix=$HOME/opt/perl5
>> make
>> make install
>> echo 'export PATH=$HOME/opt/perl5/bin:$PATH' >> ~/.bash_profile
>> source ~/.bash_profile
>>
>> #install MAKER with MPI support
>> cd $HOME/opt
>> wget
>> http://weatherby.genetics.utah.edu/maker_downloads/AD6E/A1B0/C2B7/558563CFD35E343BA1B888148294/maker-3.01.04.tgz
>> tar -zxvf maker-3.01.04.tgz
>> rm -rf maker-3.01.04.tgz
>> cd maker/src
>> perl Build.PL ##say yes to MPI question
>> ./Build installdeps ##just hit 'enter' to accept any defaults
>> ### if you get errors mentioning '--no-check-certificate', run the
>> command 'cpan' in your terminal.
>> ### If it asks to set up everything automatically, say 'yes'.
>> ### Next, whil einside cpan, run the command 'o conf init pushy_https'.
>> ### Say 'no' to the question it asks, and then run 'o conf commit' to
>> save changes and 'q' to exit.
>> ### Then retry the './Build installdeps' command
>> ./Build installexes
>> ./Build install
>> echo 'export PATH=$HOME/opt/maker/bin:$PATH' >> ~/.bash_profile
>> source ~/.bash_profile
>>
>> #test MAKER
>> cd $HOME/install_prep
>> mpiexec -n 10 maker --help ##should print a single help message
>> cp -R $HOME/opt/maker/data ./test
>> cd test
>> maker -CTL
>> mpiexec -n 10 maker --protein dpp_protein.fasta --genome dpp_contig.fasta
>> --est dpp_est.fasta ##should run MAKER under MPI
>>
>>
>>
>> —Carson
>>
>>
>> On Sep 12, 2023, at 2:22 PM, Hayley Mangelson <hayley at phasegenomics.com>
>> wrote:
>>
>> Hi Carson,
>>
>> I have tried installing MPICH and OpenMPI various ways now and am still
>> having issues. MAKER appears to be running when I am not using MPI, but I
>> am attaching the maker output (with -debug flag) and here is the error I
>> get when trying to multithread.
>>
>>
>> ===================================================================================
>> =   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
>> =   PID 35952 RUNNING AT ip-172-31-12-194
>> =   EXIT CODE: 9
>> =   CLEANING UP REMAINING PROCESSES
>> =   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
>>
>> ===================================================================================
>> YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault
>> (signal 11)
>> This typically refers to a problem with your application.
>> Please see the FAQ page for debugging suggestions
>>
>> On Thu, Aug 24, 2023 at 6:26 PM Carson Holt <carsonhh at gmail.com> wrote:
>>
>>> Once you install MPICH or OpenMPI you must launch MAKER through mpiexec
>>> and speciffy the CPU count to MPI. On a single instance you just give the
>>> CPU count off that instance, but you can also go across instances using
>>> Amazon’s Parallel Cluster.
>>>
>>> Here is an example —>
>>> https://efish.integrativebiology.msu.edu/2015/02/10/annotate.html
>>> That example uses StarCluster which is now deprecated, but the MPI
>>> install and MAKER run still apply.
>>>
>>> Here is another example of using MPi —>
>>> https://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_WGS_Assembly_and_Annotation_Winter_School_2018#MPI_Support
>>>
>>> Note that for OpenMPI you have to set the LD_PRELOAD environmental
>>> variable before launch.
>>> export LD_PRELOAD=/usr/lib64/openmpi-1.10/lib/libmpi.so
>>>
>>> —Carson
>>>
>>>
>>> On Aug 18, 2023, at 2:53 PM, Hayley Mangelson <hayley at phasegenomics.com>
>>> wrote:
>>>
>>> I am working on an AWS EC2 instance that runs Amazon Linux, and I have
>>> tried a couple of installs (MPICH and Open MPI), neither of which worked. I
>>> don't have any installed on my current instance.
>>>
>>> On Tue, Aug 15, 2023 at 5:08 PM Carson Holt <carsonhh at gmail.com> wrote:
>>>
>>>> Do you already have MPI installed on your system, and if so which MPI
>>>> and what version (i.e. MPICH, MVAPICH2, Intel MPI, Open MPI, etc)?
>>>>
>>>> —Carson
>>>>
>>>>
>>>> On Aug 15, 2023, at 9:34 AM, Hayley Mangelson <hayley at phasegenomics.com>
>>>> wrote:
>>>>
>>>> Hi Carson,
>>>>
>>>> Excellent, thanks so much! I have that part working. The multithreading
>>>> is still eluding me. Do you have a recommended method of installing a
>>>> compatible version of MPI?
>>>>
>>>> Thanks,
>>>>
>>>> Hayley
>>>>
>>>> On Fri, Aug 11, 2023 at 4:04 PM Carson Holt <carsonhh at gmail.com> wrote:
>>>>
>>>>> The error is coming from Augustus. There is a problem with the species
>>>>> file you selected. Or you put it in the wrong place.
>>>>>
>>>>> This is the error —>
>>>>> Could not find the config file
>>>>> /home/ec2-user/MAKERTOOLS/maker/exe/augustus/config/species/BUSCO_evans_trout_hap1/BUSCO_evans_trout_hap1_parameters.cfg
>>>>>
>>>>> The species info must exist under this base directory by default (it’s
>>>>> where you let MAKER install Augustus) —>
>>>>>  /home/ec2-user/MAKERTOOLS/maker/exe/augustus/config/species/
>>>>>
>>>>> You can also place the species files somewhere else but you need to
>>>>> set the AUGUSTUS_CONFIG_PATH so Augustus knows where to look.
>>>>>
>>>>> Example (in your ~/.bash_profile):
>>>>> export AUGUSTUS_CONFIG_PATH=$HOME/muy_augustus_species
>>>>>
>>>>> Thanks,
>>>>> Carson
>>>>>
>>>>>
>>>>>
>>>>> On Aug 11, 2023, at 7:20 AM, Hayley Mangelson <
>>>>> hayley at phasegenomics.com> wrote:
>>>>>
>>>>> Hi Carson,
>>>>>
>>>>> Sure thing! Sorry for the delay, I was on vacation. I reran with just
>>>>> a single scaffold. Here is the STDERR.
>>>>>
>>>>> On Mon, Jul 31, 2023 at 12:06 PM Carson Holt <carsonhh at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Can you send me the STDERR from the maker run? If you don’t have it,
>>>>>> run maker again with the output from the screen redirected to a file.
>>>>>> Example —> maker &> all_output.txt
>>>>>>
>>>>>> --Carson
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On Jul 31, 2023, at 10:04 AM, Hayley Mangelson <
>>>>>> hayley at phasegenomics.com> wrote:
>>>>>>
>>>>>> 
>>>>>> Hi!
>>>>>>
>>>>>> I am trying to get a fresh install of MAKER to work for me and am
>>>>>> having a few issues.
>>>>>>
>>>>>> 1. I got MAKER installed (I thought) and now the log is saying all
>>>>>> contigs failed. See attached log.
>>>>>> 2. Having tried several versions and methods of installation, I
>>>>>> cannot get multithreading to work.
>>>>>>
>>>>>> Could you help me work through these issues?
>>>>>> Thanks,
>>>>>>
>>>>>> Hayley
>>>>>> --
>>>>>> Hayley Mangelson
>>>>>> Senior Bioinformatics Lead
>>>>>> Phase Genomics, Inc
>>>>>> www.phasegenomics.com
>>>>>>
>>>>>> <PGA_assembly_hap1_master_datastore_index.log>
>>>>>> _______________________________________________
>>>>>> maker-devel mailing list
>>>>>> maker-devel at yandell-lab.org
>>>>>> http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Hayley Mangelson
>>>>> Senior Bioinformatics Lead
>>>>> Phase Genomics, Inc
>>>>> www.phasegenomics.com
>>>>>
>>>>> <all_output.txt>
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Hayley Mangelson
>>>> Senior Bioinformatics Lead
>>>> Phase Genomics, Inc
>>>> www.phasegenomics.com
>>>>
>>>>
>>>>
>>>
>>> --
>>> Hayley Mangelson
>>> Senior Bioinformatics Lead
>>> Phase Genomics, Inc
>>> www.phasegenomics.com
>>>
>>>
>>>
>>
>> --
>> Hayley Mangelson
>> Senior Bioinformatics Lead
>> Phase Genomics, Inc
>> www.phasegenomics.com
>>
>> <all_output.txt>
>>
>>
>>
>
> --
> Hayley Mangelson
> Senior Bioinformatics Lead
> Phase Genomics, Inc
> www.phasegenomics.com
>
>
>

-- 
Hayley Mangelson
Senior Bioinformatics Lead
Phase Genomics, Inc
www.phasegenomics.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20231106/b12f8237/attachment-0001.html>