From mcsimenc at gmail.com  Sun Apr  2 09:19:40 2017
From: mcsimenc at gmail.com (Matt Simenc)
Date: Sun, 2 Apr 2017 07:19:40 -0700
Subject: [maker-devel] MAKER RepeatMasker outputs incompatible annotations
	for single feature
Message-ID: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>

TE Superfamily RLG are aka Gypsy, but RLG is associated with Copia
annotations in our RepeatMasker annotations. Has anyone else seen this
before?


Azfi_s0077  repeatmasker match 1400976 1401715 521 + .
ID=Azfi_s0077:hit:59234:1.3.0.14;Name=species:*RLG_scAle_3_1-*
I|genus:LTR%252FCopia;Target=species:*RLG_scAle_3_*1-I|genus:*LTR%252FCopia*1522
2556 +



Matt Simenc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170402/0a6106a2/attachment.html>

From carsonhh at gmail.com  Sun Apr  2 12:38:05 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 2 Apr 2017 11:38:05 -0600
Subject: [maker-devel] MAKER RepeatMasker outputs incompatible
 annotations for single feature
In-Reply-To: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>
References: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>
Message-ID: <E34482B0-55B7-4DC1-A01C-4EDBFE2E10A9@gmail.com>

This is the RepBase entry (from this file ?/RepeatMasker/Libraries/RepeatMasker.lib) ?>

>RLG_scAle_3_1-LTR#LTR/Copia @Saccharum_hybrid_cultivar  [S:] RepbaseID: RLG_scAle_3_1-LTR
tgttagaagattatgttaatatcttctgtataaggtaggggatctgtgtg
accacgagatccggtggattggcttgcaagtcaagttgtaatctatgtaa
atctaaccttcttgttgtacaagcccaaggggctatatattaacatgtac
cgagtacctgagaggtatctcgttccatctatttctaca

There are 64 RLG entries associated with Copia in the RepBase library for RepeatMasker. I?m not a repeat expert, so I can?t address whether this is an error or not. But if you think there is something amiss, you could contact RepBase.

?Carson


> On Apr 2, 2017, at 8:19 AM, Matt Simenc <mcsimenc at gmail.com> wrote:
> 
> TE Superfamily RLG are aka Gypsy, but RLG is associated with Copia annotations in our RepeatMasker annotations. Has anyone else seen this before?
> 
> 
> Azfi_s0077  repeatmasker match
> 1400976 1401715
> 521 +
> . ID=Azfi_s0077:hit:59234:1.3.0.14;Name=species:RLG_scAle_3_1-I|genus:LTR%252FCopia;Target=species:RLG_scAle_3_1-I|genus:LTR%252FCopia1522 2556 +
> 
> 
> 
> Matt Simenc
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170402/acdc5126/attachment.html>

From carsonhh at gmail.com  Wed Apr  5 11:00:53 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Wed, 5 Apr 2017 10:00:53 -0600
Subject: [maker-devel] Some error messages in the output log of Maker
In-Reply-To: <CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
References: <CAJNrdRH1uDgtaB0C8jFi5yceZ7dcH6FRVzJV_RFgxHmtY_T12w@mail.gmail.com>
	<CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
Message-ID: <4F7ECEC6-338A-44B0-BC82-18193A494E29@gmail.com>

Augustus can sometimes produce alternate models. It normally only happens with hints and not with the raw prediction though. MAKER by default will only keeps the best scoring one (scored by AED). But EVM technically gets to see all of the raw models. Its the kind of thing where I would have to have the exact data and contig you used to run to see if this is something easily pulled apart for EVM.

?Carson


> On Apr 5, 2017, at 6:57 AM, Ray Cui <rcui at age.mpg.de> wrote:
> 
> Dear Carson,
> 
>         I think I found the reason for this here https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/ <https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/> . 
>         These messages are from EVM. The second type of message is due to incomplete gene models, so it's not a big deal. The first type is because of the overlapping gene models predicted by Augustus. This is a bit strange. Does maker allow Augustus to produce alternative spliced variants?
> 
> Best Regards,
> Ray
> 
> Dr. Rongfeng (Ray) Cui
> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
> Wissenschaftlicher MA / Postdoctoral researcher
> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
> Tel.:+49 (0)221 496 
> Mobile:           +49 0221 37970 496 <>
> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
> www.age.mpg.de <http://www.age.mpg.de/> 
> 
> 
> 
> On Wed, Apr 5, 2017 at 1:33 PM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
> Hi Carson,
> 
>       I saw these messages in the log, is this a concern? I'm using Maker beta 3.
>        Message 1:
> 
> Error with prediction: augustus_masked lend_intergenic: 730288, rend_intergenic: 664875  at /beegfs/group_dv/software/source/EVidenceModeler-1.1.1/evidence_modeler.pl <http://evidence_modeler.pl/> line 3218.
> 
>         Message 2:
> Sorry, prediction pred_gff:gth_m.PLPscf119:hit:7108:43932 fails validation.
> (GTT)    TA     683590-683765   GC      ATG
> (GCA)    AG     701228-701409   GT      AGG
> (GGA)    AG     704938-705037   GT      AGG
> (GGC)    AG     721672-721792   GT      AGG
> (TTG)    AG     730110-730288   AC      GAA
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, terminal, 730110, 730288
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 701228, 701409
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 704938, 705037
> -recovered pred_gff:gth_m.PLPspolishing proteins
> 
> 
>      PS: I am still keep being kicked out of the maker mailing list due to some reason, if you can forward the msg. there it would be great.
>          
> Best Regards,
> Ray
> 
> Dr. Rongfeng (Ray) Cui
> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
> Wissenschaftlicher MA / Postdoctoral researcher
> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
> Mobile:           +49 0221 37970 496 <>
> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
> www.age.mpg.de <http://www.age.mpg.de/> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170405/b51fcd1b/attachment.html>

From qlian003 at ucr.edu  Thu Apr  6 11:33:26 2017
From: qlian003 at ucr.edu (Qihua Liang)
Date: Fri, 7 Apr 2017 00:33:26 +0800
Subject: [maker-devel] Large dataset of RNA-Seq data
Message-ID: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>

Dear MAKER develop team,

I have some RNA-Seq data that I would like to use raw data instead of assembled transcripts in MAKER. But those raw datasets are over 800Gb. I noticed that MAKER will first read all input dataset into /tmp, and my attempts to include such large dataset failed because it exceeds the disk quota. Besides of increasing the disk quota, is there any other option to include such large datasets in MAKER annotations? Or can I redirect MAKER to read those data and use other path to temporarily store those data?

Thank you!
Best
Qihua


From carsonhh at gmail.com  Thu Apr  6 11:35:32 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 6 Apr 2017 10:35:32 -0600
Subject: [maker-devel] Large dataset of RNA-Seq data
In-Reply-To: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>
References: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>
Message-ID: <C88A1724-71D2-493F-B655-02E9A0FE3793@gmail.com>

MAKER cannot use raw RNA-Seq. It must be assembled beforehand, or processed into GFF3 using something like Stringtie.

?Carson


> On Apr 6, 2017, at 10:33 AM, Qihua Liang <qlian003 at ucr.edu> wrote:
> 
> Dear MAKER develop team,
> 
> I have some RNA-Seq data that I would like to use raw data instead of assembled transcripts in MAKER. But those raw datasets are over 800Gb. I noticed that MAKER will first read all input dataset into /tmp, and my attempts to include such large dataset failed because it exceeds the disk quota. Besides of increasing the disk quota, is there any other option to include such large datasets in MAKER annotations? Or can I redirect MAKER to read those data and use other path to temporarily store those data?
> 
> Thank you!
> Best
> Qihua
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From elyssa_garza at yahoo.com  Thu Apr  6 14:25:10 2017
From: elyssa_garza at yahoo.com (Elyssa Garza)
Date: Thu, 6 Apr 2017 19:25:10 +0000 (UTC)
Subject: [maker-devel]  merging gff files
References: <1186841770.2604454.1491506710651.ref@mail.yahoo.com>
Message-ID: <1186841770.2604454.1491506710651@mail.yahoo.com>

Hello, I currently have a MAKER-P annotated genome and would like to add additional information to the final gff file. I would like to include InDel and microsatellite genetic marker sequences/positions to the final annotation. Do you know of a reliable way to do this?

Elyssa



   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170406/09f2437b/attachment.html>

From carsonhh at gmail.com  Thu Apr  6 17:01:52 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 6 Apr 2017 16:01:52 -0600
Subject: [maker-devel] maker-devel post from elyssa_garza@yahoo.com
 requires approval
In-Reply-To: <mailman.93.1491506732.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
References: <mailman.93.1491506732.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
Message-ID: <2AE2415B-5940-45F3-800B-84F9E9E28E7E@gmail.com>

If you already have GFF3, you can merge two files using the gff3_merge script that comes with MAKER.

?Carson

> From: Elyssa Garza <elyssa_garza at yahoo.com>
> Subject: [maker-devel] merging gff files
> Date: April 6, 2017 at 1:25:10 PM MDT
> To: "dence at genetics.utah.edu" <dence at genetics.utah.edu>
> Cc: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> Reply-To: Elyssa Garza <elyssa_garza at yahoo.com>
> 
> 
> Hello, I currently have a MAKER-P annotated genome and would like to add additional information to the final gff file. I would like to include InDel and microsatellite genetic marker sequences/positions to the final annotation. Do you know of a reliable way to do this?
> 
> 
> Elyssa

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170406/4f591c05/attachment.html>

From carsonhh at gmail.com  Sun Apr  9 17:57:26 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 9 Apr 2017 16:57:26 -0600
Subject: [maker-devel] Some error messages in the output log of Maker
In-Reply-To: <CAJNrdRGfE1D-BrAQmQrRYL=1XLCPELKZNe9PZ4jneSuarAfd1g@mail.gmail.com>
References: <CAJNrdRH1uDgtaB0C8jFi5yceZ7dcH6FRVzJV_RFgxHmtY_T12w@mail.gmail.com>
	<CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
	<4F7ECEC6-338A-44B0-BC82-18193A494E29@gmail.com>
	<CAJNrdRGfE1D-BrAQmQrRYL=1XLCPELKZNe9PZ4jneSuarAfd1g@mail.gmail.com>
Message-ID: <0E1BAB87-339D-4D40-AA88-8C5C9CB895E9@gmail.com>

Most of the parameters affect the command line used to run blast/exonerate. In that case it will rerun. Some like the percent coverage parameter are post run filters. For those, it just re-parses. I don?t remember off the top of my head which ones fall into which category, but most fall into the first one.

?Carson

> On Apr 6, 2017, at 1:18 AM, Ray Cui <rcui at age.mpg.de> wrote:
> 
> Hi Carson,
> 
>     Yet another question: if I change the blast and exonerate parameters in the bopts file, will maker automatically rerun blast and exonerate (or, parse the existing output with the new cutoffs)?
> 
> Thanks
> Ray 
> 
> On Apr 5, 2017 6:01 PM, "Carson Holt" <carsonhh at gmail.com <mailto:carsonhh at gmail.com>> wrote:
> Augustus can sometimes produce alternate models. It normally only happens with hints and not with the raw prediction though. MAKER by default will only keeps the best scoring one (scored by AED). But EVM technically gets to see all of the raw models. Its the kind of thing where I would have to have the exact data and contig you used to run to see if this is something easily pulled apart for EVM.
> 
> ?Carson
> 
> 
>> On Apr 5, 2017, at 6:57 AM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
>> 
>> Dear Carson,
>> 
>>         I think I found the reason for this here https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/ <https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/> . 
>>         These messages are from EVM. The second type of message is due to incomplete gene models, so it's not a big deal. The first type is because of the overlapping gene models predicted by Augustus. This is a bit strange. Does maker allow Augustus to produce alternative spliced variants?
>> 
>> Best Regards,
>> Ray
>> 
>> Dr. Rongfeng (Ray) Cui
>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
>> Wissenschaftlicher MA / Postdoctoral researcher
>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
>> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
>> Mobile:           +49 0221 37970 496 <>
>> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
>> www.age.mpg.de <http://www.age.mpg.de/> 
>> 
>> 
>> 
>> On Wed, Apr 5, 2017 at 1:33 PM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
>> Hi Carson,
>> 
>>       I saw these messages in the log, is this a concern? I'm using Maker beta 3.
>>        Message 1:
>> 
>> Error with prediction: augustus_masked lend_intergenic: 730288, rend_intergenic: 664875  at /beegfs/group_dv/software/source/EVidenceModeler-1.1.1/evidence_modeler.pl <http://evidence_modeler.pl/> line 3218.
>> 
>>         Message 2:
>> Sorry, prediction pred_gff:gth_m.PLPscf119:hit:7108:43932 fails validation.
>> (GTT)    TA     683590-683765   GC      ATG
>> (GCA)    AG     701228-701409   GT      AGG
>> (GGA)    AG     704938-705037   GT      AGG
>> (GGC)    AG     721672-721792   GT      AGG
>> (TTG)    AG     730110-730288   AC      GAA
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, terminal, 730110, 730288
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 701228, 701409
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 704938, 705037
>> -recovered pred_gff:gth_m.PLPspolishing proteins
>> 
>> 
>>      PS: I am still keep being kicked out of the maker mailing list due to some reason, if you can forward the msg. there it would be great.
>>          
>> Best Regards,
>> Ray
>> 
>> Dr. Rongfeng (Ray) Cui
>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
>> Wissenschaftlicher MA / Postdoctoral researcher
>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
>> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
>> Mobile:           +49 0221 37970 496 <>
>> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
>> www.age.mpg.de <http://www.age.mpg.de/> 
>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170409/846f99a9/attachment.html>

From carsonhh at gmail.com  Sun Apr  9 18:06:52 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 9 Apr 2017 17:06:52 -0600
Subject: [maker-devel] maker putting _Inline directory wherever it is run
In-Reply-To: <mailman.101.1491517636.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
References: <mailman.101.1491517636.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
Message-ID: <A690B5B0-8AE5-4898-A2A9-025AF9DF4166@gmail.com>

You have either moved directories post installation, are trying to run from the ?/maker/src/bin directory instead of ?/maker/bin or you had an issue during install. You just need to start over and follow the install instructions. The few libraries MAKER must compile with Inline::C will go into ?/maker/perl/lib/

?Carson

> From: "Sobel-Sorenson, Chandler - (scar)" <scar at email.arizona.edu>
> Subject: maker putting _Inline directory wherever it is run
> Date: April 6, 2017 at 4:27:03 PM MDT
> To: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> 
> 
> I just installed the latest maker 2.31.9 on our cluster, but whenever we issue the command "maker" (even with no arguments), it creates a "_Inline" directory in the CWD..  I guess this is related to Inline perl module?  For example:
> 
> [smrtanalysis at pac makertest]$ ls -a
> .  ..
> [smrtanalysis at pac makertest]$ maker
> ERROR: Control files not found
> 
> MAKER version 2.31.9
> 
> Usage:
> 
>     maker [options] <maker_opts> <maker_bopts> <maker_exe>
> ...
> 
> [smrtanalysis at pac makertest]$ ls -a
> .  ..  _Inline
> [smrtanalysis at pac makertest]$ ls -a _Inline/
> .  ..  build  config-x86_64-linux-thread-multi-5.010001  lib  .lock
> [smrtanalysis at pac makertest]$ 
> 
> 
> This is some problem with our module installation?  It was installed using "cpanm Inline::C".  Or problem/bug with the module itself should contact the Inline team about?  Or problem with our maker installation?
> 
> 
> Thanks,
> Chandler
> Arizona Genomics Institute
> University Of Arizona
> www.genome.arizona.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170409/2842e623/attachment.html>

From wangzhennan at ioz.ac.cn  Mon Apr 10 22:08:14 2017
From: wangzhennan at ioz.ac.cn (wangzhennan at ioz.ac.cn)
Date: Tue, 11 Apr 2017 11:08:14 +0800 (GMT+08:00)
Subject: [maker-devel] How to run Maker to get the AED?
Message-ID: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>

Dear developer of Maker,

        I have used Maker to evaluate the genome annotation annotated by PASA. I have the transcript assembled by stringtie, genome (fasta), and the annotation file (gff3). How can I use the data to get the AED value from Maker? I have set the parameter of maker_opts.ctl. Can you help me to solve this? I only want to get the AED about the annotation file (gff3). Thank you very much!

        And there is also a question, I can not have a new post on https://groups.google.com/forum/#!forum/maker-devel. Why? Can you help me to solve this? Thank you!

 

         Best,

         Wangzhennan

        




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170411/9b4a2ec6/attachment.html>

From scar at email.arizona.edu  Tue Apr 11 19:21:48 2017
From: scar at email.arizona.edu (Sobel-Sorenson, Chandler - (scar))
Date: Wed, 12 Apr 2017 00:21:48 +0000
Subject: [maker-devel] run maker using qsub
Message-ID: <1491956508254.51400@email.arizona.edu>

How could we run maker using qsub with our Sun Grid Engine?  We have it working with mpiexec but this does not show slots used on the grid, so can lead to over loading our cluster.
Currently we are using command like this with mpich: mpiexec -f /opt/machinelist -n 192 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl -fix_nucleotides

Thanks
Chandler
Arizona Genomics Institute



From carsonhh at gmail.com  Sun Apr 16 14:46:35 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 16 Apr 2017 13:46:35 -0600
Subject: [maker-devel] How to run Maker to get the AED?
In-Reply-To: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>
References: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>
Message-ID: <5CD6DB76-9BB6-4DD6-9C35-A8CE0ABE1603@gmail.com>

To get just AED without maker producing it?s own predictons, you must provide your models to model_gff in valid GFF3 format (gene/mRNA/exon/CDS multi-level features). Then supply any evidence to either est=, protein=, est_gff=, or protein_gff=.

The google location is just an archive. You cannot post there. All messages must be sent to the mailing list here ?> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>

Thanks,
Carson


> On Apr 10, 2017, at 9:08 PM, wangzhennan at ioz.ac.cn wrote:
> 
> Dear developer of Maker,
> 
>         I have used Maker to evaluate the genome annotation annotated by PASA. I have the transcript assembled by stringtie, genome (fasta), and the annotation file (gff3). How can I use the data to get the AED value from Maker? I have set the parameter of maker_opts.ctl. Can you help me to solve this? I only want to get the AED about the annotation file (gff3). Thank you very much!
> 
>         And there is also a question, I can not have a new post on https://groups.google.com/forum/#!forum/maker-devel <https://groups.google.com/forum/#!forum/maker-devel>. Why? Can you help me to solve this? Thank you!
> 
>  
>          Best,
> 
>          Wangzhennan
> 
>         
> 
> 
> 
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170416/a03cd31f/attachment.html>

From carsonhh at gmail.com  Sun Apr 16 14:55:23 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 16 Apr 2017 13:55:23 -0600
Subject: [maker-devel] run maker using qsub
In-Reply-To: <1491956508254.51400@email.arizona.edu>
References: <1491956508254.51400@email.arizona.edu>
Message-ID: <6A3DAA63-06FA-4A44-9B89-EAD8AF8ED0BB@gmail.com>

You will have to look up documentation for Sun Grid Engine and the MPI flavor you wish to use. This is not a MAKER configuration issue, rather you need to setup Sun Grid for MPI according to it?s documentation. When Sun Grid is configured for MPI, it should automatically sets environmental variables and files for jobs. When set up correctly you should not even have to use the -n or -f option with mpiexec as it will be detected from values Sun Grid configures for you.

Then a command like the following will work in your batch job
?> mpiexec maker

?Carson



> On Apr 11, 2017, at 6:21 PM, Sobel-Sorenson, Chandler - (scar) <scar at email.arizona.edu> wrote:
> 
> How could we run maker using qsub with our Sun Grid Engine?  We have it working with mpiexec but this does not show slots used on the grid, so can lead to over loading our cluster.
> Currently we are using command like this with mpich: mpiexec -f /opt/machinelist -n 192 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl -fix_nucleotides
> 
> Thanks
> Chandler
> Arizona Genomics Institute
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From tfallon at mit.edu  Mon Apr 17 10:35:31 2017
From: tfallon at mit.edu (Tim Fallon)
Date: Mon, 17 Apr 2017 11:35:31 -0400
Subject: [maker-devel] Help with MPI installation and configuration
Message-ID: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>

Hi there,

I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.

See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?

"
I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.

Peter Macfarlane / WIBR IT


Description : Hi Peter,

Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?

All the best,
-Tim

On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:

Hi Peter,

I?ve tested with the following command:
bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?

Unfortunately it still seems to be giving errors and not running, with things like:
"?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??

It also doesn?t work if I try to run it directly on Tak4.

I?ve attached the relevant error log file.  Any more solutions?

All the best,
-Tim

On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:

I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:

- /nfs/apps/test/maker is now maker 3.00
- /nfs/apps/test/openmpi was used to make the aforementioned maker

I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
"
All the best,
-Tim

Timothy R. Fallon
PhD candidate
Laboratory of Jing-Ke Weng
Department of Biology
MIT

tfallon at mit.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: bsub_maker_with_mpi_log.txt
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1849 bytes
Desc: not available
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment.p7s>

From scar at email.arizona.edu  Mon Apr 17 12:29:58 2017
From: scar at email.arizona.edu (Chandler)
Date: Mon, 17 Apr 2017 10:29:58 -0700
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
Message-ID: <2940db14-712e-d318-35c5-2a7b132db842@email.arizona.edu>

Please try with MPICH instead of openmpi... we were having similar 
errors with openmpi, but when we switched to mpich it all started 
working.  Make sure only the mpich executables are in your PATH, and 
that maker is built specifying the mpich package.

Chandler / Systems Administrator
Arizona Genomics Institute
www.genome.arizona.edu





From carsonhh at gmail.com  Tue Apr 18 10:10:43 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 18 Apr 2017 09:10:43 -0600
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
Message-ID: <2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>

I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.

#this must be doen exery time you run it
export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so

# -mca btl ^openib myust be on the command line
mpiexec -mca btl ^openib -n 20 maker

#alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker

?Carson

> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu> wrote:
> 
> Hi there,
> 
> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
> 
> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
> 
> "
> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
> 
> Peter Macfarlane / WIBR IT
> 
> 
> Description : Hi Peter,
> 
> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
> 
> All the best,
> -Tim
> 
> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
> 
> Hi Peter,
> 
> I?ve tested with the following command:
> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
> 
> Unfortunately it still seems to be giving errors and not running, with things like:
> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
> 
> It also doesn?t work if I try to run it directly on Tak4.
> 
> I?ve attached the relevant error log file.  Any more solutions?
> 
> All the best,
> -Tim
> 
> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
> 
> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
> 
> - /nfs/apps/test/maker is now maker 3.00
> - /nfs/apps/test/openmpi was used to make the aforementioned maker
> 
> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
> "
> All the best,
> -Tim
> 
> Timothy R. Fallon
> PhD candidate
> Laboratory of Jing-Ke Weng
> Department of Biology
> MIT
> 
> tfallon at mit.edu <mailto:tfallon at mit.edu>
> <bsub_maker_with_mpi_log.txt>
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170418/e7ebdbc2/attachment.html>

From carsonhh at gmail.com  Tue Apr 18 10:23:04 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 18 Apr 2017 09:23:04 -0600
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
	<2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
	<CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>
Message-ID: <BDCC2D30-B9DB-4FF1-921A-307975EA8BB2@gmail.com>

MPICH3 works without any special configuration. OpenMPI is built to support infiniband which internally can drive other problems related to how registered memory is used when infiniband libraries are loaded. So you have have to set more parameters.

Both will use shared memory for communications when running on a single machine, but you will still need to set parameters on OpenMPI.

?Carson



> On Apr 18, 2017, at 9:19 AM, Tim Fallon <tfallon at mit.edu> wrote:
> 
> Hi Carson,
> 
> Thanks for the feedback, I?ll look into the maker INSTALL file to ensure things are setup correctly.
> 
> Should I be looking into MPICH3 instead?  I?m trying to run maker on a single node (~20 CPUs), rather than distribute it across our cluster.  I assume that using MPICH3 changes the mpiexec / mpirun executable, but not the commands otherwise?  I ask as I may have to configure this myself one day, so any tips appreciated.  In the meantime your comments seem straightforward, will give it a shot.
> 
> All the best,
> -Tim
> 
>> On Apr 18, 2017, at 11:10 AM, Carson Holt <carsonhh at gmail.com <mailto:carsonhh at gmail.com>> wrote:
>> 
>> I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.
>> 
>> #this must be doen exery time you run it
>> export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so
>> 
>> # -mca btl ^openib myust be on the command line
>> mpiexec -mca btl ^openib -n 20 maker
>> 
>> #alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
>> mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker
>> 
>> ?Carson
>> 
>>> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>>> 
>>> Hi there,
>>> 
>>> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
>>> 
>>> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
>>> 
>>> "
>>> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
>>> 
>>> Peter Macfarlane / WIBR IT
>>> 
>>> 
>>> Description : Hi Peter,
>>> 
>>> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
>>> 
>>> All the best,
>>> -Tim
>>> 
>>> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>>> 
>>> Hi Peter,
>>> 
>>> I?ve tested with the following command:
>>> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
>>> 
>>> Unfortunately it still seems to be giving errors and not running, with things like:
>>> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
>>> 
>>> It also doesn?t work if I try to run it directly on Tak4.
>>> 
>>> I?ve attached the relevant error log file.  Any more solutions?
>>> 
>>> All the best,
>>> -Tim
>>> 
>>> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
>>> 
>>> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
>>> 
>>> - /nfs/apps/test/maker is now maker 3.00
>>> - /nfs/apps/test/openmpi was used to make the aforementioned maker
>>> 
>>> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
>>> "
>>> All the best,
>>> -Tim
>>> 
>>> Timothy R. Fallon
>>> PhD candidate
>>> Laboratory of Jing-Ke Weng
>>> Department of Biology
>>> MIT
>>> 
>>> tfallon at mit.edu <mailto:tfallon at mit.edu>
>>> <bsub_maker_with_mpi_log.txt>
>>> _______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>
>> 
> 
> Timothy R. Fallon
> PhD candidate
> Laboratory of Jing-Ke Weng
> Department of Biology
> MIT
> 
> tfallon at mit.edu <mailto:tfallon at mit.edu>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170418/a23814d7/attachment.html>

From tfallon at mit.edu  Tue Apr 18 10:19:19 2017
From: tfallon at mit.edu (Tim Fallon)
Date: Tue, 18 Apr 2017 11:19:19 -0400
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
	<2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
Message-ID: <CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>

Hi Carson,

Thanks for the feedback, I?ll look into the maker INSTALL file to ensure things are setup correctly.

Should I be looking into MPICH3 instead?  I?m trying to run maker on a single node (~20 CPUs), rather than distribute it across our cluster.  I assume that using MPICH3 changes the mpiexec / mpirun executable, but not the commands otherwise?  I ask as I may have to configure this myself one day, so any tips appreciated.  In the meantime your comments seem straightforward, will give it a shot.

All the best,
-Tim

> On Apr 18, 2017, at 11:10 AM, Carson Holt <carsonhh at gmail.com> wrote:
> 
> I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.
> 
> #this must be doen exery time you run it
> export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so
> 
> # -mca btl ^openib myust be on the command line
> mpiexec -mca btl ^openib -n 20 maker
> 
> #alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
> mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker
> 
> ?Carson
> 
>> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>> 
>> Hi there,
>> 
>> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
>> 
>> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
>> 
>> "
>> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
>> 
>> Peter Macfarlane / WIBR IT
>> 
>> 
>> Description : Hi Peter,
>> 
>> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
>> 
>> All the best,
>> -Tim
>> 
>> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>> 
>> Hi Peter,
>> 
>> I?ve tested with the following command:
>> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
>> 
>> Unfortunately it still seems to be giving errors and not running, with things like:
>> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
>> 
>> It also doesn?t work if I try to run it directly on Tak4.
>> 
>> I?ve attached the relevant error log file.  Any more solutions?
>> 
>> All the best,
>> -Tim
>> 
>> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
>> 
>> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
>> 
>> - /nfs/apps/test/maker is now maker 3.00
>> - /nfs/apps/test/openmpi was used to make the aforementioned maker
>> 
>> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
>> "
>> All the best,
>> -Tim
>> 
>> Timothy R. Fallon
>> PhD candidate
>> Laboratory of Jing-Ke Weng
>> Department of Biology
>> MIT
>> 
>> tfallon at mit.edu <mailto:tfallon at mit.edu>
>> <bsub_maker_with_mpi_log.txt>
>> _______________________________________________
>> maker-devel mailing list
>> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
> 

Timothy R. Fallon
PhD candidate
Laboratory of Jing-Ke Weng
Department of Biology
MIT

tfallon at mit.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170418/d3ae60e9/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1849 bytes
Desc: not available
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170418/d3ae60e9/attachment.p7s>

From miche.zacharias at gmail.com  Tue Apr 25 06:35:37 2017
From: miche.zacharias at gmail.com (Miche Zacharias)
Date: Tue, 25 Apr 2017 13:35:37 +0200
Subject: [maker-devel] altEST evidence
Message-ID: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>

Hi,

Would there be any benefit to passing CDS instead of cDNA sequences as
alternative EST evidence, if both is available? Or when would you use one
or the other?

I've read in previous discussions how it's expected that the organism, from
which we use sequences as alternative EST and protein evidence, is not
closely related to the genome we are annotating.
In what terms is "closely related" defined? It's now unclear to me when you
would put EST/protein evidence as alternative and when not or put it as
direct EST/protein evidence?
Would a species from the same family classify as alternative or direct
evidence? Is ~30mya divergence too much?
What about species of the same genus?

Many thanks!
Miche
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170425/2aafa4e5/attachment.html>

From carsonhh at gmail.com  Tue Apr 25 10:30:21 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 25 Apr 2017 09:30:21 -0600
Subject: [maker-devel] altEST evidence
In-Reply-To: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>
References: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>
Message-ID: <EA7FC16A-2A92-49C1-96FF-774FD39E86DA@gmail.com>

If you have CDS, then it is not really transcription evidence, rather it was derived from protein annotations and the protein data should be used instead. The alt-EST option is only for datasets where no protein models exists, but you do have experimental evidence from transcription (i.e. not transcript predictions but actual mRNA experimentation).

Nucleotides sequence diverges more rapidly than amino acid sequence because of codon redundancy and amino acid similarity. As a result mRNA derived sequence can be compared in nucleotide space to genomic sequence of related organisms with an evolutionary difference of something like human to gorilla, but human to mouse would require amino acid level alignment to find the homology. So with alt-EST we use TBLASTX and Exonerate cdna2genome to perform a six frame translations of both the genome and assembly for alignment in amino acid space. This is of course very expensive computationally and more likely to generate spurious alignments. So for CDS sequence it would be both redundant to and less accurate than using the protein data directly.

?Carson



> On Apr 25, 2017, at 5:35 AM, Miche Zacharias <miche.zacharias at gmail.com> wrote:
> 
> Hi,
> 
> Would there be any benefit to passing CDS instead of cDNA sequences as alternative EST evidence, if both is available? Or when would you use one or the other?
> 
> I've read in previous discussions how it's expected that the organism, from which we use sequences as alternative EST and protein evidence, is not closely related to the genome we are annotating.
> In what terms is "closely related" defined? It's now unclear to me when you would put EST/protein evidence as alternative and when not or put it as direct EST/protein evidence?
> Would a species from the same family classify as alternative or direct evidence? Is ~30mya divergence too much?
> What about species of the same genus?
> 
> Many thanks!
> Miche
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From nathan.ricks at gmail.com  Fri Apr 28 10:47:55 2017
From: nathan.ricks at gmail.com (nathan.ricks at gmail.com)
Date: Fri, 28 Apr 2017 09:47:55 -0600
Subject: [maker-devel] Issues with Maker - RepBase
Message-ID: <5903642d.d6d3240a.759d3.6a01@mx.google.com>

Hi,
I?ve been having some issues getting Maker to run correctly. I don?t have a huge amount of Linux background, so perhaps that is why.
When I run it, it says 
WARNING: RepBase is not installed for RepeatMasker.
So I?ve tried getting RepBase installed, and I?m not sure what to do.
I downloaded the RepbaseRepeaMaskerEdition-20170127.tar.gz and I?ve tried unpacking , but it doesn?t give me a file I can work with, all it gives me is a folder named Library with very little in it. I?m assuming that I need to take that folder and put it somewhere in the RepeatMasker folder, but I?m not exactly sure what to do. 
Thank you for your time
Nathan Ricks


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170428/dcc2f064/attachment.html>

From carsonhh at gmail.com  Fri Apr 28 10:52:46 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Fri, 28 Apr 2017 09:52:46 -0600
Subject: [maker-devel] Issues with Maker - RepBase
In-Reply-To: <5903642d.d6d3240a.759d3.6a01@mx.google.com>
References: <5903642d.d6d3240a.759d3.6a01@mx.google.com>
Message-ID: <5E08D02E-CB26-4D63-99D1-42CD0EB2A6C0@gmail.com>

You must unpack it inside the RepeatMasker distribution per the setup instructions ?> http://www.repeatmasker.org/RMDownload.html <http://www.repeatmasker.org/RMDownload.html>

It replaces files in the ?/RepeatMasker/Library folder when unpacked.

After unpacking RepBase, you must follow the configuration instructions (same link above). Make sure to configure RepeatMasker only after unpacking RepBase.

?Carson



> On Apr 28, 2017, at 9:47 AM, <nathan.ricks at gmail.com> <nathan.ricks at gmail.com> wrote:
> 
> Hi,
> I?ve been having some issues getting Maker to run correctly. I don?t have a huge amount of Linux background, so perhaps that is why.
> When I run it, it says
> WARNING: RepBase is not installed for RepeatMasker.
> So I?ve tried getting RepBase installed, and I?m not sure what to do.
> I downloaded the RepbaseRepeaMaskerEdition-20170127.tar.gz and I?ve tried unpacking , but it doesn?t give me a file I can work with, all it gives me is a folder named Library with very little in it. I?m assuming that I need to take that folder and put it somewhere in the RepeatMasker folder, but I?m not exactly sure what to do.
> Thank you for your time
> Nathan Ricks
>  
>  
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20170428/77afe817/attachment.html>

From mcsimenc at gmail.com  Sun Apr  2 08:19:40 2017
From: mcsimenc at gmail.com (Matt Simenc)
Date: Sun, 2 Apr 2017 07:19:40 -0700
Subject: [maker-devel] MAKER RepeatMasker outputs incompatible annotations
	for single feature
Message-ID: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>

TE Superfamily RLG are aka Gypsy, but RLG is associated with Copia
annotations in our RepeatMasker annotations. Has anyone else seen this
before?


Azfi_s0077  repeatmasker match 1400976 1401715 521 + .
ID=Azfi_s0077:hit:59234:1.3.0.14;Name=species:*RLG_scAle_3_1-*
I|genus:LTR%252FCopia;Target=species:*RLG_scAle_3_*1-I|genus:*LTR%252FCopia*1522
2556 +



Matt Simenc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170402/0a6106a2/attachment-0001.html>

From carsonhh at gmail.com  Sun Apr  2 11:38:05 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 2 Apr 2017 11:38:05 -0600
Subject: [maker-devel] MAKER RepeatMasker outputs incompatible
 annotations for single feature
In-Reply-To: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>
References: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>
Message-ID: <E34482B0-55B7-4DC1-A01C-4EDBFE2E10A9@gmail.com>

This is the RepBase entry (from this file ?/RepeatMasker/Libraries/RepeatMasker.lib) ?>

>RLG_scAle_3_1-LTR#LTR/Copia @Saccharum_hybrid_cultivar  [S:] RepbaseID: RLG_scAle_3_1-LTR
tgttagaagattatgttaatatcttctgtataaggtaggggatctgtgtg
accacgagatccggtggattggcttgcaagtcaagttgtaatctatgtaa
atctaaccttcttgttgtacaagcccaaggggctatatattaacatgtac
cgagtacctgagaggtatctcgttccatctatttctaca

There are 64 RLG entries associated with Copia in the RepBase library for RepeatMasker. I?m not a repeat expert, so I can?t address whether this is an error or not. But if you think there is something amiss, you could contact RepBase.

?Carson


> On Apr 2, 2017, at 8:19 AM, Matt Simenc <mcsimenc at gmail.com> wrote:
> 
> TE Superfamily RLG are aka Gypsy, but RLG is associated with Copia annotations in our RepeatMasker annotations. Has anyone else seen this before?
> 
> 
> Azfi_s0077  repeatmasker match
> 1400976 1401715
> 521 +
> . ID=Azfi_s0077:hit:59234:1.3.0.14;Name=species:RLG_scAle_3_1-I|genus:LTR%252FCopia;Target=species:RLG_scAle_3_1-I|genus:LTR%252FCopia1522 2556 +
> 
> 
> 
> Matt Simenc
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170402/acdc5126/attachment-0001.html>

From carsonhh at gmail.com  Wed Apr  5 10:00:53 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Wed, 5 Apr 2017 10:00:53 -0600
Subject: [maker-devel] Some error messages in the output log of Maker
In-Reply-To: <CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
References: <CAJNrdRH1uDgtaB0C8jFi5yceZ7dcH6FRVzJV_RFgxHmtY_T12w@mail.gmail.com>
	<CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
Message-ID: <4F7ECEC6-338A-44B0-BC82-18193A494E29@gmail.com>

Augustus can sometimes produce alternate models. It normally only happens with hints and not with the raw prediction though. MAKER by default will only keeps the best scoring one (scored by AED). But EVM technically gets to see all of the raw models. Its the kind of thing where I would have to have the exact data and contig you used to run to see if this is something easily pulled apart for EVM.

?Carson


> On Apr 5, 2017, at 6:57 AM, Ray Cui <rcui at age.mpg.de> wrote:
> 
> Dear Carson,
> 
>         I think I found the reason for this here https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/ <https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/> . 
>         These messages are from EVM. The second type of message is due to incomplete gene models, so it's not a big deal. The first type is because of the overlapping gene models predicted by Augustus. This is a bit strange. Does maker allow Augustus to produce alternative spliced variants?
> 
> Best Regards,
> Ray
> 
> Dr. Rongfeng (Ray) Cui
> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
> Wissenschaftlicher MA / Postdoctoral researcher
> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
> Tel.:+49 (0)221 496 
> Mobile:           +49 0221 37970 496 <>
> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
> www.age.mpg.de <http://www.age.mpg.de/> 
> 
> 
> 
> On Wed, Apr 5, 2017 at 1:33 PM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
> Hi Carson,
> 
>       I saw these messages in the log, is this a concern? I'm using Maker beta 3.
>        Message 1:
> 
> Error with prediction: augustus_masked lend_intergenic: 730288, rend_intergenic: 664875  at /beegfs/group_dv/software/source/EVidenceModeler-1.1.1/evidence_modeler.pl <http://evidence_modeler.pl/> line 3218.
> 
>         Message 2:
> Sorry, prediction pred_gff:gth_m.PLPscf119:hit:7108:43932 fails validation.
> (GTT)    TA     683590-683765   GC      ATG
> (GCA)    AG     701228-701409   GT      AGG
> (GGA)    AG     704938-705037   GT      AGG
> (GGC)    AG     721672-721792   GT      AGG
> (TTG)    AG     730110-730288   AC      GAA
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, terminal, 730110, 730288
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 701228, 701409
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 704938, 705037
> -recovered pred_gff:gth_m.PLPspolishing proteins
> 
> 
>      PS: I am still keep being kicked out of the maker mailing list due to some reason, if you can forward the msg. there it would be great.
>          
> Best Regards,
> Ray
> 
> Dr. Rongfeng (Ray) Cui
> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
> Wissenschaftlicher MA / Postdoctoral researcher
> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
> Mobile:           +49 0221 37970 496 <>
> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
> www.age.mpg.de <http://www.age.mpg.de/> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170405/b51fcd1b/attachment-0001.html>

From qlian003 at ucr.edu  Thu Apr  6 10:33:26 2017
From: qlian003 at ucr.edu (Qihua Liang)
Date: Fri, 7 Apr 2017 00:33:26 +0800
Subject: [maker-devel] Large dataset of RNA-Seq data
Message-ID: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>

Dear MAKER develop team,

I have some RNA-Seq data that I would like to use raw data instead of assembled transcripts in MAKER. But those raw datasets are over 800Gb. I noticed that MAKER will first read all input dataset into /tmp, and my attempts to include such large dataset failed because it exceeds the disk quota. Besides of increasing the disk quota, is there any other option to include such large datasets in MAKER annotations? Or can I redirect MAKER to read those data and use other path to temporarily store those data?

Thank you!
Best
Qihua


From carsonhh at gmail.com  Thu Apr  6 10:35:32 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 6 Apr 2017 10:35:32 -0600
Subject: [maker-devel] Large dataset of RNA-Seq data
In-Reply-To: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>
References: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>
Message-ID: <C88A1724-71D2-493F-B655-02E9A0FE3793@gmail.com>

MAKER cannot use raw RNA-Seq. It must be assembled beforehand, or processed into GFF3 using something like Stringtie.

?Carson


> On Apr 6, 2017, at 10:33 AM, Qihua Liang <qlian003 at ucr.edu> wrote:
> 
> Dear MAKER develop team,
> 
> I have some RNA-Seq data that I would like to use raw data instead of assembled transcripts in MAKER. But those raw datasets are over 800Gb. I noticed that MAKER will first read all input dataset into /tmp, and my attempts to include such large dataset failed because it exceeds the disk quota. Besides of increasing the disk quota, is there any other option to include such large datasets in MAKER annotations? Or can I redirect MAKER to read those data and use other path to temporarily store those data?
> 
> Thank you!
> Best
> Qihua
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From elyssa_garza at yahoo.com  Thu Apr  6 13:25:10 2017
From: elyssa_garza at yahoo.com (Elyssa Garza)
Date: Thu, 6 Apr 2017 19:25:10 +0000 (UTC)
Subject: [maker-devel]  merging gff files
References: <1186841770.2604454.1491506710651.ref@mail.yahoo.com>
Message-ID: <1186841770.2604454.1491506710651@mail.yahoo.com>

Hello, I currently have a MAKER-P annotated genome and would like to add additional information to the final gff file. I would like to include InDel and microsatellite genetic marker sequences/positions to the final annotation. Do you know of a reliable way to do this?

Elyssa



   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170406/09f2437b/attachment-0001.html>

From carsonhh at gmail.com  Thu Apr  6 16:01:52 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 6 Apr 2017 16:01:52 -0600
Subject: [maker-devel] maker-devel post from elyssa_garza@yahoo.com
 requires approval
In-Reply-To: <mailman.93.1491506732.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
References: <mailman.93.1491506732.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
Message-ID: <2AE2415B-5940-45F3-800B-84F9E9E28E7E@gmail.com>

If you already have GFF3, you can merge two files using the gff3_merge script that comes with MAKER.

?Carson

> From: Elyssa Garza <elyssa_garza at yahoo.com>
> Subject: [maker-devel] merging gff files
> Date: April 6, 2017 at 1:25:10 PM MDT
> To: "dence at genetics.utah.edu" <dence at genetics.utah.edu>
> Cc: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> Reply-To: Elyssa Garza <elyssa_garza at yahoo.com>
> 
> 
> Hello, I currently have a MAKER-P annotated genome and would like to add additional information to the final gff file. I would like to include InDel and microsatellite genetic marker sequences/positions to the final annotation. Do you know of a reliable way to do this?
> 
> 
> Elyssa

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170406/4f591c05/attachment-0001.html>

From carsonhh at gmail.com  Sun Apr  9 16:57:26 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 9 Apr 2017 16:57:26 -0600
Subject: [maker-devel] Some error messages in the output log of Maker
In-Reply-To: <CAJNrdRGfE1D-BrAQmQrRYL=1XLCPELKZNe9PZ4jneSuarAfd1g@mail.gmail.com>
References: <CAJNrdRH1uDgtaB0C8jFi5yceZ7dcH6FRVzJV_RFgxHmtY_T12w@mail.gmail.com>
	<CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
	<4F7ECEC6-338A-44B0-BC82-18193A494E29@gmail.com>
	<CAJNrdRGfE1D-BrAQmQrRYL=1XLCPELKZNe9PZ4jneSuarAfd1g@mail.gmail.com>
Message-ID: <0E1BAB87-339D-4D40-AA88-8C5C9CB895E9@gmail.com>

Most of the parameters affect the command line used to run blast/exonerate. In that case it will rerun. Some like the percent coverage parameter are post run filters. For those, it just re-parses. I don?t remember off the top of my head which ones fall into which category, but most fall into the first one.

?Carson

> On Apr 6, 2017, at 1:18 AM, Ray Cui <rcui at age.mpg.de> wrote:
> 
> Hi Carson,
> 
>     Yet another question: if I change the blast and exonerate parameters in the bopts file, will maker automatically rerun blast and exonerate (or, parse the existing output with the new cutoffs)?
> 
> Thanks
> Ray 
> 
> On Apr 5, 2017 6:01 PM, "Carson Holt" <carsonhh at gmail.com <mailto:carsonhh at gmail.com>> wrote:
> Augustus can sometimes produce alternate models. It normally only happens with hints and not with the raw prediction though. MAKER by default will only keeps the best scoring one (scored by AED). But EVM technically gets to see all of the raw models. Its the kind of thing where I would have to have the exact data and contig you used to run to see if this is something easily pulled apart for EVM.
> 
> ?Carson
> 
> 
>> On Apr 5, 2017, at 6:57 AM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
>> 
>> Dear Carson,
>> 
>>         I think I found the reason for this here https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/ <https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/> . 
>>         These messages are from EVM. The second type of message is due to incomplete gene models, so it's not a big deal. The first type is because of the overlapping gene models predicted by Augustus. This is a bit strange. Does maker allow Augustus to produce alternative spliced variants?
>> 
>> Best Regards,
>> Ray
>> 
>> Dr. Rongfeng (Ray) Cui
>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
>> Wissenschaftlicher MA / Postdoctoral researcher
>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
>> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
>> Mobile:           +49 0221 37970 496 <>
>> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
>> www.age.mpg.de <http://www.age.mpg.de/> 
>> 
>> 
>> 
>> On Wed, Apr 5, 2017 at 1:33 PM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
>> Hi Carson,
>> 
>>       I saw these messages in the log, is this a concern? I'm using Maker beta 3.
>>        Message 1:
>> 
>> Error with prediction: augustus_masked lend_intergenic: 730288, rend_intergenic: 664875  at /beegfs/group_dv/software/source/EVidenceModeler-1.1.1/evidence_modeler.pl <http://evidence_modeler.pl/> line 3218.
>> 
>>         Message 2:
>> Sorry, prediction pred_gff:gth_m.PLPscf119:hit:7108:43932 fails validation.
>> (GTT)    TA     683590-683765   GC      ATG
>> (GCA)    AG     701228-701409   GT      AGG
>> (GGA)    AG     704938-705037   GT      AGG
>> (GGC)    AG     721672-721792   GT      AGG
>> (TTG)    AG     730110-730288   AC      GAA
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, terminal, 730110, 730288
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 701228, 701409
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 704938, 705037
>> -recovered pred_gff:gth_m.PLPspolishing proteins
>> 
>> 
>>      PS: I am still keep being kicked out of the maker mailing list due to some reason, if you can forward the msg. there it would be great.
>>          
>> Best Regards,
>> Ray
>> 
>> Dr. Rongfeng (Ray) Cui
>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
>> Wissenschaftlicher MA / Postdoctoral researcher
>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
>> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
>> Mobile:           +49 0221 37970 496 <>
>> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
>> www.age.mpg.de <http://www.age.mpg.de/> 
>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170409/846f99a9/attachment-0001.html>

From carsonhh at gmail.com  Sun Apr  9 17:06:52 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 9 Apr 2017 17:06:52 -0600
Subject: [maker-devel] maker putting _Inline directory wherever it is run
In-Reply-To: <mailman.101.1491517636.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
References: <mailman.101.1491517636.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
Message-ID: <A690B5B0-8AE5-4898-A2A9-025AF9DF4166@gmail.com>

You have either moved directories post installation, are trying to run from the ?/maker/src/bin directory instead of ?/maker/bin or you had an issue during install. You just need to start over and follow the install instructions. The few libraries MAKER must compile with Inline::C will go into ?/maker/perl/lib/

?Carson

> From: "Sobel-Sorenson, Chandler - (scar)" <scar at email.arizona.edu>
> Subject: maker putting _Inline directory wherever it is run
> Date: April 6, 2017 at 4:27:03 PM MDT
> To: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> 
> 
> I just installed the latest maker 2.31.9 on our cluster, but whenever we issue the command "maker" (even with no arguments), it creates a "_Inline" directory in the CWD..  I guess this is related to Inline perl module?  For example:
> 
> [smrtanalysis at pac makertest]$ ls -a
> .  ..
> [smrtanalysis at pac makertest]$ maker
> ERROR: Control files not found
> 
> MAKER version 2.31.9
> 
> Usage:
> 
>     maker [options] <maker_opts> <maker_bopts> <maker_exe>
> ...
> 
> [smrtanalysis at pac makertest]$ ls -a
> .  ..  _Inline
> [smrtanalysis at pac makertest]$ ls -a _Inline/
> .  ..  build  config-x86_64-linux-thread-multi-5.010001  lib  .lock
> [smrtanalysis at pac makertest]$ 
> 
> 
> This is some problem with our module installation?  It was installed using "cpanm Inline::C".  Or problem/bug with the module itself should contact the Inline team about?  Or problem with our maker installation?
> 
> 
> Thanks,
> Chandler
> Arizona Genomics Institute
> University Of Arizona
> www.genome.arizona.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170409/2842e623/attachment-0001.html>

From wangzhennan at ioz.ac.cn  Mon Apr 10 21:08:14 2017
From: wangzhennan at ioz.ac.cn (wangzhennan at ioz.ac.cn)
Date: Tue, 11 Apr 2017 11:08:14 +0800 (GMT+08:00)
Subject: [maker-devel] How to run Maker to get the AED?
Message-ID: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>

Dear developer of Maker,

        I have used Maker to evaluate the genome annotation annotated by PASA. I have the transcript assembled by stringtie, genome (fasta), and the annotation file (gff3). How can I use the data to get the AED value from Maker? I have set the parameter of maker_opts.ctl. Can you help me to solve this? I only want to get the AED about the annotation file (gff3). Thank you very much!

        And there is also a question, I can not have a new post on https://groups.google.com/forum/#!forum/maker-devel. Why? Can you help me to solve this? Thank you!

 

         Best,

         Wangzhennan

        




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170411/9b4a2ec6/attachment-0001.html>

From scar at email.arizona.edu  Tue Apr 11 18:21:48 2017
From: scar at email.arizona.edu (Sobel-Sorenson, Chandler - (scar))
Date: Wed, 12 Apr 2017 00:21:48 +0000
Subject: [maker-devel] run maker using qsub
Message-ID: <1491956508254.51400@email.arizona.edu>

How could we run maker using qsub with our Sun Grid Engine?  We have it working with mpiexec but this does not show slots used on the grid, so can lead to over loading our cluster.
Currently we are using command like this with mpich: mpiexec -f /opt/machinelist -n 192 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl -fix_nucleotides

Thanks
Chandler
Arizona Genomics Institute



From carsonhh at gmail.com  Sun Apr 16 13:46:35 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 16 Apr 2017 13:46:35 -0600
Subject: [maker-devel] How to run Maker to get the AED?
In-Reply-To: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>
References: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>
Message-ID: <5CD6DB76-9BB6-4DD6-9C35-A8CE0ABE1603@gmail.com>

To get just AED without maker producing it?s own predictons, you must provide your models to model_gff in valid GFF3 format (gene/mRNA/exon/CDS multi-level features). Then supply any evidence to either est=, protein=, est_gff=, or protein_gff=.

The google location is just an archive. You cannot post there. All messages must be sent to the mailing list here ?> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>

Thanks,
Carson


> On Apr 10, 2017, at 9:08 PM, wangzhennan at ioz.ac.cn wrote:
> 
> Dear developer of Maker,
> 
>         I have used Maker to evaluate the genome annotation annotated by PASA. I have the transcript assembled by stringtie, genome (fasta), and the annotation file (gff3). How can I use the data to get the AED value from Maker? I have set the parameter of maker_opts.ctl. Can you help me to solve this? I only want to get the AED about the annotation file (gff3). Thank you very much!
> 
>         And there is also a question, I can not have a new post on https://groups.google.com/forum/#!forum/maker-devel <https://groups.google.com/forum/#!forum/maker-devel>. Why? Can you help me to solve this? Thank you!
> 
>  
>          Best,
> 
>          Wangzhennan
> 
>         
> 
> 
> 
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170416/a03cd31f/attachment-0001.html>

From carsonhh at gmail.com  Sun Apr 16 13:55:23 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 16 Apr 2017 13:55:23 -0600
Subject: [maker-devel] run maker using qsub
In-Reply-To: <1491956508254.51400@email.arizona.edu>
References: <1491956508254.51400@email.arizona.edu>
Message-ID: <6A3DAA63-06FA-4A44-9B89-EAD8AF8ED0BB@gmail.com>

You will have to look up documentation for Sun Grid Engine and the MPI flavor you wish to use. This is not a MAKER configuration issue, rather you need to setup Sun Grid for MPI according to it?s documentation. When Sun Grid is configured for MPI, it should automatically sets environmental variables and files for jobs. When set up correctly you should not even have to use the -n or -f option with mpiexec as it will be detected from values Sun Grid configures for you.

Then a command like the following will work in your batch job
?> mpiexec maker

?Carson



> On Apr 11, 2017, at 6:21 PM, Sobel-Sorenson, Chandler - (scar) <scar at email.arizona.edu> wrote:
> 
> How could we run maker using qsub with our Sun Grid Engine?  We have it working with mpiexec but this does not show slots used on the grid, so can lead to over loading our cluster.
> Currently we are using command like this with mpich: mpiexec -f /opt/machinelist -n 192 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl -fix_nucleotides
> 
> Thanks
> Chandler
> Arizona Genomics Institute
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From tfallon at mit.edu  Mon Apr 17 09:35:31 2017
From: tfallon at mit.edu (Tim Fallon)
Date: Mon, 17 Apr 2017 11:35:31 -0400
Subject: [maker-devel] Help with MPI installation and configuration
Message-ID: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>

Hi there,

I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.

See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?

"
I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.

Peter Macfarlane / WIBR IT


Description : Hi Peter,

Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?

All the best,
-Tim

On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:

Hi Peter,

I?ve tested with the following command:
bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?

Unfortunately it still seems to be giving errors and not running, with things like:
"?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??

It also doesn?t work if I try to run it directly on Tak4.

I?ve attached the relevant error log file.  Any more solutions?

All the best,
-Tim

On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:

I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:

- /nfs/apps/test/maker is now maker 3.00
- /nfs/apps/test/openmpi was used to make the aforementioned maker

I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
"
All the best,
-Tim

Timothy R. Fallon
PhD candidate
Laboratory of Jing-Ke Weng
Department of Biology
MIT

tfallon at mit.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0002.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: bsub_maker_with_mpi_log.txt
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0001.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1849 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0001.p7s>

From scar at email.arizona.edu  Mon Apr 17 11:29:58 2017
From: scar at email.arizona.edu (Chandler)
Date: Mon, 17 Apr 2017 10:29:58 -0700
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
Message-ID: <2940db14-712e-d318-35c5-2a7b132db842@email.arizona.edu>

Please try with MPICH instead of openmpi... we were having similar 
errors with openmpi, but when we switched to mpich it all started 
working.  Make sure only the mpich executables are in your PATH, and 
that maker is built specifying the mpich package.

Chandler / Systems Administrator
Arizona Genomics Institute
www.genome.arizona.edu





From carsonhh at gmail.com  Tue Apr 18 09:10:43 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 18 Apr 2017 09:10:43 -0600
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
Message-ID: <2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>

I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.

#this must be doen exery time you run it
export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so

# -mca btl ^openib myust be on the command line
mpiexec -mca btl ^openib -n 20 maker

#alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker

?Carson

> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu> wrote:
> 
> Hi there,
> 
> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
> 
> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
> 
> "
> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
> 
> Peter Macfarlane / WIBR IT
> 
> 
> Description : Hi Peter,
> 
> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
> 
> All the best,
> -Tim
> 
> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
> 
> Hi Peter,
> 
> I?ve tested with the following command:
> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
> 
> Unfortunately it still seems to be giving errors and not running, with things like:
> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
> 
> It also doesn?t work if I try to run it directly on Tak4.
> 
> I?ve attached the relevant error log file.  Any more solutions?
> 
> All the best,
> -Tim
> 
> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
> 
> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
> 
> - /nfs/apps/test/maker is now maker 3.00
> - /nfs/apps/test/openmpi was used to make the aforementioned maker
> 
> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
> "
> All the best,
> -Tim
> 
> Timothy R. Fallon
> PhD candidate
> Laboratory of Jing-Ke Weng
> Department of Biology
> MIT
> 
> tfallon at mit.edu <mailto:tfallon at mit.edu>
> <bsub_maker_with_mpi_log.txt>
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/e7ebdbc2/attachment-0001.html>

From carsonhh at gmail.com  Tue Apr 18 09:23:04 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 18 Apr 2017 09:23:04 -0600
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
	<2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
	<CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>
Message-ID: <BDCC2D30-B9DB-4FF1-921A-307975EA8BB2@gmail.com>

MPICH3 works without any special configuration. OpenMPI is built to support infiniband which internally can drive other problems related to how registered memory is used when infiniband libraries are loaded. So you have have to set more parameters.

Both will use shared memory for communications when running on a single machine, but you will still need to set parameters on OpenMPI.

?Carson



> On Apr 18, 2017, at 9:19 AM, Tim Fallon <tfallon at mit.edu> wrote:
> 
> Hi Carson,
> 
> Thanks for the feedback, I?ll look into the maker INSTALL file to ensure things are setup correctly.
> 
> Should I be looking into MPICH3 instead?  I?m trying to run maker on a single node (~20 CPUs), rather than distribute it across our cluster.  I assume that using MPICH3 changes the mpiexec / mpirun executable, but not the commands otherwise?  I ask as I may have to configure this myself one day, so any tips appreciated.  In the meantime your comments seem straightforward, will give it a shot.
> 
> All the best,
> -Tim
> 
>> On Apr 18, 2017, at 11:10 AM, Carson Holt <carsonhh at gmail.com <mailto:carsonhh at gmail.com>> wrote:
>> 
>> I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.
>> 
>> #this must be doen exery time you run it
>> export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so
>> 
>> # -mca btl ^openib myust be on the command line
>> mpiexec -mca btl ^openib -n 20 maker
>> 
>> #alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
>> mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker
>> 
>> ?Carson
>> 
>>> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>>> 
>>> Hi there,
>>> 
>>> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
>>> 
>>> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
>>> 
>>> "
>>> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
>>> 
>>> Peter Macfarlane / WIBR IT
>>> 
>>> 
>>> Description : Hi Peter,
>>> 
>>> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
>>> 
>>> All the best,
>>> -Tim
>>> 
>>> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>>> 
>>> Hi Peter,
>>> 
>>> I?ve tested with the following command:
>>> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
>>> 
>>> Unfortunately it still seems to be giving errors and not running, with things like:
>>> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
>>> 
>>> It also doesn?t work if I try to run it directly on Tak4.
>>> 
>>> I?ve attached the relevant error log file.  Any more solutions?
>>> 
>>> All the best,
>>> -Tim
>>> 
>>> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
>>> 
>>> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
>>> 
>>> - /nfs/apps/test/maker is now maker 3.00
>>> - /nfs/apps/test/openmpi was used to make the aforementioned maker
>>> 
>>> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
>>> "
>>> All the best,
>>> -Tim
>>> 
>>> Timothy R. Fallon
>>> PhD candidate
>>> Laboratory of Jing-Ke Weng
>>> Department of Biology
>>> MIT
>>> 
>>> tfallon at mit.edu <mailto:tfallon at mit.edu>
>>> <bsub_maker_with_mpi_log.txt>
>>> _______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>
>> 
> 
> Timothy R. Fallon
> PhD candidate
> Laboratory of Jing-Ke Weng
> Department of Biology
> MIT
> 
> tfallon at mit.edu <mailto:tfallon at mit.edu>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/a23814d7/attachment-0001.html>

From tfallon at mit.edu  Tue Apr 18 09:19:19 2017
From: tfallon at mit.edu (Tim Fallon)
Date: Tue, 18 Apr 2017 11:19:19 -0400
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
	<2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
Message-ID: <CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>

Hi Carson,

Thanks for the feedback, I?ll look into the maker INSTALL file to ensure things are setup correctly.

Should I be looking into MPICH3 instead?  I?m trying to run maker on a single node (~20 CPUs), rather than distribute it across our cluster.  I assume that using MPICH3 changes the mpiexec / mpirun executable, but not the commands otherwise?  I ask as I may have to configure this myself one day, so any tips appreciated.  In the meantime your comments seem straightforward, will give it a shot.

All the best,
-Tim

> On Apr 18, 2017, at 11:10 AM, Carson Holt <carsonhh at gmail.com> wrote:
> 
> I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.
> 
> #this must be doen exery time you run it
> export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so
> 
> # -mca btl ^openib myust be on the command line
> mpiexec -mca btl ^openib -n 20 maker
> 
> #alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
> mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker
> 
> ?Carson
> 
>> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>> 
>> Hi there,
>> 
>> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
>> 
>> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
>> 
>> "
>> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
>> 
>> Peter Macfarlane / WIBR IT
>> 
>> 
>> Description : Hi Peter,
>> 
>> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
>> 
>> All the best,
>> -Tim
>> 
>> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>> 
>> Hi Peter,
>> 
>> I?ve tested with the following command:
>> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
>> 
>> Unfortunately it still seems to be giving errors and not running, with things like:
>> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
>> 
>> It also doesn?t work if I try to run it directly on Tak4.
>> 
>> I?ve attached the relevant error log file.  Any more solutions?
>> 
>> All the best,
>> -Tim
>> 
>> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
>> 
>> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
>> 
>> - /nfs/apps/test/maker is now maker 3.00
>> - /nfs/apps/test/openmpi was used to make the aforementioned maker
>> 
>> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
>> "
>> All the best,
>> -Tim
>> 
>> Timothy R. Fallon
>> PhD candidate
>> Laboratory of Jing-Ke Weng
>> Department of Biology
>> MIT
>> 
>> tfallon at mit.edu <mailto:tfallon at mit.edu>
>> <bsub_maker_with_mpi_log.txt>
>> _______________________________________________
>> maker-devel mailing list
>> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
> 

Timothy R. Fallon
PhD candidate
Laboratory of Jing-Ke Weng
Department of Biology
MIT

tfallon at mit.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/d3ae60e9/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1849 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/d3ae60e9/attachment-0001.p7s>

From miche.zacharias at gmail.com  Tue Apr 25 05:35:37 2017
From: miche.zacharias at gmail.com (Miche Zacharias)
Date: Tue, 25 Apr 2017 13:35:37 +0200
Subject: [maker-devel] altEST evidence
Message-ID: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>

Hi,

Would there be any benefit to passing CDS instead of cDNA sequences as
alternative EST evidence, if both is available? Or when would you use one
or the other?

I've read in previous discussions how it's expected that the organism, from
which we use sequences as alternative EST and protein evidence, is not
closely related to the genome we are annotating.
In what terms is "closely related" defined? It's now unclear to me when you
would put EST/protein evidence as alternative and when not or put it as
direct EST/protein evidence?
Would a species from the same family classify as alternative or direct
evidence? Is ~30mya divergence too much?
What about species of the same genus?

Many thanks!
Miche
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170425/2aafa4e5/attachment-0001.html>

From carsonhh at gmail.com  Tue Apr 25 09:30:21 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 25 Apr 2017 09:30:21 -0600
Subject: [maker-devel] altEST evidence
In-Reply-To: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>
References: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>
Message-ID: <EA7FC16A-2A92-49C1-96FF-774FD39E86DA@gmail.com>

If you have CDS, then it is not really transcription evidence, rather it was derived from protein annotations and the protein data should be used instead. The alt-EST option is only for datasets where no protein models exists, but you do have experimental evidence from transcription (i.e. not transcript predictions but actual mRNA experimentation).

Nucleotides sequence diverges more rapidly than amino acid sequence because of codon redundancy and amino acid similarity. As a result mRNA derived sequence can be compared in nucleotide space to genomic sequence of related organisms with an evolutionary difference of something like human to gorilla, but human to mouse would require amino acid level alignment to find the homology. So with alt-EST we use TBLASTX and Exonerate cdna2genome to perform a six frame translations of both the genome and assembly for alignment in amino acid space. This is of course very expensive computationally and more likely to generate spurious alignments. So for CDS sequence it would be both redundant to and less accurate than using the protein data directly.

?Carson



> On Apr 25, 2017, at 5:35 AM, Miche Zacharias <miche.zacharias at gmail.com> wrote:
> 
> Hi,
> 
> Would there be any benefit to passing CDS instead of cDNA sequences as alternative EST evidence, if both is available? Or when would you use one or the other?
> 
> I've read in previous discussions how it's expected that the organism, from which we use sequences as alternative EST and protein evidence, is not closely related to the genome we are annotating.
> In what terms is "closely related" defined? It's now unclear to me when you would put EST/protein evidence as alternative and when not or put it as direct EST/protein evidence?
> Would a species from the same family classify as alternative or direct evidence? Is ~30mya divergence too much?
> What about species of the same genus?
> 
> Many thanks!
> Miche
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From nathan.ricks at gmail.com  Fri Apr 28 09:47:55 2017
From: nathan.ricks at gmail.com (nathan.ricks at gmail.com)
Date: Fri, 28 Apr 2017 09:47:55 -0600
Subject: [maker-devel] Issues with Maker - RepBase
Message-ID: <5903642d.d6d3240a.759d3.6a01@mx.google.com>

Hi,
I?ve been having some issues getting Maker to run correctly. I don?t have a huge amount of Linux background, so perhaps that is why.
When I run it, it says 
WARNING: RepBase is not installed for RepeatMasker.
So I?ve tried getting RepBase installed, and I?m not sure what to do.
I downloaded the RepbaseRepeaMaskerEdition-20170127.tar.gz and I?ve tried unpacking , but it doesn?t give me a file I can work with, all it gives me is a folder named Library with very little in it. I?m assuming that I need to take that folder and put it somewhere in the RepeatMasker folder, but I?m not exactly sure what to do. 
Thank you for your time
Nathan Ricks


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170428/dcc2f064/attachment-0001.html>

From carsonhh at gmail.com  Fri Apr 28 09:52:46 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Fri, 28 Apr 2017 09:52:46 -0600
Subject: [maker-devel] Issues with Maker - RepBase
In-Reply-To: <5903642d.d6d3240a.759d3.6a01@mx.google.com>
References: <5903642d.d6d3240a.759d3.6a01@mx.google.com>
Message-ID: <5E08D02E-CB26-4D63-99D1-42CD0EB2A6C0@gmail.com>

You must unpack it inside the RepeatMasker distribution per the setup instructions ?> http://www.repeatmasker.org/RMDownload.html <http://www.repeatmasker.org/RMDownload.html>

It replaces files in the ?/RepeatMasker/Library folder when unpacked.

After unpacking RepBase, you must follow the configuration instructions (same link above). Make sure to configure RepeatMasker only after unpacking RepBase.

?Carson



> On Apr 28, 2017, at 9:47 AM, <nathan.ricks at gmail.com> <nathan.ricks at gmail.com> wrote:
> 
> Hi,
> I?ve been having some issues getting Maker to run correctly. I don?t have a huge amount of Linux background, so perhaps that is why.
> When I run it, it says
> WARNING: RepBase is not installed for RepeatMasker.
> So I?ve tried getting RepBase installed, and I?m not sure what to do.
> I downloaded the RepbaseRepeaMaskerEdition-20170127.tar.gz and I?ve tried unpacking , but it doesn?t give me a file I can work with, all it gives me is a folder named Library with very little in it. I?m assuming that I need to take that folder and put it somewhere in the RepeatMasker folder, but I?m not exactly sure what to do.
> Thank you for your time
> Nathan Ricks
>  
>  
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170428/77afe817/attachment-0001.html>

From mcsimenc at gmail.com  Sun Apr  2 08:19:40 2017
From: mcsimenc at gmail.com (Matt Simenc)
Date: Sun, 2 Apr 2017 07:19:40 -0700
Subject: [maker-devel] MAKER RepeatMasker outputs incompatible annotations
	for single feature
Message-ID: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>

TE Superfamily RLG are aka Gypsy, but RLG is associated with Copia
annotations in our RepeatMasker annotations. Has anyone else seen this
before?


Azfi_s0077  repeatmasker match 1400976 1401715 521 + .
ID=Azfi_s0077:hit:59234:1.3.0.14;Name=species:*RLG_scAle_3_1-*
I|genus:LTR%252FCopia;Target=species:*RLG_scAle_3_*1-I|genus:*LTR%252FCopia*1522
2556 +



Matt Simenc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170402/0a6106a2/attachment-0002.html>

From carsonhh at gmail.com  Sun Apr  2 11:38:05 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 2 Apr 2017 11:38:05 -0600
Subject: [maker-devel] MAKER RepeatMasker outputs incompatible
 annotations for single feature
In-Reply-To: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>
References: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>
Message-ID: <E34482B0-55B7-4DC1-A01C-4EDBFE2E10A9@gmail.com>

This is the RepBase entry (from this file ?/RepeatMasker/Libraries/RepeatMasker.lib) ?>

>RLG_scAle_3_1-LTR#LTR/Copia @Saccharum_hybrid_cultivar  [S:] RepbaseID: RLG_scAle_3_1-LTR
tgttagaagattatgttaatatcttctgtataaggtaggggatctgtgtg
accacgagatccggtggattggcttgcaagtcaagttgtaatctatgtaa
atctaaccttcttgttgtacaagcccaaggggctatatattaacatgtac
cgagtacctgagaggtatctcgttccatctatttctaca

There are 64 RLG entries associated with Copia in the RepBase library for RepeatMasker. I?m not a repeat expert, so I can?t address whether this is an error or not. But if you think there is something amiss, you could contact RepBase.

?Carson


> On Apr 2, 2017, at 8:19 AM, Matt Simenc <mcsimenc at gmail.com> wrote:
> 
> TE Superfamily RLG are aka Gypsy, but RLG is associated with Copia annotations in our RepeatMasker annotations. Has anyone else seen this before?
> 
> 
> Azfi_s0077  repeatmasker match
> 1400976 1401715
> 521 +
> . ID=Azfi_s0077:hit:59234:1.3.0.14;Name=species:RLG_scAle_3_1-I|genus:LTR%252FCopia;Target=species:RLG_scAle_3_1-I|genus:LTR%252FCopia1522 2556 +
> 
> 
> 
> Matt Simenc
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170402/acdc5126/attachment-0002.html>

From carsonhh at gmail.com  Wed Apr  5 10:00:53 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Wed, 5 Apr 2017 10:00:53 -0600
Subject: [maker-devel] Some error messages in the output log of Maker
In-Reply-To: <CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
References: <CAJNrdRH1uDgtaB0C8jFi5yceZ7dcH6FRVzJV_RFgxHmtY_T12w@mail.gmail.com>
	<CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
Message-ID: <4F7ECEC6-338A-44B0-BC82-18193A494E29@gmail.com>

Augustus can sometimes produce alternate models. It normally only happens with hints and not with the raw prediction though. MAKER by default will only keeps the best scoring one (scored by AED). But EVM technically gets to see all of the raw models. Its the kind of thing where I would have to have the exact data and contig you used to run to see if this is something easily pulled apart for EVM.

?Carson


> On Apr 5, 2017, at 6:57 AM, Ray Cui <rcui at age.mpg.de> wrote:
> 
> Dear Carson,
> 
>         I think I found the reason for this here https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/ <https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/> . 
>         These messages are from EVM. The second type of message is due to incomplete gene models, so it's not a big deal. The first type is because of the overlapping gene models predicted by Augustus. This is a bit strange. Does maker allow Augustus to produce alternative spliced variants?
> 
> Best Regards,
> Ray
> 
> Dr. Rongfeng (Ray) Cui
> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
> Wissenschaftlicher MA / Postdoctoral researcher
> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
> Tel.:+49 (0)221 496 
> Mobile:           +49 0221 37970 496 <>
> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
> www.age.mpg.de <http://www.age.mpg.de/> 
> 
> 
> 
> On Wed, Apr 5, 2017 at 1:33 PM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
> Hi Carson,
> 
>       I saw these messages in the log, is this a concern? I'm using Maker beta 3.
>        Message 1:
> 
> Error with prediction: augustus_masked lend_intergenic: 730288, rend_intergenic: 664875  at /beegfs/group_dv/software/source/EVidenceModeler-1.1.1/evidence_modeler.pl <http://evidence_modeler.pl/> line 3218.
> 
>         Message 2:
> Sorry, prediction pred_gff:gth_m.PLPscf119:hit:7108:43932 fails validation.
> (GTT)    TA     683590-683765   GC      ATG
> (GCA)    AG     701228-701409   GT      AGG
> (GGA)    AG     704938-705037   GT      AGG
> (GGC)    AG     721672-721792   GT      AGG
> (TTG)    AG     730110-730288   AC      GAA
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, terminal, 730110, 730288
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 701228, 701409
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 704938, 705037
> -recovered pred_gff:gth_m.PLPspolishing proteins
> 
> 
>      PS: I am still keep being kicked out of the maker mailing list due to some reason, if you can forward the msg. there it would be great.
>          
> Best Regards,
> Ray
> 
> Dr. Rongfeng (Ray) Cui
> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
> Wissenschaftlicher MA / Postdoctoral researcher
> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
> Mobile:           +49 0221 37970 496 <>
> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
> www.age.mpg.de <http://www.age.mpg.de/> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170405/b51fcd1b/attachment-0002.html>

From qlian003 at ucr.edu  Thu Apr  6 10:33:26 2017
From: qlian003 at ucr.edu (Qihua Liang)
Date: Fri, 7 Apr 2017 00:33:26 +0800
Subject: [maker-devel] Large dataset of RNA-Seq data
Message-ID: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>

Dear MAKER develop team,

I have some RNA-Seq data that I would like to use raw data instead of assembled transcripts in MAKER. But those raw datasets are over 800Gb. I noticed that MAKER will first read all input dataset into /tmp, and my attempts to include such large dataset failed because it exceeds the disk quota. Besides of increasing the disk quota, is there any other option to include such large datasets in MAKER annotations? Or can I redirect MAKER to read those data and use other path to temporarily store those data?

Thank you!
Best
Qihua


From carsonhh at gmail.com  Thu Apr  6 10:35:32 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 6 Apr 2017 10:35:32 -0600
Subject: [maker-devel] Large dataset of RNA-Seq data
In-Reply-To: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>
References: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>
Message-ID: <C88A1724-71D2-493F-B655-02E9A0FE3793@gmail.com>

MAKER cannot use raw RNA-Seq. It must be assembled beforehand, or processed into GFF3 using something like Stringtie.

?Carson


> On Apr 6, 2017, at 10:33 AM, Qihua Liang <qlian003 at ucr.edu> wrote:
> 
> Dear MAKER develop team,
> 
> I have some RNA-Seq data that I would like to use raw data instead of assembled transcripts in MAKER. But those raw datasets are over 800Gb. I noticed that MAKER will first read all input dataset into /tmp, and my attempts to include such large dataset failed because it exceeds the disk quota. Besides of increasing the disk quota, is there any other option to include such large datasets in MAKER annotations? Or can I redirect MAKER to read those data and use other path to temporarily store those data?
> 
> Thank you!
> Best
> Qihua
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From elyssa_garza at yahoo.com  Thu Apr  6 13:25:10 2017
From: elyssa_garza at yahoo.com (Elyssa Garza)
Date: Thu, 6 Apr 2017 19:25:10 +0000 (UTC)
Subject: [maker-devel]  merging gff files
References: <1186841770.2604454.1491506710651.ref@mail.yahoo.com>
Message-ID: <1186841770.2604454.1491506710651@mail.yahoo.com>

Hello, I currently have a MAKER-P annotated genome and would like to add additional information to the final gff file. I would like to include InDel and microsatellite genetic marker sequences/positions to the final annotation. Do you know of a reliable way to do this?

Elyssa



   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170406/09f2437b/attachment-0002.html>

From carsonhh at gmail.com  Thu Apr  6 16:01:52 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 6 Apr 2017 16:01:52 -0600
Subject: [maker-devel] maker-devel post from elyssa_garza@yahoo.com
 requires approval
In-Reply-To: <mailman.93.1491506732.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
References: <mailman.93.1491506732.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
Message-ID: <2AE2415B-5940-45F3-800B-84F9E9E28E7E@gmail.com>

If you already have GFF3, you can merge two files using the gff3_merge script that comes with MAKER.

?Carson

> From: Elyssa Garza <elyssa_garza at yahoo.com>
> Subject: [maker-devel] merging gff files
> Date: April 6, 2017 at 1:25:10 PM MDT
> To: "dence at genetics.utah.edu" <dence at genetics.utah.edu>
> Cc: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> Reply-To: Elyssa Garza <elyssa_garza at yahoo.com>
> 
> 
> Hello, I currently have a MAKER-P annotated genome and would like to add additional information to the final gff file. I would like to include InDel and microsatellite genetic marker sequences/positions to the final annotation. Do you know of a reliable way to do this?
> 
> 
> Elyssa

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170406/4f591c05/attachment-0002.html>

From carsonhh at gmail.com  Sun Apr  9 16:57:26 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 9 Apr 2017 16:57:26 -0600
Subject: [maker-devel] Some error messages in the output log of Maker
In-Reply-To: <CAJNrdRGfE1D-BrAQmQrRYL=1XLCPELKZNe9PZ4jneSuarAfd1g@mail.gmail.com>
References: <CAJNrdRH1uDgtaB0C8jFi5yceZ7dcH6FRVzJV_RFgxHmtY_T12w@mail.gmail.com>
	<CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
	<4F7ECEC6-338A-44B0-BC82-18193A494E29@gmail.com>
	<CAJNrdRGfE1D-BrAQmQrRYL=1XLCPELKZNe9PZ4jneSuarAfd1g@mail.gmail.com>
Message-ID: <0E1BAB87-339D-4D40-AA88-8C5C9CB895E9@gmail.com>

Most of the parameters affect the command line used to run blast/exonerate. In that case it will rerun. Some like the percent coverage parameter are post run filters. For those, it just re-parses. I don?t remember off the top of my head which ones fall into which category, but most fall into the first one.

?Carson

> On Apr 6, 2017, at 1:18 AM, Ray Cui <rcui at age.mpg.de> wrote:
> 
> Hi Carson,
> 
>     Yet another question: if I change the blast and exonerate parameters in the bopts file, will maker automatically rerun blast and exonerate (or, parse the existing output with the new cutoffs)?
> 
> Thanks
> Ray 
> 
> On Apr 5, 2017 6:01 PM, "Carson Holt" <carsonhh at gmail.com <mailto:carsonhh at gmail.com>> wrote:
> Augustus can sometimes produce alternate models. It normally only happens with hints and not with the raw prediction though. MAKER by default will only keeps the best scoring one (scored by AED). But EVM technically gets to see all of the raw models. Its the kind of thing where I would have to have the exact data and contig you used to run to see if this is something easily pulled apart for EVM.
> 
> ?Carson
> 
> 
>> On Apr 5, 2017, at 6:57 AM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
>> 
>> Dear Carson,
>> 
>>         I think I found the reason for this here https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/ <https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/> . 
>>         These messages are from EVM. The second type of message is due to incomplete gene models, so it's not a big deal. The first type is because of the overlapping gene models predicted by Augustus. This is a bit strange. Does maker allow Augustus to produce alternative spliced variants?
>> 
>> Best Regards,
>> Ray
>> 
>> Dr. Rongfeng (Ray) Cui
>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
>> Wissenschaftlicher MA / Postdoctoral researcher
>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
>> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
>> Mobile:           +49 0221 37970 496 <>
>> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
>> www.age.mpg.de <http://www.age.mpg.de/> 
>> 
>> 
>> 
>> On Wed, Apr 5, 2017 at 1:33 PM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
>> Hi Carson,
>> 
>>       I saw these messages in the log, is this a concern? I'm using Maker beta 3.
>>        Message 1:
>> 
>> Error with prediction: augustus_masked lend_intergenic: 730288, rend_intergenic: 664875  at /beegfs/group_dv/software/source/EVidenceModeler-1.1.1/evidence_modeler.pl <http://evidence_modeler.pl/> line 3218.
>> 
>>         Message 2:
>> Sorry, prediction pred_gff:gth_m.PLPscf119:hit:7108:43932 fails validation.
>> (GTT)    TA     683590-683765   GC      ATG
>> (GCA)    AG     701228-701409   GT      AGG
>> (GGA)    AG     704938-705037   GT      AGG
>> (GGC)    AG     721672-721792   GT      AGG
>> (TTG)    AG     730110-730288   AC      GAA
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, terminal, 730110, 730288
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 701228, 701409
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 704938, 705037
>> -recovered pred_gff:gth_m.PLPspolishing proteins
>> 
>> 
>>      PS: I am still keep being kicked out of the maker mailing list due to some reason, if you can forward the msg. there it would be great.
>>          
>> Best Regards,
>> Ray
>> 
>> Dr. Rongfeng (Ray) Cui
>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
>> Wissenschaftlicher MA / Postdoctoral researcher
>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
>> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
>> Mobile:           +49 0221 37970 496 <>
>> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
>> www.age.mpg.de <http://www.age.mpg.de/> 
>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170409/846f99a9/attachment-0002.html>

From carsonhh at gmail.com  Sun Apr  9 17:06:52 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 9 Apr 2017 17:06:52 -0600
Subject: [maker-devel] maker putting _Inline directory wherever it is run
In-Reply-To: <mailman.101.1491517636.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
References: <mailman.101.1491517636.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
Message-ID: <A690B5B0-8AE5-4898-A2A9-025AF9DF4166@gmail.com>

You have either moved directories post installation, are trying to run from the ?/maker/src/bin directory instead of ?/maker/bin or you had an issue during install. You just need to start over and follow the install instructions. The few libraries MAKER must compile with Inline::C will go into ?/maker/perl/lib/

?Carson

> From: "Sobel-Sorenson, Chandler - (scar)" <scar at email.arizona.edu>
> Subject: maker putting _Inline directory wherever it is run
> Date: April 6, 2017 at 4:27:03 PM MDT
> To: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> 
> 
> I just installed the latest maker 2.31.9 on our cluster, but whenever we issue the command "maker" (even with no arguments), it creates a "_Inline" directory in the CWD..  I guess this is related to Inline perl module?  For example:
> 
> [smrtanalysis at pac makertest]$ ls -a
> .  ..
> [smrtanalysis at pac makertest]$ maker
> ERROR: Control files not found
> 
> MAKER version 2.31.9
> 
> Usage:
> 
>     maker [options] <maker_opts> <maker_bopts> <maker_exe>
> ...
> 
> [smrtanalysis at pac makertest]$ ls -a
> .  ..  _Inline
> [smrtanalysis at pac makertest]$ ls -a _Inline/
> .  ..  build  config-x86_64-linux-thread-multi-5.010001  lib  .lock
> [smrtanalysis at pac makertest]$ 
> 
> 
> This is some problem with our module installation?  It was installed using "cpanm Inline::C".  Or problem/bug with the module itself should contact the Inline team about?  Or problem with our maker installation?
> 
> 
> Thanks,
> Chandler
> Arizona Genomics Institute
> University Of Arizona
> www.genome.arizona.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170409/2842e623/attachment-0002.html>

From wangzhennan at ioz.ac.cn  Mon Apr 10 21:08:14 2017
From: wangzhennan at ioz.ac.cn (wangzhennan at ioz.ac.cn)
Date: Tue, 11 Apr 2017 11:08:14 +0800 (GMT+08:00)
Subject: [maker-devel] How to run Maker to get the AED?
Message-ID: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>

Dear developer of Maker,

        I have used Maker to evaluate the genome annotation annotated by PASA. I have the transcript assembled by stringtie, genome (fasta), and the annotation file (gff3). How can I use the data to get the AED value from Maker? I have set the parameter of maker_opts.ctl. Can you help me to solve this? I only want to get the AED about the annotation file (gff3). Thank you very much!

        And there is also a question, I can not have a new post on https://groups.google.com/forum/#!forum/maker-devel. Why? Can you help me to solve this? Thank you!

 

         Best,

         Wangzhennan

        




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170411/9b4a2ec6/attachment-0002.html>

From scar at email.arizona.edu  Tue Apr 11 18:21:48 2017
From: scar at email.arizona.edu (Sobel-Sorenson, Chandler - (scar))
Date: Wed, 12 Apr 2017 00:21:48 +0000
Subject: [maker-devel] run maker using qsub
Message-ID: <1491956508254.51400@email.arizona.edu>

How could we run maker using qsub with our Sun Grid Engine?  We have it working with mpiexec but this does not show slots used on the grid, so can lead to over loading our cluster.
Currently we are using command like this with mpich: mpiexec -f /opt/machinelist -n 192 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl -fix_nucleotides

Thanks
Chandler
Arizona Genomics Institute



From carsonhh at gmail.com  Sun Apr 16 13:46:35 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 16 Apr 2017 13:46:35 -0600
Subject: [maker-devel] How to run Maker to get the AED?
In-Reply-To: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>
References: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>
Message-ID: <5CD6DB76-9BB6-4DD6-9C35-A8CE0ABE1603@gmail.com>

To get just AED without maker producing it?s own predictons, you must provide your models to model_gff in valid GFF3 format (gene/mRNA/exon/CDS multi-level features). Then supply any evidence to either est=, protein=, est_gff=, or protein_gff=.

The google location is just an archive. You cannot post there. All messages must be sent to the mailing list here ?> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>

Thanks,
Carson


> On Apr 10, 2017, at 9:08 PM, wangzhennan at ioz.ac.cn wrote:
> 
> Dear developer of Maker,
> 
>         I have used Maker to evaluate the genome annotation annotated by PASA. I have the transcript assembled by stringtie, genome (fasta), and the annotation file (gff3). How can I use the data to get the AED value from Maker? I have set the parameter of maker_opts.ctl. Can you help me to solve this? I only want to get the AED about the annotation file (gff3). Thank you very much!
> 
>         And there is also a question, I can not have a new post on https://groups.google.com/forum/#!forum/maker-devel <https://groups.google.com/forum/#!forum/maker-devel>. Why? Can you help me to solve this? Thank you!
> 
>  
>          Best,
> 
>          Wangzhennan
> 
>         
> 
> 
> 
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170416/a03cd31f/attachment-0002.html>

From carsonhh at gmail.com  Sun Apr 16 13:55:23 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 16 Apr 2017 13:55:23 -0600
Subject: [maker-devel] run maker using qsub
In-Reply-To: <1491956508254.51400@email.arizona.edu>
References: <1491956508254.51400@email.arizona.edu>
Message-ID: <6A3DAA63-06FA-4A44-9B89-EAD8AF8ED0BB@gmail.com>

You will have to look up documentation for Sun Grid Engine and the MPI flavor you wish to use. This is not a MAKER configuration issue, rather you need to setup Sun Grid for MPI according to it?s documentation. When Sun Grid is configured for MPI, it should automatically sets environmental variables and files for jobs. When set up correctly you should not even have to use the -n or -f option with mpiexec as it will be detected from values Sun Grid configures for you.

Then a command like the following will work in your batch job
?> mpiexec maker

?Carson



> On Apr 11, 2017, at 6:21 PM, Sobel-Sorenson, Chandler - (scar) <scar at email.arizona.edu> wrote:
> 
> How could we run maker using qsub with our Sun Grid Engine?  We have it working with mpiexec but this does not show slots used on the grid, so can lead to over loading our cluster.
> Currently we are using command like this with mpich: mpiexec -f /opt/machinelist -n 192 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl -fix_nucleotides
> 
> Thanks
> Chandler
> Arizona Genomics Institute
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From tfallon at mit.edu  Mon Apr 17 09:35:31 2017
From: tfallon at mit.edu (Tim Fallon)
Date: Mon, 17 Apr 2017 11:35:31 -0400
Subject: [maker-devel] Help with MPI installation and configuration
Message-ID: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>

Hi there,

I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.

See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?

"
I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.

Peter Macfarlane / WIBR IT


Description : Hi Peter,

Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?

All the best,
-Tim

On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:

Hi Peter,

I?ve tested with the following command:
bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?

Unfortunately it still seems to be giving errors and not running, with things like:
"?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??

It also doesn?t work if I try to run it directly on Tak4.

I?ve attached the relevant error log file.  Any more solutions?

All the best,
-Tim

On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:

I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:

- /nfs/apps/test/maker is now maker 3.00
- /nfs/apps/test/openmpi was used to make the aforementioned maker

I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
"
All the best,
-Tim

Timothy R. Fallon
PhD candidate
Laboratory of Jing-Ke Weng
Department of Biology
MIT

tfallon at mit.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0004.html>
-------------- next part --------------
[it-c05b10:00466] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00467] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00465] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00482] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00468] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00473] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00476] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00481] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00466] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00467] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00468] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00466] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00476] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00467] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00468] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00473] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00482] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00467] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00476] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00466] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00473] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00468] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00465] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00481] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00482] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00476] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00473] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00482] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00481] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00465] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00481] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00465] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:466] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:467] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:473] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:476] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:482] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[30092,1],1]
  Exit code:    1
--------------------------------------------------------------------------

------------------------------------------------------------
Sender: LSF System <lsfadmin at it-c05b10>
Subject: Job 1703425: </nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker> Exited

Job </nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker> was submitted from host <tak4> by user <tfallon> in cluster <irken>.
Job was executed on host(s) <it-c05b10>, in queue <normal>, as user <tfallon> in cluster <irken>.
</home/tfallon> was used as the home directory.
</lab/solexa_weng/Seq_data/Projects/Tim_Fallon/ppyralis_genome/Genome_project_reference_assemblies/version1/analyses/maker_mpi_troubleshooting_withmpi> was used as the working directory.
Started at Fri Apr  7 09:04:14 2017
Results reported at Fri Apr  7 09:04:18 2017

Your job looked like:

------------------------------------------------------------
# LSBATCH: User input
/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker
------------------------------------------------------------

Exited with exit code 1.

Resource usage summary:

    CPU time   :      5.99 sec.
    Max Memory :         3 MB
    Max Swap   :        37 MB

    Max Processes  :         1
    Max Threads    :         1

The output (if any) is above this job summary.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0005.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1849 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0002.p7s>

From scar at email.arizona.edu  Mon Apr 17 11:29:58 2017
From: scar at email.arizona.edu (Chandler)
Date: Mon, 17 Apr 2017 10:29:58 -0700
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
Message-ID: <2940db14-712e-d318-35c5-2a7b132db842@email.arizona.edu>

Please try with MPICH instead of openmpi... we were having similar 
errors with openmpi, but when we switched to mpich it all started 
working.  Make sure only the mpich executables are in your PATH, and 
that maker is built specifying the mpich package.

Chandler / Systems Administrator
Arizona Genomics Institute
www.genome.arizona.edu





From carsonhh at gmail.com  Tue Apr 18 09:10:43 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 18 Apr 2017 09:10:43 -0600
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
Message-ID: <2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>

I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.

#this must be doen exery time you run it
export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so

# -mca btl ^openib myust be on the command line
mpiexec -mca btl ^openib -n 20 maker

#alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker

?Carson

> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu> wrote:
> 
> Hi there,
> 
> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
> 
> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
> 
> "
> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
> 
> Peter Macfarlane / WIBR IT
> 
> 
> Description : Hi Peter,
> 
> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
> 
> All the best,
> -Tim
> 
> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
> 
> Hi Peter,
> 
> I?ve tested with the following command:
> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
> 
> Unfortunately it still seems to be giving errors and not running, with things like:
> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
> 
> It also doesn?t work if I try to run it directly on Tak4.
> 
> I?ve attached the relevant error log file.  Any more solutions?
> 
> All the best,
> -Tim
> 
> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
> 
> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
> 
> - /nfs/apps/test/maker is now maker 3.00
> - /nfs/apps/test/openmpi was used to make the aforementioned maker
> 
> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
> "
> All the best,
> -Tim
> 
> Timothy R. Fallon
> PhD candidate
> Laboratory of Jing-Ke Weng
> Department of Biology
> MIT
> 
> tfallon at mit.edu <mailto:tfallon at mit.edu>
> <bsub_maker_with_mpi_log.txt>
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/e7ebdbc2/attachment-0002.html>

From carsonhh at gmail.com  Tue Apr 18 09:23:04 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 18 Apr 2017 09:23:04 -0600
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
	<2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
	<CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>
Message-ID: <BDCC2D30-B9DB-4FF1-921A-307975EA8BB2@gmail.com>

MPICH3 works without any special configuration. OpenMPI is built to support infiniband which internally can drive other problems related to how registered memory is used when infiniband libraries are loaded. So you have have to set more parameters.

Both will use shared memory for communications when running on a single machine, but you will still need to set parameters on OpenMPI.

?Carson



> On Apr 18, 2017, at 9:19 AM, Tim Fallon <tfallon at mit.edu> wrote:
> 
> Hi Carson,
> 
> Thanks for the feedback, I?ll look into the maker INSTALL file to ensure things are setup correctly.
> 
> Should I be looking into MPICH3 instead?  I?m trying to run maker on a single node (~20 CPUs), rather than distribute it across our cluster.  I assume that using MPICH3 changes the mpiexec / mpirun executable, but not the commands otherwise?  I ask as I may have to configure this myself one day, so any tips appreciated.  In the meantime your comments seem straightforward, will give it a shot.
> 
> All the best,
> -Tim
> 
>> On Apr 18, 2017, at 11:10 AM, Carson Holt <carsonhh at gmail.com <mailto:carsonhh at gmail.com>> wrote:
>> 
>> I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.
>> 
>> #this must be doen exery time you run it
>> export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so
>> 
>> # -mca btl ^openib myust be on the command line
>> mpiexec -mca btl ^openib -n 20 maker
>> 
>> #alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
>> mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker
>> 
>> ?Carson
>> 
>>> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>>> 
>>> Hi there,
>>> 
>>> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
>>> 
>>> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
>>> 
>>> "
>>> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
>>> 
>>> Peter Macfarlane / WIBR IT
>>> 
>>> 
>>> Description : Hi Peter,
>>> 
>>> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
>>> 
>>> All the best,
>>> -Tim
>>> 
>>> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>>> 
>>> Hi Peter,
>>> 
>>> I?ve tested with the following command:
>>> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
>>> 
>>> Unfortunately it still seems to be giving errors and not running, with things like:
>>> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
>>> 
>>> It also doesn?t work if I try to run it directly on Tak4.
>>> 
>>> I?ve attached the relevant error log file.  Any more solutions?
>>> 
>>> All the best,
>>> -Tim
>>> 
>>> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
>>> 
>>> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
>>> 
>>> - /nfs/apps/test/maker is now maker 3.00
>>> - /nfs/apps/test/openmpi was used to make the aforementioned maker
>>> 
>>> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
>>> "
>>> All the best,
>>> -Tim
>>> 
>>> Timothy R. Fallon
>>> PhD candidate
>>> Laboratory of Jing-Ke Weng
>>> Department of Biology
>>> MIT
>>> 
>>> tfallon at mit.edu <mailto:tfallon at mit.edu>
>>> <bsub_maker_with_mpi_log.txt>
>>> _______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>
>> 
> 
> Timothy R. Fallon
> PhD candidate
> Laboratory of Jing-Ke Weng
> Department of Biology
> MIT
> 
> tfallon at mit.edu <mailto:tfallon at mit.edu>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/a23814d7/attachment-0002.html>

From tfallon at mit.edu  Tue Apr 18 09:19:19 2017
From: tfallon at mit.edu (Tim Fallon)
Date: Tue, 18 Apr 2017 11:19:19 -0400
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
	<2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
Message-ID: <CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>

Hi Carson,

Thanks for the feedback, I?ll look into the maker INSTALL file to ensure things are setup correctly.

Should I be looking into MPICH3 instead?  I?m trying to run maker on a single node (~20 CPUs), rather than distribute it across our cluster.  I assume that using MPICH3 changes the mpiexec / mpirun executable, but not the commands otherwise?  I ask as I may have to configure this myself one day, so any tips appreciated.  In the meantime your comments seem straightforward, will give it a shot.

All the best,
-Tim

> On Apr 18, 2017, at 11:10 AM, Carson Holt <carsonhh at gmail.com> wrote:
> 
> I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.
> 
> #this must be doen exery time you run it
> export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so
> 
> # -mca btl ^openib myust be on the command line
> mpiexec -mca btl ^openib -n 20 maker
> 
> #alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
> mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker
> 
> ?Carson
> 
>> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>> 
>> Hi there,
>> 
>> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
>> 
>> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
>> 
>> "
>> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
>> 
>> Peter Macfarlane / WIBR IT
>> 
>> 
>> Description : Hi Peter,
>> 
>> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
>> 
>> All the best,
>> -Tim
>> 
>> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>> 
>> Hi Peter,
>> 
>> I?ve tested with the following command:
>> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
>> 
>> Unfortunately it still seems to be giving errors and not running, with things like:
>> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
>> 
>> It also doesn?t work if I try to run it directly on Tak4.
>> 
>> I?ve attached the relevant error log file.  Any more solutions?
>> 
>> All the best,
>> -Tim
>> 
>> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
>> 
>> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
>> 
>> - /nfs/apps/test/maker is now maker 3.00
>> - /nfs/apps/test/openmpi was used to make the aforementioned maker
>> 
>> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
>> "
>> All the best,
>> -Tim
>> 
>> Timothy R. Fallon
>> PhD candidate
>> Laboratory of Jing-Ke Weng
>> Department of Biology
>> MIT
>> 
>> tfallon at mit.edu <mailto:tfallon at mit.edu>
>> <bsub_maker_with_mpi_log.txt>
>> _______________________________________________
>> maker-devel mailing list
>> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
> 

Timothy R. Fallon
PhD candidate
Laboratory of Jing-Ke Weng
Department of Biology
MIT

tfallon at mit.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/d3ae60e9/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1849 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/d3ae60e9/attachment-0002.p7s>

From miche.zacharias at gmail.com  Tue Apr 25 05:35:37 2017
From: miche.zacharias at gmail.com (Miche Zacharias)
Date: Tue, 25 Apr 2017 13:35:37 +0200
Subject: [maker-devel] altEST evidence
Message-ID: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>

Hi,

Would there be any benefit to passing CDS instead of cDNA sequences as
alternative EST evidence, if both is available? Or when would you use one
or the other?

I've read in previous discussions how it's expected that the organism, from
which we use sequences as alternative EST and protein evidence, is not
closely related to the genome we are annotating.
In what terms is "closely related" defined? It's now unclear to me when you
would put EST/protein evidence as alternative and when not or put it as
direct EST/protein evidence?
Would a species from the same family classify as alternative or direct
evidence? Is ~30mya divergence too much?
What about species of the same genus?

Many thanks!
Miche
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170425/2aafa4e5/attachment-0002.html>

From carsonhh at gmail.com  Tue Apr 25 09:30:21 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 25 Apr 2017 09:30:21 -0600
Subject: [maker-devel] altEST evidence
In-Reply-To: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>
References: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>
Message-ID: <EA7FC16A-2A92-49C1-96FF-774FD39E86DA@gmail.com>

If you have CDS, then it is not really transcription evidence, rather it was derived from protein annotations and the protein data should be used instead. The alt-EST option is only for datasets where no protein models exists, but you do have experimental evidence from transcription (i.e. not transcript predictions but actual mRNA experimentation).

Nucleotides sequence diverges more rapidly than amino acid sequence because of codon redundancy and amino acid similarity. As a result mRNA derived sequence can be compared in nucleotide space to genomic sequence of related organisms with an evolutionary difference of something like human to gorilla, but human to mouse would require amino acid level alignment to find the homology. So with alt-EST we use TBLASTX and Exonerate cdna2genome to perform a six frame translations of both the genome and assembly for alignment in amino acid space. This is of course very expensive computationally and more likely to generate spurious alignments. So for CDS sequence it would be both redundant to and less accurate than using the protein data directly.

?Carson



> On Apr 25, 2017, at 5:35 AM, Miche Zacharias <miche.zacharias at gmail.com> wrote:
> 
> Hi,
> 
> Would there be any benefit to passing CDS instead of cDNA sequences as alternative EST evidence, if both is available? Or when would you use one or the other?
> 
> I've read in previous discussions how it's expected that the organism, from which we use sequences as alternative EST and protein evidence, is not closely related to the genome we are annotating.
> In what terms is "closely related" defined? It's now unclear to me when you would put EST/protein evidence as alternative and when not or put it as direct EST/protein evidence?
> Would a species from the same family classify as alternative or direct evidence? Is ~30mya divergence too much?
> What about species of the same genus?
> 
> Many thanks!
> Miche
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From nathan.ricks at gmail.com  Fri Apr 28 09:47:55 2017
From: nathan.ricks at gmail.com (nathan.ricks at gmail.com)
Date: Fri, 28 Apr 2017 09:47:55 -0600
Subject: [maker-devel] Issues with Maker - RepBase
Message-ID: <5903642d.d6d3240a.759d3.6a01@mx.google.com>

Hi,
I?ve been having some issues getting Maker to run correctly. I don?t have a huge amount of Linux background, so perhaps that is why.
When I run it, it says 
WARNING: RepBase is not installed for RepeatMasker.
So I?ve tried getting RepBase installed, and I?m not sure what to do.
I downloaded the RepbaseRepeaMaskerEdition-20170127.tar.gz and I?ve tried unpacking , but it doesn?t give me a file I can work with, all it gives me is a folder named Library with very little in it. I?m assuming that I need to take that folder and put it somewhere in the RepeatMasker folder, but I?m not exactly sure what to do. 
Thank you for your time
Nathan Ricks


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170428/dcc2f064/attachment-0002.html>

From carsonhh at gmail.com  Fri Apr 28 09:52:46 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Fri, 28 Apr 2017 09:52:46 -0600
Subject: [maker-devel] Issues with Maker - RepBase
In-Reply-To: <5903642d.d6d3240a.759d3.6a01@mx.google.com>
References: <5903642d.d6d3240a.759d3.6a01@mx.google.com>
Message-ID: <5E08D02E-CB26-4D63-99D1-42CD0EB2A6C0@gmail.com>

You must unpack it inside the RepeatMasker distribution per the setup instructions ?> http://www.repeatmasker.org/RMDownload.html <http://www.repeatmasker.org/RMDownload.html>

It replaces files in the ?/RepeatMasker/Library folder when unpacked.

After unpacking RepBase, you must follow the configuration instructions (same link above). Make sure to configure RepeatMasker only after unpacking RepBase.

?Carson



> On Apr 28, 2017, at 9:47 AM, <nathan.ricks at gmail.com> <nathan.ricks at gmail.com> wrote:
> 
> Hi,
> I?ve been having some issues getting Maker to run correctly. I don?t have a huge amount of Linux background, so perhaps that is why.
> When I run it, it says
> WARNING: RepBase is not installed for RepeatMasker.
> So I?ve tried getting RepBase installed, and I?m not sure what to do.
> I downloaded the RepbaseRepeaMaskerEdition-20170127.tar.gz and I?ve tried unpacking , but it doesn?t give me a file I can work with, all it gives me is a folder named Library with very little in it. I?m assuming that I need to take that folder and put it somewhere in the RepeatMasker folder, but I?m not exactly sure what to do.
> Thank you for your time
> Nathan Ricks
>  
>  
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170428/77afe817/attachment-0002.html>

From mcsimenc at gmail.com  Sun Apr  2 08:19:40 2017
From: mcsimenc at gmail.com (Matt Simenc)
Date: Sun, 2 Apr 2017 07:19:40 -0700
Subject: [maker-devel] MAKER RepeatMasker outputs incompatible annotations
	for single feature
Message-ID: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>

TE Superfamily RLG are aka Gypsy, but RLG is associated with Copia
annotations in our RepeatMasker annotations. Has anyone else seen this
before?


Azfi_s0077  repeatmasker match 1400976 1401715 521 + .
ID=Azfi_s0077:hit:59234:1.3.0.14;Name=species:*RLG_scAle_3_1-*
I|genus:LTR%252FCopia;Target=species:*RLG_scAle_3_*1-I|genus:*LTR%252FCopia*1522
2556 +



Matt Simenc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170402/0a6106a2/attachment-0003.html>

From carsonhh at gmail.com  Sun Apr  2 11:38:05 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 2 Apr 2017 11:38:05 -0600
Subject: [maker-devel] MAKER RepeatMasker outputs incompatible
 annotations for single feature
In-Reply-To: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>
References: <CAH2F1pwVMc-q9tsWhmBsmuQfVUrw_KRJ_7D3QoG_D2q0qBWyzw@mail.gmail.com>
Message-ID: <E34482B0-55B7-4DC1-A01C-4EDBFE2E10A9@gmail.com>

This is the RepBase entry (from this file ?/RepeatMasker/Libraries/RepeatMasker.lib) ?>

>RLG_scAle_3_1-LTR#LTR/Copia @Saccharum_hybrid_cultivar  [S:] RepbaseID: RLG_scAle_3_1-LTR
tgttagaagattatgttaatatcttctgtataaggtaggggatctgtgtg
accacgagatccggtggattggcttgcaagtcaagttgtaatctatgtaa
atctaaccttcttgttgtacaagcccaaggggctatatattaacatgtac
cgagtacctgagaggtatctcgttccatctatttctaca

There are 64 RLG entries associated with Copia in the RepBase library for RepeatMasker. I?m not a repeat expert, so I can?t address whether this is an error or not. But if you think there is something amiss, you could contact RepBase.

?Carson


> On Apr 2, 2017, at 8:19 AM, Matt Simenc <mcsimenc at gmail.com> wrote:
> 
> TE Superfamily RLG are aka Gypsy, but RLG is associated with Copia annotations in our RepeatMasker annotations. Has anyone else seen this before?
> 
> 
> Azfi_s0077  repeatmasker match
> 1400976 1401715
> 521 +
> . ID=Azfi_s0077:hit:59234:1.3.0.14;Name=species:RLG_scAle_3_1-I|genus:LTR%252FCopia;Target=species:RLG_scAle_3_1-I|genus:LTR%252FCopia1522 2556 +
> 
> 
> 
> Matt Simenc
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170402/acdc5126/attachment-0003.html>

From carsonhh at gmail.com  Wed Apr  5 10:00:53 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Wed, 5 Apr 2017 10:00:53 -0600
Subject: [maker-devel] Some error messages in the output log of Maker
In-Reply-To: <CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
References: <CAJNrdRH1uDgtaB0C8jFi5yceZ7dcH6FRVzJV_RFgxHmtY_T12w@mail.gmail.com>
	<CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
Message-ID: <4F7ECEC6-338A-44B0-BC82-18193A494E29@gmail.com>

Augustus can sometimes produce alternate models. It normally only happens with hints and not with the raw prediction though. MAKER by default will only keeps the best scoring one (scored by AED). But EVM technically gets to see all of the raw models. Its the kind of thing where I would have to have the exact data and contig you used to run to see if this is something easily pulled apart for EVM.

?Carson


> On Apr 5, 2017, at 6:57 AM, Ray Cui <rcui at age.mpg.de> wrote:
> 
> Dear Carson,
> 
>         I think I found the reason for this here https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/ <https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/> . 
>         These messages are from EVM. The second type of message is due to incomplete gene models, so it's not a big deal. The first type is because of the overlapping gene models predicted by Augustus. This is a bit strange. Does maker allow Augustus to produce alternative spliced variants?
> 
> Best Regards,
> Ray
> 
> Dr. Rongfeng (Ray) Cui
> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
> Wissenschaftlicher MA / Postdoctoral researcher
> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
> Tel.:+49 (0)221 496 
> Mobile:           +49 0221 37970 496 <>
> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
> www.age.mpg.de <http://www.age.mpg.de/> 
> 
> 
> 
> On Wed, Apr 5, 2017 at 1:33 PM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
> Hi Carson,
> 
>       I saw these messages in the log, is this a concern? I'm using Maker beta 3.
>        Message 1:
> 
> Error with prediction: augustus_masked lend_intergenic: 730288, rend_intergenic: 664875  at /beegfs/group_dv/software/source/EVidenceModeler-1.1.1/evidence_modeler.pl <http://evidence_modeler.pl/> line 3218.
> 
>         Message 2:
> Sorry, prediction pred_gff:gth_m.PLPscf119:hit:7108:43932 fails validation.
> (GTT)    TA     683590-683765   GC      ATG
> (GCA)    AG     701228-701409   GT      AGG
> (GGA)    AG     704938-705037   GT      AGG
> (GGC)    AG     721672-721792   GT      AGG
> (TTG)    AG     730110-730288   AC      GAA
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, terminal, 730110, 730288
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 701228, 701409
> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 704938, 705037
> -recovered pred_gff:gth_m.PLPspolishing proteins
> 
> 
>      PS: I am still keep being kicked out of the maker mailing list due to some reason, if you can forward the msg. there it would be great.
>          
> Best Regards,
> Ray
> 
> Dr. Rongfeng (Ray) Cui
> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
> Wissenschaftlicher MA / Postdoctoral researcher
> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
> Mobile:           +49 0221 37970 496 <>
> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
> www.age.mpg.de <http://www.age.mpg.de/> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170405/b51fcd1b/attachment-0003.html>

From qlian003 at ucr.edu  Thu Apr  6 10:33:26 2017
From: qlian003 at ucr.edu (Qihua Liang)
Date: Fri, 7 Apr 2017 00:33:26 +0800
Subject: [maker-devel] Large dataset of RNA-Seq data
Message-ID: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>

Dear MAKER develop team,

I have some RNA-Seq data that I would like to use raw data instead of assembled transcripts in MAKER. But those raw datasets are over 800Gb. I noticed that MAKER will first read all input dataset into /tmp, and my attempts to include such large dataset failed because it exceeds the disk quota. Besides of increasing the disk quota, is there any other option to include such large datasets in MAKER annotations? Or can I redirect MAKER to read those data and use other path to temporarily store those data?

Thank you!
Best
Qihua


From carsonhh at gmail.com  Thu Apr  6 10:35:32 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 6 Apr 2017 10:35:32 -0600
Subject: [maker-devel] Large dataset of RNA-Seq data
In-Reply-To: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>
References: <C1581A15-6E02-4273-9001-E47BF8D68D16@ucr.edu>
Message-ID: <C88A1724-71D2-493F-B655-02E9A0FE3793@gmail.com>

MAKER cannot use raw RNA-Seq. It must be assembled beforehand, or processed into GFF3 using something like Stringtie.

?Carson


> On Apr 6, 2017, at 10:33 AM, Qihua Liang <qlian003 at ucr.edu> wrote:
> 
> Dear MAKER develop team,
> 
> I have some RNA-Seq data that I would like to use raw data instead of assembled transcripts in MAKER. But those raw datasets are over 800Gb. I noticed that MAKER will first read all input dataset into /tmp, and my attempts to include such large dataset failed because it exceeds the disk quota. Besides of increasing the disk quota, is there any other option to include such large datasets in MAKER annotations? Or can I redirect MAKER to read those data and use other path to temporarily store those data?
> 
> Thank you!
> Best
> Qihua
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From elyssa_garza at yahoo.com  Thu Apr  6 13:25:10 2017
From: elyssa_garza at yahoo.com (Elyssa Garza)
Date: Thu, 6 Apr 2017 19:25:10 +0000 (UTC)
Subject: [maker-devel]  merging gff files
References: <1186841770.2604454.1491506710651.ref@mail.yahoo.com>
Message-ID: <1186841770.2604454.1491506710651@mail.yahoo.com>

Hello, I currently have a MAKER-P annotated genome and would like to add additional information to the final gff file. I would like to include InDel and microsatellite genetic marker sequences/positions to the final annotation. Do you know of a reliable way to do this?

Elyssa



   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170406/09f2437b/attachment-0003.html>

From carsonhh at gmail.com  Thu Apr  6 16:01:52 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 6 Apr 2017 16:01:52 -0600
Subject: [maker-devel] maker-devel post from elyssa_garza@yahoo.com
 requires approval
In-Reply-To: <mailman.93.1491506732.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
References: <mailman.93.1491506732.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
Message-ID: <2AE2415B-5940-45F3-800B-84F9E9E28E7E@gmail.com>

If you already have GFF3, you can merge two files using the gff3_merge script that comes with MAKER.

?Carson

> From: Elyssa Garza <elyssa_garza at yahoo.com>
> Subject: [maker-devel] merging gff files
> Date: April 6, 2017 at 1:25:10 PM MDT
> To: "dence at genetics.utah.edu" <dence at genetics.utah.edu>
> Cc: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> Reply-To: Elyssa Garza <elyssa_garza at yahoo.com>
> 
> 
> Hello, I currently have a MAKER-P annotated genome and would like to add additional information to the final gff file. I would like to include InDel and microsatellite genetic marker sequences/positions to the final annotation. Do you know of a reliable way to do this?
> 
> 
> Elyssa

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170406/4f591c05/attachment-0003.html>

From carsonhh at gmail.com  Sun Apr  9 16:57:26 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 9 Apr 2017 16:57:26 -0600
Subject: [maker-devel] Some error messages in the output log of Maker
In-Reply-To: <CAJNrdRGfE1D-BrAQmQrRYL=1XLCPELKZNe9PZ4jneSuarAfd1g@mail.gmail.com>
References: <CAJNrdRH1uDgtaB0C8jFi5yceZ7dcH6FRVzJV_RFgxHmtY_T12w@mail.gmail.com>
	<CAJNrdRFPDgFLbngyxAKPE4T06iq4o587gCDpQX0qFouvtPgT4A@mail.gmail.com>
	<4F7ECEC6-338A-44B0-BC82-18193A494E29@gmail.com>
	<CAJNrdRGfE1D-BrAQmQrRYL=1XLCPELKZNe9PZ4jneSuarAfd1g@mail.gmail.com>
Message-ID: <0E1BAB87-339D-4D40-AA88-8C5C9CB895E9@gmail.com>

Most of the parameters affect the command line used to run blast/exonerate. In that case it will rerun. Some like the percent coverage parameter are post run filters. For those, it just re-parses. I don?t remember off the top of my head which ones fall into which category, but most fall into the first one.

?Carson

> On Apr 6, 2017, at 1:18 AM, Ray Cui <rcui at age.mpg.de> wrote:
> 
> Hi Carson,
> 
>     Yet another question: if I change the blast and exonerate parameters in the bopts file, will maker automatically rerun blast and exonerate (or, parse the existing output with the new cutoffs)?
> 
> Thanks
> Ray 
> 
> On Apr 5, 2017 6:01 PM, "Carson Holt" <carsonhh at gmail.com <mailto:carsonhh at gmail.com>> wrote:
> Augustus can sometimes produce alternate models. It normally only happens with hints and not with the raw prediction though. MAKER by default will only keeps the best scoring one (scored by AED). But EVM technically gets to see all of the raw models. Its the kind of thing where I would have to have the exact data and contig you used to run to see if this is something easily pulled apart for EVM.
> 
> ?Carson
> 
> 
>> On Apr 5, 2017, at 6:57 AM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
>> 
>> Dear Carson,
>> 
>>         I think I found the reason for this here https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/ <https://sourceforge.net/p/evidencemodeler/mailman/message/27284428/> . 
>>         These messages are from EVM. The second type of message is due to incomplete gene models, so it's not a big deal. The first type is because of the overlapping gene models predicted by Augustus. This is a bit strange. Does maker allow Augustus to produce alternative spliced variants?
>> 
>> Best Regards,
>> Ray
>> 
>> Dr. Rongfeng (Ray) Cui
>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
>> Wissenschaftlicher MA / Postdoctoral researcher
>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
>> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
>> Mobile:           +49 0221 37970 496 <>
>> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
>> www.age.mpg.de <http://www.age.mpg.de/> 
>> 
>> 
>> 
>> On Wed, Apr 5, 2017 at 1:33 PM, Ray Cui <rcui at age.mpg.de <mailto:rcui at age.mpg.de>> wrote:
>> Hi Carson,
>> 
>>       I saw these messages in the log, is this a concern? I'm using Maker beta 3.
>>        Message 1:
>> 
>> Error with prediction: augustus_masked lend_intergenic: 730288, rend_intergenic: 664875  at /beegfs/group_dv/software/source/EVidenceModeler-1.1.1/evidence_modeler.pl <http://evidence_modeler.pl/> line 3218.
>> 
>>         Message 2:
>> Sorry, prediction pred_gff:gth_m.PLPscf119:hit:7108:43932 fails validation.
>> (GTT)    TA     683590-683765   GC      ATG
>> (GCA)    AG     701228-701409   GT      AGG
>> (GGA)    AG     704938-705037   GT      AGG
>> (GGC)    AG     721672-721792   GT      AGG
>> (TTG)    AG     730110-730288   AC      GAA
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, terminal, 730110, 730288
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 701228, 701409
>> -recovered pred_gff:gth_m.PLPscf119:hit:7108:43932, internal, 704938, 705037
>> -recovered pred_gff:gth_m.PLPspolishing proteins
>> 
>> 
>>      PS: I am still keep being kicked out of the maker mailing list due to some reason, if you can forward the msg. there it would be great.
>>          
>> Best Regards,
>> Ray
>> 
>> Dr. Rongfeng (Ray) Cui
>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing
>> Wissenschaftlicher MA / Postdoctoral researcher
>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne
>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne
>> Tel.:+49 (0)221 496 <tel:+49%20221%20496> 
>> Mobile:           +49 0221 37970 496 <>
>> rcui at age.mpg.de <mailto:rcui at age.mpg.de>
>> www.age.mpg.de <http://www.age.mpg.de/> 
>> 
>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170409/846f99a9/attachment-0003.html>

From carsonhh at gmail.com  Sun Apr  9 17:06:52 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 9 Apr 2017 17:06:52 -0600
Subject: [maker-devel] maker putting _Inline directory wherever it is run
In-Reply-To: <mailman.101.1491517636.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
References: <mailman.101.1491517636.25547.maker-devel_yandell-lab.org@box290.bluehost.com>
Message-ID: <A690B5B0-8AE5-4898-A2A9-025AF9DF4166@gmail.com>

You have either moved directories post installation, are trying to run from the ?/maker/src/bin directory instead of ?/maker/bin or you had an issue during install. You just need to start over and follow the install instructions. The few libraries MAKER must compile with Inline::C will go into ?/maker/perl/lib/

?Carson

> From: "Sobel-Sorenson, Chandler - (scar)" <scar at email.arizona.edu>
> Subject: maker putting _Inline directory wherever it is run
> Date: April 6, 2017 at 4:27:03 PM MDT
> To: "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
> 
> 
> I just installed the latest maker 2.31.9 on our cluster, but whenever we issue the command "maker" (even with no arguments), it creates a "_Inline" directory in the CWD..  I guess this is related to Inline perl module?  For example:
> 
> [smrtanalysis at pac makertest]$ ls -a
> .  ..
> [smrtanalysis at pac makertest]$ maker
> ERROR: Control files not found
> 
> MAKER version 2.31.9
> 
> Usage:
> 
>     maker [options] <maker_opts> <maker_bopts> <maker_exe>
> ...
> 
> [smrtanalysis at pac makertest]$ ls -a
> .  ..  _Inline
> [smrtanalysis at pac makertest]$ ls -a _Inline/
> .  ..  build  config-x86_64-linux-thread-multi-5.010001  lib  .lock
> [smrtanalysis at pac makertest]$ 
> 
> 
> This is some problem with our module installation?  It was installed using "cpanm Inline::C".  Or problem/bug with the module itself should contact the Inline team about?  Or problem with our maker installation?
> 
> 
> Thanks,
> Chandler
> Arizona Genomics Institute
> University Of Arizona
> www.genome.arizona.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170409/2842e623/attachment-0003.html>

From wangzhennan at ioz.ac.cn  Mon Apr 10 21:08:14 2017
From: wangzhennan at ioz.ac.cn (wangzhennan at ioz.ac.cn)
Date: Tue, 11 Apr 2017 11:08:14 +0800 (GMT+08:00)
Subject: [maker-devel] How to run Maker to get the AED?
Message-ID: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>

Dear developer of Maker,

        I have used Maker to evaluate the genome annotation annotated by PASA. I have the transcript assembled by stringtie, genome (fasta), and the annotation file (gff3). How can I use the data to get the AED value from Maker? I have set the parameter of maker_opts.ctl. Can you help me to solve this? I only want to get the AED about the annotation file (gff3). Thank you very much!

        And there is also a question, I can not have a new post on https://groups.google.com/forum/#!forum/maker-devel. Why? Can you help me to solve this? Thank you!

 

         Best,

         Wangzhennan

        




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170411/9b4a2ec6/attachment-0003.html>

From scar at email.arizona.edu  Tue Apr 11 18:21:48 2017
From: scar at email.arizona.edu (Sobel-Sorenson, Chandler - (scar))
Date: Wed, 12 Apr 2017 00:21:48 +0000
Subject: [maker-devel] run maker using qsub
Message-ID: <1491956508254.51400@email.arizona.edu>

How could we run maker using qsub with our Sun Grid Engine?  We have it working with mpiexec but this does not show slots used on the grid, so can lead to over loading our cluster.
Currently we are using command like this with mpich: mpiexec -f /opt/machinelist -n 192 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl -fix_nucleotides

Thanks
Chandler
Arizona Genomics Institute



From carsonhh at gmail.com  Sun Apr 16 13:46:35 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 16 Apr 2017 13:46:35 -0600
Subject: [maker-devel] How to run Maker to get the AED?
In-Reply-To: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>
References: <107b45b.92d.15b5afbab48.Coremail.wangzhennan@ioz.ac.cn>
Message-ID: <5CD6DB76-9BB6-4DD6-9C35-A8CE0ABE1603@gmail.com>

To get just AED without maker producing it?s own predictons, you must provide your models to model_gff in valid GFF3 format (gene/mRNA/exon/CDS multi-level features). Then supply any evidence to either est=, protein=, est_gff=, or protein_gff=.

The google location is just an archive. You cannot post there. All messages must be sent to the mailing list here ?> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>

Thanks,
Carson


> On Apr 10, 2017, at 9:08 PM, wangzhennan at ioz.ac.cn wrote:
> 
> Dear developer of Maker,
> 
>         I have used Maker to evaluate the genome annotation annotated by PASA. I have the transcript assembled by stringtie, genome (fasta), and the annotation file (gff3). How can I use the data to get the AED value from Maker? I have set the parameter of maker_opts.ctl. Can you help me to solve this? I only want to get the AED about the annotation file (gff3). Thank you very much!
> 
>         And there is also a question, I can not have a new post on https://groups.google.com/forum/#!forum/maker-devel <https://groups.google.com/forum/#!forum/maker-devel>. Why? Can you help me to solve this? Thank you!
> 
>  
>          Best,
> 
>          Wangzhennan
> 
>         
> 
> 
> 
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170416/a03cd31f/attachment-0003.html>

From carsonhh at gmail.com  Sun Apr 16 13:55:23 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Sun, 16 Apr 2017 13:55:23 -0600
Subject: [maker-devel] run maker using qsub
In-Reply-To: <1491956508254.51400@email.arizona.edu>
References: <1491956508254.51400@email.arizona.edu>
Message-ID: <6A3DAA63-06FA-4A44-9B89-EAD8AF8ED0BB@gmail.com>

You will have to look up documentation for Sun Grid Engine and the MPI flavor you wish to use. This is not a MAKER configuration issue, rather you need to setup Sun Grid for MPI according to it?s documentation. When Sun Grid is configured for MPI, it should automatically sets environmental variables and files for jobs. When set up correctly you should not even have to use the -n or -f option with mpiexec as it will be detected from values Sun Grid configures for you.

Then a command like the following will work in your batch job
?> mpiexec maker

?Carson



> On Apr 11, 2017, at 6:21 PM, Sobel-Sorenson, Chandler - (scar) <scar at email.arizona.edu> wrote:
> 
> How could we run maker using qsub with our Sun Grid Engine?  We have it working with mpiexec but this does not show slots used on the grid, so can lead to over loading our cluster.
> Currently we are using command like this with mpich: mpiexec -f /opt/machinelist -n 192 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl -fix_nucleotides
> 
> Thanks
> Chandler
> Arizona Genomics Institute
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From tfallon at mit.edu  Mon Apr 17 09:35:31 2017
From: tfallon at mit.edu (Tim Fallon)
Date: Mon, 17 Apr 2017 11:35:31 -0400
Subject: [maker-devel] Help with MPI installation and configuration
Message-ID: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>

Hi there,

I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.

See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?

"
I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.

Peter Macfarlane / WIBR IT


Description : Hi Peter,

Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?

All the best,
-Tim

On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:

Hi Peter,

I?ve tested with the following command:
bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?

Unfortunately it still seems to be giving errors and not running, with things like:
"?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??

It also doesn?t work if I try to run it directly on Tak4.

I?ve attached the relevant error log file.  Any more solutions?

All the best,
-Tim

On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:

I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:

- /nfs/apps/test/maker is now maker 3.00
- /nfs/apps/test/openmpi was used to make the aforementioned maker

I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
"
All the best,
-Tim

Timothy R. Fallon
PhD candidate
Laboratory of Jing-Ke Weng
Department of Biology
MIT

tfallon at mit.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0006.html>
-------------- next part --------------
[it-c05b10:00466] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00467] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00465] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00482] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00468] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00473] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00476] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00481] mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)
[it-c05b10:00466] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00467] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00468] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00466] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00476] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00467] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00468] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00473] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00482] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00467] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00476] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00466] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00473] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00468] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00465] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00481] mca_base_component_repository_open: unable to open mca_shmem_mmap: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_mmap.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00482] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00476] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00473] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00482] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00481] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00465] mca_base_component_repository_open: unable to open mca_shmem_posix: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_posix.so: undefined symbol: opal_shmem_base_framework (ignored)
[it-c05b10:00481] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
[it-c05b10:00465] mca_base_component_repository_open: unable to open mca_shmem_sysv: /nfs/apps/test/openmpi/lib/openmpi/mca_shmem_sysv.so: undefined symbol: opal_show_help (ignored)
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like orte_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_init failed
  --> Returned value Error (-1) instead of ORTE_SUCCESS
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:466] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:467] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:473] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:476] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
***    and potentially your MPI job)
[it-c05b10:482] Local abort before MPI_INIT completed completed successfully, but am not able to aggregate error messages, and not able to guarantee that all other processes were killed!
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems.  This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):

  ompi_mpi_init: ompi_rte_init failed
  --> Returned "Error" (-1) instead of "Success" (0)
--------------------------------------------------------------------------
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[30092,1],1]
  Exit code:    1
--------------------------------------------------------------------------

------------------------------------------------------------
Sender: LSF System <lsfadmin at it-c05b10>
Subject: Job 1703425: </nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker> Exited

Job </nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker> was submitted from host <tak4> by user <tfallon> in cluster <irken>.
Job was executed on host(s) <it-c05b10>, in queue <normal>, as user <tfallon> in cluster <irken>.
</home/tfallon> was used as the home directory.
</lab/solexa_weng/Seq_data/Projects/Tim_Fallon/ppyralis_genome/Genome_project_reference_assemblies/version1/analyses/maker_mpi_troubleshooting_withmpi> was used as the working directory.
Started at Fri Apr  7 09:04:14 2017
Results reported at Fri Apr  7 09:04:18 2017

Your job looked like:

------------------------------------------------------------
# LSBATCH: User input
/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker
------------------------------------------------------------

Exited with exit code 1.

Resource usage summary:

    CPU time   :      5.99 sec.
    Max Memory :         3 MB
    Max Swap   :        37 MB

    Max Processes  :         1
    Max Threads    :         1

The output (if any) is above this job summary.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0007.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1849 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170417/e6eb4917/attachment-0003.p7s>

From scar at email.arizona.edu  Mon Apr 17 11:29:58 2017
From: scar at email.arizona.edu (Chandler)
Date: Mon, 17 Apr 2017 10:29:58 -0700
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
Message-ID: <2940db14-712e-d318-35c5-2a7b132db842@email.arizona.edu>

Please try with MPICH instead of openmpi... we were having similar 
errors with openmpi, but when we switched to mpich it all started 
working.  Make sure only the mpich executables are in your PATH, and 
that maker is built specifying the mpich package.

Chandler / Systems Administrator
Arizona Genomics Institute
www.genome.arizona.edu





From carsonhh at gmail.com  Tue Apr 18 09:10:43 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 18 Apr 2017 09:10:43 -0600
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
Message-ID: <2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>

I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.

#this must be doen exery time you run it
export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so

# -mca btl ^openib myust be on the command line
mpiexec -mca btl ^openib -n 20 maker

#alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker

?Carson

> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu> wrote:
> 
> Hi there,
> 
> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
> 
> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
> 
> "
> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
> 
> Peter Macfarlane / WIBR IT
> 
> 
> Description : Hi Peter,
> 
> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
> 
> All the best,
> -Tim
> 
> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
> 
> Hi Peter,
> 
> I?ve tested with the following command:
> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
> 
> Unfortunately it still seems to be giving errors and not running, with things like:
> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
> 
> It also doesn?t work if I try to run it directly on Tak4.
> 
> I?ve attached the relevant error log file.  Any more solutions?
> 
> All the best,
> -Tim
> 
> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
> 
> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
> 
> - /nfs/apps/test/maker is now maker 3.00
> - /nfs/apps/test/openmpi was used to make the aforementioned maker
> 
> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
> "
> All the best,
> -Tim
> 
> Timothy R. Fallon
> PhD candidate
> Laboratory of Jing-Ke Weng
> Department of Biology
> MIT
> 
> tfallon at mit.edu <mailto:tfallon at mit.edu>
> <bsub_maker_with_mpi_log.txt>
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/e7ebdbc2/attachment-0003.html>

From carsonhh at gmail.com  Tue Apr 18 09:23:04 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 18 Apr 2017 09:23:04 -0600
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
	<2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
	<CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>
Message-ID: <BDCC2D30-B9DB-4FF1-921A-307975EA8BB2@gmail.com>

MPICH3 works without any special configuration. OpenMPI is built to support infiniband which internally can drive other problems related to how registered memory is used when infiniband libraries are loaded. So you have have to set more parameters.

Both will use shared memory for communications when running on a single machine, but you will still need to set parameters on OpenMPI.

?Carson



> On Apr 18, 2017, at 9:19 AM, Tim Fallon <tfallon at mit.edu> wrote:
> 
> Hi Carson,
> 
> Thanks for the feedback, I?ll look into the maker INSTALL file to ensure things are setup correctly.
> 
> Should I be looking into MPICH3 instead?  I?m trying to run maker on a single node (~20 CPUs), rather than distribute it across our cluster.  I assume that using MPICH3 changes the mpiexec / mpirun executable, but not the commands otherwise?  I ask as I may have to configure this myself one day, so any tips appreciated.  In the meantime your comments seem straightforward, will give it a shot.
> 
> All the best,
> -Tim
> 
>> On Apr 18, 2017, at 11:10 AM, Carson Holt <carsonhh at gmail.com <mailto:carsonhh at gmail.com>> wrote:
>> 
>> I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.
>> 
>> #this must be doen exery time you run it
>> export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so
>> 
>> # -mca btl ^openib myust be on the command line
>> mpiexec -mca btl ^openib -n 20 maker
>> 
>> #alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
>> mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker
>> 
>> ?Carson
>> 
>>> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>>> 
>>> Hi there,
>>> 
>>> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
>>> 
>>> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
>>> 
>>> "
>>> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
>>> 
>>> Peter Macfarlane / WIBR IT
>>> 
>>> 
>>> Description : Hi Peter,
>>> 
>>> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
>>> 
>>> All the best,
>>> -Tim
>>> 
>>> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>>> 
>>> Hi Peter,
>>> 
>>> I?ve tested with the following command:
>>> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
>>> 
>>> Unfortunately it still seems to be giving errors and not running, with things like:
>>> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
>>> 
>>> It also doesn?t work if I try to run it directly on Tak4.
>>> 
>>> I?ve attached the relevant error log file.  Any more solutions?
>>> 
>>> All the best,
>>> -Tim
>>> 
>>> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
>>> 
>>> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
>>> 
>>> - /nfs/apps/test/maker is now maker 3.00
>>> - /nfs/apps/test/openmpi was used to make the aforementioned maker
>>> 
>>> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
>>> "
>>> All the best,
>>> -Tim
>>> 
>>> Timothy R. Fallon
>>> PhD candidate
>>> Laboratory of Jing-Ke Weng
>>> Department of Biology
>>> MIT
>>> 
>>> tfallon at mit.edu <mailto:tfallon at mit.edu>
>>> <bsub_maker_with_mpi_log.txt>
>>> _______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>
>> 
> 
> Timothy R. Fallon
> PhD candidate
> Laboratory of Jing-Ke Weng
> Department of Biology
> MIT
> 
> tfallon at mit.edu <mailto:tfallon at mit.edu>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/a23814d7/attachment-0003.html>

From tfallon at mit.edu  Tue Apr 18 09:19:19 2017
From: tfallon at mit.edu (Tim Fallon)
Date: Tue, 18 Apr 2017 11:19:19 -0400
Subject: [maker-devel] Help with MPI installation and configuration
In-Reply-To: <2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
References: <B94C170E-926C-4150-936E-66BE8F979A17@mit.edu>
	<2D04DEB0-11AE-411A-8B8D-57877E1C55E7@gmail.com>
Message-ID: <CAD49A34-A6C5-43C3-9D61-29A2FB36EA59@mit.edu>

Hi Carson,

Thanks for the feedback, I?ll look into the maker INSTALL file to ensure things are setup correctly.

Should I be looking into MPICH3 instead?  I?m trying to run maker on a single node (~20 CPUs), rather than distribute it across our cluster.  I assume that using MPICH3 changes the mpiexec / mpirun executable, but not the commands otherwise?  I ask as I may have to configure this myself one day, so any tips appreciated.  In the meantime your comments seem straightforward, will give it a shot.

All the best,
-Tim

> On Apr 18, 2017, at 11:10 AM, Carson Holt <carsonhh at gmail.com> wrote:
> 
> I agree that just using MPICH3 is the easiest solution. But to use OpenMPI you will need to export the environmental variables add the command line flags to mpiexec explained in the ../maker/INSTALL file.
> 
> #this must be doen exery time you run it
> export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so
> 
> # -mca btl ^openib myust be on the command line
> mpiexec -mca btl ^openib -n 20 maker
> 
> #alternatively add --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 (use ib0 or whatever adapter interface exists on your cluster)
> mpiexec --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 -n 20 maker
> 
> ?Carson
> 
>> On Apr 17, 2017, at 9:35 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>> 
>> Hi there,
>> 
>> I?ve been troubleshooting the maker 3.0.0 beta plus MPI installation and configuration with our institute?s IT department.  Despite a lot of tries and recompilations from scratch, we haven?t been able to get it to work (non-MPI maker works fine, albeit a bit slow).  I?ve attached our most recent crash log when trying to run maker with MPI.  Any tips?  I?ve never used MPI before maker so not that familiar with troubleshooting it.
>> 
>> See below for my most recent contact with IT on it.  I?ve also attached the most recent log file.  We are trying to execute it through the LSF cluster submission tool, but I don?t think that is related to the problem.  Any ideas?
>> 
>> "
>> I think one option is to try a build on a host without the full suite of cluster software.  It may actually help, but if nothing else we can say that our problem exists with a (relatively) vanilla ubuntu linux host.
>> 
>> Peter Macfarlane / WIBR IT
>> 
>> 
>> Description : Hi Peter,
>> 
>> Hoping to hear back from you on this.  It does seem like we?ve exhausted the reasonable troubleshooting for this maker MPI execution, but maybe worth another shot before I ask the developers?
>> 
>> All the best,
>> -Tim
>> 
>> On Apr 7, 2017, at 9:11 AM, Tim Fallon <tfallon at mit.edu <mailto:tfallon at mit.edu>> wrote:
>> 
>> Hi Peter,
>> 
>> I?ve tested with the following command:
>> bsub -o bsub_maker_with_mpi_log.txt  "/nfs/apps/test/openmpi/bin/mpirun -n 8 /nfs/apps/test/maker/bin/maker?
>> 
>> Unfortunately it still seems to be giving errors and not running, with things like:
>> "?mca_base_component_repository_open: unable to open mca_patcher_overwrite: /nfs/apps/test/openmpi/lib/openmpi/mca_patcher_overwrite.so: undefined symbol: mca_patcher_base_patch_t_class (ignored)??
>> 
>> It also doesn?t work if I try to run it directly on Tak4.
>> 
>> I?ve attached the relevant error log file.  Any more solutions?
>> 
>> All the best,
>> -Tim
>> 
>> On Apr 5, 2017, at 3:50 PM, IT Systems Group <unix-help at wi.mit.edu <mailto:unix-help at wi.mit.edu>> wrote:
>> 
>> I tried rebuilding maker 3 with it's own also-built-from-source openmpi instance.  So with that:
>> 
>> - /nfs/apps/test/maker is now maker 3.00
>> - /nfs/apps/test/openmpi was used to make the aforementioned maker
>> 
>> I think that all you'll need to do differently here is invoke the version of mpirun in /nfs/apps/test/openmpi/bin
>> "
>> All the best,
>> -Tim
>> 
>> Timothy R. Fallon
>> PhD candidate
>> Laboratory of Jing-Ke Weng
>> Department of Biology
>> MIT
>> 
>> tfallon at mit.edu <mailto:tfallon at mit.edu>
>> <bsub_maker_with_mpi_log.txt>
>> _______________________________________________
>> maker-devel mailing list
>> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
> 

Timothy R. Fallon
PhD candidate
Laboratory of Jing-Ke Weng
Department of Biology
MIT

tfallon at mit.edu

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/d3ae60e9/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1849 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170418/d3ae60e9/attachment-0003.p7s>

From miche.zacharias at gmail.com  Tue Apr 25 05:35:37 2017
From: miche.zacharias at gmail.com (Miche Zacharias)
Date: Tue, 25 Apr 2017 13:35:37 +0200
Subject: [maker-devel] altEST evidence
Message-ID: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>

Hi,

Would there be any benefit to passing CDS instead of cDNA sequences as
alternative EST evidence, if both is available? Or when would you use one
or the other?

I've read in previous discussions how it's expected that the organism, from
which we use sequences as alternative EST and protein evidence, is not
closely related to the genome we are annotating.
In what terms is "closely related" defined? It's now unclear to me when you
would put EST/protein evidence as alternative and when not or put it as
direct EST/protein evidence?
Would a species from the same family classify as alternative or direct
evidence? Is ~30mya divergence too much?
What about species of the same genus?

Many thanks!
Miche
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170425/2aafa4e5/attachment-0003.html>

From carsonhh at gmail.com  Tue Apr 25 09:30:21 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 25 Apr 2017 09:30:21 -0600
Subject: [maker-devel] altEST evidence
In-Reply-To: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>
References: <CABhRG6K4sbZS0+wtytkvfvaSAQjNSji_pPeCQUKo6Xe+tGRHEA@mail.gmail.com>
Message-ID: <EA7FC16A-2A92-49C1-96FF-774FD39E86DA@gmail.com>

If you have CDS, then it is not really transcription evidence, rather it was derived from protein annotations and the protein data should be used instead. The alt-EST option is only for datasets where no protein models exists, but you do have experimental evidence from transcription (i.e. not transcript predictions but actual mRNA experimentation).

Nucleotides sequence diverges more rapidly than amino acid sequence because of codon redundancy and amino acid similarity. As a result mRNA derived sequence can be compared in nucleotide space to genomic sequence of related organisms with an evolutionary difference of something like human to gorilla, but human to mouse would require amino acid level alignment to find the homology. So with alt-EST we use TBLASTX and Exonerate cdna2genome to perform a six frame translations of both the genome and assembly for alignment in amino acid space. This is of course very expensive computationally and more likely to generate spurious alignments. So for CDS sequence it would be both redundant to and less accurate than using the protein data directly.

?Carson



> On Apr 25, 2017, at 5:35 AM, Miche Zacharias <miche.zacharias at gmail.com> wrote:
> 
> Hi,
> 
> Would there be any benefit to passing CDS instead of cDNA sequences as alternative EST evidence, if both is available? Or when would you use one or the other?
> 
> I've read in previous discussions how it's expected that the organism, from which we use sequences as alternative EST and protein evidence, is not closely related to the genome we are annotating.
> In what terms is "closely related" defined? It's now unclear to me when you would put EST/protein evidence as alternative and when not or put it as direct EST/protein evidence?
> Would a species from the same family classify as alternative or direct evidence? Is ~30mya divergence too much?
> What about species of the same genus?
> 
> Many thanks!
> Miche
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From nathan.ricks at gmail.com  Fri Apr 28 09:47:55 2017
From: nathan.ricks at gmail.com (nathan.ricks at gmail.com)
Date: Fri, 28 Apr 2017 09:47:55 -0600
Subject: [maker-devel] Issues with Maker - RepBase
Message-ID: <5903642d.d6d3240a.759d3.6a01@mx.google.com>

Hi,
I?ve been having some issues getting Maker to run correctly. I don?t have a huge amount of Linux background, so perhaps that is why.
When I run it, it says 
WARNING: RepBase is not installed for RepeatMasker.
So I?ve tried getting RepBase installed, and I?m not sure what to do.
I downloaded the RepbaseRepeaMaskerEdition-20170127.tar.gz and I?ve tried unpacking , but it doesn?t give me a file I can work with, all it gives me is a folder named Library with very little in it. I?m assuming that I need to take that folder and put it somewhere in the RepeatMasker folder, but I?m not exactly sure what to do. 
Thank you for your time
Nathan Ricks


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170428/dcc2f064/attachment-0003.html>

From carsonhh at gmail.com  Fri Apr 28 09:52:46 2017
From: carsonhh at gmail.com (Carson Holt)
Date: Fri, 28 Apr 2017 09:52:46 -0600
Subject: [maker-devel] Issues with Maker - RepBase
In-Reply-To: <5903642d.d6d3240a.759d3.6a01@mx.google.com>
References: <5903642d.d6d3240a.759d3.6a01@mx.google.com>
Message-ID: <5E08D02E-CB26-4D63-99D1-42CD0EB2A6C0@gmail.com>

You must unpack it inside the RepeatMasker distribution per the setup instructions ?> http://www.repeatmasker.org/RMDownload.html <http://www.repeatmasker.org/RMDownload.html>

It replaces files in the ?/RepeatMasker/Library folder when unpacked.

After unpacking RepBase, you must follow the configuration instructions (same link above). Make sure to configure RepeatMasker only after unpacking RepBase.

?Carson



> On Apr 28, 2017, at 9:47 AM, <nathan.ricks at gmail.com> <nathan.ricks at gmail.com> wrote:
> 
> Hi,
> I?ve been having some issues getting Maker to run correctly. I don?t have a huge amount of Linux background, so perhaps that is why.
> When I run it, it says
> WARNING: RepBase is not installed for RepeatMasker.
> So I?ve tried getting RepBase installed, and I?m not sure what to do.
> I downloaded the RepbaseRepeaMaskerEdition-20170127.tar.gz and I?ve tried unpacking , but it doesn?t give me a file I can work with, all it gives me is a folder named Library with very little in it. I?m assuming that I need to take that folder and put it somewhere in the RepeatMasker folder, but I?m not exactly sure what to do.
> Thank you for your time
> Nathan Ricks
>  
>  
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com <mailto:maker-devel at box290.bluehost.com>
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org <http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20170428/77afe817/attachment-0003.html>