From carsonhh at gmail.com Tue Jul 3 09:12:57 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 03 Jul 2012 10:12:57 -0400 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: <4FEC6428.3010108@stud-mail.uni-wuerzburg.de> Message-ID: Have you tried 2.25 to see if the error still occurs in that version? Thanks, Carson On 12-06-28 10:03 AM, "Markus Ankenbrand" wrote: >Hi all, > >I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >only using coding sequences of a reference species as "altest". Without >any obvious reason annotation fails for some contigs. It seems to have >nothing to do with the contig length. I also tried to run maker with >different reference ESTs and GeneMark. Depending on the setup different >contigs fail. > >In case of failure, the log is: >... >cleaning clusters.... >total clusters:3 now processing 0 > ...processing 0 of 3 > ...processing 1 of 3 >total clusters:3 now processing 0 >total clusters:3 now processing 0 > ...processing 0 of 2 >Can't call method "start" on an undefined valueERROR: Failed while >preparing evidence clusters for annotations >ERROR: Chunk failed at level:0, tier_type:3 >FAILED CONTIG:contig1 > >ERROR: Chunk failed at level:7, tier_type:0 >FAILED CONTIG:contig1 > >--Next Contig-- >... > >If someone has any idea how to solve this problem, help would be very >much apriciated. > >Greetings, >Markus > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Jul 3 12:27:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 03 Jul 2012 13:27:56 -0400 Subject: [maker-devel] maker leaving large numbers of defunct zombies In-Reply-To: <68CF9F49-540C-4018-A849-1D1B5751943B@slu.se> Message-ID: MAKER doesn't lock the SQL database except on initial creation and population. However, SQLite does it's own advisory locking apart from MAKER (and it may be trying to do this with every transaction). This could be problematic on some NFS systems where advisory locking may be Buggy. I'm going to try something. Just update the SVN repository one more time. What I've done is to add a check to see if you are running in an NFS mounted directory. If so, maker will try and localize the SQLite db file to /tmp or wherever your TMP variable is set to in the maker_opts.ctl file (maker will check the NFS status on that as well before attempting the copy so as not to waste time in just copying to another NFS). So if you set TMP or your linux TMPDIR variable is set to an NFS mounted locations, try changing it to a location that will be local to each machine. Usually /tmp (Linux default) or you can use the memory mounted virtual directory /dev/shm. Maker will clean up temporary files it creates afterwards. Automatically. By using a local rather than NFS location in this step, advisory locks should work properly for SQLite (if that is the issue you are experiencing). I've added checks and optimizations so the file will not be copied if it has already been localized to a given machine, even if it was by another node in the MPI job. So the overall space overhead should be limited, and you'll probably even get a slight performance boost as well (local SQLite databases run much faster than NFS located ones). Thanks, Carson On 12-06-28 4:16 PM, "Mikael Brandstr?m Durling" wrote: >Sorry, but I might have been slightly to quick to my conclusions. All the >initialisation and initial database creation works fine. However, when >running I start to see locking error from SQLite. Is GFFDB supposed to >use the NFSLock to lock the database at all accesses? After running for a >while now, I see locking errors (ie the query or transaction failes) from >SQLite at these lines in GFFDB.pm: 107/109, 529 and 678. > >Mikael From markus.ankenbrand at stud-mail.uni-wuerzburg.de Thu Jul 5 06:22:15 2012 From: markus.ankenbrand at stud-mail.uni-wuerzburg.de (Markus Ankenbrand) Date: Thu, 05 Jul 2012 13:22:15 +0200 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: References: Message-ID: <4FF578E7.7060700@stud-mail.uni-wuerzburg.de> Hi Carson, thanks for your response. Version 2.25 works without any errors. May I have a changelog for 2.25, if available? Thanks, Markus Am 03.07.2012 16:12, schrieb Carson Holt: > Have you tried 2.25 to see if the error still occurs in that version? > > Thanks, > Carson > > > > On 12-06-28 10:03 AM, "Markus Ankenbrand" > wrote: > >> Hi all, >> >> I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >> only using coding sequences of a reference species as "altest". Without >> any obvious reason annotation fails for some contigs. It seems to have >> nothing to do with the contig length. I also tried to run maker with >> different reference ESTs and GeneMark. Depending on the setup different >> contigs fail. >> >> In case of failure, the log is: >> ... >> cleaning clusters.... >> total clusters:3 now processing 0 >> ...processing 0 of 3 >> ...processing 1 of 3 >> total clusters:3 now processing 0 >> total clusters:3 now processing 0 >> ...processing 0 of 2 >> Can't call method "start" on an undefined valueERROR: Failed while >> preparing evidence clusters for annotations >> ERROR: Chunk failed at level:0, tier_type:3 >> FAILED CONTIG:contig1 >> >> ERROR: Chunk failed at level:7, tier_type:0 >> FAILED CONTIG:contig1 >> >> --Next Contig-- >> ... >> >> If someone has any idea how to solve this problem, help would be very >> much apriciated. >> >> Greetings, >> Markus >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Thu Jul 5 09:00:54 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 10:00:54 -0400 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: <4FF578E7.7060700@stud-mail.uni-wuerzburg.de> Message-ID: Here are comments from subversion between 2.24 and 2.25 Renaming and minor fixes to some modules improved gff3_merge script fix est passthrough bug made GFFDB module faster Thanks, Carson On 12-07-05 7:22 AM, "Markus Ankenbrand" wrote: >Hi Carson, > >thanks for your response. >Version 2.25 works without any errors. >May I have a changelog for 2.25, if available? > >Thanks, >Markus > > >Am 03.07.2012 16:12, schrieb Carson Holt: >> Have you tried 2.25 to see if the error still occurs in that version? >> >> Thanks, >> Carson >> >> >> >> On 12-06-28 10:03 AM, "Markus Ankenbrand" >> wrote: >> >>> Hi all, >>> >>> I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >>> only using coding sequences of a reference species as "altest". Without >>> any obvious reason annotation fails for some contigs. It seems to have >>> nothing to do with the contig length. I also tried to run maker with >>> different reference ESTs and GeneMark. Depending on the setup different >>> contigs fail. >>> >>> In case of failure, the log is: >>> ... >>> cleaning clusters.... >>> total clusters:3 now processing 0 >>> ...processing 0 of 3 >>> ...processing 1 of 3 >>> total clusters:3 now processing 0 >>> total clusters:3 now processing 0 >>> ...processing 0 of 2 >>> Can't call method "start" on an undefined valueERROR: Failed while >>> preparing evidence clusters for annotations >>> ERROR: Chunk failed at level:0, tier_type:3 >>> FAILED CONTIG:contig1 >>> >>> ERROR: Chunk failed at level:7, tier_type:0 >>> FAILED CONTIG:contig1 >>> >>> --Next Contig-- >>> ... >>> >>> If someone has any idea how to solve this problem, help would be very >>> much apriciated. >>> >>> Greetings, >>> Markus >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > From chrisi.hahni at gmail.com Thu Jul 5 10:06:38 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Thu, 05 Jul 2012 17:06:38 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap Message-ID: <4FF5AD7E.7010805@gmail.com> Dear MAKER developers and users, I am new to MAKER and during my first attempts I unfortunately encountert some problems. It would be great if anyone could help me out! Here's my problem: I have used gm_es.pl from Genemark to create es.mod for my assembly. I would like to give this model to MAKER, so I add the path to es.mod in the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I am running maker it cannot find the gmhmm_wrap script. Here s what it says: #-------------------------------# Can't open perl script "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wrap": No such file or directory ERROR: Genemark failed Apparently it is looking for the script in the wrong place. I have found it in: /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap The other paths are correct. Is there a way to manually tell Maker where to look for this script? I should mention also that I have maker (2.10) running on a cluster. Any help is highly appreciated! Thanks in advance!! much obliged, Christoph University of Oslo From carsonhh at gmail.com Thu Jul 5 11:18:45 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 12:18:45 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF5AD7E.7010805@gmail.com> Message-ID: Your cluster is apparently using the CPAN module local::lib to define where perl libraries and executables get installed, this is overriding MAKER's own installation parameters, thus putting things in the wrong place. This only happens in 2.10, the 2.25 version of MAKER knows how to keep local::lib from messing with it's relative installation paths. So install 2.25 and make sure to remove the mis-located MAKER executables in /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. Thanks, Carson On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >Dear MAKER developers and users, > >I am new to MAKER and during my first attempts I unfortunately >encountert some problems. It would be great if anyone could help me out! >Here's my problem: >I have used gm_es.pl from Genemark to create es.mod for my assembly. I >would like to give this model to MAKER, so I add the path to es.mod in >the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >am running maker it cannot find the gmhmm_wrap script. Here s what it >says: > >#-------------------------------# >Can't open perl script >"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wr >ap": >No such file or directory >ERROR: Genemark failed > >Apparently it is looking for the script in the wrong place. I have found >it in: >/site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >The other paths are correct. > >Is there a way to manually tell Maker where to look for this script? I >should mention also that I have maker (2.10) running on a cluster. > >Any help is highly appreciated! Thanks in advance!! > >much obliged, >Christoph > >University of Oslo > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From chrisi.hahni at gmail.com Thu Jul 5 11:55:19 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Thu, 05 Jul 2012 18:55:19 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: References: Message-ID: <4FF5C6F7.3090601@gmail.com> Dear Carson, Thanks for your reply! I have already requested the installation of maker 2.25 from the cluster admin, but unfortunately this will take a while. I guess there is no way for me to fix that for the moment in maker 2.10?? Thanks again! cheers, Christoph On 07/05/2012 06:18 PM, Carson Holt wrote: > Your cluster is apparently using the CPAN module local::lib to define > where perl libraries and executables get installed, this is overriding > MAKER's own installation parameters, thus putting things in the wrong > place. This only happens in 2.10, the 2.25 version of MAKER knows how to > keep local::lib from messing with it's relative installation paths. So > install 2.25 and make sure to remove the mis-located MAKER executables in > /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. > > Thanks, > Carson > > > > On 12-07-05 11:06 AM, "Christoph Hahn" wrote: > >> Dear MAKER developers and users, >> >> I am new to MAKER and during my first attempts I unfortunately >> encountert some problems. It would be great if anyone could help me out! >> Here's my problem: >> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >> would like to give this model to MAKER, so I add the path to es.mod in >> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >> am running maker it cannot find the gmhmm_wrap script. Here s what it >> says: >> >> #-------------------------------# >> Can't open perl script >> "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wr >> ap": >> No such file or directory >> ERROR: Genemark failed >> >> Apparently it is looking for the script in the wrong place. I have found >> it in: >> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >> The other paths are correct. >> >> Is there a way to manually tell Maker where to look for this script? I >> should mention also that I have maker (2.10) running on a cluster. >> >> Any help is highly appreciated! Thanks in advance!! >> >> much obliged, >> Christoph >> >> University of Oslo >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > From carsonhh at gmail.com Thu Jul 5 12:00:23 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 13:00:23 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF5C6F7.3090601@gmail.com> Message-ID: You could create the directory maker/bin for the current installation and move all executables there from the /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the executable installed in the maker/bin directory. MAKER is expecting to go back one directory to find libraries and scripts in the MAKER base directory i.e. ../lib/Widget/genemark/gmhmm_wrap Thanks, Carson On 12-07-05 12:55 PM, "Christoph Hahn" wrote: >Dear Carson, > >Thanks for your reply! I have already requested the installation of >maker 2.25 from the cluster admin, but unfortunately this will take a >while. I guess there is no way for me to fix that for the moment in >maker 2.10?? > >Thanks again! >cheers, >Christoph > >On 07/05/2012 06:18 PM, Carson Holt wrote: >> Your cluster is apparently using the CPAN module local::lib to define >> where perl libraries and executables get installed, this is overriding >> MAKER's own installation parameters, thus putting things in the wrong >> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>to >> keep local::lib from messing with it's relative installation paths. So >> install 2.25 and make sure to remove the mis-located MAKER executables >>in >> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >> >> Thanks, >> Carson >> >> >> >> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >> >>> Dear MAKER developers and users, >>> >>> I am new to MAKER and during my first attempts I unfortunately >>> encountert some problems. It would be great if anyone could help me >>>out! >>> Here's my problem: >>> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >>> would like to give this model to MAKER, so I add the path to es.mod in >>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>> says: >>> >>> #-------------------------------# >>> Can't open perl script >>> >>>"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_ >>>wr >>> ap": >>> No such file or directory >>> ERROR: Genemark failed >>> >>> Apparently it is looking for the script in the wrong place. I have >>>found >>> it in: >>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>> The other paths are correct. >>> >>> Is there a way to manually tell Maker where to look for this script? I >>> should mention also that I have maker (2.10) running on a cluster. >>> >>> Any help is highly appreciated! Thanks in advance!! >>> >>> much obliged, >>> Christoph >>> >>> University of Oslo >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > From cjfields at illinois.edu Thu Jul 5 23:19:59 2012 From: cjfields at illinois.edu (Fields, Christopher J) Date: Fri, 6 Jul 2012 04:19:59 +0000 Subject: [maker-devel] Transcriptome assembly annotation? Message-ID: <8AC559A1-AAD1-4090-B48F-0346DF6A7524@illinois.edu> Just curious, but I haven't found the answer to this in the mail list. Has anyone used MAKER for transcriptome annotation? Obviously the ab initio gene predictors wouldn't make much sense, but I could see a tool like ESTScan or similar acting in a similar capacity. chris Chris Fields Technical Lead in Genome Informatics High Performance Computing in Biology W.M. Keck Center for Comparative and Functional Genomics Institute for Genomic Biology From chrisi.hahni at gmail.com Fri Jul 6 11:03:39 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Fri, 06 Jul 2012 18:03:39 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: References: Message-ID: <4FF70C5B.5000804@gmail.com> Dear Carson, I have installed MAKER 2.25 locally on the cluster now and as you have predicted the problem disappeared. Thanks again for your help!! cheers, Christoph On 07/05/2012 07:00 PM, Carson Holt wrote: > You could create the directory maker/bin for the current installation and > move all executables there from the > /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the > executable installed in the maker/bin directory. MAKER is expecting to go > back one directory to find libraries and scripts in the MAKER base > directory i.e. ../lib/Widget/genemark/gmhmm_wrap > > Thanks, > Carson > > > > On 12-07-05 12:55 PM, "Christoph Hahn" wrote: > >> Dear Carson, >> >> Thanks for your reply! I have already requested the installation of >> maker 2.25 from the cluster admin, but unfortunately this will take a >> while. I guess there is no way for me to fix that for the moment in >> maker 2.10?? >> >> Thanks again! >> cheers, >> Christoph >> >> On 07/05/2012 06:18 PM, Carson Holt wrote: >>> Your cluster is apparently using the CPAN module local::lib to define >>> where perl libraries and executables get installed, this is overriding >>> MAKER's own installation parameters, thus putting things in the wrong >>> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>> to >>> keep local::lib from messing with it's relative installation paths. So >>> install 2.25 and make sure to remove the mis-located MAKER executables >>> in >>> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >>> >>> Thanks, >>> Carson >>> >>> >>> >>> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >>> >>>> Dear MAKER developers and users, >>>> >>>> I am new to MAKER and during my first attempts I unfortunately >>>> encountert some problems. It would be great if anyone could help me >>>> out! >>>> Here's my problem: >>>> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >>>> would like to give this model to MAKER, so I add the path to es.mod in >>>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >>>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>>> says: >>>> >>>> #-------------------------------# >>>> Can't open perl script >>>> >>>> "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_ >>>> wr >>>> ap": >>>> No such file or directory >>>> ERROR: Genemark failed >>>> >>>> Apparently it is looking for the script in the wrong place. I have >>>> found >>>> it in: >>>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>>> The other paths are correct. >>>> >>>> Is there a way to manually tell Maker where to look for this script? I >>>> should mention also that I have maker (2.10) running on a cluster. >>>> >>>> Any help is highly appreciated! Thanks in advance!! >>>> >>>> much obliged, >>>> Christoph >>>> >>>> University of Oslo >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > From carsonhh at gmail.com Fri Jul 6 11:05:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 06 Jul 2012 12:05:25 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF70C5B.5000804@gmail.com> Message-ID: I'm glad it's working. Thanks, Carson On 12-07-06 12:03 PM, "Christoph Hahn" wrote: >Dear Carson, > >I have installed MAKER 2.25 locally on the cluster now and as you have >predicted the problem disappeared. >Thanks again for your help!! > >cheers, >Christoph > > >On 07/05/2012 07:00 PM, Carson Holt wrote: >> You could create the directory maker/bin for the current installation >>and >> move all executables there from the >> /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the >> executable installed in the maker/bin directory. MAKER is expecting to >>go >> back one directory to find libraries and scripts in the MAKER base >> directory i.e. ../lib/Widget/genemark/gmhmm_wrap >> >> Thanks, >> Carson >> >> >> >> On 12-07-05 12:55 PM, "Christoph Hahn" wrote: >> >>> Dear Carson, >>> >>> Thanks for your reply! I have already requested the installation of >>> maker 2.25 from the cluster admin, but unfortunately this will take a >>> while. I guess there is no way for me to fix that for the moment in >>> maker 2.10?? >>> >>> Thanks again! >>> cheers, >>> Christoph >>> >>> On 07/05/2012 06:18 PM, Carson Holt wrote: >>>> Your cluster is apparently using the CPAN module local::lib to define >>>> where perl libraries and executables get installed, this is overriding >>>> MAKER's own installation parameters, thus putting things in the wrong >>>> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>>> to >>>> keep local::lib from messing with it's relative installation paths. >>>>So >>>> install 2.25 and make sure to remove the mis-located MAKER executables >>>> in >>>> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> >>>> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >>>> >>>>> Dear MAKER developers and users, >>>>> >>>>> I am new to MAKER and during my first attempts I unfortunately >>>>> encountert some problems. It would be great if anyone could help me >>>>> out! >>>>> Here's my problem: >>>>> I have used gm_es.pl from Genemark to create es.mod for my assembly. >>>>>I >>>>> would like to give this model to MAKER, so I add the path to es.mod >>>>>in >>>>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, >>>>>when I >>>>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>>>> says: >>>>> >>>>> #-------------------------------# >>>>> Can't open perl script >>>>> >>>>> >>>>>"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhm >>>>>m_ >>>>> wr >>>>> ap": >>>>> No such file or directory >>>>> ERROR: Genemark failed >>>>> >>>>> Apparently it is looking for the script in the wrong place. I have >>>>> found >>>>> it in: >>>>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>>>> The other paths are correct. >>>>> >>>>> Is there a way to manually tell Maker where to look for this script? >>>>>I >>>>> should mention also that I have maker (2.10) running on a cluster. >>>>> >>>>> Any help is highly appreciated! Thanks in advance!! >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> University of Oslo >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> >>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>g >>> >> > > From carsonhh at gmail.com Tue Jul 10 07:35:10 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 10 Jul 2012 08:35:10 -0400 Subject: [maker-devel] Help: min_contig length In-Reply-To: Message-ID: There will be no negative effects in restarting. The min_contig option just speeds things up by skipping contigs that are likely too small to accurately annotate. Gene predictors generally require a certain amount of sequence upstream and downstream of a gene to work accurately, so for most Eukaryotic organisms, contigs of size of less than 10,000 will not produce results, but if the genes are smaller in your organism you can reduce this. The basic idea is to just skip contigs that won't produce results anyways. Thanks, Carson From: Diego Calderon Date: Monday, 9 July, 2012 7:20 PM To: ?? Li, Qing Cc: Carson Holt Subject: Re: Help: min_contig length Just a little add on: If we stop the currently running mpi maker, change the option from the default 1 to 500 or 1000, then restart maker, will it have any (negative) effect on maker (for example having to restart the analysis or something)? I'm guessing that it will not, but we just wanted to make sure. Best, Diego On Mon, Jul 9, 2012 at 5:11 PM, ?? Li, Qing wrote: > Hi Carson, > Very sorry for disturbing you! Diego and I are running maker for whole genome > annotation now, and we have a question: should the min_contig=500 or 1000? We > asked several people but get different answers,,, So we think you may be the > best one to ask,,, > Thank you! > Best, > Qing -------------- next part -------------- An HTML attachment was scrubbed... URL: From carson.holt at genetics.utah.edu Fri Jul 13 14:13:56 2012 From: carson.holt at genetics.utah.edu (Carson Holt) Date: Fri, 13 Jul 2012 19:13:56 +0000 Subject: [maker-devel] adding a new gene predictor to maker 2 In-Reply-To: <81f8d5b1-c8e8-46d2-8506-640d9dc18205@googlegroups.com> Message-ID: If you convert it to standard GFF3 format, MAKER will take it as input. GlimmerHMM would go into the pred_gff option. Thanks, Carson From: ckuanglim > Date: Friday, 13 July, 2012 3:01 AM To: Carson Holt > Subject: Re: [maker-devel] adding a new gene predictor to maker 2 Can I pass the output from GlimmerHMM into MAKER? Regards, Chan On Friday, July 30, 2010 9:43:53 AM UTC+8, Carson Holt wrote: mRNA-seq data can be passed to MAKER by first processing the data and then putting it into GFF3 format, i.e. process with something like tophat and cufflinks and then convert the output into GFF3. MAKER comes with tophat2gff3 and cufflinks2gff3 conversion scripts. Supply the results to the est_gff option in the maker_opt.ctl file. For ab initio gene predictors, do the same. Convert the output into GFF3 format then supply the data to pred_gff in the maker_opt.ctl file. Using GFF3 pass-through you can supply MAKER with evidence from virtually any external source of your choice. You just have to convert it into GFF3 so MAKER understands how to use it. Thanks, Carson On 7/29/10 6:43 PM, "Dieter Best" > wrote: Hello, is there a way to plugin another gene predictor into the maker2 pipeline or can we only choose between snap, augustus etc _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From alangfan at gmail.com Sun Jul 15 01:54:39 2012 From: alangfan at gmail.com (=?Big5?B?RmFuO1dlbi1MYW5nIK1TpOWtpg==?=) Date: Sun, 15 Jul 2012 14:54:39 +0800 Subject: [maker-devel] Could someone help me to set up MAKER2? Message-ID: Dear All, I refer to Maker website http://gmod.org/wiki/MAKER_Tutorial to set up Maker2. But the SNAP ( http://homepage.mac.com/iankorf/) website is broken. Could someone kindly tell me where to get SNAP software or alternative scheme? -- Alang -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Sun Jul 15 11:10:42 2012 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 15 Jul 2012 12:10:42 -0400 Subject: [maker-devel] Could someone help me to set up MAKER2? In-Reply-To: Message-ID: The old site was a victim of apple discontinuing MobileMe. SNAP can now be found on the Korf lab's website here --> http://korflab.ucdavis.edu/software.html Thanks, Carson From: "Fan;Wen-Lang ???" Date: Sunday, 15 July, 2012 2:54 AM To: Subject: [maker-devel] Could someone help me to set up MAKER2? Dear All, I refer to Maker website http://gmod.org/wiki/MAKER_Tutorial to set up Maker2. But the SNAP (http://homepage.mac.com/iankorf/) website is broken. Could someone kindly tell me where to get SNAP software or alternative scheme? -- Alang _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Tue Jul 17 00:49:36 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Tue, 17 Jul 2012 05:49:36 +0000 Subject: [maker-devel] maker and repeatproteinmask Message-ID: Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 17 16:05:58 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 17 Jul 2012 17:05:58 -0400 Subject: [maker-devel] maker and repeatproteinmask In-Reply-To: Message-ID: MAKER calls the 'RepeatMasker' script and is thus dependent on anything RepeatMasker decides to call internally. MAKER will never call TRF or RepeatProteinMask directly, but I can't say whether or not RepeatMasker will always do so as part of it's normal processing. The only way to know for sure is to try. I removed TRF from my installation and RepeatMasker shows as completing successfully, so I imagine you would probably be fine without it. Thanks, Carson From: Mr Thomas Schumann Date: Tuesday, 17 July, 2012 1:49 AM To: "maker-devel at yandell-lab.org" Cc: "c.chan10 at uq.edu.au" Subject: [maker-devel] maker and repeatproteinmask Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Wed Jul 18 17:03:10 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Wed, 18 Jul 2012 22:03:10 +0000 Subject: [maker-devel] maker and repeatproteinmask In-Reply-To: References: , Message-ID: Ok thanks. I looked through all the RepeatMasker source files and it doesn't look like anything calls RepeatProteinMask, and RepeatProteinMask is the only program which calls trf, so it looks like MAKER doesn't depend on trf at all then. Thanks, Tom Schumann ________________________________ From: Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 18 July 2012 7:05 AM To: Mr Thomas Schumann; maker-devel at yandell-lab.org Cc: c.chan10 at uq.edu.au Subject: Re: [maker-devel] maker and repeatproteinmask MAKER calls the 'RepeatMasker' script and is thus dependent on anything RepeatMasker decides to call internally. MAKER will never call TRF or RepeatProteinMask directly, but I can't say whether or not RepeatMasker will always do so as part of it's normal processing. The only way to know for sure is to try. I removed TRF from my installation and RepeatMasker shows as completing successfully, so I imagine you would probably be fine without it. Thanks, Carson From: Mr Thomas Schumann > Date: Tuesday, 17 July, 2012 1:49 AM To: "maker-devel at yandell-lab.org" > Cc: "c.chan10 at uq.edu.au" > Subject: [maker-devel] maker and repeatproteinmask Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Jul 23 13:45:59 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 23 Jul 2012 14:45:59 -0400 Subject: [maker-devel] Transcriptome assembly annotation? In-Reply-To: <8AC559A1-AAD1-4090-B48F-0346DF6A7524@illinois.edu> Message-ID: Never tried it, but I think I would probably just skip maker and perhaps do something like run cufflinks or trinity followed by intperproscan and blast2go. There is a wrapper for interproscan packaged with MAKER that might be useful. --Carson On 12-07-06 12:19 AM, "Fields, Christopher J" wrote: >Just curious, but I haven't found the answer to this in the mail list. >Has anyone used MAKER for transcriptome annotation? Obviously the ab >initio gene predictors wouldn't make much sense, but I could see a tool >like ESTScan or similar acting in a similar capacity. > >chris > >Chris Fields >Technical Lead in Genome Informatics >High Performance Computing in Biology >W.M. Keck Center for Comparative and Functional Genomics >Institute for Genomic Biology > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Jul 24 09:20:43 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 24 Jul 2012 10:20:43 -0400 Subject: [maker-devel] MAKER 2.26 Message-ID: FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Tue Jul 24 17:06:11 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Tue, 24 Jul 2012 22:06:11 +0000 Subject: [maker-devel] MAKER 2.26 In-Reply-To: References: Message-ID: Hi Carson, Where can I download it? The Yandell Lab site still only lists v2.10 and v2.26 beta. Thanks, Tom Schumann ________________________________ From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 25 July 2012 12:20 AM To: maker-devel at yandell-lab.org Subject: [maker-devel] MAKER 2.26 FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 24 18:10:21 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 24 Jul 2012 19:10:21 -0400 Subject: [maker-devel] MAKER 2.26 In-Reply-To: Message-ID: That's the one. The beta tag will remain for a short time. The previous 2.25 version was also labelled with the beta tag. Thanks, Carson From: Mr Thomas Schumann Date: Tuesday, 24 July, 2012 6:06 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: RE: MAKER 2.26 Hi Carson, Where can I download it? The Yandell Lab site still only lists v2.10 and v2.26 beta. Thanks, Tom Schumann From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 25 July 2012 12:20 AM To: maker-devel at yandell-lab.org Subject: [maker-devel] MAKER 2.26 FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From guoyunfei1989 at gmail.com Wed Jul 25 14:15:59 2012 From: guoyunfei1989 at gmail.com (Yunfei Guo) Date: Wed, 25 Jul 2012 12:15:59 -0700 Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) Message-ID: Hi everyone, I ran maker2.25 without a problem, but with maker2.26, I encountered the following error after running it for ~8 hr with 2 nodes and 24 cpus, do you have any idea what's going on here? Some contigs did get finished, maybe this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system is SGE. Thanks! running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/scaffold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx #-------------------------------# deleted:-1 hits SIGCHLD handler "DEFAULT" not defined. SIGCHLD handler "DEFAULT" not defined. running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 #-------------------------------# Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fffa3a2e740) failed dequeue_and_set_error(596): Communication error with rank 18 running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/scaffold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx #-------------------------------# running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/scaffold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1.tblastx #-------------------------------# running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 #-------------------------------# cleaning blastx... in cluster::shadow_cluster... ...finished clustering. cleaning clusters.... total clusters:1 now processing 0 ...processing 0 of 2 deleted:0 hits ...processing 1 of 2 running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/scaffold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6.tblastx #-------------------------------# deleted:0 hits running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr%7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.9.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/ scaffold2590.1-3712.tr%7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 #-------------------------------# deleted:0 hits cleaning blastx... cleaning clusters.... total clusters:1 now processing 0 cleaning clusters.... total clusters:1 now processing 0 deleted:-1 hits deleted:-1 hits deleted:-6 hits deleted:-3 hits deleted:-2 hits Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Yunfei -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Jul 25 14:46:17 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 25 Jul 2012 15:46:17 -0400 Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) In-Reply-To: Message-ID: MPI is notorious for unexplicable communication errors, so first I would suggest just restarting and seeing if it happens again (MAKER will pick up where it left off on restart, so no need to alter settings or files). If it happens again, we can look into it, but no component of the MPI communication framework changed between 2.25 and 2.26 (100% identical), so my first instinct is that this was just what the message said, a"Communication error with rank 18". If it happens again I can try and add some extra messages so we can see the hostname of rank 18. That way we can identify if it's constantly a specific node on your cluster. Let me know if you see it again. Thanks, Carson From: Yunfei Guo Date: Wednesday, 25 July, 2012 3:15 PM To: Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) Hi everyone, I ran maker2.25 without a problem, but with maker2.26, I encountered the following error after running it for ~8 hr with 2 nodes and 24 cpus, do you have any idea what's going on here? Some contigs did get finished, maybe this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system is SGE. Thanks! running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/sc affold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx #-------------------------------# deleted:-1 hits SIGCHLD handler "DEFAULT" not defined. SIGCHLD handler "DEFAULT" not defined. running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp %7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 #-------------------------------# Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fffa3a2e740) failed dequeue_and_set_error(596): Communication error with rank 18 running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/sc affold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx #-------------------------------# running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/sc affold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_ Rubri%2Efasta.mpi.10.1.tblastx #-------------------------------# running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp %7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 #-------------------------------# cleaning blastx... in cluster::shadow_cluster... ...finished clustering. cleaning clusters.... total clusters:1 now processing 0 ...processing 0 of 2 deleted:0 hits ...processing 1 of 2 running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/sc affold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_ Rubri%2Efasta.mpi.10.6.tblastx #-------------------------------# deleted:0 hits running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr %7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.9.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.tr %7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 #-------------------------------# deleted:0 hits cleaning blastx... cleaning clusters.... total clusters:1 now processing 0 cleaning clusters.... total clusters:1 now processing 0 deleted:-1 hits deleted:-1 hits deleted:-6 hits deleted:-3 hits deleted:-2 hits Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Yunfei _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From guoyunfei1989 at gmail.com Thu Jul 26 10:10:42 2012 From: guoyunfei1989 at gmail.com (Yunfei Guo) Date: Thu, 26 Jul 2012 08:10:42 -0700 Subject: [maker-devel] Fwd: ERROR: MPI_Recv(186), dequeue_and_set_error(596) In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: Yunfei Guo Date: Thu, Jul 26, 2012 at 8:10 AM Subject: Re: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) To: Carson Holt Hi Carson, same error occurred again. What should I do to check if it was caused by the same node? Also, if I ran maker on a single node instead of two nodes, will the same error appear again? Thank you. #-------------------------------# SIGCHLD handler "DEFAULT" not defined. Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fff1c3dd3b0, count=2, MPI_ INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fff1c3dd3 90) failed dequeue_and_set_error(596): Communication error with rank 21 running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro /run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datast ore/81/43/scaffold5780//theVoid.scaffold5780/tr%7CG3N4L5%7CG3N4L5_GASA C.for.6527-8832.2.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir _Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/81/4 3/scaffold5780//theVoid.scaffold5780/scaffold5780.6527-8832.2.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --sho wcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigr o-53k_part.maker.output/Nigro-53k_part_datastore/81/43/scaffold5780//t heVoid.scaffold5780/scaffold5780.6527-8832.tr%7CG3N4L5%7CG3N4L5_GASAC. p_exonerate.2 #-------------------------------# Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached ... Yunfei On Wed, Jul 25, 2012 at 1:43 PM, Yunfei Guo wrote: > Thanks, Carson. Actually I already set clean_try=1. > > > On Wed, Jul 25, 2012 at 1:34 PM, Carson Holt wrote: > >> That second error from 2.25, Seems to be thrown by Perl's Storable >> module. There may be some weird partial serialization of the data that >> occurred on the first failure and is now causing failures on retry. You >> can set clean_try=1 in MAKER, to let it wipe out data for a failed contig >> before retrying. That can sometimes help get around weird hard failures. >> >> Thanks, >> Carson >> >> >> From: Yunfei Guo >> Date: Wednesday, 25 July, 2012 4:26 PM >> To: Carson Holt >> Subject: Re: [maker-devel] ERROR: MPI_Recv(186), >> dequeue_and_set_error(596) >> >> Thank you, Carson. I'm rerunning maker2.26 now. I just tried maker2.25 as >> well, it failed this time, with similar errors below. I guess it might be >> caused by the cluster (or node) itself, like you said, because we just >> added a few nodes and more memories. I'll ask the admin to see whether he >> can explain this. >> #-------------------------------# >> Thread 1 terminated abnormally: >> ------------- EXCEPTION: Bio::Root::Exception ------------- >> MSG: no data for midline Sequence with id BL_ORD_ID:126195 no longer >> exists in database...alignment skipped >> STACK: Error::throw >> STACK: Bio::Root::Root::throw >> /home/yunfeiguo/perl5/lib/perl5/Bio/Root/Root.pm:472 >> STACK: Bio::SearchIO::blast::next_result >> /home/yunfeiguo/perl5/lib/perl5/Bio/SearchIO/blast.pm:1888 >> STACK: Widget::tblastx::keepers >> /home/yunfeiguo/Downloads/maker/bin/../lib/Widget/tblastx.pm:114 >> STACK: Widget::tblastx::parse >> /home/yunfeiguo/Downloads/maker/bin/../lib/Widget/tblastx.pm:95 >> STACK: GI::tblastx_as_chunks >> /home/yunfeiguo/Downloads/maker/bin/../lib/GI.pm:2612 >> STACK: Process::MpiChunk::_go >> /home/yunfeiguo/Downloads/maker/bin/../lib/Process/MpiChunk.pm:1829 >> STACK: Process::MpiChunk::run >> /home/yunfeiguo/Downloads/maker/bin/../lib/Process/MpiChunk.pm:331 >> STACK: main::node_thread /home/yunfeiguo/Downloads/maker/bin/maker:1308 >> STACK: threads::new >> /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/forks.pm:799 >> STACK: /home/yunfeiguo/Downloads/maker/bin/maker:804 >> ----------------------------------------------------------- >> Cannot restore overloading on HASH(0x1b7f9b60) (package >> Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at >> /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/Storable.pm line >> 416, at /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/ >> forks.pm line 2256. >> Compilation failed in require at >> /home/yunfeiguo/Downloads/maker/bin/maker line 11. >> BEGIN failed--compilation aborted at >> /home/yunfeiguo/Downloads/maker/bin/maker line 11. >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> deleted:0 hits >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff05c78630, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff05c78610) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1414fb00, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1414fae0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff25d86c00, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff25d86be0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> cleaning tblastx... >> cleaning clusters.... >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1b71f1f0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1b71f1d0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffc99d29c0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffc99d29a0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffc4aaf720, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffc4aaf700) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> in cluster::shadow_cluster... >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff317862a0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff31786280) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> ...finished clustering. >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff8abb8e50, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff8abb8e30) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1d1ff180, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1d1ff160) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff4d865850, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff4d865830) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffbec98150, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffbec98130) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffa4ead990, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffa4ead970) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> >> On Wed, Jul 25, 2012 at 12:46 PM, Carson Holt wrote: >> >>> MPI is notorious for unexplicable communication errors, so first I would >>> suggest just restarting and seeing if it happens again (MAKER will pick up >>> where it left off on restart, so no need to alter settings or files). >>> >>> If it happens again, we can look into it, but no component of the MPI >>> communication framework changed between 2.25 and 2.26 (100% identical), so >>> my first instinct is that this was just what the message said, >>> a"Communication error with rank 18". If it happens again I can try and add >>> some extra messages so we can see the hostname of rank 18. That way we can >>> identify if it's constantly a specific node on your cluster. >>> >>> Let me know if you see it again. >>> >>> Thanks, >>> Carson >>> >>> >>> >>> From: Yunfei Guo >>> Date: Wednesday, 25 July, 2012 3:15 PM >>> To: >>> Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) >>> >>> Hi everyone, >>> >>> I ran maker2.25 without a problem, but with maker2.26, I encountered the >>> following error after running it for ~8 hr with 2 nodes and 24 cpus, do you >>> have any idea what's going on here? Some contigs did get finished, maybe >>> this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system >>> is SGE. Thanks! >>> >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query >>> /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 >>> -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/scaffold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx >>> #-------------------------------# >>> deleted:-1 hits >>> SIGCHLD handler "DEFAULT" not defined. >>> SIGCHLD handler "DEFAULT" not defined. >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 >>> #-------------------------------# >>> Fatal error in MPI_Recv: Other MPI error, error stack: >>> MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, >>> MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, >>> status=0x7fffa3a2e740) failed >>> dequeue_and_set_error(596): Communication error with rank 18 >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query >>> /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 >>> -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/scaffold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::tblastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 >>> -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments >>> 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp >>> 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true >>> -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/scaffold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1.tblastx >>> #-------------------------------# >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 >>> #-------------------------------# >>> cleaning blastx... >>> in cluster::shadow_cluster... >>> ...finished clustering. >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> ...processing 0 of 2 >>> deleted:0 hits >>> ...processing 1 of 2 >>> running blast search. >>> #--------- command -------------# >>> Widget::tblastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 >>> -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments >>> 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp >>> 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true >>> -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/scaffold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6.tblastx >>> #-------------------------------# >>> deleted:0 hits >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr%7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.9.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/ >>> scaffold2590.1-3712.tr%7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 >>> #-------------------------------# >>> deleted:0 hits >>> cleaning blastx... >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> deleted:-1 hits >>> deleted:-1 hits >>> deleted:-6 hits >>> deleted:-3 hits >>> deleted:-2 hits >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> >>> Yunfei >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fourie.joubert at up.ac.za Fri Jul 27 09:30:33 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Fri, 27 Jul 2012 16:30:33 +0200 Subject: [maker-devel] Passing extra options to augustus in maker? Message-ID: <5012A609.7060101@up.ac.za> Hi Could anyone advise me regarding passing additional options to augustus when running maker? I have generated a series of hints files (http://bioinf.uni-greifswald.de/bioinf/wiki/pmwiki.php?n=Augustus.IncorporateProteins) that work fine when running augustus stand-alone, but would like to be able to pass the necessary parameters to augustus when running maker so that augustus uses the hints files. Alternatively, if I need to run augustus stand-alone with the hints files, which option should I use to load the resulting augustus gff files into maker (protein_gff, other_gff, ...)? Thanks and regards! Fourie -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Fri Jul 27 11:34:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 27 Jul 2012 12:34:30 -0400 Subject: [maker-devel] Passing extra options to augustus in maker? In-Reply-To: <5012A609.7060101@up.ac.za> Message-ID: You can't pass externally generated augustus hint files into maker. But if you do decide to run augustus alone using the hint file you just generated, you can then pass the results in to pred_gff (after any necessary formatting fixes for correct GFF3). The results of running exonerate2hints.pl are essentially the same as letting MAKER generate its own hints file because maker is using exonerate to generate the alignment. So the effect of an external hint file would be essentially redundant. When running augustus in MAKER, it is ran a minimum of 2 times. The first time it is run without hints and only using the training parameters from the augustus species files. The second time MAKER internally generates hints based on exonerate alignments and then runs augustus using those hints. If you also pass in your external run, you would then have 3 augustus runs on the genome with MAKER selecting the model from the pool of runs that best matches the evidence alignments. Thanks, Carson On 12-07-27 10:30 AM, "Fourie Joubert" wrote: >Hi > >Could anyone advise me regarding passing additional options to augustus >when running maker? > >I have generated a series of hints files >(http://bioinf.uni-greifswald.de/bioinf/wiki/pmwiki.php?n=Augustus.Incorpo >rateProteins) >that work fine when running augustus stand-alone, but would like to be >able to pass the necessary parameters to augustus when running maker so >that augustus uses the hints files. > >Alternatively, if I need to run augustus stand-alone with the hints >files, which option should I use to load the resulting augustus gff >files into maker (protein_gff, other_gff, ...)? > >Thanks and regards! > >Fourie > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Fri Jul 27 11:38:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 27 Jul 2012 12:38:56 -0400 Subject: [maker-devel] Advice for optimizing augustus training on fungal genome? In-Reply-To: <4FEC7423.3050001@up.ac.za> Message-ID: Regarding this older post, if you aren't getting good results using CEGMA for training augustus for a new species, then One option would be just to make a copy Neurospora's species directory. Edit the necessary file contents to make it list as a different species, then run the augustus training steps as before but this use the Neurospora copy as the base so augustus will be optimizing Neurospora's parameters to be more like your species of interest. Thanks, Carson On 12-06-28 11:11 AM, "Fourie Joubert" wrote: >Hi Everyone > >Apologies if this is not the relevant list to mail to. > >I am looking for advice in training augustus for a novel fungal genome. > >I generated a gene set using CEGMA (below), and have subsequently been >following the instructions at >http://www.molecularevolution.org/molevolfiles/exercises/augustus/scipio.h >tml >and at >http://www.molecularevolution.org/molevolfiles/exercises/augustus/training >.html. > >My training set is 339 genes and the test set is 100 genes. > >My initial output is below. > >It does not improve much with optimize_augustus. > >When using the training paramters to predict genes in the genome, I seem >to only find around 2,000 of the known ~16,000 genes. When I use the >training data from a distantly related fungus (Neurospora), I get >roughly the correct number of genes. > >I am obviously doing something wrong here... (commands below). > >I would really appreciate any advice on where to start looking for >improvement. > >Kindest regards! > >Fourie > > > > > >Augustus commands (Editedmyspecies_parameters.cfg and >setstopCodonExcludedFromCDS to true.): > >> etraining --species=myspecies genes.gb.train > >> augustus --species=myspecies genes.gb.test | tee firsttest.out > >> grep -A 22 Evaluation firsttest.out > >> optimize_augustus.pl --species=myspecies genes.gb.train > >> etraining --species=myspecies genes.gb.train > >> augustus --species=myspecies genes.gb.test | tee secondtest.out > >> grep -A 22 Evaluation secondtest.out > > > >CEGMA output: > ># Statistics of the completeness of the genome based on 248 CEGs > # > > #Prots %Completeness - #Total Average %Ortho > > Complete 240 96.77 - 278 1.16 11.67 > > Group 1 64 96.97 - 72 1.12 7.81 > Group 2 54 96.43 - 66 1.22 18.52 > Group 3 58 95.08 - 70 1.21 13.79 > Group 4 64 98.46 - 70 1.09 7.81 > > Partial 245 98.79 - 290 1.18 13.88 > > Group 1 65 98.48 - 73 1.12 7.69 > Group 2 56 100.00 - 70 1.25 21.43 > Group 3 59 96.72 - 75 1.27 18.64 > Group 4 65 100.00 - 72 1.11 9.23 > > > > >Augustus output: > >******* Evaluation of gene prediction ******* > >---------------------------------------------\ > > | sensitivity | specificity | > >---------------------------------------------| > >nucleotide level | 0.933 | 0.772 | > >---------------------------------------------/ > >-------------------------------------------------------------------------- >--------------------------------\ > > | #pred | #anno | | FP = false pos. | FN = false >neg. | | | > > | total/ | total/ | TP >|--------------------|--------------------| sensitivity | specificity | > > | unique | unique | | part | ovlp | wrng | part | ovlp | >wrng | | | > >-------------------------------------------------------------------------- >--------------------------------| > > | | | | 229 | > 85 | | | > >exon level | 475 | 331 | 246 | ------------------ | >------------------ | 0.743 | 0.518 | > > | 475 | 331 | | 59 | 9 | 161 | 56 | 2 | > 27 | | | > >-------------------------------------------------------------------------- >--------------------------------/ > >-------------------------------------------------------------------------- >--\ > >transcript | #pred | #anno | TP | FP | FN | sensitivity | >specificity | > >-------------------------------------------------------------------------- >--| > >gene level | 158 | 100 | 45 | 113 | 55 | 0.45 | >0.285 | > >-------------------------------------------------------------------------- >--/ > > > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From jason.stajich at gmail.com Fri Jul 27 17:52:07 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Fri, 27 Jul 2012 15:52:07 -0700 Subject: [maker-devel] Advice for optimizing augustus training on fungal genome? In-Reply-To: References: Message-ID: <4457959E-443D-4A17-8EC6-5D16FD58B2C7@gmail.com> Best option is to get RNA-Seq or some transcript evidence of course so you have something empirical. I'm really surprised you are getting such a low count - this is just from augustus running on the cmdline? Alternatively if you have 16,000 known proteins, derived from protein alignments of related species aligned to your genome I expect? You could also try and generate a best set of gene predictions from this. When I did this for Coprinus I gathered full length alignments of fungal proteins and used at the time genomewise and genewise to get the best set of loci with gene calls. Today I'd probably just use FASTA to find the locus and exonerate to get the spliced gene model out and provide these as training. Basically this is doing a fungal-specific CEGMA, something we ought to build, but just don't have time to do. Do you get similar results when running genemarkHMM with its selftraining? jason On Jul 27, 2012, at 9:38 AM, Carson Holt wrote: > Regarding this older post, if you aren't getting good results using CEGMA > for training augustus for a new species, then One option would be just to > make a copy Neurospora's species directory. Edit the necessary file > contents to make it list as a different species, then run the augustus > training steps as before but this use the Neurospora copy as the base so > augustus will be optimizing Neurospora's parameters to be more like your > species of interest. > > Thanks, > Carson > > > > On 12-06-28 11:11 AM, "Fourie Joubert" wrote: > >> Hi Everyone >> >> Apologies if this is not the relevant list to mail to. >> >> I am looking for advice in training augustus for a novel fungal genome. >> >> I generated a gene set using CEGMA (below), and have subsequently been >> following the instructions at >> http://www.molecularevolution.org/molevolfiles/exercises/augustus/scipio.h >> tml >> and at >> http://www.molecularevolution.org/molevolfiles/exercises/augustus/training >> .html. >> >> My training set is 339 genes and the test set is 100 genes. >> >> My initial output is below. >> >> It does not improve much with optimize_augustus. >> >> When using the training paramters to predict genes in the genome, I seem >> to only find around 2,000 of the known ~16,000 genes. When I use the >> training data from a distantly related fungus (Neurospora), I get >> roughly the correct number of genes. >> >> I am obviously doing something wrong here... (commands below). >> >> I would really appreciate any advice on where to start looking for >> improvement. >> >> Kindest regards! >> >> Fourie >> >> >> >> >> >> Augustus commands (Editedmyspecies_parameters.cfg and >> setstopCodonExcludedFromCDS to true.): >> >>> etraining --species=myspecies genes.gb.train >> >>> augustus --species=myspecies genes.gb.test | tee firsttest.out >> >>> grep -A 22 Evaluation firsttest.out >> >>> optimize_augustus.pl --species=myspecies genes.gb.train >> >>> etraining --species=myspecies genes.gb.train >> >>> augustus --species=myspecies genes.gb.test | tee secondtest.out >> >>> grep -A 22 Evaluation secondtest.out >> >> >> >> CEGMA output: >> >> # Statistics of the completeness of the genome based on 248 CEGs >> # >> >> #Prots %Completeness - #Total Average %Ortho >> >> Complete 240 96.77 - 278 1.16 11.67 >> >> Group 1 64 96.97 - 72 1.12 7.81 >> Group 2 54 96.43 - 66 1.22 18.52 >> Group 3 58 95.08 - 70 1.21 13.79 >> Group 4 64 98.46 - 70 1.09 7.81 >> >> Partial 245 98.79 - 290 1.18 13.88 >> >> Group 1 65 98.48 - 73 1.12 7.69 >> Group 2 56 100.00 - 70 1.25 21.43 >> Group 3 59 96.72 - 75 1.27 18.64 >> Group 4 65 100.00 - 72 1.11 9.23 >> >> >> >> >> Augustus output: >> >> ******* Evaluation of gene prediction ******* >> >> ---------------------------------------------\ >> >> | sensitivity | specificity | >> >> ---------------------------------------------| >> >> nucleotide level | 0.933 | 0.772 | >> >> ---------------------------------------------/ >> >> -------------------------------------------------------------------------- >> --------------------------------\ >> >> | #pred | #anno | | FP = false pos. | FN = false >> neg. | | | >> >> | total/ | total/ | TP >> |--------------------|--------------------| sensitivity | specificity | >> >> | unique | unique | | part | ovlp | wrng | part | ovlp | >> wrng | | | >> >> -------------------------------------------------------------------------- >> --------------------------------| >> >> | | | | 229 | >> 85 | | | >> >> exon level | 475 | 331 | 246 | ------------------ | >> ------------------ | 0.743 | 0.518 | >> >> | 475 | 331 | | 59 | 9 | 161 | 56 | 2 | >> 27 | | | >> >> -------------------------------------------------------------------------- >> --------------------------------/ >> >> -------------------------------------------------------------------------- >> --\ >> >> transcript | #pred | #anno | TP | FP | FN | sensitivity | >> specificity | >> >> -------------------------------------------------------------------------- >> --| >> >> gene level | 158 | 100 | 45 | 113 | 55 | 0.45 | >> 0.285 | >> >> -------------------------------------------------------------------------- >> --/ >> >> >> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org From sbrubaker at solazyme.com Mon Jul 30 18:52:57 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Mon, 30 Jul 2012 23:52:57 +0000 Subject: [maker-devel] CDS file? Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 31 08:01:07 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 31 Jul 2012 09:01:07 -0400 Subject: [maker-devel] CDS file? In-Reply-To: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 12:16:45 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 17:16:45 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE361@EXCHANGE-05.internal.solazyme.com> Thanks! From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 12:17:19 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 17:17:19 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com> One more question - do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 31 12:58:51 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 31 Jul 2012 13:58:51 -0400 Subject: [maker-devel] CDS file? In-Reply-To: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com> Message-ID: Not really, but the functional annotation tools included with maker might be useful for some of the downstream work. Although I guess there is a way I could fool maker's direct protein2genome gene creation into generating gene models with a little code modification. --Carson From: Shane Brubaker Date: Tuesday, 31 July, 2012 1:17 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: RE: [maker-devel] CDS file? One more question ? do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 14:13:55 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 19:13:55 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com>, Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE4C4@EXCHANGE-05.internal.solazyme.com> Got it, thanks for your help! ________________________________________ From: Carson Holt [carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 10:58 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Not really, but the functional annotation tools included with maker might be useful for some of the downstream work. Although I guess there is a way I could fool maker's direct protein2genome gene creation into generating gene models with a little code modification. --Carson From: Shane Brubaker > Date: Tuesday, 31 July, 2012 1:17 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: RE: [maker-devel] CDS file? One more question ? do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Jul 3 08:12:57 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 03 Jul 2012 10:12:57 -0400 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: <4FEC6428.3010108@stud-mail.uni-wuerzburg.de> Message-ID: Have you tried 2.25 to see if the error still occurs in that version? Thanks, Carson On 12-06-28 10:03 AM, "Markus Ankenbrand" wrote: >Hi all, > >I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >only using coding sequences of a reference species as "altest". Without >any obvious reason annotation fails for some contigs. It seems to have >nothing to do with the contig length. I also tried to run maker with >different reference ESTs and GeneMark. Depending on the setup different >contigs fail. > >In case of failure, the log is: >... >cleaning clusters.... >total clusters:3 now processing 0 > ...processing 0 of 3 > ...processing 1 of 3 >total clusters:3 now processing 0 >total clusters:3 now processing 0 > ...processing 0 of 2 >Can't call method "start" on an undefined valueERROR: Failed while >preparing evidence clusters for annotations >ERROR: Chunk failed at level:0, tier_type:3 >FAILED CONTIG:contig1 > >ERROR: Chunk failed at level:7, tier_type:0 >FAILED CONTIG:contig1 > >--Next Contig-- >... > >If someone has any idea how to solve this problem, help would be very >much apriciated. > >Greetings, >Markus > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Jul 3 11:27:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 03 Jul 2012 13:27:56 -0400 Subject: [maker-devel] maker leaving large numbers of defunct zombies In-Reply-To: <68CF9F49-540C-4018-A849-1D1B5751943B@slu.se> Message-ID: MAKER doesn't lock the SQL database except on initial creation and population. However, SQLite does it's own advisory locking apart from MAKER (and it may be trying to do this with every transaction). This could be problematic on some NFS systems where advisory locking may be Buggy. I'm going to try something. Just update the SVN repository one more time. What I've done is to add a check to see if you are running in an NFS mounted directory. If so, maker will try and localize the SQLite db file to /tmp or wherever your TMP variable is set to in the maker_opts.ctl file (maker will check the NFS status on that as well before attempting the copy so as not to waste time in just copying to another NFS). So if you set TMP or your linux TMPDIR variable is set to an NFS mounted locations, try changing it to a location that will be local to each machine. Usually /tmp (Linux default) or you can use the memory mounted virtual directory /dev/shm. Maker will clean up temporary files it creates afterwards. Automatically. By using a local rather than NFS location in this step, advisory locks should work properly for SQLite (if that is the issue you are experiencing). I've added checks and optimizations so the file will not be copied if it has already been localized to a given machine, even if it was by another node in the MPI job. So the overall space overhead should be limited, and you'll probably even get a slight performance boost as well (local SQLite databases run much faster than NFS located ones). Thanks, Carson On 12-06-28 4:16 PM, "Mikael Brandstr?m Durling" wrote: >Sorry, but I might have been slightly to quick to my conclusions. All the >initialisation and initial database creation works fine. However, when >running I start to see locking error from SQLite. Is GFFDB supposed to >use the NFSLock to lock the database at all accesses? After running for a >while now, I see locking errors (ie the query or transaction failes) from >SQLite at these lines in GFFDB.pm: 107/109, 529 and 678. > >Mikael From markus.ankenbrand at stud-mail.uni-wuerzburg.de Thu Jul 5 05:22:15 2012 From: markus.ankenbrand at stud-mail.uni-wuerzburg.de (Markus Ankenbrand) Date: Thu, 05 Jul 2012 13:22:15 +0200 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: References: Message-ID: <4FF578E7.7060700@stud-mail.uni-wuerzburg.de> Hi Carson, thanks for your response. Version 2.25 works without any errors. May I have a changelog for 2.25, if available? Thanks, Markus Am 03.07.2012 16:12, schrieb Carson Holt: > Have you tried 2.25 to see if the error still occurs in that version? > > Thanks, > Carson > > > > On 12-06-28 10:03 AM, "Markus Ankenbrand" > wrote: > >> Hi all, >> >> I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >> only using coding sequences of a reference species as "altest". Without >> any obvious reason annotation fails for some contigs. It seems to have >> nothing to do with the contig length. I also tried to run maker with >> different reference ESTs and GeneMark. Depending on the setup different >> contigs fail. >> >> In case of failure, the log is: >> ... >> cleaning clusters.... >> total clusters:3 now processing 0 >> ...processing 0 of 3 >> ...processing 1 of 3 >> total clusters:3 now processing 0 >> total clusters:3 now processing 0 >> ...processing 0 of 2 >> Can't call method "start" on an undefined valueERROR: Failed while >> preparing evidence clusters for annotations >> ERROR: Chunk failed at level:0, tier_type:3 >> FAILED CONTIG:contig1 >> >> ERROR: Chunk failed at level:7, tier_type:0 >> FAILED CONTIG:contig1 >> >> --Next Contig-- >> ... >> >> If someone has any idea how to solve this problem, help would be very >> much apriciated. >> >> Greetings, >> Markus >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Thu Jul 5 08:00:54 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 10:00:54 -0400 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: <4FF578E7.7060700@stud-mail.uni-wuerzburg.de> Message-ID: Here are comments from subversion between 2.24 and 2.25 Renaming and minor fixes to some modules improved gff3_merge script fix est passthrough bug made GFFDB module faster Thanks, Carson On 12-07-05 7:22 AM, "Markus Ankenbrand" wrote: >Hi Carson, > >thanks for your response. >Version 2.25 works without any errors. >May I have a changelog for 2.25, if available? > >Thanks, >Markus > > >Am 03.07.2012 16:12, schrieb Carson Holt: >> Have you tried 2.25 to see if the error still occurs in that version? >> >> Thanks, >> Carson >> >> >> >> On 12-06-28 10:03 AM, "Markus Ankenbrand" >> wrote: >> >>> Hi all, >>> >>> I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >>> only using coding sequences of a reference species as "altest". Without >>> any obvious reason annotation fails for some contigs. It seems to have >>> nothing to do with the contig length. I also tried to run maker with >>> different reference ESTs and GeneMark. Depending on the setup different >>> contigs fail. >>> >>> In case of failure, the log is: >>> ... >>> cleaning clusters.... >>> total clusters:3 now processing 0 >>> ...processing 0 of 3 >>> ...processing 1 of 3 >>> total clusters:3 now processing 0 >>> total clusters:3 now processing 0 >>> ...processing 0 of 2 >>> Can't call method "start" on an undefined valueERROR: Failed while >>> preparing evidence clusters for annotations >>> ERROR: Chunk failed at level:0, tier_type:3 >>> FAILED CONTIG:contig1 >>> >>> ERROR: Chunk failed at level:7, tier_type:0 >>> FAILED CONTIG:contig1 >>> >>> --Next Contig-- >>> ... >>> >>> If someone has any idea how to solve this problem, help would be very >>> much apriciated. >>> >>> Greetings, >>> Markus >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > From chrisi.hahni at gmail.com Thu Jul 5 09:06:38 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Thu, 05 Jul 2012 17:06:38 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap Message-ID: <4FF5AD7E.7010805@gmail.com> Dear MAKER developers and users, I am new to MAKER and during my first attempts I unfortunately encountert some problems. It would be great if anyone could help me out! Here's my problem: I have used gm_es.pl from Genemark to create es.mod for my assembly. I would like to give this model to MAKER, so I add the path to es.mod in the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I am running maker it cannot find the gmhmm_wrap script. Here s what it says: #-------------------------------# Can't open perl script "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wrap": No such file or directory ERROR: Genemark failed Apparently it is looking for the script in the wrong place. I have found it in: /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap The other paths are correct. Is there a way to manually tell Maker where to look for this script? I should mention also that I have maker (2.10) running on a cluster. Any help is highly appreciated! Thanks in advance!! much obliged, Christoph University of Oslo From carsonhh at gmail.com Thu Jul 5 10:18:45 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 12:18:45 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF5AD7E.7010805@gmail.com> Message-ID: Your cluster is apparently using the CPAN module local::lib to define where perl libraries and executables get installed, this is overriding MAKER's own installation parameters, thus putting things in the wrong place. This only happens in 2.10, the 2.25 version of MAKER knows how to keep local::lib from messing with it's relative installation paths. So install 2.25 and make sure to remove the mis-located MAKER executables in /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. Thanks, Carson On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >Dear MAKER developers and users, > >I am new to MAKER and during my first attempts I unfortunately >encountert some problems. It would be great if anyone could help me out! >Here's my problem: >I have used gm_es.pl from Genemark to create es.mod for my assembly. I >would like to give this model to MAKER, so I add the path to es.mod in >the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >am running maker it cannot find the gmhmm_wrap script. Here s what it >says: > >#-------------------------------# >Can't open perl script >"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wr >ap": >No such file or directory >ERROR: Genemark failed > >Apparently it is looking for the script in the wrong place. I have found >it in: >/site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >The other paths are correct. > >Is there a way to manually tell Maker where to look for this script? I >should mention also that I have maker (2.10) running on a cluster. > >Any help is highly appreciated! Thanks in advance!! > >much obliged, >Christoph > >University of Oslo > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From chrisi.hahni at gmail.com Thu Jul 5 10:55:19 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Thu, 05 Jul 2012 18:55:19 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: References: Message-ID: <4FF5C6F7.3090601@gmail.com> Dear Carson, Thanks for your reply! I have already requested the installation of maker 2.25 from the cluster admin, but unfortunately this will take a while. I guess there is no way for me to fix that for the moment in maker 2.10?? Thanks again! cheers, Christoph On 07/05/2012 06:18 PM, Carson Holt wrote: > Your cluster is apparently using the CPAN module local::lib to define > where perl libraries and executables get installed, this is overriding > MAKER's own installation parameters, thus putting things in the wrong > place. This only happens in 2.10, the 2.25 version of MAKER knows how to > keep local::lib from messing with it's relative installation paths. So > install 2.25 and make sure to remove the mis-located MAKER executables in > /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. > > Thanks, > Carson > > > > On 12-07-05 11:06 AM, "Christoph Hahn" wrote: > >> Dear MAKER developers and users, >> >> I am new to MAKER and during my first attempts I unfortunately >> encountert some problems. It would be great if anyone could help me out! >> Here's my problem: >> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >> would like to give this model to MAKER, so I add the path to es.mod in >> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >> am running maker it cannot find the gmhmm_wrap script. Here s what it >> says: >> >> #-------------------------------# >> Can't open perl script >> "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wr >> ap": >> No such file or directory >> ERROR: Genemark failed >> >> Apparently it is looking for the script in the wrong place. I have found >> it in: >> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >> The other paths are correct. >> >> Is there a way to manually tell Maker where to look for this script? I >> should mention also that I have maker (2.10) running on a cluster. >> >> Any help is highly appreciated! Thanks in advance!! >> >> much obliged, >> Christoph >> >> University of Oslo >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > From carsonhh at gmail.com Thu Jul 5 11:00:23 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 13:00:23 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF5C6F7.3090601@gmail.com> Message-ID: You could create the directory maker/bin for the current installation and move all executables there from the /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the executable installed in the maker/bin directory. MAKER is expecting to go back one directory to find libraries and scripts in the MAKER base directory i.e. ../lib/Widget/genemark/gmhmm_wrap Thanks, Carson On 12-07-05 12:55 PM, "Christoph Hahn" wrote: >Dear Carson, > >Thanks for your reply! I have already requested the installation of >maker 2.25 from the cluster admin, but unfortunately this will take a >while. I guess there is no way for me to fix that for the moment in >maker 2.10?? > >Thanks again! >cheers, >Christoph > >On 07/05/2012 06:18 PM, Carson Holt wrote: >> Your cluster is apparently using the CPAN module local::lib to define >> where perl libraries and executables get installed, this is overriding >> MAKER's own installation parameters, thus putting things in the wrong >> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>to >> keep local::lib from messing with it's relative installation paths. So >> install 2.25 and make sure to remove the mis-located MAKER executables >>in >> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >> >> Thanks, >> Carson >> >> >> >> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >> >>> Dear MAKER developers and users, >>> >>> I am new to MAKER and during my first attempts I unfortunately >>> encountert some problems. It would be great if anyone could help me >>>out! >>> Here's my problem: >>> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >>> would like to give this model to MAKER, so I add the path to es.mod in >>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>> says: >>> >>> #-------------------------------# >>> Can't open perl script >>> >>>"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_ >>>wr >>> ap": >>> No such file or directory >>> ERROR: Genemark failed >>> >>> Apparently it is looking for the script in the wrong place. I have >>>found >>> it in: >>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>> The other paths are correct. >>> >>> Is there a way to manually tell Maker where to look for this script? I >>> should mention also that I have maker (2.10) running on a cluster. >>> >>> Any help is highly appreciated! Thanks in advance!! >>> >>> much obliged, >>> Christoph >>> >>> University of Oslo >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > From cjfields at illinois.edu Thu Jul 5 22:19:59 2012 From: cjfields at illinois.edu (Fields, Christopher J) Date: Fri, 6 Jul 2012 04:19:59 +0000 Subject: [maker-devel] Transcriptome assembly annotation? Message-ID: <8AC559A1-AAD1-4090-B48F-0346DF6A7524@illinois.edu> Just curious, but I haven't found the answer to this in the mail list. Has anyone used MAKER for transcriptome annotation? Obviously the ab initio gene predictors wouldn't make much sense, but I could see a tool like ESTScan or similar acting in a similar capacity. chris Chris Fields Technical Lead in Genome Informatics High Performance Computing in Biology W.M. Keck Center for Comparative and Functional Genomics Institute for Genomic Biology From chrisi.hahni at gmail.com Fri Jul 6 10:03:39 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Fri, 06 Jul 2012 18:03:39 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: References: Message-ID: <4FF70C5B.5000804@gmail.com> Dear Carson, I have installed MAKER 2.25 locally on the cluster now and as you have predicted the problem disappeared. Thanks again for your help!! cheers, Christoph On 07/05/2012 07:00 PM, Carson Holt wrote: > You could create the directory maker/bin for the current installation and > move all executables there from the > /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the > executable installed in the maker/bin directory. MAKER is expecting to go > back one directory to find libraries and scripts in the MAKER base > directory i.e. ../lib/Widget/genemark/gmhmm_wrap > > Thanks, > Carson > > > > On 12-07-05 12:55 PM, "Christoph Hahn" wrote: > >> Dear Carson, >> >> Thanks for your reply! I have already requested the installation of >> maker 2.25 from the cluster admin, but unfortunately this will take a >> while. I guess there is no way for me to fix that for the moment in >> maker 2.10?? >> >> Thanks again! >> cheers, >> Christoph >> >> On 07/05/2012 06:18 PM, Carson Holt wrote: >>> Your cluster is apparently using the CPAN module local::lib to define >>> where perl libraries and executables get installed, this is overriding >>> MAKER's own installation parameters, thus putting things in the wrong >>> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>> to >>> keep local::lib from messing with it's relative installation paths. So >>> install 2.25 and make sure to remove the mis-located MAKER executables >>> in >>> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >>> >>> Thanks, >>> Carson >>> >>> >>> >>> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >>> >>>> Dear MAKER developers and users, >>>> >>>> I am new to MAKER and during my first attempts I unfortunately >>>> encountert some problems. It would be great if anyone could help me >>>> out! >>>> Here's my problem: >>>> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >>>> would like to give this model to MAKER, so I add the path to es.mod in >>>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >>>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>>> says: >>>> >>>> #-------------------------------# >>>> Can't open perl script >>>> >>>> "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_ >>>> wr >>>> ap": >>>> No such file or directory >>>> ERROR: Genemark failed >>>> >>>> Apparently it is looking for the script in the wrong place. I have >>>> found >>>> it in: >>>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>>> The other paths are correct. >>>> >>>> Is there a way to manually tell Maker where to look for this script? I >>>> should mention also that I have maker (2.10) running on a cluster. >>>> >>>> Any help is highly appreciated! Thanks in advance!! >>>> >>>> much obliged, >>>> Christoph >>>> >>>> University of Oslo >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > From carsonhh at gmail.com Fri Jul 6 10:05:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 06 Jul 2012 12:05:25 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF70C5B.5000804@gmail.com> Message-ID: I'm glad it's working. Thanks, Carson On 12-07-06 12:03 PM, "Christoph Hahn" wrote: >Dear Carson, > >I have installed MAKER 2.25 locally on the cluster now and as you have >predicted the problem disappeared. >Thanks again for your help!! > >cheers, >Christoph > > >On 07/05/2012 07:00 PM, Carson Holt wrote: >> You could create the directory maker/bin for the current installation >>and >> move all executables there from the >> /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the >> executable installed in the maker/bin directory. MAKER is expecting to >>go >> back one directory to find libraries and scripts in the MAKER base >> directory i.e. ../lib/Widget/genemark/gmhmm_wrap >> >> Thanks, >> Carson >> >> >> >> On 12-07-05 12:55 PM, "Christoph Hahn" wrote: >> >>> Dear Carson, >>> >>> Thanks for your reply! I have already requested the installation of >>> maker 2.25 from the cluster admin, but unfortunately this will take a >>> while. I guess there is no way for me to fix that for the moment in >>> maker 2.10?? >>> >>> Thanks again! >>> cheers, >>> Christoph >>> >>> On 07/05/2012 06:18 PM, Carson Holt wrote: >>>> Your cluster is apparently using the CPAN module local::lib to define >>>> where perl libraries and executables get installed, this is overriding >>>> MAKER's own installation parameters, thus putting things in the wrong >>>> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>>> to >>>> keep local::lib from messing with it's relative installation paths. >>>>So >>>> install 2.25 and make sure to remove the mis-located MAKER executables >>>> in >>>> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> >>>> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >>>> >>>>> Dear MAKER developers and users, >>>>> >>>>> I am new to MAKER and during my first attempts I unfortunately >>>>> encountert some problems. It would be great if anyone could help me >>>>> out! >>>>> Here's my problem: >>>>> I have used gm_es.pl from Genemark to create es.mod for my assembly. >>>>>I >>>>> would like to give this model to MAKER, so I add the path to es.mod >>>>>in >>>>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, >>>>>when I >>>>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>>>> says: >>>>> >>>>> #-------------------------------# >>>>> Can't open perl script >>>>> >>>>> >>>>>"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhm >>>>>m_ >>>>> wr >>>>> ap": >>>>> No such file or directory >>>>> ERROR: Genemark failed >>>>> >>>>> Apparently it is looking for the script in the wrong place. I have >>>>> found >>>>> it in: >>>>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>>>> The other paths are correct. >>>>> >>>>> Is there a way to manually tell Maker where to look for this script? >>>>>I >>>>> should mention also that I have maker (2.10) running on a cluster. >>>>> >>>>> Any help is highly appreciated! Thanks in advance!! >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> University of Oslo >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> >>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>g >>> >> > > From carsonhh at gmail.com Tue Jul 10 06:35:10 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 10 Jul 2012 08:35:10 -0400 Subject: [maker-devel] Help: min_contig length In-Reply-To: Message-ID: There will be no negative effects in restarting. The min_contig option just speeds things up by skipping contigs that are likely too small to accurately annotate. Gene predictors generally require a certain amount of sequence upstream and downstream of a gene to work accurately, so for most Eukaryotic organisms, contigs of size of less than 10,000 will not produce results, but if the genes are smaller in your organism you can reduce this. The basic idea is to just skip contigs that won't produce results anyways. Thanks, Carson From: Diego Calderon Date: Monday, 9 July, 2012 7:20 PM To: ?? Li, Qing Cc: Carson Holt Subject: Re: Help: min_contig length Just a little add on: If we stop the currently running mpi maker, change the option from the default 1 to 500 or 1000, then restart maker, will it have any (negative) effect on maker (for example having to restart the analysis or something)? I'm guessing that it will not, but we just wanted to make sure. Best, Diego On Mon, Jul 9, 2012 at 5:11 PM, ?? Li, Qing wrote: > Hi Carson, > Very sorry for disturbing you! Diego and I are running maker for whole genome > annotation now, and we have a question: should the min_contig=500 or 1000? We > asked several people but get different answers,,, So we think you may be the > best one to ask,,, > Thank you! > Best, > Qing -------------- next part -------------- An HTML attachment was scrubbed... URL: From carson.holt at genetics.utah.edu Fri Jul 13 13:13:56 2012 From: carson.holt at genetics.utah.edu (Carson Holt) Date: Fri, 13 Jul 2012 19:13:56 +0000 Subject: [maker-devel] adding a new gene predictor to maker 2 In-Reply-To: <81f8d5b1-c8e8-46d2-8506-640d9dc18205@googlegroups.com> Message-ID: If you convert it to standard GFF3 format, MAKER will take it as input. GlimmerHMM would go into the pred_gff option. Thanks, Carson From: ckuanglim > Date: Friday, 13 July, 2012 3:01 AM To: Carson Holt > Subject: Re: [maker-devel] adding a new gene predictor to maker 2 Can I pass the output from GlimmerHMM into MAKER? Regards, Chan On Friday, July 30, 2010 9:43:53 AM UTC+8, Carson Holt wrote: mRNA-seq data can be passed to MAKER by first processing the data and then putting it into GFF3 format, i.e. process with something like tophat and cufflinks and then convert the output into GFF3. MAKER comes with tophat2gff3 and cufflinks2gff3 conversion scripts. Supply the results to the est_gff option in the maker_opt.ctl file. For ab initio gene predictors, do the same. Convert the output into GFF3 format then supply the data to pred_gff in the maker_opt.ctl file. Using GFF3 pass-through you can supply MAKER with evidence from virtually any external source of your choice. You just have to convert it into GFF3 so MAKER understands how to use it. Thanks, Carson On 7/29/10 6:43 PM, "Dieter Best" > wrote: Hello, is there a way to plugin another gene predictor into the maker2 pipeline or can we only choose between snap, augustus etc _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From alangfan at gmail.com Sun Jul 15 00:54:39 2012 From: alangfan at gmail.com (=?Big5?B?RmFuO1dlbi1MYW5nIK1TpOWtpg==?=) Date: Sun, 15 Jul 2012 14:54:39 +0800 Subject: [maker-devel] Could someone help me to set up MAKER2? Message-ID: Dear All, I refer to Maker website http://gmod.org/wiki/MAKER_Tutorial to set up Maker2. But the SNAP ( http://homepage.mac.com/iankorf/) website is broken. Could someone kindly tell me where to get SNAP software or alternative scheme? -- Alang -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Sun Jul 15 10:10:42 2012 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 15 Jul 2012 12:10:42 -0400 Subject: [maker-devel] Could someone help me to set up MAKER2? In-Reply-To: Message-ID: The old site was a victim of apple discontinuing MobileMe. SNAP can now be found on the Korf lab's website here --> http://korflab.ucdavis.edu/software.html Thanks, Carson From: "Fan;Wen-Lang ???" Date: Sunday, 15 July, 2012 2:54 AM To: Subject: [maker-devel] Could someone help me to set up MAKER2? Dear All, I refer to Maker website http://gmod.org/wiki/MAKER_Tutorial to set up Maker2. But the SNAP (http://homepage.mac.com/iankorf/) website is broken. Could someone kindly tell me where to get SNAP software or alternative scheme? -- Alang _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Mon Jul 16 23:49:36 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Tue, 17 Jul 2012 05:49:36 +0000 Subject: [maker-devel] maker and repeatproteinmask Message-ID: Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 17 15:05:58 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 17 Jul 2012 17:05:58 -0400 Subject: [maker-devel] maker and repeatproteinmask In-Reply-To: Message-ID: MAKER calls the 'RepeatMasker' script and is thus dependent on anything RepeatMasker decides to call internally. MAKER will never call TRF or RepeatProteinMask directly, but I can't say whether or not RepeatMasker will always do so as part of it's normal processing. The only way to know for sure is to try. I removed TRF from my installation and RepeatMasker shows as completing successfully, so I imagine you would probably be fine without it. Thanks, Carson From: Mr Thomas Schumann Date: Tuesday, 17 July, 2012 1:49 AM To: "maker-devel at yandell-lab.org" Cc: "c.chan10 at uq.edu.au" Subject: [maker-devel] maker and repeatproteinmask Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Wed Jul 18 16:03:10 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Wed, 18 Jul 2012 22:03:10 +0000 Subject: [maker-devel] maker and repeatproteinmask In-Reply-To: References: , Message-ID: Ok thanks. I looked through all the RepeatMasker source files and it doesn't look like anything calls RepeatProteinMask, and RepeatProteinMask is the only program which calls trf, so it looks like MAKER doesn't depend on trf at all then. Thanks, Tom Schumann ________________________________ From: Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 18 July 2012 7:05 AM To: Mr Thomas Schumann; maker-devel at yandell-lab.org Cc: c.chan10 at uq.edu.au Subject: Re: [maker-devel] maker and repeatproteinmask MAKER calls the 'RepeatMasker' script and is thus dependent on anything RepeatMasker decides to call internally. MAKER will never call TRF or RepeatProteinMask directly, but I can't say whether or not RepeatMasker will always do so as part of it's normal processing. The only way to know for sure is to try. I removed TRF from my installation and RepeatMasker shows as completing successfully, so I imagine you would probably be fine without it. Thanks, Carson From: Mr Thomas Schumann > Date: Tuesday, 17 July, 2012 1:49 AM To: "maker-devel at yandell-lab.org" > Cc: "c.chan10 at uq.edu.au" > Subject: [maker-devel] maker and repeatproteinmask Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Jul 23 12:45:59 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 23 Jul 2012 14:45:59 -0400 Subject: [maker-devel] Transcriptome assembly annotation? In-Reply-To: <8AC559A1-AAD1-4090-B48F-0346DF6A7524@illinois.edu> Message-ID: Never tried it, but I think I would probably just skip maker and perhaps do something like run cufflinks or trinity followed by intperproscan and blast2go. There is a wrapper for interproscan packaged with MAKER that might be useful. --Carson On 12-07-06 12:19 AM, "Fields, Christopher J" wrote: >Just curious, but I haven't found the answer to this in the mail list. >Has anyone used MAKER for transcriptome annotation? Obviously the ab >initio gene predictors wouldn't make much sense, but I could see a tool >like ESTScan or similar acting in a similar capacity. > >chris > >Chris Fields >Technical Lead in Genome Informatics >High Performance Computing in Biology >W.M. Keck Center for Comparative and Functional Genomics >Institute for Genomic Biology > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Jul 24 08:20:43 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 24 Jul 2012 10:20:43 -0400 Subject: [maker-devel] MAKER 2.26 Message-ID: FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Tue Jul 24 16:06:11 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Tue, 24 Jul 2012 22:06:11 +0000 Subject: [maker-devel] MAKER 2.26 In-Reply-To: References: Message-ID: Hi Carson, Where can I download it? The Yandell Lab site still only lists v2.10 and v2.26 beta. Thanks, Tom Schumann ________________________________ From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 25 July 2012 12:20 AM To: maker-devel at yandell-lab.org Subject: [maker-devel] MAKER 2.26 FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 24 17:10:21 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 24 Jul 2012 19:10:21 -0400 Subject: [maker-devel] MAKER 2.26 In-Reply-To: Message-ID: That's the one. The beta tag will remain for a short time. The previous 2.25 version was also labelled with the beta tag. Thanks, Carson From: Mr Thomas Schumann Date: Tuesday, 24 July, 2012 6:06 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: RE: MAKER 2.26 Hi Carson, Where can I download it? The Yandell Lab site still only lists v2.10 and v2.26 beta. Thanks, Tom Schumann From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 25 July 2012 12:20 AM To: maker-devel at yandell-lab.org Subject: [maker-devel] MAKER 2.26 FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From guoyunfei1989 at gmail.com Wed Jul 25 13:15:59 2012 From: guoyunfei1989 at gmail.com (Yunfei Guo) Date: Wed, 25 Jul 2012 12:15:59 -0700 Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) Message-ID: Hi everyone, I ran maker2.25 without a problem, but with maker2.26, I encountered the following error after running it for ~8 hr with 2 nodes and 24 cpus, do you have any idea what's going on here? Some contigs did get finished, maybe this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system is SGE. Thanks! running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/scaffold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx #-------------------------------# deleted:-1 hits SIGCHLD handler "DEFAULT" not defined. SIGCHLD handler "DEFAULT" not defined. running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 #-------------------------------# Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fffa3a2e740) failed dequeue_and_set_error(596): Communication error with rank 18 running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/scaffold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx #-------------------------------# running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/scaffold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1.tblastx #-------------------------------# running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 #-------------------------------# cleaning blastx... in cluster::shadow_cluster... ...finished clustering. cleaning clusters.... total clusters:1 now processing 0 ...processing 0 of 2 deleted:0 hits ...processing 1 of 2 running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/scaffold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6.tblastx #-------------------------------# deleted:0 hits running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr%7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.9.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/ scaffold2590.1-3712.tr%7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 #-------------------------------# deleted:0 hits cleaning blastx... cleaning clusters.... total clusters:1 now processing 0 cleaning clusters.... total clusters:1 now processing 0 deleted:-1 hits deleted:-1 hits deleted:-6 hits deleted:-3 hits deleted:-2 hits Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Yunfei -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Jul 25 13:46:17 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 25 Jul 2012 15:46:17 -0400 Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) In-Reply-To: Message-ID: MPI is notorious for unexplicable communication errors, so first I would suggest just restarting and seeing if it happens again (MAKER will pick up where it left off on restart, so no need to alter settings or files). If it happens again, we can look into it, but no component of the MPI communication framework changed between 2.25 and 2.26 (100% identical), so my first instinct is that this was just what the message said, a"Communication error with rank 18". If it happens again I can try and add some extra messages so we can see the hostname of rank 18. That way we can identify if it's constantly a specific node on your cluster. Let me know if you see it again. Thanks, Carson From: Yunfei Guo Date: Wednesday, 25 July, 2012 3:15 PM To: Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) Hi everyone, I ran maker2.25 without a problem, but with maker2.26, I encountered the following error after running it for ~8 hr with 2 nodes and 24 cpus, do you have any idea what's going on here? Some contigs did get finished, maybe this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system is SGE. Thanks! running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/sc affold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx #-------------------------------# deleted:-1 hits SIGCHLD handler "DEFAULT" not defined. SIGCHLD handler "DEFAULT" not defined. running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp %7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 #-------------------------------# Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fffa3a2e740) failed dequeue_and_set_error(596): Communication error with rank 18 running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/sc affold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx #-------------------------------# running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/sc affold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_ Rubri%2Efasta.mpi.10.1.tblastx #-------------------------------# running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp %7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 #-------------------------------# cleaning blastx... in cluster::shadow_cluster... ...finished clustering. cleaning clusters.... total clusters:1 now processing 0 ...processing 0 of 2 deleted:0 hits ...processing 1 of 2 running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/sc affold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_ Rubri%2Efasta.mpi.10.6.tblastx #-------------------------------# deleted:0 hits running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr %7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.9.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.tr %7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 #-------------------------------# deleted:0 hits cleaning blastx... cleaning clusters.... total clusters:1 now processing 0 cleaning clusters.... total clusters:1 now processing 0 deleted:-1 hits deleted:-1 hits deleted:-6 hits deleted:-3 hits deleted:-2 hits Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Yunfei _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From guoyunfei1989 at gmail.com Thu Jul 26 09:10:42 2012 From: guoyunfei1989 at gmail.com (Yunfei Guo) Date: Thu, 26 Jul 2012 08:10:42 -0700 Subject: [maker-devel] Fwd: ERROR: MPI_Recv(186), dequeue_and_set_error(596) In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: Yunfei Guo Date: Thu, Jul 26, 2012 at 8:10 AM Subject: Re: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) To: Carson Holt Hi Carson, same error occurred again. What should I do to check if it was caused by the same node? Also, if I ran maker on a single node instead of two nodes, will the same error appear again? Thank you. #-------------------------------# SIGCHLD handler "DEFAULT" not defined. Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fff1c3dd3b0, count=2, MPI_ INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fff1c3dd3 90) failed dequeue_and_set_error(596): Communication error with rank 21 running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro /run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datast ore/81/43/scaffold5780//theVoid.scaffold5780/tr%7CG3N4L5%7CG3N4L5_GASA C.for.6527-8832.2.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir _Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/81/4 3/scaffold5780//theVoid.scaffold5780/scaffold5780.6527-8832.2.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --sho wcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigr o-53k_part.maker.output/Nigro-53k_part_datastore/81/43/scaffold5780//t heVoid.scaffold5780/scaffold5780.6527-8832.tr%7CG3N4L5%7CG3N4L5_GASAC. p_exonerate.2 #-------------------------------# Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached ... Yunfei On Wed, Jul 25, 2012 at 1:43 PM, Yunfei Guo wrote: > Thanks, Carson. Actually I already set clean_try=1. > > > On Wed, Jul 25, 2012 at 1:34 PM, Carson Holt wrote: > >> That second error from 2.25, Seems to be thrown by Perl's Storable >> module. There may be some weird partial serialization of the data that >> occurred on the first failure and is now causing failures on retry. You >> can set clean_try=1 in MAKER, to let it wipe out data for a failed contig >> before retrying. That can sometimes help get around weird hard failures. >> >> Thanks, >> Carson >> >> >> From: Yunfei Guo >> Date: Wednesday, 25 July, 2012 4:26 PM >> To: Carson Holt >> Subject: Re: [maker-devel] ERROR: MPI_Recv(186), >> dequeue_and_set_error(596) >> >> Thank you, Carson. I'm rerunning maker2.26 now. I just tried maker2.25 as >> well, it failed this time, with similar errors below. I guess it might be >> caused by the cluster (or node) itself, like you said, because we just >> added a few nodes and more memories. I'll ask the admin to see whether he >> can explain this. >> #-------------------------------# >> Thread 1 terminated abnormally: >> ------------- EXCEPTION: Bio::Root::Exception ------------- >> MSG: no data for midline Sequence with id BL_ORD_ID:126195 no longer >> exists in database...alignment skipped >> STACK: Error::throw >> STACK: Bio::Root::Root::throw >> /home/yunfeiguo/perl5/lib/perl5/Bio/Root/Root.pm:472 >> STACK: Bio::SearchIO::blast::next_result >> /home/yunfeiguo/perl5/lib/perl5/Bio/SearchIO/blast.pm:1888 >> STACK: Widget::tblastx::keepers >> /home/yunfeiguo/Downloads/maker/bin/../lib/Widget/tblastx.pm:114 >> STACK: Widget::tblastx::parse >> /home/yunfeiguo/Downloads/maker/bin/../lib/Widget/tblastx.pm:95 >> STACK: GI::tblastx_as_chunks >> /home/yunfeiguo/Downloads/maker/bin/../lib/GI.pm:2612 >> STACK: Process::MpiChunk::_go >> /home/yunfeiguo/Downloads/maker/bin/../lib/Process/MpiChunk.pm:1829 >> STACK: Process::MpiChunk::run >> /home/yunfeiguo/Downloads/maker/bin/../lib/Process/MpiChunk.pm:331 >> STACK: main::node_thread /home/yunfeiguo/Downloads/maker/bin/maker:1308 >> STACK: threads::new >> /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/forks.pm:799 >> STACK: /home/yunfeiguo/Downloads/maker/bin/maker:804 >> ----------------------------------------------------------- >> Cannot restore overloading on HASH(0x1b7f9b60) (package >> Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at >> /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/Storable.pm line >> 416, at /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/ >> forks.pm line 2256. >> Compilation failed in require at >> /home/yunfeiguo/Downloads/maker/bin/maker line 11. >> BEGIN failed--compilation aborted at >> /home/yunfeiguo/Downloads/maker/bin/maker line 11. >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> deleted:0 hits >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff05c78630, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff05c78610) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1414fb00, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1414fae0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff25d86c00, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff25d86be0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> cleaning tblastx... >> cleaning clusters.... >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1b71f1f0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1b71f1d0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffc99d29c0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffc99d29a0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffc4aaf720, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffc4aaf700) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> in cluster::shadow_cluster... >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff317862a0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff31786280) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> ...finished clustering. >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff8abb8e50, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff8abb8e30) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1d1ff180, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1d1ff160) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff4d865850, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff4d865830) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffbec98150, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffbec98130) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffa4ead990, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffa4ead970) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> >> On Wed, Jul 25, 2012 at 12:46 PM, Carson Holt wrote: >> >>> MPI is notorious for unexplicable communication errors, so first I would >>> suggest just restarting and seeing if it happens again (MAKER will pick up >>> where it left off on restart, so no need to alter settings or files). >>> >>> If it happens again, we can look into it, but no component of the MPI >>> communication framework changed between 2.25 and 2.26 (100% identical), so >>> my first instinct is that this was just what the message said, >>> a"Communication error with rank 18". If it happens again I can try and add >>> some extra messages so we can see the hostname of rank 18. That way we can >>> identify if it's constantly a specific node on your cluster. >>> >>> Let me know if you see it again. >>> >>> Thanks, >>> Carson >>> >>> >>> >>> From: Yunfei Guo >>> Date: Wednesday, 25 July, 2012 3:15 PM >>> To: >>> Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) >>> >>> Hi everyone, >>> >>> I ran maker2.25 without a problem, but with maker2.26, I encountered the >>> following error after running it for ~8 hr with 2 nodes and 24 cpus, do you >>> have any idea what's going on here? Some contigs did get finished, maybe >>> this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system >>> is SGE. Thanks! >>> >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query >>> /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 >>> -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/scaffold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx >>> #-------------------------------# >>> deleted:-1 hits >>> SIGCHLD handler "DEFAULT" not defined. >>> SIGCHLD handler "DEFAULT" not defined. >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 >>> #-------------------------------# >>> Fatal error in MPI_Recv: Other MPI error, error stack: >>> MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, >>> MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, >>> status=0x7fffa3a2e740) failed >>> dequeue_and_set_error(596): Communication error with rank 18 >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query >>> /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 >>> -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/scaffold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::tblastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 >>> -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments >>> 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp >>> 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true >>> -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/scaffold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1.tblastx >>> #-------------------------------# >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 >>> #-------------------------------# >>> cleaning blastx... >>> in cluster::shadow_cluster... >>> ...finished clustering. >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> ...processing 0 of 2 >>> deleted:0 hits >>> ...processing 1 of 2 >>> running blast search. >>> #--------- command -------------# >>> Widget::tblastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 >>> -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments >>> 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp >>> 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true >>> -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/scaffold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6.tblastx >>> #-------------------------------# >>> deleted:0 hits >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr%7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.9.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/ >>> scaffold2590.1-3712.tr%7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 >>> #-------------------------------# >>> deleted:0 hits >>> cleaning blastx... >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> deleted:-1 hits >>> deleted:-1 hits >>> deleted:-6 hits >>> deleted:-3 hits >>> deleted:-2 hits >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> >>> Yunfei >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fourie.joubert at up.ac.za Fri Jul 27 08:30:33 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Fri, 27 Jul 2012 16:30:33 +0200 Subject: [maker-devel] Passing extra options to augustus in maker? Message-ID: <5012A609.7060101@up.ac.za> Hi Could anyone advise me regarding passing additional options to augustus when running maker? I have generated a series of hints files (http://bioinf.uni-greifswald.de/bioinf/wiki/pmwiki.php?n=Augustus.IncorporateProteins) that work fine when running augustus stand-alone, but would like to be able to pass the necessary parameters to augustus when running maker so that augustus uses the hints files. Alternatively, if I need to run augustus stand-alone with the hints files, which option should I use to load the resulting augustus gff files into maker (protein_gff, other_gff, ...)? Thanks and regards! Fourie -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Fri Jul 27 10:34:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 27 Jul 2012 12:34:30 -0400 Subject: [maker-devel] Passing extra options to augustus in maker? In-Reply-To: <5012A609.7060101@up.ac.za> Message-ID: You can't pass externally generated augustus hint files into maker. But if you do decide to run augustus alone using the hint file you just generated, you can then pass the results in to pred_gff (after any necessary formatting fixes for correct GFF3). The results of running exonerate2hints.pl are essentially the same as letting MAKER generate its own hints file because maker is using exonerate to generate the alignment. So the effect of an external hint file would be essentially redundant. When running augustus in MAKER, it is ran a minimum of 2 times. The first time it is run without hints and only using the training parameters from the augustus species files. The second time MAKER internally generates hints based on exonerate alignments and then runs augustus using those hints. If you also pass in your external run, you would then have 3 augustus runs on the genome with MAKER selecting the model from the pool of runs that best matches the evidence alignments. Thanks, Carson On 12-07-27 10:30 AM, "Fourie Joubert" wrote: >Hi > >Could anyone advise me regarding passing additional options to augustus >when running maker? > >I have generated a series of hints files >(http://bioinf.uni-greifswald.de/bioinf/wiki/pmwiki.php?n=Augustus.Incorpo >rateProteins) >that work fine when running augustus stand-alone, but would like to be >able to pass the necessary parameters to augustus when running maker so >that augustus uses the hints files. > >Alternatively, if I need to run augustus stand-alone with the hints >files, which option should I use to load the resulting augustus gff >files into maker (protein_gff, other_gff, ...)? > >Thanks and regards! > >Fourie > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Fri Jul 27 10:38:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 27 Jul 2012 12:38:56 -0400 Subject: [maker-devel] Advice for optimizing augustus training on fungal genome? In-Reply-To: <4FEC7423.3050001@up.ac.za> Message-ID: Regarding this older post, if you aren't getting good results using CEGMA for training augustus for a new species, then One option would be just to make a copy Neurospora's species directory. Edit the necessary file contents to make it list as a different species, then run the augustus training steps as before but this use the Neurospora copy as the base so augustus will be optimizing Neurospora's parameters to be more like your species of interest. Thanks, Carson On 12-06-28 11:11 AM, "Fourie Joubert" wrote: >Hi Everyone > >Apologies if this is not the relevant list to mail to. > >I am looking for advice in training augustus for a novel fungal genome. > >I generated a gene set using CEGMA (below), and have subsequently been >following the instructions at >http://www.molecularevolution.org/molevolfiles/exercises/augustus/scipio.h >tml >and at >http://www.molecularevolution.org/molevolfiles/exercises/augustus/training >.html. > >My training set is 339 genes and the test set is 100 genes. > >My initial output is below. > >It does not improve much with optimize_augustus. > >When using the training paramters to predict genes in the genome, I seem >to only find around 2,000 of the known ~16,000 genes. When I use the >training data from a distantly related fungus (Neurospora), I get >roughly the correct number of genes. > >I am obviously doing something wrong here... (commands below). > >I would really appreciate any advice on where to start looking for >improvement. > >Kindest regards! > >Fourie > > > > > >Augustus commands (Editedmyspecies_parameters.cfg and >setstopCodonExcludedFromCDS to true.): > >> etraining --species=myspecies genes.gb.train > >> augustus --species=myspecies genes.gb.test | tee firsttest.out > >> grep -A 22 Evaluation firsttest.out > >> optimize_augustus.pl --species=myspecies genes.gb.train > >> etraining --species=myspecies genes.gb.train > >> augustus --species=myspecies genes.gb.test | tee secondtest.out > >> grep -A 22 Evaluation secondtest.out > > > >CEGMA output: > ># Statistics of the completeness of the genome based on 248 CEGs > # > > #Prots %Completeness - #Total Average %Ortho > > Complete 240 96.77 - 278 1.16 11.67 > > Group 1 64 96.97 - 72 1.12 7.81 > Group 2 54 96.43 - 66 1.22 18.52 > Group 3 58 95.08 - 70 1.21 13.79 > Group 4 64 98.46 - 70 1.09 7.81 > > Partial 245 98.79 - 290 1.18 13.88 > > Group 1 65 98.48 - 73 1.12 7.69 > Group 2 56 100.00 - 70 1.25 21.43 > Group 3 59 96.72 - 75 1.27 18.64 > Group 4 65 100.00 - 72 1.11 9.23 > > > > >Augustus output: > >******* Evaluation of gene prediction ******* > >---------------------------------------------\ > > | sensitivity | specificity | > >---------------------------------------------| > >nucleotide level | 0.933 | 0.772 | > >---------------------------------------------/ > >-------------------------------------------------------------------------- >--------------------------------\ > > | #pred | #anno | | FP = false pos. | FN = false >neg. | | | > > | total/ | total/ | TP >|--------------------|--------------------| sensitivity | specificity | > > | unique | unique | | part | ovlp | wrng | part | ovlp | >wrng | | | > >-------------------------------------------------------------------------- >--------------------------------| > > | | | | 229 | > 85 | | | > >exon level | 475 | 331 | 246 | ------------------ | >------------------ | 0.743 | 0.518 | > > | 475 | 331 | | 59 | 9 | 161 | 56 | 2 | > 27 | | | > >-------------------------------------------------------------------------- >--------------------------------/ > >-------------------------------------------------------------------------- >--\ > >transcript | #pred | #anno | TP | FP | FN | sensitivity | >specificity | > >-------------------------------------------------------------------------- >--| > >gene level | 158 | 100 | 45 | 113 | 55 | 0.45 | >0.285 | > >-------------------------------------------------------------------------- >--/ > > > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From jason.stajich at gmail.com Fri Jul 27 16:52:07 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Fri, 27 Jul 2012 15:52:07 -0700 Subject: [maker-devel] Advice for optimizing augustus training on fungal genome? In-Reply-To: References: Message-ID: <4457959E-443D-4A17-8EC6-5D16FD58B2C7@gmail.com> Best option is to get RNA-Seq or some transcript evidence of course so you have something empirical. I'm really surprised you are getting such a low count - this is just from augustus running on the cmdline? Alternatively if you have 16,000 known proteins, derived from protein alignments of related species aligned to your genome I expect? You could also try and generate a best set of gene predictions from this. When I did this for Coprinus I gathered full length alignments of fungal proteins and used at the time genomewise and genewise to get the best set of loci with gene calls. Today I'd probably just use FASTA to find the locus and exonerate to get the spliced gene model out and provide these as training. Basically this is doing a fungal-specific CEGMA, something we ought to build, but just don't have time to do. Do you get similar results when running genemarkHMM with its selftraining? jason On Jul 27, 2012, at 9:38 AM, Carson Holt wrote: > Regarding this older post, if you aren't getting good results using CEGMA > for training augustus for a new species, then One option would be just to > make a copy Neurospora's species directory. Edit the necessary file > contents to make it list as a different species, then run the augustus > training steps as before but this use the Neurospora copy as the base so > augustus will be optimizing Neurospora's parameters to be more like your > species of interest. > > Thanks, > Carson > > > > On 12-06-28 11:11 AM, "Fourie Joubert" wrote: > >> Hi Everyone >> >> Apologies if this is not the relevant list to mail to. >> >> I am looking for advice in training augustus for a novel fungal genome. >> >> I generated a gene set using CEGMA (below), and have subsequently been >> following the instructions at >> http://www.molecularevolution.org/molevolfiles/exercises/augustus/scipio.h >> tml >> and at >> http://www.molecularevolution.org/molevolfiles/exercises/augustus/training >> .html. >> >> My training set is 339 genes and the test set is 100 genes. >> >> My initial output is below. >> >> It does not improve much with optimize_augustus. >> >> When using the training paramters to predict genes in the genome, I seem >> to only find around 2,000 of the known ~16,000 genes. When I use the >> training data from a distantly related fungus (Neurospora), I get >> roughly the correct number of genes. >> >> I am obviously doing something wrong here... (commands below). >> >> I would really appreciate any advice on where to start looking for >> improvement. >> >> Kindest regards! >> >> Fourie >> >> >> >> >> >> Augustus commands (Editedmyspecies_parameters.cfg and >> setstopCodonExcludedFromCDS to true.): >> >>> etraining --species=myspecies genes.gb.train >> >>> augustus --species=myspecies genes.gb.test | tee firsttest.out >> >>> grep -A 22 Evaluation firsttest.out >> >>> optimize_augustus.pl --species=myspecies genes.gb.train >> >>> etraining --species=myspecies genes.gb.train >> >>> augustus --species=myspecies genes.gb.test | tee secondtest.out >> >>> grep -A 22 Evaluation secondtest.out >> >> >> >> CEGMA output: >> >> # Statistics of the completeness of the genome based on 248 CEGs >> # >> >> #Prots %Completeness - #Total Average %Ortho >> >> Complete 240 96.77 - 278 1.16 11.67 >> >> Group 1 64 96.97 - 72 1.12 7.81 >> Group 2 54 96.43 - 66 1.22 18.52 >> Group 3 58 95.08 - 70 1.21 13.79 >> Group 4 64 98.46 - 70 1.09 7.81 >> >> Partial 245 98.79 - 290 1.18 13.88 >> >> Group 1 65 98.48 - 73 1.12 7.69 >> Group 2 56 100.00 - 70 1.25 21.43 >> Group 3 59 96.72 - 75 1.27 18.64 >> Group 4 65 100.00 - 72 1.11 9.23 >> >> >> >> >> Augustus output: >> >> ******* Evaluation of gene prediction ******* >> >> ---------------------------------------------\ >> >> | sensitivity | specificity | >> >> ---------------------------------------------| >> >> nucleotide level | 0.933 | 0.772 | >> >> ---------------------------------------------/ >> >> -------------------------------------------------------------------------- >> --------------------------------\ >> >> | #pred | #anno | | FP = false pos. | FN = false >> neg. | | | >> >> | total/ | total/ | TP >> |--------------------|--------------------| sensitivity | specificity | >> >> | unique | unique | | part | ovlp | wrng | part | ovlp | >> wrng | | | >> >> -------------------------------------------------------------------------- >> --------------------------------| >> >> | | | | 229 | >> 85 | | | >> >> exon level | 475 | 331 | 246 | ------------------ | >> ------------------ | 0.743 | 0.518 | >> >> | 475 | 331 | | 59 | 9 | 161 | 56 | 2 | >> 27 | | | >> >> -------------------------------------------------------------------------- >> --------------------------------/ >> >> -------------------------------------------------------------------------- >> --\ >> >> transcript | #pred | #anno | TP | FP | FN | sensitivity | >> specificity | >> >> -------------------------------------------------------------------------- >> --| >> >> gene level | 158 | 100 | 45 | 113 | 55 | 0.45 | >> 0.285 | >> >> -------------------------------------------------------------------------- >> --/ >> >> >> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org From sbrubaker at solazyme.com Mon Jul 30 17:52:57 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Mon, 30 Jul 2012 23:52:57 +0000 Subject: [maker-devel] CDS file? Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 31 07:01:07 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 31 Jul 2012 09:01:07 -0400 Subject: [maker-devel] CDS file? In-Reply-To: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 11:16:45 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 17:16:45 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE361@EXCHANGE-05.internal.solazyme.com> Thanks! From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 11:17:19 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 17:17:19 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com> One more question - do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 31 11:58:51 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 31 Jul 2012 13:58:51 -0400 Subject: [maker-devel] CDS file? In-Reply-To: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com> Message-ID: Not really, but the functional annotation tools included with maker might be useful for some of the downstream work. Although I guess there is a way I could fool maker's direct protein2genome gene creation into generating gene models with a little code modification. --Carson From: Shane Brubaker Date: Tuesday, 31 July, 2012 1:17 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: RE: [maker-devel] CDS file? One more question ? do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 13:13:55 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 19:13:55 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com>, Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE4C4@EXCHANGE-05.internal.solazyme.com> Got it, thanks for your help! ________________________________________ From: Carson Holt [carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 10:58 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Not really, but the functional annotation tools included with maker might be useful for some of the downstream work. Although I guess there is a way I could fool maker's direct protein2genome gene creation into generating gene models with a little code modification. --Carson From: Shane Brubaker > Date: Tuesday, 31 July, 2012 1:17 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: RE: [maker-devel] CDS file? One more question ? do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Jul 3 08:12:57 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 03 Jul 2012 10:12:57 -0400 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: <4FEC6428.3010108@stud-mail.uni-wuerzburg.de> Message-ID: Have you tried 2.25 to see if the error still occurs in that version? Thanks, Carson On 12-06-28 10:03 AM, "Markus Ankenbrand" wrote: >Hi all, > >I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >only using coding sequences of a reference species as "altest". Without >any obvious reason annotation fails for some contigs. It seems to have >nothing to do with the contig length. I also tried to run maker with >different reference ESTs and GeneMark. Depending on the setup different >contigs fail. > >In case of failure, the log is: >... >cleaning clusters.... >total clusters:3 now processing 0 > ...processing 0 of 3 > ...processing 1 of 3 >total clusters:3 now processing 0 >total clusters:3 now processing 0 > ...processing 0 of 2 >Can't call method "start" on an undefined valueERROR: Failed while >preparing evidence clusters for annotations >ERROR: Chunk failed at level:0, tier_type:3 >FAILED CONTIG:contig1 > >ERROR: Chunk failed at level:7, tier_type:0 >FAILED CONTIG:contig1 > >--Next Contig-- >... > >If someone has any idea how to solve this problem, help would be very >much apriciated. > >Greetings, >Markus > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Jul 3 11:27:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 03 Jul 2012 13:27:56 -0400 Subject: [maker-devel] maker leaving large numbers of defunct zombies In-Reply-To: <68CF9F49-540C-4018-A849-1D1B5751943B@slu.se> Message-ID: MAKER doesn't lock the SQL database except on initial creation and population. However, SQLite does it's own advisory locking apart from MAKER (and it may be trying to do this with every transaction). This could be problematic on some NFS systems where advisory locking may be Buggy. I'm going to try something. Just update the SVN repository one more time. What I've done is to add a check to see if you are running in an NFS mounted directory. If so, maker will try and localize the SQLite db file to /tmp or wherever your TMP variable is set to in the maker_opts.ctl file (maker will check the NFS status on that as well before attempting the copy so as not to waste time in just copying to another NFS). So if you set TMP or your linux TMPDIR variable is set to an NFS mounted locations, try changing it to a location that will be local to each machine. Usually /tmp (Linux default) or you can use the memory mounted virtual directory /dev/shm. Maker will clean up temporary files it creates afterwards. Automatically. By using a local rather than NFS location in this step, advisory locks should work properly for SQLite (if that is the issue you are experiencing). I've added checks and optimizations so the file will not be copied if it has already been localized to a given machine, even if it was by another node in the MPI job. So the overall space overhead should be limited, and you'll probably even get a slight performance boost as well (local SQLite databases run much faster than NFS located ones). Thanks, Carson On 12-06-28 4:16 PM, "Mikael Brandstr?m Durling" wrote: >Sorry, but I might have been slightly to quick to my conclusions. All the >initialisation and initial database creation works fine. However, when >running I start to see locking error from SQLite. Is GFFDB supposed to >use the NFSLock to lock the database at all accesses? After running for a >while now, I see locking errors (ie the query or transaction failes) from >SQLite at these lines in GFFDB.pm: 107/109, 529 and 678. > >Mikael From markus.ankenbrand at stud-mail.uni-wuerzburg.de Thu Jul 5 05:22:15 2012 From: markus.ankenbrand at stud-mail.uni-wuerzburg.de (Markus Ankenbrand) Date: Thu, 05 Jul 2012 13:22:15 +0200 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: References: Message-ID: <4FF578E7.7060700@stud-mail.uni-wuerzburg.de> Hi Carson, thanks for your response. Version 2.25 works without any errors. May I have a changelog for 2.25, if available? Thanks, Markus Am 03.07.2012 16:12, schrieb Carson Holt: > Have you tried 2.25 to see if the error still occurs in that version? > > Thanks, > Carson > > > > On 12-06-28 10:03 AM, "Markus Ankenbrand" > wrote: > >> Hi all, >> >> I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >> only using coding sequences of a reference species as "altest". Without >> any obvious reason annotation fails for some contigs. It seems to have >> nothing to do with the contig length. I also tried to run maker with >> different reference ESTs and GeneMark. Depending on the setup different >> contigs fail. >> >> In case of failure, the log is: >> ... >> cleaning clusters.... >> total clusters:3 now processing 0 >> ...processing 0 of 3 >> ...processing 1 of 3 >> total clusters:3 now processing 0 >> total clusters:3 now processing 0 >> ...processing 0 of 2 >> Can't call method "start" on an undefined valueERROR: Failed while >> preparing evidence clusters for annotations >> ERROR: Chunk failed at level:0, tier_type:3 >> FAILED CONTIG:contig1 >> >> ERROR: Chunk failed at level:7, tier_type:0 >> FAILED CONTIG:contig1 >> >> --Next Contig-- >> ... >> >> If someone has any idea how to solve this problem, help would be very >> much apriciated. >> >> Greetings, >> Markus >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Thu Jul 5 08:00:54 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 10:00:54 -0400 Subject: [maker-devel] maker-2.24 fails while preparing evidence clusters In-Reply-To: <4FF578E7.7060700@stud-mail.uni-wuerzburg.de> Message-ID: Here are comments from subversion between 2.24 and 2.25 Renaming and minor fixes to some modules improved gff3_merge script fix est passthrough bug made GFFDB module faster Thanks, Carson On 12-07-05 7:22 AM, "Markus Ankenbrand" wrote: >Hi Carson, > >thanks for your response. >Version 2.25 works without any errors. >May I have a changelog for 2.25, if available? > >Thanks, >Markus > > >Am 03.07.2012 16:12, schrieb Carson Holt: >> Have you tried 2.25 to see if the error still occurs in that version? >> >> Thanks, >> Carson >> >> >> >> On 12-06-28 10:03 AM, "Markus Ankenbrand" >> wrote: >> >>> Hi all, >>> >>> I'm using maker-2.24 to annotate contigs of a procaryotic organism. I'm >>> only using coding sequences of a reference species as "altest". Without >>> any obvious reason annotation fails for some contigs. It seems to have >>> nothing to do with the contig length. I also tried to run maker with >>> different reference ESTs and GeneMark. Depending on the setup different >>> contigs fail. >>> >>> In case of failure, the log is: >>> ... >>> cleaning clusters.... >>> total clusters:3 now processing 0 >>> ...processing 0 of 3 >>> ...processing 1 of 3 >>> total clusters:3 now processing 0 >>> total clusters:3 now processing 0 >>> ...processing 0 of 2 >>> Can't call method "start" on an undefined valueERROR: Failed while >>> preparing evidence clusters for annotations >>> ERROR: Chunk failed at level:0, tier_type:3 >>> FAILED CONTIG:contig1 >>> >>> ERROR: Chunk failed at level:7, tier_type:0 >>> FAILED CONTIG:contig1 >>> >>> --Next Contig-- >>> ... >>> >>> If someone has any idea how to solve this problem, help would be very >>> much apriciated. >>> >>> Greetings, >>> Markus >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > From chrisi.hahni at gmail.com Thu Jul 5 09:06:38 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Thu, 05 Jul 2012 17:06:38 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap Message-ID: <4FF5AD7E.7010805@gmail.com> Dear MAKER developers and users, I am new to MAKER and during my first attempts I unfortunately encountert some problems. It would be great if anyone could help me out! Here's my problem: I have used gm_es.pl from Genemark to create es.mod for my assembly. I would like to give this model to MAKER, so I add the path to es.mod in the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I am running maker it cannot find the gmhmm_wrap script. Here s what it says: #-------------------------------# Can't open perl script "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wrap": No such file or directory ERROR: Genemark failed Apparently it is looking for the script in the wrong place. I have found it in: /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap The other paths are correct. Is there a way to manually tell Maker where to look for this script? I should mention also that I have maker (2.10) running on a cluster. Any help is highly appreciated! Thanks in advance!! much obliged, Christoph University of Oslo From carsonhh at gmail.com Thu Jul 5 10:18:45 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 12:18:45 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF5AD7E.7010805@gmail.com> Message-ID: Your cluster is apparently using the CPAN module local::lib to define where perl libraries and executables get installed, this is overriding MAKER's own installation parameters, thus putting things in the wrong place. This only happens in 2.10, the 2.25 version of MAKER knows how to keep local::lib from messing with it's relative installation paths. So install 2.25 and make sure to remove the mis-located MAKER executables in /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. Thanks, Carson On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >Dear MAKER developers and users, > >I am new to MAKER and during my first attempts I unfortunately >encountert some problems. It would be great if anyone could help me out! >Here's my problem: >I have used gm_es.pl from Genemark to create es.mod for my assembly. I >would like to give this model to MAKER, so I add the path to es.mod in >the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >am running maker it cannot find the gmhmm_wrap script. Here s what it >says: > >#-------------------------------# >Can't open perl script >"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wr >ap": >No such file or directory >ERROR: Genemark failed > >Apparently it is looking for the script in the wrong place. I have found >it in: >/site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >The other paths are correct. > >Is there a way to manually tell Maker where to look for this script? I >should mention also that I have maker (2.10) running on a cluster. > >Any help is highly appreciated! Thanks in advance!! > >much obliged, >Christoph > >University of Oslo > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From chrisi.hahni at gmail.com Thu Jul 5 10:55:19 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Thu, 05 Jul 2012 18:55:19 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: References: Message-ID: <4FF5C6F7.3090601@gmail.com> Dear Carson, Thanks for your reply! I have already requested the installation of maker 2.25 from the cluster admin, but unfortunately this will take a while. I guess there is no way for me to fix that for the moment in maker 2.10?? Thanks again! cheers, Christoph On 07/05/2012 06:18 PM, Carson Holt wrote: > Your cluster is apparently using the CPAN module local::lib to define > where perl libraries and executables get installed, this is overriding > MAKER's own installation parameters, thus putting things in the wrong > place. This only happens in 2.10, the 2.25 version of MAKER knows how to > keep local::lib from messing with it's relative installation paths. So > install 2.25 and make sure to remove the mis-located MAKER executables in > /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. > > Thanks, > Carson > > > > On 12-07-05 11:06 AM, "Christoph Hahn" wrote: > >> Dear MAKER developers and users, >> >> I am new to MAKER and during my first attempts I unfortunately >> encountert some problems. It would be great if anyone could help me out! >> Here's my problem: >> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >> would like to give this model to MAKER, so I add the path to es.mod in >> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >> am running maker it cannot find the gmhmm_wrap script. Here s what it >> says: >> >> #-------------------------------# >> Can't open perl script >> "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_wr >> ap": >> No such file or directory >> ERROR: Genemark failed >> >> Apparently it is looking for the script in the wrong place. I have found >> it in: >> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >> The other paths are correct. >> >> Is there a way to manually tell Maker where to look for this script? I >> should mention also that I have maker (2.10) running on a cluster. >> >> Any help is highly appreciated! Thanks in advance!! >> >> much obliged, >> Christoph >> >> University of Oslo >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > From carsonhh at gmail.com Thu Jul 5 11:00:23 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 05 Jul 2012 13:00:23 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF5C6F7.3090601@gmail.com> Message-ID: You could create the directory maker/bin for the current installation and move all executables there from the /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the executable installed in the maker/bin directory. MAKER is expecting to go back one directory to find libraries and scripts in the MAKER base directory i.e. ../lib/Widget/genemark/gmhmm_wrap Thanks, Carson On 12-07-05 12:55 PM, "Christoph Hahn" wrote: >Dear Carson, > >Thanks for your reply! I have already requested the installation of >maker 2.25 from the cluster admin, but unfortunately this will take a >while. I guess there is no way for me to fix that for the moment in >maker 2.10?? > >Thanks again! >cheers, >Christoph > >On 07/05/2012 06:18 PM, Carson Holt wrote: >> Your cluster is apparently using the CPAN module local::lib to define >> where perl libraries and executables get installed, this is overriding >> MAKER's own installation parameters, thus putting things in the wrong >> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>to >> keep local::lib from messing with it's relative installation paths. So >> install 2.25 and make sure to remove the mis-located MAKER executables >>in >> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >> >> Thanks, >> Carson >> >> >> >> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >> >>> Dear MAKER developers and users, >>> >>> I am new to MAKER and during my first attempts I unfortunately >>> encountert some problems. It would be great if anyone could help me >>>out! >>> Here's my problem: >>> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >>> would like to give this model to MAKER, so I add the path to es.mod in >>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>> says: >>> >>> #-------------------------------# >>> Can't open perl script >>> >>>"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_ >>>wr >>> ap": >>> No such file or directory >>> ERROR: Genemark failed >>> >>> Apparently it is looking for the script in the wrong place. I have >>>found >>> it in: >>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>> The other paths are correct. >>> >>> Is there a way to manually tell Maker where to look for this script? I >>> should mention also that I have maker (2.10) running on a cluster. >>> >>> Any help is highly appreciated! Thanks in advance!! >>> >>> much obliged, >>> Christoph >>> >>> University of Oslo >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > From cjfields at illinois.edu Thu Jul 5 22:19:59 2012 From: cjfields at illinois.edu (Fields, Christopher J) Date: Fri, 6 Jul 2012 04:19:59 +0000 Subject: [maker-devel] Transcriptome assembly annotation? Message-ID: <8AC559A1-AAD1-4090-B48F-0346DF6A7524@illinois.edu> Just curious, but I haven't found the answer to this in the mail list. Has anyone used MAKER for transcriptome annotation? Obviously the ab initio gene predictors wouldn't make much sense, but I could see a tool like ESTScan or similar acting in a similar capacity. chris Chris Fields Technical Lead in Genome Informatics High Performance Computing in Biology W.M. Keck Center for Comparative and Functional Genomics Institute for Genomic Biology From chrisi.hahni at gmail.com Fri Jul 6 10:03:39 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Fri, 06 Jul 2012 18:03:39 +0200 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: References: Message-ID: <4FF70C5B.5000804@gmail.com> Dear Carson, I have installed MAKER 2.25 locally on the cluster now and as you have predicted the problem disappeared. Thanks again for your help!! cheers, Christoph On 07/05/2012 07:00 PM, Carson Holt wrote: > You could create the directory maker/bin for the current installation and > move all executables there from the > /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the > executable installed in the maker/bin directory. MAKER is expecting to go > back one directory to find libraries and scripts in the MAKER base > directory i.e. ../lib/Widget/genemark/gmhmm_wrap > > Thanks, > Carson > > > > On 12-07-05 12:55 PM, "Christoph Hahn" wrote: > >> Dear Carson, >> >> Thanks for your reply! I have already requested the installation of >> maker 2.25 from the cluster admin, but unfortunately this will take a >> while. I guess there is no way for me to fix that for the moment in >> maker 2.10?? >> >> Thanks again! >> cheers, >> Christoph >> >> On 07/05/2012 06:18 PM, Carson Holt wrote: >>> Your cluster is apparently using the CPAN module local::lib to define >>> where perl libraries and executables get installed, this is overriding >>> MAKER's own installation parameters, thus putting things in the wrong >>> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>> to >>> keep local::lib from messing with it's relative installation paths. So >>> install 2.25 and make sure to remove the mis-located MAKER executables >>> in >>> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >>> >>> Thanks, >>> Carson >>> >>> >>> >>> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >>> >>>> Dear MAKER developers and users, >>>> >>>> I am new to MAKER and during my first attempts I unfortunately >>>> encountert some problems. It would be great if anyone could help me >>>> out! >>>> Here's my problem: >>>> I have used gm_es.pl from Genemark to create es.mod for my assembly. I >>>> would like to give this model to MAKER, so I add the path to es.mod in >>>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, when I >>>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>>> says: >>>> >>>> #-------------------------------# >>>> Can't open perl script >>>> >>>> "/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhmm_ >>>> wr >>>> ap": >>>> No such file or directory >>>> ERROR: Genemark failed >>>> >>>> Apparently it is looking for the script in the wrong place. I have >>>> found >>>> it in: >>>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>>> The other paths are correct. >>>> >>>> Is there a way to manually tell Maker where to look for this script? I >>>> should mention also that I have maker (2.10) running on a cluster. >>>> >>>> Any help is highly appreciated! Thanks in advance!! >>>> >>>> much obliged, >>>> Christoph >>>> >>>> University of Oslo >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > From carsonhh at gmail.com Fri Jul 6 10:05:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 06 Jul 2012 12:05:25 -0400 Subject: [maker-devel] change the PATH to gmhmm_wrap In-Reply-To: <4FF70C5B.5000804@gmail.com> Message-ID: I'm glad it's working. Thanks, Carson On 12-07-06 12:03 PM, "Christoph Hahn" wrote: >Dear Carson, > >I have installed MAKER 2.25 locally on the cluster now and as you have >predicted the problem disappeared. >Thanks again for your help!! > >cheers, >Christoph > > >On 07/05/2012 07:00 PM, Carson Holt wrote: >> You could create the directory maker/bin for the current installation >>and >> move all executables there from the >> /xanadu/site/common/VERSIONS/perl-1.7/bin/ directory. Then run the >> executable installed in the maker/bin directory. MAKER is expecting to >>go >> back one directory to find libraries and scripts in the MAKER base >> directory i.e. ../lib/Widget/genemark/gmhmm_wrap >> >> Thanks, >> Carson >> >> >> >> On 12-07-05 12:55 PM, "Christoph Hahn" wrote: >> >>> Dear Carson, >>> >>> Thanks for your reply! I have already requested the installation of >>> maker 2.25 from the cluster admin, but unfortunately this will take a >>> while. I guess there is no way for me to fix that for the moment in >>> maker 2.10?? >>> >>> Thanks again! >>> cheers, >>> Christoph >>> >>> On 07/05/2012 06:18 PM, Carson Holt wrote: >>>> Your cluster is apparently using the CPAN module local::lib to define >>>> where perl libraries and executables get installed, this is overriding >>>> MAKER's own installation parameters, thus putting things in the wrong >>>> place. This only happens in 2.10, the 2.25 version of MAKER knows how >>>> to >>>> keep local::lib from messing with it's relative installation paths. >>>>So >>>> install 2.25 and make sure to remove the mis-located MAKER executables >>>> in >>>> /xanadu/site/common/VERSIONS/perl-1.7/bin/ before trying to run again. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> >>>> On 12-07-05 11:06 AM, "Christoph Hahn" wrote: >>>> >>>>> Dear MAKER developers and users, >>>>> >>>>> I am new to MAKER and during my first attempts I unfortunately >>>>> encountert some problems. It would be great if anyone could help me >>>>> out! >>>>> Here's my problem: >>>>> I have used gm_es.pl from Genemark to create es.mod for my assembly. >>>>>I >>>>> would like to give this model to MAKER, so I add the path to es.mod >>>>>in >>>>> the maker_opts.ctl file at gmhmm. So far so good. Unfortunately, >>>>>when I >>>>> am running maker it cannot find the gmhmm_wrap script. Here s what it >>>>> says: >>>>> >>>>> #-------------------------------# >>>>> Can't open perl script >>>>> >>>>> >>>>>"/xanadu/site/common/VERSIONS/perl-1.7/bin/../lib/Widget/genemark/gmhm >>>>>m_ >>>>> wr >>>>> ap": >>>>> No such file or directory >>>>> ERROR: Genemark failed >>>>> >>>>> Apparently it is looking for the script in the wrong place. I have >>>>> found >>>>> it in: >>>>> /site/VERSIONS/perl-1.7/perl/lib/MAKER/Widget/genemark/gmhmm_wrap >>>>> The other paths are correct. >>>>> >>>>> Is there a way to manually tell Maker where to look for this script? >>>>>I >>>>> should mention also that I have maker (2.10) running on a cluster. >>>>> >>>>> Any help is highly appreciated! Thanks in advance!! >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> University of Oslo >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> >>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>g >>> >> > > From carsonhh at gmail.com Tue Jul 10 06:35:10 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 10 Jul 2012 08:35:10 -0400 Subject: [maker-devel] Help: min_contig length In-Reply-To: Message-ID: There will be no negative effects in restarting. The min_contig option just speeds things up by skipping contigs that are likely too small to accurately annotate. Gene predictors generally require a certain amount of sequence upstream and downstream of a gene to work accurately, so for most Eukaryotic organisms, contigs of size of less than 10,000 will not produce results, but if the genes are smaller in your organism you can reduce this. The basic idea is to just skip contigs that won't produce results anyways. Thanks, Carson From: Diego Calderon Date: Monday, 9 July, 2012 7:20 PM To: ?? Li, Qing Cc: Carson Holt Subject: Re: Help: min_contig length Just a little add on: If we stop the currently running mpi maker, change the option from the default 1 to 500 or 1000, then restart maker, will it have any (negative) effect on maker (for example having to restart the analysis or something)? I'm guessing that it will not, but we just wanted to make sure. Best, Diego On Mon, Jul 9, 2012 at 5:11 PM, ?? Li, Qing wrote: > Hi Carson, > Very sorry for disturbing you! Diego and I are running maker for whole genome > annotation now, and we have a question: should the min_contig=500 or 1000? We > asked several people but get different answers,,, So we think you may be the > best one to ask,,, > Thank you! > Best, > Qing -------------- next part -------------- An HTML attachment was scrubbed... URL: From carson.holt at genetics.utah.edu Fri Jul 13 13:13:56 2012 From: carson.holt at genetics.utah.edu (Carson Holt) Date: Fri, 13 Jul 2012 19:13:56 +0000 Subject: [maker-devel] adding a new gene predictor to maker 2 In-Reply-To: <81f8d5b1-c8e8-46d2-8506-640d9dc18205@googlegroups.com> Message-ID: If you convert it to standard GFF3 format, MAKER will take it as input. GlimmerHMM would go into the pred_gff option. Thanks, Carson From: ckuanglim > Date: Friday, 13 July, 2012 3:01 AM To: Carson Holt > Subject: Re: [maker-devel] adding a new gene predictor to maker 2 Can I pass the output from GlimmerHMM into MAKER? Regards, Chan On Friday, July 30, 2010 9:43:53 AM UTC+8, Carson Holt wrote: mRNA-seq data can be passed to MAKER by first processing the data and then putting it into GFF3 format, i.e. process with something like tophat and cufflinks and then convert the output into GFF3. MAKER comes with tophat2gff3 and cufflinks2gff3 conversion scripts. Supply the results to the est_gff option in the maker_opt.ctl file. For ab initio gene predictors, do the same. Convert the output into GFF3 format then supply the data to pred_gff in the maker_opt.ctl file. Using GFF3 pass-through you can supply MAKER with evidence from virtually any external source of your choice. You just have to convert it into GFF3 so MAKER understands how to use it. Thanks, Carson On 7/29/10 6:43 PM, "Dieter Best" > wrote: Hello, is there a way to plugin another gene predictor into the maker2 pipeline or can we only choose between snap, augustus etc _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From alangfan at gmail.com Sun Jul 15 00:54:39 2012 From: alangfan at gmail.com (=?Big5?B?RmFuO1dlbi1MYW5nIK1TpOWtpg==?=) Date: Sun, 15 Jul 2012 14:54:39 +0800 Subject: [maker-devel] Could someone help me to set up MAKER2? Message-ID: Dear All, I refer to Maker website http://gmod.org/wiki/MAKER_Tutorial to set up Maker2. But the SNAP ( http://homepage.mac.com/iankorf/) website is broken. Could someone kindly tell me where to get SNAP software or alternative scheme? -- Alang -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Sun Jul 15 10:10:42 2012 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 15 Jul 2012 12:10:42 -0400 Subject: [maker-devel] Could someone help me to set up MAKER2? In-Reply-To: Message-ID: The old site was a victim of apple discontinuing MobileMe. SNAP can now be found on the Korf lab's website here --> http://korflab.ucdavis.edu/software.html Thanks, Carson From: "Fan;Wen-Lang ???" Date: Sunday, 15 July, 2012 2:54 AM To: Subject: [maker-devel] Could someone help me to set up MAKER2? Dear All, I refer to Maker website http://gmod.org/wiki/MAKER_Tutorial to set up Maker2. But the SNAP (http://homepage.mac.com/iankorf/) website is broken. Could someone kindly tell me where to get SNAP software or alternative scheme? -- Alang _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Mon Jul 16 23:49:36 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Tue, 17 Jul 2012 05:49:36 +0000 Subject: [maker-devel] maker and repeatproteinmask Message-ID: Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 17 15:05:58 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 17 Jul 2012 17:05:58 -0400 Subject: [maker-devel] maker and repeatproteinmask In-Reply-To: Message-ID: MAKER calls the 'RepeatMasker' script and is thus dependent on anything RepeatMasker decides to call internally. MAKER will never call TRF or RepeatProteinMask directly, but I can't say whether or not RepeatMasker will always do so as part of it's normal processing. The only way to know for sure is to try. I removed TRF from my installation and RepeatMasker shows as completing successfully, so I imagine you would probably be fine without it. Thanks, Carson From: Mr Thomas Schumann Date: Tuesday, 17 July, 2012 1:49 AM To: "maker-devel at yandell-lab.org" Cc: "c.chan10 at uq.edu.au" Subject: [maker-devel] maker and repeatproteinmask Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Wed Jul 18 16:03:10 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Wed, 18 Jul 2012 22:03:10 +0000 Subject: [maker-devel] maker and repeatproteinmask In-Reply-To: References: , Message-ID: Ok thanks. I looked through all the RepeatMasker source files and it doesn't look like anything calls RepeatProteinMask, and RepeatProteinMask is the only program which calls trf, so it looks like MAKER doesn't depend on trf at all then. Thanks, Tom Schumann ________________________________ From: Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 18 July 2012 7:05 AM To: Mr Thomas Schumann; maker-devel at yandell-lab.org Cc: c.chan10 at uq.edu.au Subject: Re: [maker-devel] maker and repeatproteinmask MAKER calls the 'RepeatMasker' script and is thus dependent on anything RepeatMasker decides to call internally. MAKER will never call TRF or RepeatProteinMask directly, but I can't say whether or not RepeatMasker will always do so as part of it's normal processing. The only way to know for sure is to try. I removed TRF from my installation and RepeatMasker shows as completing successfully, so I imagine you would probably be fine without it. Thanks, Carson From: Mr Thomas Schumann > Date: Tuesday, 17 July, 2012 1:49 AM To: "maker-devel at yandell-lab.org" > Cc: "c.chan10 at uq.edu.au" > Subject: [maker-devel] maker and repeatproteinmask Hi, Does MAKER require the RepeatProteinMask script from the RepeatMasker package to function? I've tried running MAKER with default parameters without this script and it appears to work fine but I'm not sure if different parameters require RepeatProteinMask. And if it is required for MAKER, what file is it called from? Basically I can't use trf (Tandem Repeat Finder) which is needed for the RepeatProteinMask script and I'd like to know if trf is required for MAKER and, if it is, whether the absence of trf has any impact on the results. Thanks, Tom Schumann _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Jul 23 12:45:59 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 23 Jul 2012 14:45:59 -0400 Subject: [maker-devel] Transcriptome assembly annotation? In-Reply-To: <8AC559A1-AAD1-4090-B48F-0346DF6A7524@illinois.edu> Message-ID: Never tried it, but I think I would probably just skip maker and perhaps do something like run cufflinks or trinity followed by intperproscan and blast2go. There is a wrapper for interproscan packaged with MAKER that might be useful. --Carson On 12-07-06 12:19 AM, "Fields, Christopher J" wrote: >Just curious, but I haven't found the answer to this in the mail list. >Has anyone used MAKER for transcriptome annotation? Obviously the ab >initio gene predictors wouldn't make much sense, but I could see a tool >like ESTScan or similar acting in a similar capacity. > >chris > >Chris Fields >Technical Lead in Genome Informatics >High Performance Computing in Biology >W.M. Keck Center for Comparative and Functional Genomics >Institute for Genomic Biology > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Jul 24 08:20:43 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 24 Jul 2012 10:20:43 -0400 Subject: [maker-devel] MAKER 2.26 Message-ID: FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From thomas.schumann at uqconnect.edu.au Tue Jul 24 16:06:11 2012 From: thomas.schumann at uqconnect.edu.au (Mr Thomas Schumann) Date: Tue, 24 Jul 2012 22:06:11 +0000 Subject: [maker-devel] MAKER 2.26 In-Reply-To: References: Message-ID: Hi Carson, Where can I download it? The Yandell Lab site still only lists v2.10 and v2.26 beta. Thanks, Tom Schumann ________________________________ From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 25 July 2012 12:20 AM To: maker-devel at yandell-lab.org Subject: [maker-devel] MAKER 2.26 FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 24 17:10:21 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 24 Jul 2012 19:10:21 -0400 Subject: [maker-devel] MAKER 2.26 In-Reply-To: Message-ID: That's the one. The beta tag will remain for a short time. The previous 2.25 version was also labelled with the beta tag. Thanks, Carson From: Mr Thomas Schumann Date: Tuesday, 24 July, 2012 6:06 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: RE: MAKER 2.26 Hi Carson, Where can I download it? The Yandell Lab site still only lists v2.10 and v2.26 beta. Thanks, Tom Schumann From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Carson Holt [carsonhh at gmail.com] Sent: Wednesday, 25 July 2012 12:20 AM To: maker-devel at yandell-lab.org Subject: [maker-devel] MAKER 2.26 FYI. MAKER is now updated to 2.26. *Improvements with very long alignments. *Fixes for installation when using the Local::Lib module. *Other minor bug fixes. *There is also an option to better handle false merging of genes in fungi caused by overlapping UTR (mRNAseq reads get misassembled by things like trinity and cufflinks). Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From guoyunfei1989 at gmail.com Wed Jul 25 13:15:59 2012 From: guoyunfei1989 at gmail.com (Yunfei Guo) Date: Wed, 25 Jul 2012 12:15:59 -0700 Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) Message-ID: Hi everyone, I ran maker2.25 without a problem, but with maker2.26, I encountered the following error after running it for ~8 hr with 2 nodes and 24 cpus, do you have any idea what's going on here? Some contigs did get finished, maybe this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system is SGE. Thanks! running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/scaffold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx #-------------------------------# deleted:-1 hits SIGCHLD handler "DEFAULT" not defined. SIGCHLD handler "DEFAULT" not defined. running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 #-------------------------------# Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fffa3a2e740) failed dequeue_and_set_error(596): Communication error with rank 18 running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/scaffold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx #-------------------------------# running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/scaffold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1.tblastx #-------------------------------# running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 #-------------------------------# cleaning blastx... in cluster::shadow_cluster... ...finished clustering. cleaning clusters.... total clusters:1 now processing 0 ...processing 0 of 2 deleted:0 hits ...processing 1 of 2 running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/scaffold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6.tblastx #-------------------------------# deleted:0 hits running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr%7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.9.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/ scaffold2590.1-3712.tr%7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 #-------------------------------# deleted:0 hits cleaning blastx... cleaning clusters.... total clusters:1 now processing 0 cleaning clusters.... total clusters:1 now processing 0 deleted:-1 hits deleted:-1 hits deleted:-6 hits deleted:-3 hits deleted:-2 hits Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Yunfei -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Jul 25 13:46:17 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 25 Jul 2012 15:46:17 -0400 Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) In-Reply-To: Message-ID: MPI is notorious for unexplicable communication errors, so first I would suggest just restarting and seeing if it happens again (MAKER will pick up where it left off on restart, so no need to alter settings or files). If it happens again, we can look into it, but no component of the MPI communication framework changed between 2.25 and 2.26 (100% identical), so my first instinct is that this was just what the message said, a"Communication error with rank 18". If it happens again I can try and add some extra messages so we can see the hostname of rank 18. That way we can identify if it's constantly a specific node on your cluster. Let me know if you see it again. Thanks, Carson From: Yunfei Guo Date: Wednesday, 25 July, 2012 3:15 PM To: Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) Hi everyone, I ran maker2.25 without a problem, but with maker2.26, I encountered the following error after running it for ~8 hr with 2 nodes and 24 cpus, do you have any idea what's going on here? Some contigs did get finished, maybe this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system is SGE. Thanks! running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/sc affold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx #-------------------------------# deleted:-1 hits SIGCHLD handler "DEFAULT" not defined. SIGCHLD handler "DEFAULT" not defined. running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp %7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 #-------------------------------# Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fffa3a2e740) failed dequeue_and_set_error(596): Communication error with rank 18 running blast search. #--------- command -------------# Widget::blastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/sc affold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx #-------------------------------# running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/sc affold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_ Rubri%2Efasta.mpi.10.1.tblastx #-------------------------------# running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp %7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.8.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 #-------------------------------# cleaning blastx... in cluster::shadow_cluster... ...finished clustering. cleaning clusters.... total clusters:1 now processing 0 ...processing 0 of 2 deleted:0 hits ...processing 1 of 2 running blast search. #--------- command -------------# Widget::tblastx: /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true -show_gis -out /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/sc affold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_ Rubri%2Efasta.mpi.10.6.tblastx #-------------------------------# deleted:0 hits running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr %7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.9.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --showcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker .output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sc affold2590.1-3712.tr %7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 #-------------------------------# deleted:0 hits cleaning blastx... cleaning clusters.... total clusters:1 now processing 0 cleaning clusters.... total clusters:1 now processing 0 deleted:-1 hits deleted:-1 hits deleted:-6 hits deleted:-3 hits deleted:-2 hits Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached Yunfei _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From guoyunfei1989 at gmail.com Thu Jul 26 09:10:42 2012 From: guoyunfei1989 at gmail.com (Yunfei Guo) Date: Thu, 26 Jul 2012 08:10:42 -0700 Subject: [maker-devel] Fwd: ERROR: MPI_Recv(186), dequeue_and_set_error(596) In-Reply-To: References: Message-ID: ---------- Forwarded message ---------- From: Yunfei Guo Date: Thu, Jul 26, 2012 at 8:10 AM Subject: Re: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) To: Carson Holt Hi Carson, same error occurred again. What should I do to check if it was caused by the same node? Also, if I ran maker on a single node instead of two nodes, will the same error appear again? Thank you. #-------------------------------# SIGCHLD handler "DEFAULT" not defined. Fatal error in MPI_Recv: Other MPI error, error stack: MPI_Recv(186).............: MPI_Recv(buf=0x7fff1c3dd3b0, count=2, MPI_ INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, status=0x7fff1c3dd3 90) failed dequeue_and_set_error(596): Communication error with rank 21 running exonerate search. #--------- command -------------# Widget::exonerate::protein2genome: /home/username/usr/bin/exonerate -q /home/yunfeiguo/projects/fish/Nigro /run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datast ore/81/43/scaffold5780//theVoid.scaffold5780/tr%7CG3N4L5%7CG3N4L5_GASA C.for.6527-8832.2.fasta -t /home/yunfeiguo/projects/fish/Nigro/run/dir _Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/81/4 3/scaffold5780//theVoid.scaffold5780/scaffold5780.6527-8832.2.fasta -Q protein -T dna -m protein2genome --softmasktarget --percent 20 --sho wcigar > /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigr o-53k_part.maker.output/Nigro-53k_part_datastore/81/43/scaffold5780//t heVoid.scaffold5780/scaffold5780.6527-8832.tr%7CG3N4L5%7CG3N4L5_GASAC. p_exonerate.2 #-------------------------------# Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached ... Yunfei On Wed, Jul 25, 2012 at 1:43 PM, Yunfei Guo wrote: > Thanks, Carson. Actually I already set clean_try=1. > > > On Wed, Jul 25, 2012 at 1:34 PM, Carson Holt wrote: > >> That second error from 2.25, Seems to be thrown by Perl's Storable >> module. There may be some weird partial serialization of the data that >> occurred on the first failure and is now causing failures on retry. You >> can set clean_try=1 in MAKER, to let it wipe out data for a failed contig >> before retrying. That can sometimes help get around weird hard failures. >> >> Thanks, >> Carson >> >> >> From: Yunfei Guo >> Date: Wednesday, 25 July, 2012 4:26 PM >> To: Carson Holt >> Subject: Re: [maker-devel] ERROR: MPI_Recv(186), >> dequeue_and_set_error(596) >> >> Thank you, Carson. I'm rerunning maker2.26 now. I just tried maker2.25 as >> well, it failed this time, with similar errors below. I guess it might be >> caused by the cluster (or node) itself, like you said, because we just >> added a few nodes and more memories. I'll ask the admin to see whether he >> can explain this. >> #-------------------------------# >> Thread 1 terminated abnormally: >> ------------- EXCEPTION: Bio::Root::Exception ------------- >> MSG: no data for midline Sequence with id BL_ORD_ID:126195 no longer >> exists in database...alignment skipped >> STACK: Error::throw >> STACK: Bio::Root::Root::throw >> /home/yunfeiguo/perl5/lib/perl5/Bio/Root/Root.pm:472 >> STACK: Bio::SearchIO::blast::next_result >> /home/yunfeiguo/perl5/lib/perl5/Bio/SearchIO/blast.pm:1888 >> STACK: Widget::tblastx::keepers >> /home/yunfeiguo/Downloads/maker/bin/../lib/Widget/tblastx.pm:114 >> STACK: Widget::tblastx::parse >> /home/yunfeiguo/Downloads/maker/bin/../lib/Widget/tblastx.pm:95 >> STACK: GI::tblastx_as_chunks >> /home/yunfeiguo/Downloads/maker/bin/../lib/GI.pm:2612 >> STACK: Process::MpiChunk::_go >> /home/yunfeiguo/Downloads/maker/bin/../lib/Process/MpiChunk.pm:1829 >> STACK: Process::MpiChunk::run >> /home/yunfeiguo/Downloads/maker/bin/../lib/Process/MpiChunk.pm:331 >> STACK: main::node_thread /home/yunfeiguo/Downloads/maker/bin/maker:1308 >> STACK: threads::new >> /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/forks.pm:799 >> STACK: /home/yunfeiguo/Downloads/maker/bin/maker:804 >> ----------------------------------------------------------- >> Cannot restore overloading on HASH(0x1b7f9b60) (package >> Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at >> /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/Storable.pm line >> 416, at /home/yunfeiguo/perl5/lib/perl5/x86_64-linux-thread-multi/ >> forks.pm line 2256. >> Compilation failed in require at >> /home/yunfeiguo/Downloads/maker/bin/maker line 11. >> BEGIN failed--compilation aborted at >> /home/yunfeiguo/Downloads/maker/bin/maker line 11. >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> deleted:0 hits >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff05c78630, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff05c78610) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1414fb00, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1414fae0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff25d86c00, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff25d86be0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> cleaning tblastx... >> cleaning clusters.... >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1b71f1f0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1b71f1d0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffc99d29c0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffc99d29a0) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffc4aaf720, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffc4aaf700) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> in cluster::shadow_cluster... >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff317862a0, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff31786280) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> ...finished clustering. >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff8abb8e50, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff8abb8e30) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff1d1ff180, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff1d1ff160) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fff4d865850, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fff4d865830) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffbec98150, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffbec98130) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Fatal error in MPI_Recv: Other MPI error, error stack: >> MPI_Recv(186).............: MPI_Recv(buf=0x7fffa4ead990, count=2, >> MPI_INT, src=0, tag=5555, MPI_COMM_WORLD, status=0x7fffa4ead970) failed >> dequeue_and_set_error(596): Communication error with rank 0 >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> Perl exited with active threads: >> 1 running and unjoined >> 0 finished and unjoined >> 0 running and detached >> >> On Wed, Jul 25, 2012 at 12:46 PM, Carson Holt wrote: >> >>> MPI is notorious for unexplicable communication errors, so first I would >>> suggest just restarting and seeing if it happens again (MAKER will pick up >>> where it left off on restart, so no need to alter settings or files). >>> >>> If it happens again, we can look into it, but no component of the MPI >>> communication framework changed between 2.25 and 2.26 (100% identical), so >>> my first instinct is that this was just what the message said, >>> a"Communication error with rank 18". If it happens again I can try and add >>> some extra messages so we can see the hostname of rank 18. That way we can >>> identify if it's constantly a specific node on your cluster. >>> >>> Let me know if you see it again. >>> >>> Thanks, >>> Carson >>> >>> >>> >>> From: Yunfei Guo >>> Date: Wednesday, 25 July, 2012 3:15 PM >>> To: >>> Subject: [maker-devel] ERROR: MPI_Recv(186), dequeue_and_set_error(596) >>> >>> Hi everyone, >>> >>> I ran maker2.25 without a problem, but with maker2.26, I encountered the >>> following error after running it for ~8 hr with 2 nodes and 24 cpus, do you >>> have any idea what's going on here? Some contigs did get finished, maybe >>> this is not a big problem. My mpich2 version 1.4.1p1, job scheduling system >>> is SGE. Thanks! >>> >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.4 -query >>> /tmp/6480.1.all.q/maker_PQOTIq/rank3/scaffold2602.0 -num_alignments 10000 >>> -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/7A/37/scaffold2602//theVoid.scaffold2602/scaffold2602.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.4.blastx >>> #-------------------------------# >>> deleted:-1 hits >>> SIGCHLD handler "DEFAULT" not defined. >>> SIGCHLD handler "DEFAULT" not defined. >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8N8A2%7CANR44_HUMAN.for.1-3712.8.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8N8A2%7CANR44_HUMAN.p_exonerate.8 >>> #-------------------------------# >>> Fatal error in MPI_Recv: Other MPI error, error stack: >>> MPI_Recv(186).............: MPI_Recv(buf=0x7fffa3a2e760, count=2, >>> MPI_INT, src=MPI_ANY_SOURCE, tag=1111, MPI_COMM_WORLD, >>> status=0x7fffa3a2e740) failed >>> dequeue_and_set_error(596): Communication error with rank 18 >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/blastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/concatPro%2Etxt.mpi.10.8 -query >>> /tmp/6480.1.all.q/maker_PQOTIq/rank11/scaffold2575.0 -num_alignments 10000 >>> -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F0/AE/scaffold2575//theVoid.scaffold2575/scaffold2575.0.concatPro%2Etxt.blastx.temp_dir/concatPro%2Etxt.mpi.10.8.blastx >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::tblastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1 >>> -query /tmp/6480.1.all.q/maker_PQOTIq/rank7/scaffold2620.0 -num_alignments >>> 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp >>> 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true >>> -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/6B/FB/scaffold2620//theVoid.scaffold2620/scaffold2620.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.1.tblastx >>> #-------------------------------# >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/sp%7CQ8NB46%7CANR52_HUMAN.for.1-3712.8.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.8.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.sp%7CQ8NB46%7CANR52_HUMAN.p_exonerate.8 >>> #-------------------------------# >>> cleaning blastx... >>> in cluster::shadow_cluster... >>> ...finished clustering. >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> ...processing 0 of 2 >>> deleted:0 hits >>> ...processing 1 of 2 >>> running blast search. >>> #--------- command -------------# >>> Widget::tblastx: >>> /home/yunfeiguo/Downloads/maker/bin/../exe/blast/bin/tblastx -db >>> /tmp/6480.1.all.q/maker_PQOTIq/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6 >>> -query /tmp/6480.1.all.q/maker_PQOTIq/rank9/scaffold2615.0 -num_alignments >>> 10000 -num_descriptions 10000 -evalue 1e-10 -dbsize 1000 -searchsp >>> 500000000 -num_threads 1 -lcase_masking -seg yes -soft_masking true >>> -show_gis -out >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/E2/6E/scaffold2615//theVoid.scaffold2615/scaffold2615.0.AllSebESTs_plus_Rubri%2Efasta.tblastx.temp_dir/AllSebESTs_plus_Rubri%2Efasta.mpi.10.6.tblastx >>> #-------------------------------# >>> deleted:0 hits >>> running exonerate search. >>> #--------- command -------------# >>> Widget::exonerate::protein2genome: >>> /home/username/usr/bin/exonerate -q >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/tr%7CE7F7S0%7CE7F7S0_DANRE.for.1-3712.9.fasta >>> -t >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/scaffold2590.1-3712.9.fasta >>> -Q protein -T dna -m protein2genome --softmasktarget --percent 20 >>> --showcigar > >>> /home/yunfeiguo/projects/fish/Nigro/run/dir_Nigro-53k00/Nigro-53k_part.maker.output/Nigro-53k_part_datastore/F9/9B/scaffold2590//theVoid.scaffold2590/ >>> scaffold2590.1-3712.tr%7CE7F7S0%7CE7F7S0_DANRE.p_exonerate.9 >>> #-------------------------------# >>> deleted:0 hits >>> cleaning blastx... >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> cleaning clusters.... >>> total clusters:1 now processing 0 >>> deleted:-1 hits >>> deleted:-1 hits >>> deleted:-6 hits >>> deleted:-3 hits >>> deleted:-2 hits >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> Perl exited with active threads: >>> 1 running and unjoined >>> 0 finished and unjoined >>> 0 running and detached >>> >>> Yunfei >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fourie.joubert at up.ac.za Fri Jul 27 08:30:33 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Fri, 27 Jul 2012 16:30:33 +0200 Subject: [maker-devel] Passing extra options to augustus in maker? Message-ID: <5012A609.7060101@up.ac.za> Hi Could anyone advise me regarding passing additional options to augustus when running maker? I have generated a series of hints files (http://bioinf.uni-greifswald.de/bioinf/wiki/pmwiki.php?n=Augustus.IncorporateProteins) that work fine when running augustus stand-alone, but would like to be able to pass the necessary parameters to augustus when running maker so that augustus uses the hints files. Alternatively, if I need to run augustus stand-alone with the hints files, which option should I use to load the resulting augustus gff files into maker (protein_gff, other_gff, ...)? Thanks and regards! Fourie -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Fri Jul 27 10:34:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 27 Jul 2012 12:34:30 -0400 Subject: [maker-devel] Passing extra options to augustus in maker? In-Reply-To: <5012A609.7060101@up.ac.za> Message-ID: You can't pass externally generated augustus hint files into maker. But if you do decide to run augustus alone using the hint file you just generated, you can then pass the results in to pred_gff (after any necessary formatting fixes for correct GFF3). The results of running exonerate2hints.pl are essentially the same as letting MAKER generate its own hints file because maker is using exonerate to generate the alignment. So the effect of an external hint file would be essentially redundant. When running augustus in MAKER, it is ran a minimum of 2 times. The first time it is run without hints and only using the training parameters from the augustus species files. The second time MAKER internally generates hints based on exonerate alignments and then runs augustus using those hints. If you also pass in your external run, you would then have 3 augustus runs on the genome with MAKER selecting the model from the pool of runs that best matches the evidence alignments. Thanks, Carson On 12-07-27 10:30 AM, "Fourie Joubert" wrote: >Hi > >Could anyone advise me regarding passing additional options to augustus >when running maker? > >I have generated a series of hints files >(http://bioinf.uni-greifswald.de/bioinf/wiki/pmwiki.php?n=Augustus.Incorpo >rateProteins) >that work fine when running augustus stand-alone, but would like to be >able to pass the necessary parameters to augustus when running maker so >that augustus uses the hints files. > >Alternatively, if I need to run augustus stand-alone with the hints >files, which option should I use to load the resulting augustus gff >files into maker (protein_gff, other_gff, ...)? > >Thanks and regards! > >Fourie > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Fri Jul 27 10:38:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 27 Jul 2012 12:38:56 -0400 Subject: [maker-devel] Advice for optimizing augustus training on fungal genome? In-Reply-To: <4FEC7423.3050001@up.ac.za> Message-ID: Regarding this older post, if you aren't getting good results using CEGMA for training augustus for a new species, then One option would be just to make a copy Neurospora's species directory. Edit the necessary file contents to make it list as a different species, then run the augustus training steps as before but this use the Neurospora copy as the base so augustus will be optimizing Neurospora's parameters to be more like your species of interest. Thanks, Carson On 12-06-28 11:11 AM, "Fourie Joubert" wrote: >Hi Everyone > >Apologies if this is not the relevant list to mail to. > >I am looking for advice in training augustus for a novel fungal genome. > >I generated a gene set using CEGMA (below), and have subsequently been >following the instructions at >http://www.molecularevolution.org/molevolfiles/exercises/augustus/scipio.h >tml >and at >http://www.molecularevolution.org/molevolfiles/exercises/augustus/training >.html. > >My training set is 339 genes and the test set is 100 genes. > >My initial output is below. > >It does not improve much with optimize_augustus. > >When using the training paramters to predict genes in the genome, I seem >to only find around 2,000 of the known ~16,000 genes. When I use the >training data from a distantly related fungus (Neurospora), I get >roughly the correct number of genes. > >I am obviously doing something wrong here... (commands below). > >I would really appreciate any advice on where to start looking for >improvement. > >Kindest regards! > >Fourie > > > > > >Augustus commands (Editedmyspecies_parameters.cfg and >setstopCodonExcludedFromCDS to true.): > >> etraining --species=myspecies genes.gb.train > >> augustus --species=myspecies genes.gb.test | tee firsttest.out > >> grep -A 22 Evaluation firsttest.out > >> optimize_augustus.pl --species=myspecies genes.gb.train > >> etraining --species=myspecies genes.gb.train > >> augustus --species=myspecies genes.gb.test | tee secondtest.out > >> grep -A 22 Evaluation secondtest.out > > > >CEGMA output: > ># Statistics of the completeness of the genome based on 248 CEGs > # > > #Prots %Completeness - #Total Average %Ortho > > Complete 240 96.77 - 278 1.16 11.67 > > Group 1 64 96.97 - 72 1.12 7.81 > Group 2 54 96.43 - 66 1.22 18.52 > Group 3 58 95.08 - 70 1.21 13.79 > Group 4 64 98.46 - 70 1.09 7.81 > > Partial 245 98.79 - 290 1.18 13.88 > > Group 1 65 98.48 - 73 1.12 7.69 > Group 2 56 100.00 - 70 1.25 21.43 > Group 3 59 96.72 - 75 1.27 18.64 > Group 4 65 100.00 - 72 1.11 9.23 > > > > >Augustus output: > >******* Evaluation of gene prediction ******* > >---------------------------------------------\ > > | sensitivity | specificity | > >---------------------------------------------| > >nucleotide level | 0.933 | 0.772 | > >---------------------------------------------/ > >-------------------------------------------------------------------------- >--------------------------------\ > > | #pred | #anno | | FP = false pos. | FN = false >neg. | | | > > | total/ | total/ | TP >|--------------------|--------------------| sensitivity | specificity | > > | unique | unique | | part | ovlp | wrng | part | ovlp | >wrng | | | > >-------------------------------------------------------------------------- >--------------------------------| > > | | | | 229 | > 85 | | | > >exon level | 475 | 331 | 246 | ------------------ | >------------------ | 0.743 | 0.518 | > > | 475 | 331 | | 59 | 9 | 161 | 56 | 2 | > 27 | | | > >-------------------------------------------------------------------------- >--------------------------------/ > >-------------------------------------------------------------------------- >--\ > >transcript | #pred | #anno | TP | FP | FN | sensitivity | >specificity | > >-------------------------------------------------------------------------- >--| > >gene level | 158 | 100 | 45 | 113 | 55 | 0.45 | >0.285 | > >-------------------------------------------------------------------------- >--/ > > > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From jason.stajich at gmail.com Fri Jul 27 16:52:07 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Fri, 27 Jul 2012 15:52:07 -0700 Subject: [maker-devel] Advice for optimizing augustus training on fungal genome? In-Reply-To: References: Message-ID: <4457959E-443D-4A17-8EC6-5D16FD58B2C7@gmail.com> Best option is to get RNA-Seq or some transcript evidence of course so you have something empirical. I'm really surprised you are getting such a low count - this is just from augustus running on the cmdline? Alternatively if you have 16,000 known proteins, derived from protein alignments of related species aligned to your genome I expect? You could also try and generate a best set of gene predictions from this. When I did this for Coprinus I gathered full length alignments of fungal proteins and used at the time genomewise and genewise to get the best set of loci with gene calls. Today I'd probably just use FASTA to find the locus and exonerate to get the spliced gene model out and provide these as training. Basically this is doing a fungal-specific CEGMA, something we ought to build, but just don't have time to do. Do you get similar results when running genemarkHMM with its selftraining? jason On Jul 27, 2012, at 9:38 AM, Carson Holt wrote: > Regarding this older post, if you aren't getting good results using CEGMA > for training augustus for a new species, then One option would be just to > make a copy Neurospora's species directory. Edit the necessary file > contents to make it list as a different species, then run the augustus > training steps as before but this use the Neurospora copy as the base so > augustus will be optimizing Neurospora's parameters to be more like your > species of interest. > > Thanks, > Carson > > > > On 12-06-28 11:11 AM, "Fourie Joubert" wrote: > >> Hi Everyone >> >> Apologies if this is not the relevant list to mail to. >> >> I am looking for advice in training augustus for a novel fungal genome. >> >> I generated a gene set using CEGMA (below), and have subsequently been >> following the instructions at >> http://www.molecularevolution.org/molevolfiles/exercises/augustus/scipio.h >> tml >> and at >> http://www.molecularevolution.org/molevolfiles/exercises/augustus/training >> .html. >> >> My training set is 339 genes and the test set is 100 genes. >> >> My initial output is below. >> >> It does not improve much with optimize_augustus. >> >> When using the training paramters to predict genes in the genome, I seem >> to only find around 2,000 of the known ~16,000 genes. When I use the >> training data from a distantly related fungus (Neurospora), I get >> roughly the correct number of genes. >> >> I am obviously doing something wrong here... (commands below). >> >> I would really appreciate any advice on where to start looking for >> improvement. >> >> Kindest regards! >> >> Fourie >> >> >> >> >> >> Augustus commands (Editedmyspecies_parameters.cfg and >> setstopCodonExcludedFromCDS to true.): >> >>> etraining --species=myspecies genes.gb.train >> >>> augustus --species=myspecies genes.gb.test | tee firsttest.out >> >>> grep -A 22 Evaluation firsttest.out >> >>> optimize_augustus.pl --species=myspecies genes.gb.train >> >>> etraining --species=myspecies genes.gb.train >> >>> augustus --species=myspecies genes.gb.test | tee secondtest.out >> >>> grep -A 22 Evaluation secondtest.out >> >> >> >> CEGMA output: >> >> # Statistics of the completeness of the genome based on 248 CEGs >> # >> >> #Prots %Completeness - #Total Average %Ortho >> >> Complete 240 96.77 - 278 1.16 11.67 >> >> Group 1 64 96.97 - 72 1.12 7.81 >> Group 2 54 96.43 - 66 1.22 18.52 >> Group 3 58 95.08 - 70 1.21 13.79 >> Group 4 64 98.46 - 70 1.09 7.81 >> >> Partial 245 98.79 - 290 1.18 13.88 >> >> Group 1 65 98.48 - 73 1.12 7.69 >> Group 2 56 100.00 - 70 1.25 21.43 >> Group 3 59 96.72 - 75 1.27 18.64 >> Group 4 65 100.00 - 72 1.11 9.23 >> >> >> >> >> Augustus output: >> >> ******* Evaluation of gene prediction ******* >> >> ---------------------------------------------\ >> >> | sensitivity | specificity | >> >> ---------------------------------------------| >> >> nucleotide level | 0.933 | 0.772 | >> >> ---------------------------------------------/ >> >> -------------------------------------------------------------------------- >> --------------------------------\ >> >> | #pred | #anno | | FP = false pos. | FN = false >> neg. | | | >> >> | total/ | total/ | TP >> |--------------------|--------------------| sensitivity | specificity | >> >> | unique | unique | | part | ovlp | wrng | part | ovlp | >> wrng | | | >> >> -------------------------------------------------------------------------- >> --------------------------------| >> >> | | | | 229 | >> 85 | | | >> >> exon level | 475 | 331 | 246 | ------------------ | >> ------------------ | 0.743 | 0.518 | >> >> | 475 | 331 | | 59 | 9 | 161 | 56 | 2 | >> 27 | | | >> >> -------------------------------------------------------------------------- >> --------------------------------/ >> >> -------------------------------------------------------------------------- >> --\ >> >> transcript | #pred | #anno | TP | FP | FN | sensitivity | >> specificity | >> >> -------------------------------------------------------------------------- >> --| >> >> gene level | 158 | 100 | 45 | 113 | 55 | 0.45 | >> 0.285 | >> >> -------------------------------------------------------------------------- >> --/ >> >> >> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org From sbrubaker at solazyme.com Mon Jul 30 17:52:57 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Mon, 30 Jul 2012 23:52:57 +0000 Subject: [maker-devel] CDS file? Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 31 07:01:07 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 31 Jul 2012 09:01:07 -0400 Subject: [maker-devel] CDS file? In-Reply-To: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 11:16:45 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 17:16:45 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE361@EXCHANGE-05.internal.solazyme.com> Thanks! From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 11:17:19 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 17:17:19 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDDE0F@EXCHANGE-05.internal.solazyme.com> Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com> One more question - do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Jul 31 11:58:51 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 31 Jul 2012 13:58:51 -0400 Subject: [maker-devel] CDS file? In-Reply-To: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com> Message-ID: Not really, but the functional annotation tools included with maker might be useful for some of the downstream work. Although I guess there is a way I could fool maker's direct protein2genome gene creation into generating gene models with a little code modification. --Carson From: Shane Brubaker Date: Tuesday, 31 July, 2012 1:17 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: RE: [maker-devel] CDS file? One more question ? do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From sbrubaker at solazyme.com Tue Jul 31 13:13:55 2012 From: sbrubaker at solazyme.com (Shane Brubaker) Date: Tue, 31 Jul 2012 19:13:55 +0000 Subject: [maker-devel] CDS file? In-Reply-To: References: <61D01ACB70C1E141A150BA9F586D5BFA03FDE37A@EXCHANGE-05.internal.solazyme.com>, Message-ID: <61D01ACB70C1E141A150BA9F586D5BFA03FDE4C4@EXCHANGE-05.internal.solazyme.com> Got it, thanks for your help! ________________________________________ From: Carson Holt [carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 10:58 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Not really, but the functional annotation tools included with maker might be useful for some of the downstream work. Although I guess there is a way I could fool maker's direct protein2genome gene creation into generating gene models with a little code modification. --Carson From: Shane Brubaker > Date: Tuesday, 31 July, 2012 1:17 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: RE: [maker-devel] CDS file? One more question ? do people ever run MAKER on transcriptomes? From: Carson Holt [mailto:carsonhh at gmail.com] Sent: Tuesday, July 31, 2012 6:01 AM To: Shane Brubaker; maker-devel at yandell-lab.org Subject: Re: [maker-devel] CDS file? Use the fasta_tool script that comes with MAKER together with the --trim_maker_utr option. Thanks, Carson From: Shane Brubaker > Date: Monday, 30 July, 2012 7:52 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] CDS file? Hi, does anyone know if maker outputs a file of just the CDS portion of transcripts? I want to use it to determine a codon usage table. Thanks, Shane _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org