From mike.thon at gmail.com Mon Oct 1 00:53:23 2012 From: mike.thon at gmail.com (Michael Thon) Date: Mon, 1 Oct 2012 07:53:23 +0200 Subject: [maker-devel] model_gff question Message-ID: Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. -Mike -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Oct 1 09:01:01 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 01 Oct 2012 10:01:01 -0400 Subject: [maker-devel] model_gff question In-Reply-To: Message-ID: They can be replaced under two circumstances. 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. Thanks, Carson From: Michael Thon Date: Monday, 1 October, 2012 1:53 AM To: Subject: [maker-devel] model_gff question Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. -Mike _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.thon at gmail.com Tue Oct 2 00:01:09 2012 From: mike.thon at gmail.com (Michael Thon) Date: Tue, 2 Oct 2012 07:01:09 +0200 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: I looked at two cases in which the model_gff disappeared and they occurred in regions where there are multiple overlapping cufflinks features. One model that I'm looking at right now has overlapping protein2genome and a SNAP feature overlapping it but it was still not included in the output. it could be a problem in MAKER or it could be a problem with my RNA Seq data. I aligned the RNA Seq data using tophat/cufflinks and converted the transcripts.gtf file to gff using cufflinks2gff3 script. Is it better to use RNA Seq feature from tophat or cufflinks? On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > They can be replaced under two circumstances. > 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. > 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). > > MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. > > Thanks, > Carson > > > From: Michael Thon > Date: Monday, 1 October, 2012 1:53 AM > To: > Subject: [maker-devel] model_gff question > > Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: > https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion > > That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. > -Mike > > _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.thon at gmail.com Tue Oct 2 02:50:04 2012 From: mike.thon at gmail.com (Michael Thon) Date: Tue, 2 Oct 2012 09:50:04 +0200 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: <8E7B4629-B071-44A6-8E7F-2B1133A946D0@gmail.com> It seems to have disappeared completely. I'm running MAKER again now using the tophat alignments that I fed to cufflinks, instead of the cufflinks data. So far the two models visually checked as missing with the cufflinks data are present as they should be. I have to wait for the run to finish to get a whole genome count though. Maybe I need to look more closely at the cufflinks run that I did. The RNA-Seq data are from the NCBI SRA and I didn't do anything to clean them up before I ran tophat. On Oct 2, 2012, at 9:10 AM, Daniel Hughes wrote: > Did the whole model vanish or just the protein product - contaminated rnaseq that hasn't been cleaned up enough will regularly cause the later to become part of a bad utr. > > Dan > > On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: > I looked at two cases in which the model_gff disappeared and they occurred in regions where there are multiple overlapping cufflinks features. One model that I'm looking at right now has overlapping protein2genome and a SNAP feature overlapping it but it was still not included in the output. it could be a problem in MAKER or it could be a problem with my RNA Seq data. I aligned the RNA Seq data using tophat/cufflinks and converted the transcripts.gtf file to gff using cufflinks2gff3 script. > > Is it better to use RNA Seq feature from tophat or cufflinks? > > > On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > >> They can be replaced under two circumstances. >> 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. >> 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). >> >> MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. >> >> Thanks, >> Carson >> >> >> From: Michael Thon >> Date: Monday, 1 October, 2012 1:53 AM >> To: >> Subject: [maker-devel] model_gff question >> >> Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: >> https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion >> >> That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. >> -Mike >> >> _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Oct 2 13:05:04 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 02 Oct 2012 14:05:04 -0400 Subject: [maker-devel] model_gff question In-Reply-To: <8E7B4629-B071-44A6-8E7F-2B1133A946D0@gmail.com> Message-ID: If it overlaps the UTR of some other chosen model it might have been excluded for that reason as well. Sometimes this can happen when you have RNA-seq and high gene density (so models wander into each other). Try setting the the correct_est_fusion option to 1. This will take steps to trim UTR that might cause neighboring models to be left out because of UTR overlap (it also helps with false fusions caused by cufflinks results). I would not be surprised if the model being left out was because of UTR overlap. I recommend using the cufflinks data and leaving tophat out. Tophat results tend to be very noisy and can span large rather weird regions. Thanks, Carson From: Michael Thon Date: Tuesday, 2 October, 2012 3:50 AM To: Daniel Hughes Cc: Michael Thon , , Carson Holt Subject: Re: [maker-devel] model_gff question It seems to have disappeared completely. I'm running MAKER again now using the tophat alignments that I fed to cufflinks, instead of the cufflinks data. So far the two models visually checked as missing with the cufflinks data are present as they should be. I have to wait for the run to finish to get a whole genome count though. Maybe I need to look more closely at the cufflinks run that I did. The RNA-Seq data are from the NCBI SRA and I didn't do anything to clean them up before I ran tophat. On Oct 2, 2012, at 9:10 AM, Daniel Hughes wrote: > > Did the whole model vanish or just the protein product - contaminated rnaseq > that hasn't been cleaned up enough will regularly cause the later to become > part of a bad utr. > > Dan > > On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: >> I looked at two cases in which the model_gff disappeared and they occurred in >> regions where there are multiple overlapping cufflinks features. One model >> that I'm looking at right now has overlapping protein2genome and a SNAP >> feature overlapping it but it was still not included in the output. it could >> be a problem in MAKER or it could be a problem with my RNA Seq data. I >> aligned the RNA Seq data using tophat/cufflinks and converted the >> transcripts.gtf file to gff using cufflinks2gff3 script. >> >> Is it better to use RNA Seq feature from tophat or cufflinks? >> >> >> On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: >> >>> They can be replaced under two circumstances. >>> 1. If you provide two model_gff files (comma separated list), in which case >>> MAKER thinks it is merging legacy annotations and will only keep one or the >>> other if models overlap. >>> 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this >>> as a cue that if these other programs produce a better model, it can replace >>> the current model. If you set map_forward=1, MAKER will conserve the name >>> of the previous model (so models change structure but names are conserved); >>> otherwise, it gets a new name. Sometimes groups like to rename models every >>> time their is a structural change. I think you are supposed to get the >>> Alias attribute set when you don't get names mapped forward though (I can't >>> remember if I added this or just planned on adding the Alias mapping >>> though). >>> >>> MAKER should never drop a model_gff model. It can only replace it if >>> something better comes along, but it should not disappear. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Michael Thon >>> Date: Monday, 1 October, 2012 1:53 AM >>> To: >>> Subject: [maker-devel] model_gff question >>> >>> Under what circumstances will maker not include a gene model from the >>> model_gff file in its final output? It was my understanding from this post: >>> https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion >>> >>> That maker will keep or replace models in model_gff and never remove them. >>> I'm reannotating a fungal genome and in model_gff I'm providing the gene >>> models originally made by the sequencing center. I have 12006 models in the >>> file I specify in model_gff but maker's final annotation has only 10727 >>> models in it. >>> -Mike >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m >>> aker-devel_yandell-lab.org >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dsthughes at gmail.com Tue Oct 2 02:10:19 2012 From: dsthughes at gmail.com (Daniel Hughes) Date: Tue, 2 Oct 2012 08:10:19 +0100 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: Did the whole model vanish or just the protein product - contaminated rnaseq that hasn't been cleaned up enough will regularly cause the later to become part of a bad utr. Dan On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: > I looked at two cases in which the model_gff disappeared and they occurred > in regions where there are multiple overlapping cufflinks features. One > model that I'm looking at right now has overlapping protein2genome and a > SNAP feature overlapping it but it was still not included in the output. > it could be a problem in MAKER or it could be a problem with my RNA Seq > data. I aligned the RNA Seq data using tophat/cufflinks and converted the > transcripts.gtf file to gff using cufflinks2gff3 script. > > Is it better to use RNA Seq feature from tophat or cufflinks? > > > On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > > They can be replaced under two circumstances. > 1. If you provide two model_gff files (comma separated list), in which > case MAKER thinks it is merging legacy annotations and will only keep one > or the other if models overlap. > 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees > this as a cue that if these other programs produce a better model, it can > replace the current model. If you set map_forward=1, MAKER will conserve > the name of the previous model (so models change structure but names are > conserved); otherwise, it gets a new name. Sometimes groups like to rename > models every time their is a structural change. I think you are supposed > to get the Alias attribute set when you don't get names mapped forward > though (I can't remember if I added this or just planned on adding the > Alias mapping though). > > MAKER should never drop a model_gff model. It can only replace it if > something better comes along, but it should not disappear. > > Thanks, > Carson > > > From: Michael Thon > Date: Monday, 1 October, 2012 1:53 AM > To: > Subject: [maker-devel] model_gff question > > Under what circumstances will maker not include a gene model from the > model_gff file in its final output? It was my understanding from this post: > https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion > > That maker will keep or replace models in model_gff and never remove them. > I'm reannotating a fungal genome and in model_gff I'm providing the gene > models originally made by the sequencing center. I have 12006 models in > the file I specify in model_gff but maker's final annotation has only 10727 > models in it. > -Mike > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amelia.ireland at gmod.org Tue Oct 2 16:00:57 2012 From: amelia.ireland at gmod.org (Amelia Ireland) Date: Tue, 2 Oct 2012 14:00:57 -0700 Subject: [maker-devel] problem installing maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868AF3A@EXMBX05.austin.utexas.edu> References: <1B4E4BB25B711A4FBC93E4B3AFEB2A3218689DC6@EXMBX05.austin.utexas.edu> <1B4E4BB25B711A4FBC93E4B3AFEB2A321868AF3A@EXMBX05.austin.utexas.edu> Message-ID: Hello Oscar, I'm forwarding your email on to the MAKER mailing list and one of the developers; they should be able to answer your query. Thanks, Amelia. On Tue, Oct 2, 2012 at 1:55 PM, Dian "Oscar" Jiao wrote: > Hi, > > I was trying to install Maker. All the prerequisite seemed installed okay. > However, when I ran maker or mpimaker, I got the following error: > /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_File/DB_File.so: > undefined symbol: db_version". I have edited config.in before compiling to > make sure the paths are correct for db.h and libdb. Do you have any idea > what is causing this? > > It seems to have to do with the berkeleydb and DB_File. I have BerkeleyDB > 5.3.21 installed and set the paths correctly for db.h and libdb in config.in > when I was compiling DB_File, but it still failed. Do you know what is > causing this? > > Thanks > Oscar > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > -- > You received this message because you are subscribed to the Google Groups > "GMOD Help Desk" group. > To post to this group, send email to gmod-help-desk at googlegroups.com. > To unsubscribe from this group, send email to > gmod-help-desk+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > -- Amelia Ireland GMOD Community Support Specialist || http://gmod.org From carsonhh at gmail.com Wed Oct 3 09:13:20 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 03 Oct 2012 10:13:20 -0400 Subject: [maker-devel] problem installing maker In-Reply-To: Message-ID: You may need to completely delete this directory before reinstalling DB_File. --> /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_Fil e Also if that doesn't work you can skip DB_file completely, and use a different database backend During the MAKER install do the following: perl ./Build.PL --AnyDBM_ISA SDBM_File ./Build install Then test maker once on a new job. The --AnyDBM_ISA sets a different database than Berkley DB (All of which are slower than Berkley DB). --Carson On 12-10-02 5:00 PM, "Amelia Ireland" wrote: >Hello Oscar, > >I'm forwarding your email on to the MAKER mailing list and one of the >developers; they should be able to answer your query. > >Thanks, >Amelia. > >On Tue, Oct 2, 2012 at 1:55 PM, Dian "Oscar" Jiao >wrote: >> Hi, >> >> I was trying to install Maker. All the prerequisite seemed installed >>okay. >> However, when I ran maker or mpimaker, I got the following error: >> >>/work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_F >>ile/DB_File.so: >> undefined symbol: db_version". I have edited config.in before compiling >>to >> make sure the paths are correct for db.h and libdb. Do you have any idea >> what is causing this? >> >> It seems to have to do with the berkeleydb and DB_File. I have >>BerkeleyDB >> 5.3.21 installed and set the paths correctly for db.h and libdb in >>config.in >> when I was compiling DB_File, but it still failed. Do you know what is >> causing this? >> >> Thanks >> Oscar >> -- >> Dian (Oscar) Jiao, Ph.D., >> Research Associate, Life Sciences Computing Group >> Texas Advanced Computing Center >> University of Texas at Austin >> jiao at tacc.utexas.edu | (832) 303-1166 >> >> -- >> You received this message because you are subscribed to the Google >>Groups >> "GMOD Help Desk" group. >> To post to this group, send email to gmod-help-desk at googlegroups.com. >> To unsubscribe from this group, send email to >> gmod-help-desk+unsubscribe at googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > > >-- >Amelia Ireland >GMOD Community Support Specialist || http://gmod.org > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From parulk at caltech.edu Wed Oct 3 21:14:36 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Wed, 3 Oct 2012 19:14:36 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs Message-ID: <2553.131.215.15.234.1349316876.squirrel@webmail.caltech.edu> I am running maker on example data that comes along with installation and is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note Please advice on the aforementioned error. --------------------- Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_proteins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est%2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_protein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_proteins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est%2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_protein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ------------------------------------ Many thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jiao at tacc.utexas.edu Wed Oct 3 23:04:32 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 04:04:32 +0000 Subject: [maker-devel] problem running maker Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868CD39@EXMBX05.austin.utexas.edu> Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 07:27:05 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 08:27:05 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868CD39@EXMBX05.austin.utexas.edu> Message-ID: Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 07:34:38 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 08:34:38 -0400 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <2553.131.215.15.234.1349316876.squirrel@webmail.caltech.edu> Message-ID: You probably need to reinstall Bio-perl. Other things that can cause the same error are setting TMP in the maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or sometimes setting it to an NFS mount. A full drive can cause this as well or a broken Berkley DB. Use df -h to see if any of the drives used are full (either current working directory or TMP location). You can also swap Berkley DB for a different backend by setting AnyDBM_ISA during setup. Example: cd ../maker/src/ perl Build.PL --AnyDBM_ISA SDBM_File ./Build install --Carson On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >I am running maker on example data that comes along with installation and >is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note > >Please advice on the aforementioned error. > >--------------------- >Maker is now finished!!! > >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >eins%2Efasta.repeatrunner >deleted:0 hits > in cluster:shadow cluster... > i_size:0 j_size:0 > sorting hits in shadow cluster... >... finished. >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >%2Efasta.blastn >deleted:-1 hits >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >tein%2Efasta.blastx >WARNING: Multiple MAKER processes have been started in the >same directory. > >deleted:0 hits >WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >stop here:dpp-mRNA-4 >ERROR: Fasta index error > >FATAL ERROR >Maker is now finished!!! > >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >eins%2Efasta.repeatrunner >deleted:0 hits > in cluster:shadow cluster... > i_size:0 j_size:0 > sorting hits in shadow cluster... >... finished. >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >%2Efasta.blastn >deleted:-1 hits >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >tein%2Efasta.blastx >WARNING: Multiple MAKER processes have been started in the >same directory. > >deleted:0 hits >WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >stop here:dpp-mRNA-4 >ERROR: Fasta index error > >FATAL ERROR >ERROR: Failed while polishig ESTs!! > >ERROR: Chunk failed at level 14 >!! >FAILED CONTIG:contig-dpp-500-500 > > > >ERROR: Chunk failed at level 14 >!! >FAILED CONTIG:contig-dpp-500-500 >------------------------------------ > > >Many thanks, >Parul Kudtarkar > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From jiao at tacc.utexas.edu Thu Oct 4 09:02:14 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 14:02:14 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D305@EXMBX05.austin.utexas.edu> 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 09:03:01 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 10:03:01 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D305@EXMBX05.austin.utexas.edu> Message-ID: Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 10:36:22 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 15:36:22 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 12:17:08 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 13:17:08 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Message-ID: Seg faults really only happen from C and not Perl, so there may be a perl module that is broken on your machine that use C internally. Here are some modules that MAKER can use that I know use C and that you might want to reinstall from CPAN. DB_File Forks Proc::ProcessTable Inline::C Also during the maker install say no to the "use MPI" question. It's best to make the installation as simple as possible before getting into MPI configuration. Also after installing those and rerunning maker's 'perl Build.PL' and './Build install' steps do your first run with 'maker --debug' (redirect STDERR to a file). Then if it fails, you can send me the error log. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 11:36 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 12:23:52 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 17:23:52 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D36D@EXMBX05.austin.utexas.edu> I tried the debug trick and it worked (at least it finished). I however when I looked at the stderr, I again noticed the "can't locate GDBM_File for @AnyDBM_File::ISA" error. Although it did not affect the output, is it what is causing the segmentation error? Because this is the only error other than a bunch of "UNKOWN Bio" messages. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 12:17 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Seg faults really only happen from C and not Perl, so there may be a perl module that is broken on your machine that use C internally. Here are some modules that MAKER can use that I know use C and that you might want to reinstall from CPAN. DB_File Forks Proc::ProcessTable Inline::C Also during the maker install say no to the "use MPI" question. It's best to make the installation as simple as possible before getting into MPI configuration. Also after installing those and rerunning maker's 'perl Build.PL' and './Build install' steps do your first run with 'maker --debug' (redirect STDERR to a file). Then if it fails, you can send me the error log. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 11:36 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:52:51 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 14:52:51 -0400 Subject: [maker-devel] Expected format for EST GFF3 Message-ID: Greetings. I would like to use Maker's *est_gff* option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 13:58:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 14:58:56 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Match/match_part features are expected. MAKER expects these to be polished (I.e. correctly aligned around splice sites). For example exonerate, BLAT, and cufflinks results will be correct around splice sites and can be used; BLAST results on the other hand will not be correct and should not be used. The Gap attribute is used by maker if available, but is not required (Gap describes how to reconstruct an alignment for gaps and mismatches). Otherwise MAKER assumes all positions are matches to the reference. Thanks, Carson From: Daniel Standage Date: Thursday, 4 October, 2012 2:52 PM To: Maker Mailing List Subject: [maker-devel] Expected format for EST GFF3 Greetings. I would like to use Maker's est_gff option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 14:07:18 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:07:18 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the *transcript* features to *match* and the *exon* features to *match_part *and make sure the parent/child relationships are maintained with the *Parent* and *ID* attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > Match/match_part features are expected. MAKER expects these to be > polished (I.e. correctly aligned around splice sites). For example > exonerate, BLAT, and cufflinks results will be correct around splice sites > and can be used; BLAST results on the other hand will not be correct and > should not be used. > > The Gap attribute is used by maker if available, but is not required (Gap > describes how to reconstruct an alignment for gaps and mismatches). > Otherwise MAKER assumes all positions are matches to the reference. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM > To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 > > Greetings. > > I would like to use Maker's *est_gff* option to include EST evidence from > an external GFF3 file. In addition to being a valid, well-formed GFF3 file, > what expectations does Maker assume about the contents of this file? Which > feature types are expected and/or supported? Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 14:09:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 15:09:25 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Use the cufflinks2gff3 script that comes with MAKER. Thanks, Carson From: Daniel Standage Date: Thursday, 4 October, 2012 3:07 PM To: Carson Holt Cc: Maker Mailing List Subject: Re: [maker-devel] Expected format for EST GFF3 Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only transcript and exon features. So I'm assuming I can simply convert the transcript features to match and the exon features to match_part and make sure the parent/child relationships are maintained with the Parent and ID attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > Match/match_part features are expected. MAKER expects these to be polished > (I.e. correctly aligned around splice sites). For example exonerate, BLAT, > and cufflinks results will be correct around splice sites and can be used; > BLAST results on the other hand will not be correct and should not be used. > > The Gap attribute is used by maker if available, but is not required (Gap > describes how to reconstruct an alignment for gaps and mismatches). Otherwise > MAKER assumes all positions are matches to the reference. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM > To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 > > Greetings. > > I would like to use Maker's est_gff option to include EST evidence from an > external GFF3 file. In addition to being a valid, well-formed GFF3 file, what > expectations does Maker assume about the contents of this file? Which feature > types are expected and/or supported? Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/mak > er-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 14:16:53 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:16:53 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Does the cufflinks2gff3 script filter out single-exon transcripts? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt wrote: > Use the cufflinks2gff3 script that comes with MAKER. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 3:07 PM > To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 > > Great. I am using PE Illumina reads mapped by Tophat and assembled by > Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the > *transcript* features to *match* and the *exon* features to *match_part *and > make sure the parent/child relationships are maintained with the *Parent* and > *ID* attributes? > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > >> Match/match_part features are expected. MAKER expects these to be >> polished (I.e. correctly aligned around splice sites). For example >> exonerate, BLAT, and cufflinks results will be correct around splice sites >> and can be used; BLAST results on the other hand will not be correct and >> should not be used. >> >> The Gap attribute is used by maker if available, but is not required (Gap >> describes how to reconstruct an alignment for gaps and mismatches). >> Otherwise MAKER assumes all positions are matches to the reference. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Thursday, 4 October, 2012 2:52 PM >> To: Maker Mailing List >> Subject: [maker-devel] Expected format for EST GFF3 >> >> Greetings. >> >> I would like to use Maker's *est_gff* option to include EST evidence >> from an external GFF3 file. In addition to being a valid, well-formed GFF3 >> file, what expectations does Maker assume about the contents of this file? >> Which feature types are expected and/or supported? Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 14:21:52 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:21:52 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Good to know. Thanks for your help! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:19 PM, Carson Holt wrote: > Yes. You really have to because the single exon transcripts produced > from mRNA-seq assembly are strandless (you don't know where they belong). > They also tend to be heavily weighted to pseudogenes and transposons. > > Thanks, > Carson > > > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 3:16 PM > > To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 > > Does the cufflinks2gff3 script filter out single-exon transcripts? > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt wrote: > >> Use the cufflinks2gff3 script that comes with MAKER. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Thursday, 4 October, 2012 3:07 PM >> To: Carson Holt >> Cc: Maker Mailing List >> Subject: Re: [maker-devel] Expected format for EST GFF3 >> >> Great. I am using PE Illumina reads mapped by Tophat and assembled by >> Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the >> *transcript* features to *match* and the *exon* features to *match_part *and >> make sure the parent/child relationships are maintained with the *Parent* and >> *ID* attributes? >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: >> >>> Match/match_part features are expected. MAKER expects these to be >>> polished (I.e. correctly aligned around splice sites). For example >>> exonerate, BLAT, and cufflinks results will be correct around splice sites >>> and can be used; BLAST results on the other hand will not be correct and >>> should not be used. >>> >>> The Gap attribute is used by maker if available, but is not required >>> (Gap describes how to reconstruct an alignment for gaps and mismatches). >>> Otherwise MAKER assumes all positions are matches to the reference. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Thursday, 4 October, 2012 2:52 PM >>> To: Maker Mailing List >>> Subject: [maker-devel] Expected format for EST GFF3 >>> >>> Greetings. >>> >>> I would like to use Maker's *est_gff* option to include EST evidence >>> from an external GFF3 file. In addition to being a valid, well-formed GFF3 >>> file, what expectations does Maker assume about the contents of this file? >>> Which feature types are expected and/or supported? Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> _______________________________________________ maker-devel mailing >>> list maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy.semeiks at utsw.edu Thu Oct 4 14:39:25 2012 From: jeremy.semeiks at utsw.edu (Jeremy Semeiks) Date: Thu, 4 Oct 2012 14:39:25 -0500 Subject: [maker-devel] maker hung on a contig Message-ID: Hi, I ran maker 2.26 beta on a mammalian assembly containing ~200,000 scaffolds. I ran like this: $ /usr/bin/time mpiexec -n 30 maker -q < /dev/null > maker.oe 2>&1 & disown -h After 1--2 weeks, all but one contig had finished successfully, although maker needed a retry on ~80 others. The one contig that failed is small (1,146 nt) and not obviously weird. maker hung on this one contig, outputting the following type of stack-trace-looking thing to maker.oe, over and over: """ eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xdd9d480)', 'Error::Simple=HASH(0xa9a56f8)', undef, 'ARRAY(0x9ecc710)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xf1b6a40)', 'HASH(0xdd9d480)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0xab33f30)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0xab33f30)', 'SCALAR(0x17cd3a90)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xb3a58e0)', 'Error::Simple=HASH(0xab33f30)', undef, 'ARRAY(0x77ffae0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xbe47098)', 'HASH(0xb3a58e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0xc86fb88)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0xc86fb88)', 'SCALAR(0xb504490)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xc2d7d00)', 'Error::Simple=HASH(0xc86fb88)', undef, 'ARRAY(0x767a290)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0x5ab13f0)', 'HASH(0xc2d7d00)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0x18943010)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0x18943010)', 'SCALAR(0xa6e7cd0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 """ Every 100,000 lines or so, that sort of pattern is interrupted by this sort of pattern: """ Process::MpiTiers::__ANON__('Error::Simple=HASH(0xd9816c8)', 'SCALAR(0x2558c48)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0x1bd77ab8)', 'Error::Simple=HASH(0xd9816c8)', undef, 'ARRAY(0x2561948)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xf48b0d8)', 'HASH(0x1bd77ab8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 285 Process::MpiTiers::run_all('Process::MpiTiers=HASH(0xc10a7e0)', 0) calledERROR: Failed while builing masking tiers ERROR: Can not get next level ERROR: Can't open seq file: /export/home9/jrs/bmy/maker02/bmy.min1e3.maker.output/bmy.min1e3_datastore/E0/1F/C16738816//theVoid.C16738816/query.masked.gff.seq No such file or directory at /home/jrs/maker-2.26-beta/bin/../lib/Dumper/GFF/GFFV3.pm line 173. Dumper::GFF::GFFV3::finalize('Dumper::GFF::GFFV3=HASH(0x4c328c8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 700 Process::MpiChunk::__ANON__() called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 415 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0xfc1d208)', 'HASH(0xfc1fbd8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 3768 Process::MpiChunk::_go('Process::MpiChunk=HASH(0x47162a0)', 'flow', 'HASH(0x93ba680)', 2, 0) called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 369 """ The makers kept outputting these things until I killed them all, and also mpiexec, with kill -s SIGKILL. These strack traces go on for at least the last million lines of maker.oe, which is ~100 GB. Because the log is so big, without knowing what to grep for it's hard for me to suggest what originally went wrong. The contig's datastore directory does not contain anything obvious. Its run.log indicates only that it was the second try for this contig. None of the files in this directory (including theVoid) had been updated since 3 days ago. So this contig itself may even be a red herring. maker's protein output for all the other contigs seems sane, and this one contig is probably too small to contain any proteins, so it doesn't matter to me if maker finishes it successfully. I just want to figure out how I can keep maker from hanging. Suggestions? Thanks, Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 15:42:52 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 20:42:52 +0000 Subject: [maker-devel] maker mpi failed Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D692@EXMBX05.austin.utexas.edu> Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 15:53:08 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 16:53:08 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D692@EXMBX05.austin.utexas.edu> Message-ID: You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 16:20:48 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 21:20:48 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D7C8@EXMBX05.austin.utexas.edu> Hi Carson, I didn't know maker does not work with MVAPICH2. The cluster I was installing maker on only has mvapich2. So I guess I will have to install mpich2 under my directory. However, I installed perl threading based on mvapich2. Does that mean I need to reinstall perl with mpich2 before reinstalling maker? Oscar -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Thu Oct 4 18:06:13 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Thu, 4 Oct 2012 16:06:13 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: References: Message-ID: <1764.131.215.15.234.1349391973.squirrel@webmail.caltech.edu> Hi, Thanks, reinstalling BioPerl fixed the issue. Now we are training our genome to generate training dataset for SNAP and Augustus, as per the instructions provided at http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors However the training dataset would have multiple gff3 prediction files, one for each contig. Is it recommended to cat(command) all the gff3 files to generate a single file and finally generate hmm file with the steps mentioned in the tutorial. Would the same hmm file work as training dataset for AUGUSTUS? Many thanks, Parul Kudtarkar > You probably need to reinstall Bio-perl. > > Other things that can cause the same error are setting TMP in the > maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or > sometimes setting it to an NFS mount. > A full drive can cause this as well or a broken Berkley DB. Use df -h to > see if any of the drives used are full (either current working directory > or TMP location). You can also swap Berkley DB for a different backend by > setting AnyDBM_ISA during setup. > > Example: > cd ../maker/src/ > perl Build.PL --AnyDBM_ISA SDBM_File > ./Build install > > --Carson > > > > On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: > >>I am running maker on example data that comes along with installation and >>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >> >>Please advice on the aforementioned error. >> >>--------------------- >>Maker is now finished!!! >> >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >>eins%2Efasta.repeatrunner >>deleted:0 hits >> in cluster:shadow cluster... >> i_size:0 j_size:0 >> sorting hits in shadow cluster... >>... finished. >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >>%2Efasta.blastn >>deleted:-1 hits >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >>tein%2Efasta.blastx >>WARNING: Multiple MAKER processes have been started in the >>same directory. >> >>deleted:0 hits >>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>stop here:dpp-mRNA-4 >>ERROR: Fasta index error >> >>FATAL ERROR >>Maker is now finished!!! >> >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >>eins%2Efasta.repeatrunner >>deleted:0 hits >> in cluster:shadow cluster... >> i_size:0 j_size:0 >> sorting hits in shadow cluster... >>... finished. >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >>%2Efasta.blastn >>deleted:-1 hits >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >>tein%2Efasta.blastx >>WARNING: Multiple MAKER processes have been started in the >>same directory. >> >>deleted:0 hits >>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>stop here:dpp-mRNA-4 >>ERROR: Fasta index error >> >>FATAL ERROR >>ERROR: Failed while polishig ESTs!! >> >>ERROR: Chunk failed at level 14 >>!! >>FAILED CONTIG:contig-dpp-500-500 >> >> >> >>ERROR: Chunk failed at level 14 >>!! >>FAILED CONTIG:contig-dpp-500-500 >>------------------------------------ >> >> >>Many thanks, >>Parul Kudtarkar >> >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org >> >> >>_______________________________________________ >>maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From david.powell at monash.edu Thu Oct 4 18:44:32 2012 From: david.powell at monash.edu (David Powell) Date: Fri, 5 Oct 2012 09:44:32 +1000 Subject: [maker-devel] problem running maker In-Reply-To: References: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Message-ID: I've also had the problem with a segfault when running MAKER 2.26. I'm not sure it is the one experienced here, but in the end I was able to work around it. I tracked the problem to lib/FastA.pm in the _safe_new function. Specifically, in the local handler installed for warnings. By removing the "die" call, I was able to stop the perl process from segfaulting. I suspect this is not a great fix since it will ignore any warnings from Bio::DB::Fasta, but it worked for me (I did check the warnings generated were not important). Cheers, -- David Powell On 5 October 2012 03:17, Carson Holt wrote: > Seg faults really only happen from C and not Perl, so there may be a perl > module that is broken on your machine that use C internally. > > Here are some modules that MAKER can use that I know use C and that you > might want to reinstall from CPAN. > > DB_File > Forks > Proc::ProcessTable > Inline::C > > Also during the maker install say no to the "use MPI" question. It's best > to make the installation as simple as possible before getting into MPI > configuration. > > Also after installing those and rerunning maker's 'perl Build.PL' and > './Build install' steps do your first run with 'maker --debug' (redirect > STDERR to a file). Then if it fails, you can send me the error log. > > Thanks, > Carson > > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 11:36 AM > > To: Carson Holt , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Ok. So I installed 2.26 instead. When I ran maker again I got segmentation > fault with no additional message. I am using dpp_contig.fasta, > dpp_est.fasta and dpp_protein.fasta. > > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM > To: Oscar Jiao , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Try 2.26 and let me know. > > Thanks, > Carson > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM > To: Carson Holt , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > 2.10 > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM > To: Oscar Jiao , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Are you trying to install 2.26 or 2.10? > > --Carson > > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM > To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker > > Hi, > > I just installed maker and tried to run it with these dpp example fasta > files that came with the package. However when I do "maker maker_opts.ctl > maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like > there are three errors: (1) defined array deprecated; (2) unknown state in > Signal.pm and (3) GDBM_File (etc.) missing > > The first line seems to be just warning. I got it even when I do just > "maker" or "maker ctl". The control files did get generated. And what is > the relationship between the three perl modules, GDBM/NDBM/SDBM_File, > AnyDBM and DB_File? > > Oscar > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 21:24:16 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 22:24:16 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D7C8@EXMBX05.austin.utexas.edu> Message-ID: I don't think you will need to reinstall perl (you may but I don't think so). MVAPICH2 doesn't work because they have overridden malloc with their own version which causes weirdness with shared libraries in Perl. I've submitted some bug reports to the MVAPICH2 developers and made modifications to the development version of MAKER, and had some progress, but I am still not getting reliable behavior in MVAPICH2 when compiling for the OFA-IB-Nemesis or OFA-IB-CH3 interfaces. So for now using MPICH2 is your best choice. You lose some of the infiniband optimizations but IO rather than messages passing is MAKER's true bottleneck (so you won't get a performance hit by using MPICH2). --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 5:20 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Hi Carson, I didn't know maker does not work with MVAPICH2. The cluster I was installing maker on only has mvapich2. So I guess I will have to install mpich2 under my directory. However, I installed perl threading based on mvapich2. Does that mean I need to reinstall perl with mpich2 before reinstalling maker? Oscar -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 21:51:19 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 22:51:19 -0400 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <1764.131.215.15.234.1349391973.squirrel@webmail.caltech.edu> Message-ID: Just concatenating won't work. Use the gff3_merge tool to safely merge separate GFF3 files. In the tutorial dataset their should only be one contig to train with, but with real data you will have multiple contigs and will want to merge the GFF3 files. Augustus has a separate training procedure that is a little more complicated than SNAP's. You will need to read their documentation on training which is a little cryptic. I know Jason Stajich developed a tool for training augustus from snap results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be willing to share it with you ;-) Thanks, Carson On 12-10-04 7:06 PM, "Parul Kudtarkar" wrote: >Hi, > >Thanks, reinstalling BioPerl fixed the issue. Now we are training our >genome to generate training dataset for SNAP and Augustus, as per the >instructions provided at >http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors > >However the training dataset would have multiple gff3 prediction files, >one for each contig. Is it recommended to cat(command) all the gff3 files >to generate a single file and finally generate hmm file with the steps >mentioned in the tutorial. Would the same hmm file work as training >dataset for AUGUSTUS? > >Many thanks, >Parul Kudtarkar > >> You probably need to reinstall Bio-perl. >> >> Other things that can cause the same error are setting TMP in the >> maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or >> sometimes setting it to an NFS mount. >> A full drive can cause this as well or a broken Berkley DB. Use df -h >>to >> see if any of the drives used are full (either current working directory >> or TMP location). You can also swap Berkley DB for a different backend >>by >> setting AnyDBM_ISA during setup. >> >> Example: >> cd ../maker/src/ >> perl Build.PL --AnyDBM_ISA SDBM_File >> ./Build install >> >> --Carson >> >> >> >> On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >> >>>I am running maker on example data that comes along with installation >>>and >>>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >>> >>>Please advice on the aforementioned error. >>> >>>--------------------- >>>Maker is now finished!!! >>> >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>ot >>>eins%2Efasta.repeatrunner >>>deleted:0 hits >>> in cluster:shadow cluster... >>> i_size:0 j_size:0 >>> sorting hits in shadow cluster... >>>... finished. >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>st >>>%2Efasta.blastn >>>deleted:-1 hits >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>ro >>>tein%2Efasta.blastx >>>WARNING: Multiple MAKER processes have been started in the >>>same directory. >>> >>>deleted:0 hits >>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>stop here:dpp-mRNA-4 >>>ERROR: Fasta index error >>> >>>FATAL ERROR >>>Maker is now finished!!! >>> >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>ot >>>eins%2Efasta.repeatrunner >>>deleted:0 hits >>> in cluster:shadow cluster... >>> i_size:0 j_size:0 >>> sorting hits in shadow cluster... >>>... finished. >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>st >>>%2Efasta.blastn >>>deleted:-1 hits >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>ro >>>tein%2Efasta.blastx >>>WARNING: Multiple MAKER processes have been started in the >>>same directory. >>> >>>deleted:0 hits >>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>stop here:dpp-mRNA-4 >>>ERROR: Fasta index error >>> >>>FATAL ERROR >>>ERROR: Failed while polishig ESTs!! >>> >>>ERROR: Chunk failed at level 14 >>>!! >>>FAILED CONTIG:contig-dpp-500-500 >>> >>> >>> >>>ERROR: Chunk failed at level 14 >>>!! >>>FAILED CONTIG:contig-dpp-500-500 >>>------------------------------------ >>> >>> >>>Many thanks, >>>Parul Kudtarkar >>> >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org >>> >>> >>>_______________________________________________ >>>maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > From jiao at tacc.utexas.edu Fri Oct 5 00:06:48 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 05:06:48 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868DBE8@EXMBX05.austin.utexas.edu> I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Fri Oct 5 00:34:37 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Thu, 4 Oct 2012 22:34:37 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: References: Message-ID: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> Carson, thanks for very quick response. Jason, could you please elaborate/share document if any on using zff2augustus_gbk.pl Thanks, Parul Kudtarkar > Just concatenating won't work. Use the gff3_merge tool to safely merge > separate GFF3 files. > > In the tutorial dataset their should only be one contig to train with, but > with real data you will have multiple contigs and will want to merge the > GFF3 files. > > Augustus has a separate training procedure that is a little more > complicated than SNAP's. You will need to read their documentation on > training which is a little cryptic. > > I know Jason Stajich developed a tool for training augustus from snap > results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be > willing to share it with you ;-) > > Thanks, > Carson > > > > On 12-10-04 7:06 PM, "Parul Kudtarkar" wrote: > >>Hi, >> >>Thanks, reinstalling BioPerl fixed the issue. Now we are training our >>genome to generate training dataset for SNAP and Augustus, as per the >>instructions provided at >>http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors >> >>However the training dataset would have multiple gff3 prediction files, >>one for each contig. Is it recommended to cat(command) all the gff3 files >>to generate a single file and finally generate hmm file with the steps >>mentioned in the tutorial. Would the same hmm file work as training >>dataset for AUGUSTUS? >> >>Many thanks, >>Parul Kudtarkar >> >>> You probably need to reinstall Bio-perl. >>> >>> Other things that can cause the same error are setting TMP in the >>> maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) >>> or >>> sometimes setting it to an NFS mount. >>> A full drive can cause this as well or a broken Berkley DB. Use df -h >>>to >>> see if any of the drives used are full (either current working >>> directory >>> or TMP location). You can also swap Berkley DB for a different backend >>>by >>> setting AnyDBM_ISA during setup. >>> >>> Example: >>> cd ../maker/src/ >>> perl Build.PL --AnyDBM_ISA SDBM_File >>> ./Build install >>> >>> --Carson >>> >>> >>> >>> On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >>> >>>>I am running maker on example data that comes along with installation >>>>and >>>>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >>>> >>>>Please advice on the aforementioned error. >>>> >>>>--------------------- >>>>Maker is now finished!!! >>>> >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>>ot >>>>eins%2Efasta.repeatrunner >>>>deleted:0 hits >>>> in cluster:shadow cluster... >>>> i_size:0 j_size:0 >>>> sorting hits in shadow cluster... >>>>... finished. >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>>st >>>>%2Efasta.blastn >>>>deleted:-1 hits >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>>ro >>>>tein%2Efasta.blastx >>>>WARNING: Multiple MAKER processes have been started in the >>>>same directory. >>>> >>>>deleted:0 hits >>>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>>stop here:dpp-mRNA-4 >>>>ERROR: Fasta index error >>>> >>>>FATAL ERROR >>>>Maker is now finished!!! >>>> >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>>ot >>>>eins%2Efasta.repeatrunner >>>>deleted:0 hits >>>> in cluster:shadow cluster... >>>> i_size:0 j_size:0 >>>> sorting hits in shadow cluster... >>>>... finished. >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>>st >>>>%2Efasta.blastn >>>>deleted:-1 hits >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>>ro >>>>tein%2Efasta.blastx >>>>WARNING: Multiple MAKER processes have been started in the >>>>same directory. >>>> >>>>deleted:0 hits >>>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>>stop here:dpp-mRNA-4 >>>>ERROR: Fasta index error >>>> >>>>FATAL ERROR >>>>ERROR: Failed while polishig ESTs!! >>>> >>>>ERROR: Chunk failed at level 14 >>>>!! >>>>FAILED CONTIG:contig-dpp-500-500 >>>> >>>> >>>> >>>>ERROR: Chunk failed at level 14 >>>>!! >>>>FAILED CONTIG:contig-dpp-500-500 >>>>------------------------------------ >>>> >>>> >>>>Many thanks, >>>>Parul Kudtarkar >>>> >>>>-- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org >>>> >>>> >>>>_______________________________________________ >>>>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> >> >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org >> > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jason.stajich at ucr.edu Fri Oct 5 02:02:50 2012 From: jason.stajich at ucr.edu (Jason E Stajich) Date: Fri, 5 Oct 2012 07:02:50 +0000 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> References: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> Message-ID: <41A4465BD48EEE47A449FA03108C852A073651E2@EXCH-MBOX-3.exch.ucr.edu> I wrote up how to use it on this list msg - http://brie4.cshl.edu/pipermail/gmod-help/2012-June/001724.html Script is in this github repo - https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl Patches and documentation updates are always welcomed, I'll get around to that eventually. Jason On Oct 4, 2012, at 10:34 PM, Parul Kudtarkar > wrote: Carson, thanks for very quick response. Jason, could you please elaborate/share document if any on using zff2augustus_gbk.pl Thanks, Parul Kudtarkar Just concatenating won't work. Use the gff3_merge tool to safely merge separate GFF3 files. In the tutorial dataset their should only be one contig to train with, but with real data you will have multiple contigs and will want to merge the GFF3 files. Augustus has a separate training procedure that is a little more complicated than SNAP's. You will need to read their documentation on training which is a little cryptic. I know Jason Stajich developed a tool for training augustus from snap results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be willing to share it with you ;-) Thanks, Carson On 12-10-04 7:06 PM, "Parul Kudtarkar" > wrote: Hi, Thanks, reinstalling BioPerl fixed the issue. Now we are training our genome to generate training dataset for SNAP and Augustus, as per the instructions provided at http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors However the training dataset would have multiple gff3 prediction files, one for each contig. Is it recommended to cat(command) all the gff3 files to generate a single file and finally generate hmm file with the steps mentioned in the tutorial. Would the same hmm file work as training dataset for AUGUSTUS? Many thanks, Parul Kudtarkar You probably need to reinstall Bio-perl. Other things that can cause the same error are setting TMP in the maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or sometimes setting it to an NFS mount. A full drive can cause this as well or a broken Berkley DB. Use df -h to see if any of the drives used are full (either current working directory or TMP location). You can also swap Berkley DB for a different backend by setting AnyDBM_ISA during setup. Example: cd ../maker/src/ perl Build.PL --AnyDBM_ISA SDBM_File ./Build install --Carson On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: I am running maker on example data that comes along with installation and is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note Please advice on the aforementioned error. --------------------- Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr ot eins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e st %2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p ro tein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr ot eins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e st %2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p ro tein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ------------------------------------ Many thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -- Jason E Stajich, PhD Assistant Professor Plant Pathology & Microbiology University of California, Riverside 951.827.2363 http://lab.stajich.org http://fungalgenomes.org http://fungidb.org http://1000.fungalgenomes.org/ twitter @stajichlab @hyphaltip @fungalgenomes @fungidb http://plantpathology.ucr.edu http://genomics.ucr.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From Carson.Holt at oicr.on.ca Thu Oct 4 14:19:57 2012 From: Carson.Holt at oicr.on.ca (Carson Holt) Date: Thu, 4 Oct 2012 19:19:57 +0000 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Yes. You really have to because the single exon transcripts produced from mRNA-seq assembly are strandless (you don't know where they belong). They also tend to be heavily weighted to pseudogenes and transposons. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 3:16 PM To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 Does the cufflinks2gff3 script filter out single-exon transcripts? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt > wrote: Use the cufflinks2gff3 script that comes with MAKER. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 3:07 PM To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only transcript and exon features. So I'm assuming I can simply convert the transcript features to match and the exon features to match_part and make sure the parent/child relationships are maintained with the Parent and ID attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt > wrote: Match/match_part features are expected. MAKER expects these to be polished (I.e. correctly aligned around splice sites). For example exonerate, BLAT, and cufflinks results will be correct around splice sites and can be used; BLAST results on the other hand will not be correct and should not be used. The Gap attribute is used by maker if available, but is not required (Gap describes how to reconstruct an alignment for gaps and mismatches). Otherwise MAKER assumes all positions are matches to the reference. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 Greetings. I would like to use Maker's est_gff option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 11:05:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 12:05:56 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868DBE8@EXMBX05.austin.utexas.edu> Message-ID: Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 11:34:12 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 12:34:12 -0400 Subject: [maker-devel] editing MAKER mailing list options Message-ID: Just a small note to subscribers to the MAKER e-mail list. Being as there are spurts of high volume list activity that may be a nuisance to some, if you want to edit your mailing list options visit this link --> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org You can then change to digest mode if you don't want to receive every single message posted to the list, or you can set your options to be subscribed to but not receive messages from the list (so you can post and get help but won't get everyone else's posts). Enter your e-mail address in the box at the very very bottom of the page to edit option. Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 13:19:15 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 14:19:15 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E519@EXMBX05.austin.utexas.edu> Message-ID: Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 13:16:41 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 18:16:41 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E519@EXMBX05.austin.utexas.edu> See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: maker_opts.ctl Type: application/octet-stream Size: 4468 bytes Desc: maker_opts.ctl URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mpich-maker-debug.log Type: application/octet-stream Size: 315735 bytes Desc: mpich-maker-debug.log URL: From jiao at tacc.utexas.edu Fri Oct 5 14:55:39 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 19:55:39 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E7A3@EXMBX05.austin.utexas.edu> I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 15:03:09 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 16:03:09 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E7A3@EXMBX05.austin.utexas.edu> Message-ID: What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Ap plication/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.p m line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/M PI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 15:06:02 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 20:06:02 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E866@EXMBX05.austin.utexas.edu> Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 15:07:18 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 16:07:18 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E866@EXMBX05.austin.utexas.edu> Message-ID: Is /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2 a new install of mpich2 or is it the install you did with 2.26 just copied over? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 4:06 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Ap plication/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.p m line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/M PI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 15:08:35 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 20:08:35 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E8D6@EXMBX05.austin.utexas.edu> It is a new install. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:07 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Is /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2 a new install of mpich2 or is it the install you did with 2.26 just copied over? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 4:06 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisi.hahni at gmail.com Tue Oct 9 06:07:14 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 09 Oct 2012 13:07:14 +0200 Subject: [maker-devel] install maker on cluster Message-ID: <50740562.9010904@gmail.com> Hello maker-team, I am trying to install maker 2.25 on a cluster, without root privileges (I managed before, but now after migration to a new cluster I cant seem to get it to work). All necessary prerequisite programs are installed and in the path (I followed the steps in the INSTALL file that comes with maker2.25). When I do: perl Build.PL, I get: Checking prerequisites... requires: ! DBD::SQLite is not installed ! IO::All is not installed ! Inline::C is not installed ! Perl::Unsafe::Signals is not installed ! Proc::ProcessTable is not installed ! Want is not installed ! forks is not installed ! forks::shared is not installed build_requires: ! LWP::Simple is not installed recommends: * DBD::Pg is not installed ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the versions of the modules indicated above before proceeding with this installation Run 'Build installdeps' to install missing prerequisites. MAKER supports distributed parallelization via MPI. Would you like to configure MAKER for MPI (This requires that you have an MPI client installed)? [N ]n WARNING: Apache cannot be located. The optional web based interface to MAKER will not be available to you. Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'MAKER' version '2.25' The file 'Build' has been created for you to finish installing MAKER. ============================================================================== STATUS MAKER 2.25 ============================================================================== PERL Dependencies: MISSING ! forks ! IO::All ! forks::shared ! Want ! DBD::SQLite ! Proc::ProcessTable ! Inline::C ! Perl::Unsafe::Signals External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: DISABLED MWAS Web Interface: DISABLED MAKER PACKAGE: MISSING PREREQUISITES Important Commands: ./Build installdeps #installs missing PERL dependencies ./Build installexes #installs all missing external programs ./Build install #installs MAKER ./Build status #Shows this status menu Other Commands: ./Build repeatmasker #installs RepeatMasker (asks for RepBase) ./Build blast #installs BLAST (NCBI BLAST+) ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) ./Build snap #installs SNAP ./Build augustus #installs Augustus ./Build apollo #installs Apollo ./Build gbrowse #installs GBrowse (must be root) ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) ./Build mpich2 #installs MPICH2 (but manual install recommended) So it seems some Perl Dependencies are missing. As indicated in INSTALL I followed the quick and dirty installation for Bioperl, so I am not sure what I did wrong/missed out. When I run ./Build installdeps, I get: You do not have write access to install missing Modules. I can try and install these locally (i.e. only for MAKER) in the .../maker/perl/lib directory, or you can run './Build installdeps' as root or using sudo and try again. Do want MAKER to try and build a local installation? [N ]y mkdir /usit/titan: Permission denied at /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm line 59 Sorry to bother you with this minor things!! Any help would me much appreciated! much obliged, Christoph From carsonhh at gmail.com Tue Oct 9 11:27:36 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 09 Oct 2012 12:27:36 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <50740562.9010904@gmail.com> Message-ID: You may need to reconfigure your cpan preferences. You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, or by starting cpan from your command line (command is cpan). Then run 'o conf init' inside the cpan ui. --Carson On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >Hello maker-team, > >I am trying to install maker 2.25 on a cluster, without root privileges >(I managed before, but now after migration to a new cluster I cant seem >to get it to work). All necessary prerequisite programs are installed >and in the path (I followed the steps in the INSTALL file that comes >with maker2.25). When I do: perl Build.PL, I get: >Checking prerequisites... > requires: > ! DBD::SQLite is not installed > ! IO::All is not installed > ! Inline::C is not installed > ! Perl::Unsafe::Signals is not installed > ! Proc::ProcessTable is not installed > ! Want is not installed > ! forks is not installed > ! forks::shared is not installed > build_requires: > ! LWP::Simple is not installed > recommends: > * DBD::Pg is not installed > >ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >versions >of the modules indicated above before proceeding with this installation > >Run 'Build installdeps' to install missing prerequisites. > >MAKER supports distributed parallelization via MPI. >Would you like to configure MAKER for MPI (This >requires that you have an MPI client installed)? [N ]n > >WARNING: Apache cannot be located. The optional web based >interface to MAKER will not be available to you. > >Created MYMETA.yml and MYMETA.json >Creating new 'Build' script for 'MAKER' version '2.25' > > >The file 'Build' has been created for you to finish installing MAKER. > > >========================================================================== >==== >STATUS MAKER 2.25 >========================================================================== >==== >PERL Dependencies: MISSING > ! forks > ! IO::All > ! forks::shared > ! Want > ! DBD::SQLite > ! Proc::ProcessTable > ! Inline::C > ! Perl::Unsafe::Signals > >External Programs: VERIFIED >External C Libraries: VERIFIED >MPI SUPPORT: DISABLED >MWAS Web Interface: DISABLED >MAKER PACKAGE: MISSING PREREQUISITES > > >Important Commands: > ./Build installdeps #installs missing PERL dependencies > ./Build installexes #installs all missing external programs > ./Build install #installs MAKER > ./Build status #Shows this status menu > >Other Commands: > ./Build repeatmasker #installs RepeatMasker (asks for RepBase) > ./Build blast #installs BLAST (NCBI BLAST+) > ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on >Mac OSX) > ./Build snap #installs SNAP > ./Build augustus #installs Augustus > ./Build apollo #installs Apollo > ./Build gbrowse #installs GBrowse (must be root) > ./Build jbrowse #installs JBrowse (MAKER copy, not web >accecible) > ./Build mpich2 #installs MPICH2 (but manual install >recommended) > >So it seems some Perl Dependencies are missing. As indicated in INSTALL >I followed the quick and dirty installation for Bioperl, so I am not >sure what I did wrong/missed out. > >When I run ./Build installdeps, I get: >You do not have write access to install missing Modules. >I can try and install these locally (i.e. only for MAKER) >in the .../maker/perl/lib directory, or you can run >'./Build installdeps' as root or using sudo and try again. >Do want MAKER to try and build a local installation? [N ]y >mkdir /usit/titan: Permission denied at >/cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm line >59 > >Sorry to bother you with this minor things!! Any help would me much >appreciated! > >much obliged, >Christoph > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From kippjohnson at uchicago.edu Thu Oct 11 14:43:32 2012 From: kippjohnson at uchicago.edu (Kipp Johnson) Date: Thu, 11 Oct 2012 14:43:32 -0500 Subject: [maker-devel] Interpreting Maker Results Message-ID: Hi Carson, I'm trying to get my genetic annotation out of maker. My maker run on a non-model eukaryote finished, and I used your gff3_merge script to merge the resulting files. This file is enormous, because I used snap, augustus, genemark, repeatmasker, exonerate, and blast, and has a lot of entries from all of these different programs. I want to extract only the genetic regions predicted by maker, so I used the gff3_merge script with the "-g" option. However, when I do this, I get a maker file that only has about 12,000 genes, while I was expecting around 20,000 genes for our genome. However, when I use the fasta merge tool, however, I get output files (for example, "merged.fasta.all.maker.non_overlapping_ab_initio.proteins.fasta") with about 21,000 proteins, which is closer to the gene number that I was expecting. Does the "-g" option ignore evidence from blast/exonerate or similar? How should I extract the complete set of genetic regions to blast against, so that I can go about further working on the annotation? Also, what is maker using to find these 9,000 extra proteins? Are these these all alternately sliced or something along those lines? I can't find any documentation online for how to actually get the final annotations out of maker correctly. Thanks for your time! Best, Kipp Johnson kippjohnson at uchicago.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 11 17:21:49 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 11 Oct 2012 18:21:49 -0400 Subject: [maker-devel] Interpreting Maker Results In-Reply-To: Message-ID: The non_overlapping_ab_initio.proteins.fasta file contain models that were rejected for lack of homology evidence that do not overlap the maker models. So if you accept everything it is not 21,000 proteins rather it's 33,000 (12,000 + 21,000). This is because the contents of that file do not occupy the same regions as the gene models (I.e. non-overlapping). Ab initio predictors have a tendency to overpredict which is why there are so many models. If you think you are missing genes you can try and add additional evidence to the maker run (I.e all proteins from a few related species). Also try running Cegma (http://korflab.ucdavis.edu/datasets/cegma/) to estimate the completeness of you assembly. Sometime a lower number than expected can be attributed to an incomplete assembly. Also you can run something like InterProScan to identify models in the non_overlapping_ab_initio.proteins.fasta file that contain Uniprot protein domains (likely to be real genes), then add them to your results as a second step using maker's model_gff option. I've attached a couple of scripts that can help with that. gff3_preds2models will turn a set of match/match_part features to gene/mRNA/exon/CDS features. It doesn't check translation of CDS though, so only use gene predictions which should be all CDS. gff3_select is used to select some subset of features from a GFF3 file. Useful for slicing sections of data from a GFF3 file. Thanks, Carson From: Kipp Johnson Date: Thursday, 11 October, 2012 3:43 PM To: , Carson Holt Subject: Interpreting Maker Results Hi Carson, I'm trying to get my genetic annotation out of maker. My maker run on a non-model eukaryote finished, and I used your gff3_merge script to merge the resulting files. This file is enormous, because I used snap, augustus, genemark, repeatmasker, exonerate, and blast, and has a lot of entries from all of these different programs. I want to extract only the genetic regions predicted by maker, so I used the gff3_merge script with the "-g" option. However, when I do this, I get a maker file that only has about 12,000 genes, while I was expecting around 20,000 genes for our genome. However, when I use the fasta merge tool, however, I get output files (for example, "merged.fasta.all.maker.non_overlapping_ab_initio.proteins.fasta") with about 21,000 proteins, which is closer to the gene number that I was expecting. Does the "-g" option ignore evidence from blast/exonerate or similar? How should I extract the complete set of genetic regions to blast against, so that I can go about further working on the annotation? Also, what is maker using to find these 9,000 extra proteins? Are these these all alternately sliced or something along those lines? I can't find any documentation online for how to actually get the final annotations out of maker correctly. Thanks for your time! Best, Kipp Johnson kippjohnson at uchicago.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gff3_preds2models Type: application/octet-stream Size: 4777 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gff3_select Type: application/octet-stream Size: 3236 bytes Desc: not available URL: From parulk at caltech.edu Fri Oct 12 16:46:21 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Fri, 12 Oct 2012 14:46:21 -0700 (PDT) Subject: [maker-devel] Conensus gene model Message-ID: <4092.131.215.15.234.1350078381.squirrel@webmail.caltech.edu> Hi, We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 file as input). The output that we get is combination of various gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? Thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig2106.gff Type: application/octet-stream Size: 10562 bytes Desc: not available URL: From parulk at caltech.edu Mon Oct 15 12:41:54 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 15 Oct 2012 10:41:54 -0700 (PDT) Subject: [maker-devel] Conensus gene model Message-ID: <2992.131.215.15.234.1350322914.squirrel@webmail.caltech.edu> Hello, Please advice on the aforementioned query? Thanks, Parul Kudtarkar ---------------------------- Original Message ---------------------------- Subject: [maker-devel] Conensus gene model From: "Parul Kudtarkar" Date: Fri, October 12, 2012 2:46 pm To: maker-devel at yandell-lab.org -------------------------------------------------------------------------- Hi, We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 file as input). The output that we get is combination of various gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? Thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org_______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig2106.gff Type: application/octet-stream Size: 10562 bytes Desc: not available URL: From carsonhh at gmail.com Mon Oct 15 12:58:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 15 Oct 2012 13:58:25 -0400 Subject: [maker-devel] Conensus gene model In-Reply-To: <2992.131.215.15.234.1350322914.squirrel@webmail.caltech.edu> Message-ID: The contig in question is really too small to get much out of it (only 5 kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb length or longer contigs to glean enough information for optimizing your parameters. The general suggestions for any maker run are to use proteins from a closely related organism or a couple of closely related organisms for the protein= option in maker. Also leave single_exon set to 0, except for certain eukaryotes that have a bias for single exon transcripts (i.e. some fungi and oomycetes). And leave keep_preds set to 0 because ab initio predictors tend to over-predict by a wide margin (lots of false positives). Additional training would really depend on what your other contigs look like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. Thanks, Carson On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >Hello, > >Please advice on the aforementioned query? > >Thanks, >Parul Kudtarkar >---------------------------- Original Message ---------------------------- >Subject: [maker-devel] Conensus gene model >From: "Parul Kudtarkar" >Date: Fri, October 12, 2012 2:46 pm >To: maker-devel at yandell-lab.org >-------------------------------------------------------------------------- > >Hi, > >We are using snap(training set[hmm file] generated using est,protein and >contig file), agustus,genemarkE(we ran it outside maker and have gff3 file >as input). The output that we get is combination of various >gene-predictors and evidences. I have attached sample result file. What >would you recommend to get consensus result set? Bootstrapping the >resulting gff3 file (rerunning maker)? > >Thanks, >Parul Kudtarkar >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Mon Oct 15 15:10:00 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 15 Oct 2012 16:10:00 -0400 Subject: [maker-devel] Conensus gene model In-Reply-To: <4445.131.215.15.234.1350330328.squirrel@webmail.caltech.edu> Message-ID: One thing you seem to be missing is protein evidence. Is this a sea urchin (I looked up some of the ESTs)? If so, I would recommend adding all proteins from the Strongylocentrotus purpuratus genome, then throw in another Deuterstome of your choice. Perhaps you should also add a couple of outgroup organisms like Nematostella vectensis (cnidaria) and a protostome of your choice. Be careful if adding adding to many protostome outgroups (i.e. C. elegans and Drosophila) because a big part of their evolution is gene loss (so distant cnidaria often match deuterstomes better than most protostomes do). You could take the maker results when protein data is included and use it to retrain SNAP again. Even a 22 kb contig is still really short. Is this genome primarily constituted by short contigs like this? I would recommend running CEGMA once on this genome to get an appropriate estimate of how recoverable the genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). Cegma will give you an estimate for genome completeness as well as estimates of what percentage of genes will be found in their entirety and what percent will be partial genes. This is important to do if your genome is fragmented as it will give you a reasonable expectation of what you can expected to recover (as short contigs don't annotate very well - you tend to loose a lot). Thanks, Carson On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >Hi Carson, > >Thanks. I have attached another contig which is 22 kb, with as many as 3 >exons EST alignments. Could you please recommend additional training. We >are currently running maker on the entire contig set and eventually merge >all the gff3 contig predictions. The using suggested parameter/methods we >would like to get a consensus gene-set with minimal false >positives/negatives. > >Thanks, >Parul > >> The contig in question is really too small to get much out of it (only 5 >kb). There was only one single exon EST alignments and a couple of >predictions with no evidence support. Anything smaller than 10 kb is >mostly useless for annotation purposes. You would really need a few >100kb >> length or longer contigs to glean enough information for optimizing your >parameters. >> >> The general suggestions for any maker run are to use proteins from a >closely related organism or a couple of closely related organisms for >the >> protein= option in maker. Also leave single_exon set to 0, except for >certain eukaryotes that have a bias for single exon transcripts (i.e. >some >> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >predictors tend to over-predict by a wide margin (lots of false >> positives). >> >> Additional training would really depend on what your other contigs look >like. Do you have any large contigs? I could look at one of those and >give suggestions but the provided contig is just too short to glean >much. >> >> Thanks, >> Carson >> >> >> >> >> >> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >> >>>Hello, >>>Please advice on the aforementioned query? >>>Thanks, >>>Parul Kudtarkar >>>---------------------------- Original Message >>> ---------------------------- >>>Subject: [maker-devel] Conensus gene model >>>From: "Parul Kudtarkar" >>>Date: Fri, October 12, 2012 2:46 pm >>>To: maker-devel at yandell-lab.org >>>------------------------------------------------------------------------ >>>-- >Hi, >>>We are using snap(training set[hmm file] generated using est,protein and >contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>> file >>>as input). The output that we get is combination of various >>>gene-predictors and evidences. I have attached sample result file. What >would you recommend to get consensus result set? Bootstrapping the >resulting gff3 file (rerunning maker)? >>>Thanks, >>>Parul Kudtarkar >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org_______________________________________________ >maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org_______________________________________________ >maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > From chrisi.hahni at gmail.com Tue Oct 16 09:02:36 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 16:02:36 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D68FC.10201@gmail.com> Thanks for your help Carson! I tried maker2.26, but the error still remains. I also tried to change the TMP= option in the maker_opts file, but no change either.. STATUS: Parsing control files... ERROR: The HMM file[s] provided for gmhmm do not exist: When building the maker2.26 I saw that it is complaining about one more (recommended) dependency (I must have overlooked that before): Checking prerequisites... recommends: * DBD::Pg is not installed I am having problems to get that one.. Is that causing the error? much obliged, Christoph Am 15.10.2012 20:05, schrieb Carson Holt: > It looks like either an NFS issue or perhaps an issue with the location > provided for the TMP= in the maker_opts.ctl file. > > First I would suggest trying maker 2.26 to see if the issue still happens > there (there are a few updates to how NFS locks are handled in 2.26). > > Second if you are setting TMP make sure your disk is not full. Or if TMP > was set to a tmpfs location (in memory filesystem), it could be a full > memory issue in which case you could just set TMP to somewhere else. > > Let me know if you still see the issue in 2.26 or after changing the > location of TMP. > > Thanks, > Carson > > > On 12-10-12 1:09 PM, "Christoph Hahn" wrote: > >> Hi Carson, >> >> Thanks for your reply. I managed to install maker2.25 properly now, but >> when running it I am getting strange errors (two kinds): >> type one: >> STATUS: Parsing control files... >> ERROR: The HMM file[s] provided for gmhmm do not exist: >> >> type two: >> STATUS: Parsing control files... >> ERROR: Cannot get initialization lock. >> >> I have divided the draft assembly to several files, each containing a >> subset of contigs on which I am running a maker instance. I am getting >> the above named error type one for the majority of the instances, >> although the HMM file it is complaining about (es.mod) is actually in >> the path it is naming (not shown above). In a small number of instances >> I am getting the type two error mentioned above. >> >> Can you help me? >> >> much obliged, >> Christoph >> >> >> >> On 09.10.2012 18:27, Carson Holt wrote: >>> You may need to reconfigure your cpan preferences. >>> >>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>> or >>> by starting cpan from your command line (command is cpan). Then run 'o >>> conf init' inside the cpan ui. >>> >>> --Carson >>> >>> >>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>> >>>> Hello maker-team, >>>> >>>> I am trying to install maker 2.25 on a cluster, without root privileges >>>> (I managed before, but now after migration to a new cluster I cant seem >>>> to get it to work). All necessary prerequisite programs are installed >>>> and in the path (I followed the steps in the INSTALL file that comes >>>> with maker2.25). When I do: perl Build.PL, I get: >>>> Checking prerequisites... >>>> requires: >>>> ! DBD::SQLite is not installed >>>> ! IO::All is not installed >>>> ! Inline::C is not installed >>>> ! Perl::Unsafe::Signals is not installed >>>> ! Proc::ProcessTable is not installed >>>> ! Want is not installed >>>> ! forks is not installed >>>> ! forks::shared is not installed >>>> build_requires: >>>> ! LWP::Simple is not installed >>>> recommends: >>>> * DBD::Pg is not installed >>>> >>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>> versions >>>> of the modules indicated above before proceeding with this installation >>>> >>>> Run 'Build installdeps' to install missing prerequisites. >>>> >>>> MAKER supports distributed parallelization via MPI. >>>> Would you like to configure MAKER for MPI (This >>>> requires that you have an MPI client installed)? [N ]n >>>> >>>> WARNING: Apache cannot be located. The optional web based >>>> interface to MAKER will not be available to you. >>>> >>>> Created MYMETA.yml and MYMETA.json >>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>> >>>> >>>> The file 'Build' has been created for you to finish installing MAKER. >>>> >>>> >>>> >>>> ======================================================================== >>>> == >>>> ==== >>>> STATUS MAKER 2.25 >>>> >>>> ======================================================================== >>>> == >>>> ==== >>>> PERL Dependencies: MISSING >>>> ! forks >>>> ! IO::All >>>> ! forks::shared >>>> ! Want >>>> ! DBD::SQLite >>>> ! Proc::ProcessTable >>>> ! Inline::C >>>> ! Perl::Unsafe::Signals >>>> >>>> External Programs: VERIFIED >>>> External C Libraries: VERIFIED >>>> MPI SUPPORT: DISABLED >>>> MWAS Web Interface: DISABLED >>>> MAKER PACKAGE: MISSING PREREQUISITES >>>> >>>> >>>> Important Commands: >>>> ./Build installdeps #installs missing PERL dependencies >>>> ./Build installexes #installs all missing external >>>> programs >>>> ./Build install #installs MAKER >>>> ./Build status #Shows this status menu >>>> >>>> Other Commands: >>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>> RepBase) >>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>> ./Build exonerate #installs Exonerate (v2 on UNIX / v1 >>>> on >>>> Mac OSX) >>>> ./Build snap #installs SNAP >>>> ./Build augustus #installs Augustus >>>> ./Build apollo #installs Apollo >>>> ./Build gbrowse #installs GBrowse (must be root) >>>> ./Build jbrowse #installs JBrowse (MAKER copy, not web >>>> accecible) >>>> ./Build mpich2 #installs MPICH2 (but manual install >>>> recommended) >>>> >>>> So it seems some Perl Dependencies are missing. As indicated in INSTALL >>>> I followed the quick and dirty installation for Bioperl, so I am not >>>> sure what I did wrong/missed out. >>>> >>>> When I run ./Build installdeps, I get: >>>> You do not have write access to install missing Modules. >>>> I can try and install these locally (i.e. only for MAKER) >>>> in the .../maker/perl/lib directory, or you can run >>>> './Build installdeps' as root or using sudo and try again. >>>> Do want MAKER to try and build a local installation? [N ]y >>>> mkdir /usit/titan: Permission denied at >>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>> line >>>> 59 >>>> >>>> Sorry to bother you with this minor things!! Any help would me much >>>> appreciated! >>>> >>>> much obliged, >>>> Christoph >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> > From parulk at caltech.edu Mon Oct 15 14:45:28 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 15 Oct 2012 12:45:28 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: Message-ID: <4445.131.215.15.234.1350330328.squirrel@webmail.caltech.edu> Hi Carson, Thanks. I have attached another contig which is 22 kb, with as many as 3 exons EST alignments. Could you please recommend additional training. We are currently running maker on the entire contig set and eventually merge all the gff3 contig predictions. The using suggested parameter/methods we would like to get a consensus gene-set with minimal false positives/negatives. Thanks, Parul > The contig in question is really too small to get much out of it (only 5 kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb > length or longer contigs to glean enough information for optimizing your parameters. > > The general suggestions for any maker run are to use proteins from a closely related organism or a couple of closely related organisms for the > protein= option in maker. Also leave single_exon set to 0, except for certain eukaryotes that have a bias for single exon transcripts (i.e. some > fungi and oomycetes). And leave keep_preds set to 0 because ab initio predictors tend to over-predict by a wide margin (lots of false > positives). > > Additional training would really depend on what your other contigs look like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. > > Thanks, > Carson > > > > > > On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: > >>Hello, >>Please advice on the aforementioned query? >>Thanks, >>Parul Kudtarkar >>---------------------------- Original Message >> ---------------------------- >>Subject: [maker-devel] Conensus gene model >>From: "Parul Kudtarkar" >>Date: Fri, October 12, 2012 2:46 pm >>To: maker-devel at yandell-lab.org >>-------------------------------------------------------------------------- Hi, >>We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 >> file >>as input). The output that we get is combination of various >>gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? >>Thanks, >>Parul Kudtarkar >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org_______________________________________________ maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org_______________________________________________ maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig7.gff Type: application/octet-stream Size: 68447 bytes Desc: not available URL: From carsonhh at gmail.com Tue Oct 16 09:19:21 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 16 Oct 2012 10:19:21 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <507D68FC.10201@gmail.com> Message-ID: No. that's just a recommendation for the maker2chado script to work. Is es.mod the actual HMM or a soft link to the HMM? After the error message their should be a list of the actual files. Does it print that? Are there trailing commas in you maker_opts.ctl file after the hmm file name in the control files? Are their ':' characters in the path name to the file? --Carson On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >Thanks for your help Carson! > >I tried maker2.26, but the error still remains. I also tried to change >the TMP= option in the maker_opts file, but no change either.. > >STATUS: Parsing control files... >ERROR: The HMM file[s] provided for gmhmm do not exist: > >When building the maker2.26 I saw that it is complaining about one more >(recommended) dependency (I must have overlooked that before): > Checking prerequisites... > recommends: > * DBD::Pg is not installed > >I am having problems to get that one.. Is that causing the error? > >much obliged, >Christoph > >Am 15.10.2012 20:05, schrieb Carson Holt: >> It looks like either an NFS issue or perhaps an issue with the location >> provided for the TMP= in the maker_opts.ctl file. >> >> First I would suggest trying maker 2.26 to see if the issue still >>happens >> there (there are a few updates to how NFS locks are handled in 2.26). >> >> Second if you are setting TMP make sure your disk is not full. Or if >>TMP >> was set to a tmpfs location (in memory filesystem), it could be a full >> memory issue in which case you could just set TMP to somewhere else. >> >> Let me know if you still see the issue in 2.26 or after changing the >> location of TMP. >> >> Thanks, >> Carson >> >> >> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >> >>> Hi Carson, >>> >>> Thanks for your reply. I managed to install maker2.25 properly now, but >>> when running it I am getting strange errors (two kinds): >>> type one: >>> STATUS: Parsing control files... >>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>> >>> type two: >>> STATUS: Parsing control files... >>> ERROR: Cannot get initialization lock. >>> >>> I have divided the draft assembly to several files, each containing a >>> subset of contigs on which I am running a maker instance. I am getting >>> the above named error type one for the majority of the instances, >>> although the HMM file it is complaining about (es.mod) is actually in >>> the path it is naming (not shown above). In a small number of instances >>> I am getting the type two error mentioned above. >>> >>> Can you help me? >>> >>> much obliged, >>> Christoph >>> >>> >>> >>> On 09.10.2012 18:27, Carson Holt wrote: >>>> You may need to reconfigure your cpan preferences. >>>> >>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>>> or >>>> by starting cpan from your command line (command is cpan). Then run >>>>'o >>>> conf init' inside the cpan ui. >>>> >>>> --Carson >>>> >>>> >>>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>>> >>>>> Hello maker-team, >>>>> >>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>privileges >>>>> (I managed before, but now after migration to a new cluster I cant >>>>>seem >>>>> to get it to work). All necessary prerequisite programs are installed >>>>> and in the path (I followed the steps in the INSTALL file that comes >>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>> Checking prerequisites... >>>>> requires: >>>>> ! DBD::SQLite is not installed >>>>> ! IO::All is not installed >>>>> ! Inline::C is not installed >>>>> ! Perl::Unsafe::Signals is not installed >>>>> ! Proc::ProcessTable is not installed >>>>> ! Want is not installed >>>>> ! forks is not installed >>>>> ! forks::shared is not installed >>>>> build_requires: >>>>> ! LWP::Simple is not installed >>>>> recommends: >>>>> * DBD::Pg is not installed >>>>> >>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>>> versions >>>>> of the modules indicated above before proceeding with this >>>>>installation >>>>> >>>>> Run 'Build installdeps' to install missing prerequisites. >>>>> >>>>> MAKER supports distributed parallelization via MPI. >>>>> Would you like to configure MAKER for MPI (This >>>>> requires that you have an MPI client installed)? [N ]n >>>>> >>>>> WARNING: Apache cannot be located. The optional web based >>>>> interface to MAKER will not be available to you. >>>>> >>>>> Created MYMETA.yml and MYMETA.json >>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>> >>>>> >>>>> The file 'Build' has been created for you to finish installing MAKER. >>>>> >>>>> >>>>> >>>>> >>>>>====================================================================== >>>>>== >>>>> == >>>>> ==== >>>>> STATUS MAKER 2.25 >>>>> >>>>> >>>>>====================================================================== >>>>>== >>>>> == >>>>> ==== >>>>> PERL Dependencies: MISSING >>>>> ! forks >>>>> ! IO::All >>>>> ! forks::shared >>>>> ! Want >>>>> ! DBD::SQLite >>>>> ! Proc::ProcessTable >>>>> ! Inline::C >>>>> ! Perl::Unsafe::Signals >>>>> >>>>> External Programs: VERIFIED >>>>> External C Libraries: VERIFIED >>>>> MPI SUPPORT: DISABLED >>>>> MWAS Web Interface: DISABLED >>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>> >>>>> >>>>> Important Commands: >>>>> ./Build installdeps #installs missing PERL dependencies >>>>> ./Build installexes #installs all missing external >>>>> programs >>>>> ./Build install #installs MAKER >>>>> ./Build status #Shows this status menu >>>>> >>>>> Other Commands: >>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>> RepBase) >>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>> ./Build exonerate #installs Exonerate (v2 on UNIX / >>>>>v1 >>>>> on >>>>> Mac OSX) >>>>> ./Build snap #installs SNAP >>>>> ./Build augustus #installs Augustus >>>>> ./Build apollo #installs Apollo >>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>> ./Build jbrowse #installs JBrowse (MAKER copy, not >>>>>web >>>>> accecible) >>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>install >>>>> recommended) >>>>> >>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>INSTALL >>>>> I followed the quick and dirty installation for Bioperl, so I am not >>>>> sure what I did wrong/missed out. >>>>> >>>>> When I run ./Build installdeps, I get: >>>>> You do not have write access to install missing Modules. >>>>> I can try and install these locally (i.e. only for MAKER) >>>>> in the .../maker/perl/lib directory, or you can run >>>>> './Build installdeps' as root or using sudo and try again. >>>>> Do want MAKER to try and build a local installation? [N ]y >>>>> mkdir /usit/titan: Permission denied at >>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>> line >>>>> 59 >>>>> >>>>> Sorry to bother you with this minor things!! Any help would me much >>>>> appreciated! >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> >>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>g >>>> >> > > From chrisi.hahni at gmail.com Tue Oct 16 10:14:36 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 17:14:36 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D79DC.5090702@gmail.com> Hi Carson, It was a link. The path was not correct any more because of the migration to the new cluster. I could not change it so I just gave the path to the real file in the maker_opts file. That has resolved this error, but now I am getting the following: STATUS: Parsing control files... ERROR: Cannot get initialization lock. Probably not relevant any more, but: No trailing commas, no ':' characters in the path. Thanks for your help! Christoph Am 16.10.2012 16:19, schrieb Carson Holt: > No. that's just a recommendation for the maker2chado script to work. > > Is es.mod the actual HMM or a soft link to the HMM? > After the error message their should be a list of the actual files. Does > it print that? > Are there trailing commas in you maker_opts.ctl file after the hmm file > name in the control files? > Are their ':' characters in the path name to the file? > > --Carson > > > > On 12-10-16 10:02 AM, "Christoph Hahn" wrote: > >> Thanks for your help Carson! >> >> I tried maker2.26, but the error still remains. I also tried to change >> the TMP= option in the maker_opts file, but no change either.. >> >> STATUS: Parsing control files... >> ERROR: The HMM file[s] provided for gmhmm do not exist: >> >> When building the maker2.26 I saw that it is complaining about one more >> (recommended) dependency (I must have overlooked that before): >> Checking prerequisites... >> recommends: >> * DBD::Pg is not installed >> >> I am having problems to get that one.. Is that causing the error? >> >> much obliged, >> Christoph >> >> Am 15.10.2012 20:05, schrieb Carson Holt: >>> It looks like either an NFS issue or perhaps an issue with the location >>> provided for the TMP= in the maker_opts.ctl file. >>> >>> First I would suggest trying maker 2.26 to see if the issue still >>> happens >>> there (there are a few updates to how NFS locks are handled in 2.26). >>> >>> Second if you are setting TMP make sure your disk is not full. Or if >>> TMP >>> was set to a tmpfs location (in memory filesystem), it could be a full >>> memory issue in which case you could just set TMP to somewhere else. >>> >>> Let me know if you still see the issue in 2.26 or after changing the >>> location of TMP. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>> >>>> Hi Carson, >>>> >>>> Thanks for your reply. I managed to install maker2.25 properly now, but >>>> when running it I am getting strange errors (two kinds): >>>> type one: >>>> STATUS: Parsing control files... >>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>> >>>> type two: >>>> STATUS: Parsing control files... >>>> ERROR: Cannot get initialization lock. >>>> >>>> I have divided the draft assembly to several files, each containing a >>>> subset of contigs on which I am running a maker instance. I am getting >>>> the above named error type one for the majority of the instances, >>>> although the HMM file it is complaining about (es.mod) is actually in >>>> the path it is naming (not shown above). In a small number of instances >>>> I am getting the type two error mentioned above. >>>> >>>> Can you help me? >>>> >>>> much obliged, >>>> Christoph >>>> >>>> >>>> >>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>> You may need to reconfigure your cpan preferences. >>>>> >>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>>>> or >>>>> by starting cpan from your command line (command is cpan). Then run >>>>> 'o >>>>> conf init' inside the cpan ui. >>>>> >>>>> --Carson >>>>> >>>>> >>>>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>>>> >>>>>> Hello maker-team, >>>>>> >>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>> privileges >>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>> seem >>>>>> to get it to work). All necessary prerequisite programs are installed >>>>>> and in the path (I followed the steps in the INSTALL file that comes >>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>> Checking prerequisites... >>>>>> requires: >>>>>> ! DBD::SQLite is not installed >>>>>> ! IO::All is not installed >>>>>> ! Inline::C is not installed >>>>>> ! Perl::Unsafe::Signals is not installed >>>>>> ! Proc::ProcessTable is not installed >>>>>> ! Want is not installed >>>>>> ! forks is not installed >>>>>> ! forks::shared is not installed >>>>>> build_requires: >>>>>> ! LWP::Simple is not installed >>>>>> recommends: >>>>>> * DBD::Pg is not installed >>>>>> >>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>>>> versions >>>>>> of the modules indicated above before proceeding with this >>>>>> installation >>>>>> >>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>> >>>>>> MAKER supports distributed parallelization via MPI. >>>>>> Would you like to configure MAKER for MPI (This >>>>>> requires that you have an MPI client installed)? [N ]n >>>>>> >>>>>> WARNING: Apache cannot be located. The optional web based >>>>>> interface to MAKER will not be available to you. >>>>>> >>>>>> Created MYMETA.yml and MYMETA.json >>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>> >>>>>> >>>>>> The file 'Build' has been created for you to finish installing MAKER. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ====================================================================== >>>>>> == >>>>>> == >>>>>> ==== >>>>>> STATUS MAKER 2.25 >>>>>> >>>>>> >>>>>> ====================================================================== >>>>>> == >>>>>> == >>>>>> ==== >>>>>> PERL Dependencies: MISSING >>>>>> ! forks >>>>>> ! IO::All >>>>>> ! forks::shared >>>>>> ! Want >>>>>> ! DBD::SQLite >>>>>> ! Proc::ProcessTable >>>>>> ! Inline::C >>>>>> ! Perl::Unsafe::Signals >>>>>> >>>>>> External Programs: VERIFIED >>>>>> External C Libraries: VERIFIED >>>>>> MPI SUPPORT: DISABLED >>>>>> MWAS Web Interface: DISABLED >>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>> >>>>>> >>>>>> Important Commands: >>>>>> ./Build installdeps #installs missing PERL dependencies >>>>>> ./Build installexes #installs all missing external >>>>>> programs >>>>>> ./Build install #installs MAKER >>>>>> ./Build status #Shows this status menu >>>>>> >>>>>> Other Commands: >>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>> RepBase) >>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX / >>>>>> v1 >>>>>> on >>>>>> Mac OSX) >>>>>> ./Build snap #installs SNAP >>>>>> ./Build augustus #installs Augustus >>>>>> ./Build apollo #installs Apollo >>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, not >>>>>> web >>>>>> accecible) >>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>> install >>>>>> recommended) >>>>>> >>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>> INSTALL >>>>>> I followed the quick and dirty installation for Bioperl, so I am not >>>>>> sure what I did wrong/missed out. >>>>>> >>>>>> When I run ./Build installdeps, I get: >>>>>> You do not have write access to install missing Modules. >>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>> in the .../maker/perl/lib directory, or you can run >>>>>> './Build installdeps' as root or using sudo and try again. >>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>> mkdir /usit/titan: Permission denied at >>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>> line >>>>>> 59 >>>>>> >>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>> appreciated! >>>>>> >>>>>> much obliged, >>>>>> Christoph >>>>>> >>>>>> _______________________________________________ >>>>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>> g >> > From carsonhh at gmail.com Tue Oct 16 10:18:48 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 16 Oct 2012 11:18:48 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <507D79DC.5090702@gmail.com> Message-ID: Delete any file in the maker output directory that start with the name .init or .NFS then rerun. Thanks, Carson On 12-10-16 11:14 AM, "Christoph Hahn" wrote: >Hi Carson, > >It was a link. The path was not correct any more because of the >migration to the new cluster. I could not change it so I just gave the >path to the real file in the maker_opts file. That has resolved this >error, but now I am getting the following: >STATUS: Parsing control files... >ERROR: Cannot get initialization lock. > >Probably not relevant any more, but: No trailing commas, no ':' >characters in the path. > >Thanks for your help! >Christoph > >Am 16.10.2012 16:19, schrieb Carson Holt: >> No. that's just a recommendation for the maker2chado script to work. >> >> Is es.mod the actual HMM or a soft link to the HMM? >> After the error message their should be a list of the actual files. >>Does >> it print that? >> Are there trailing commas in you maker_opts.ctl file after the hmm file >> name in the control files? >> Are their ':' characters in the path name to the file? >> >> --Carson >> >> >> >> On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >> >>> Thanks for your help Carson! >>> >>> I tried maker2.26, but the error still remains. I also tried to change >>> the TMP= option in the maker_opts file, but no change either.. >>> >>> STATUS: Parsing control files... >>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>> >>> When building the maker2.26 I saw that it is complaining about one more >>> (recommended) dependency (I must have overlooked that before): >>> Checking prerequisites... >>> recommends: >>> * DBD::Pg is not installed >>> >>> I am having problems to get that one.. Is that causing the error? >>> >>> much obliged, >>> Christoph >>> >>> Am 15.10.2012 20:05, schrieb Carson Holt: >>>> It looks like either an NFS issue or perhaps an issue with the >>>>location >>>> provided for the TMP= in the maker_opts.ctl file. >>>> >>>> First I would suggest trying maker 2.26 to see if the issue still >>>> happens >>>> there (there are a few updates to how NFS locks are handled in 2.26). >>>> >>>> Second if you are setting TMP make sure your disk is not full. Or if >>>> TMP >>>> was set to a tmpfs location (in memory filesystem), it could be a full >>>> memory issue in which case you could just set TMP to somewhere else. >>>> >>>> Let me know if you still see the issue in 2.26 or after changing the >>>> location of TMP. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>>> >>>>> Hi Carson, >>>>> >>>>> Thanks for your reply. I managed to install maker2.25 properly now, >>>>>but >>>>> when running it I am getting strange errors (two kinds): >>>>> type one: >>>>> STATUS: Parsing control files... >>>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>>> >>>>> type two: >>>>> STATUS: Parsing control files... >>>>> ERROR: Cannot get initialization lock. >>>>> >>>>> I have divided the draft assembly to several files, each containing a >>>>> subset of contigs on which I am running a maker instance. I am >>>>>getting >>>>> the above named error type one for the majority of the instances, >>>>> although the HMM file it is complaining about (es.mod) is actually in >>>>> the path it is naming (not shown above). In a small number of >>>>>instances >>>>> I am getting the type two error mentioned above. >>>>> >>>>> Can you help me? >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> >>>>> >>>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>>> You may need to reconfigure your cpan preferences. >>>>>> >>>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm >>>>>>file, >>>>>> or >>>>>> by starting cpan from your command line (command is cpan). Then run >>>>>> 'o >>>>>> conf init' inside the cpan ui. >>>>>> >>>>>> --Carson >>>>>> >>>>>> >>>>>> On 12-10-09 7:07 AM, "Christoph Hahn" >>>>>>wrote: >>>>>> >>>>>>> Hello maker-team, >>>>>>> >>>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>>> privileges >>>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>>> seem >>>>>>> to get it to work). All necessary prerequisite programs are >>>>>>>installed >>>>>>> and in the path (I followed the steps in the INSTALL file that >>>>>>>comes >>>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>>> Checking prerequisites... >>>>>>> requires: >>>>>>> ! DBD::SQLite is not installed >>>>>>> ! IO::All is not installed >>>>>>> ! Inline::C is not installed >>>>>>> ! Perl::Unsafe::Signals is not installed >>>>>>> ! Proc::ProcessTable is not installed >>>>>>> ! Want is not installed >>>>>>> ! forks is not installed >>>>>>> ! forks::shared is not installed >>>>>>> build_requires: >>>>>>> ! LWP::Simple is not installed >>>>>>> recommends: >>>>>>> * DBD::Pg is not installed >>>>>>> >>>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install >>>>>>>the >>>>>>> versions >>>>>>> of the modules indicated above before proceeding with this >>>>>>> installation >>>>>>> >>>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>>> >>>>>>> MAKER supports distributed parallelization via MPI. >>>>>>> Would you like to configure MAKER for MPI (This >>>>>>> requires that you have an MPI client installed)? [N ]n >>>>>>> >>>>>>> WARNING: Apache cannot be located. The optional web based >>>>>>> interface to MAKER will not be available to you. >>>>>>> >>>>>>> Created MYMETA.yml and MYMETA.json >>>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>>> >>>>>>> >>>>>>> The file 'Build' has been created for you to finish installing >>>>>>>MAKER. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>== >>>>>>> == >>>>>>> == >>>>>>> ==== >>>>>>> STATUS MAKER 2.25 >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>== >>>>>>> == >>>>>>> == >>>>>>> ==== >>>>>>> PERL Dependencies: MISSING >>>>>>> ! forks >>>>>>> ! IO::All >>>>>>> ! forks::shared >>>>>>> ! Want >>>>>>> ! DBD::SQLite >>>>>>> ! Proc::ProcessTable >>>>>>> ! Inline::C >>>>>>> ! Perl::Unsafe::Signals >>>>>>> >>>>>>> External Programs: VERIFIED >>>>>>> External C Libraries: VERIFIED >>>>>>> MPI SUPPORT: DISABLED >>>>>>> MWAS Web Interface: DISABLED >>>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>>> >>>>>>> >>>>>>> Important Commands: >>>>>>> ./Build installdeps #installs missing PERL >>>>>>>dependencies >>>>>>> ./Build installexes #installs all missing external >>>>>>> programs >>>>>>> ./Build install #installs MAKER >>>>>>> ./Build status #Shows this status menu >>>>>>> >>>>>>> Other Commands: >>>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>>> RepBase) >>>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX >>>>>>>/ >>>>>>> v1 >>>>>>> on >>>>>>> Mac OSX) >>>>>>> ./Build snap #installs SNAP >>>>>>> ./Build augustus #installs Augustus >>>>>>> ./Build apollo #installs Apollo >>>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, >>>>>>>not >>>>>>> web >>>>>>> accecible) >>>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>>> install >>>>>>> recommended) >>>>>>> >>>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>>> INSTALL >>>>>>> I followed the quick and dirty installation for Bioperl, so I am >>>>>>>not >>>>>>> sure what I did wrong/missed out. >>>>>>> >>>>>>> When I run ./Build installdeps, I get: >>>>>>> You do not have write access to install missing Modules. >>>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>>> in the .../maker/perl/lib directory, or you can run >>>>>>> './Build installdeps' as root or using sudo and try again. >>>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>>> mkdir /usit/titan: Permission denied at >>>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>>> line >>>>>>> 59 >>>>>>> >>>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>>> appreciated! >>>>>>> >>>>>>> much obliged, >>>>>>> Christoph >>>>>>> >>>>>>> _______________________________________________ >>>>>>> maker-devel mailing list >>>>>>> maker-devel at box290.bluehost.com >>>>>>> >>>>>>> >>>>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab. >>>>>>>or >>>>>>> g >>> >> > > From chrisi.hahni at gmail.com Tue Oct 16 11:12:21 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 18:12:21 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D8765.8010909@gmail.com> The maker output directory contains only one thing: .NFSLock.init_lock.NFSLock, which is a link to .NFSLock.init_lock.tmp.3358.21603.1715.48578929183 in the same directory. When I delete it and rerun maker I get the same error and another .NFSLock.init_lock.NFSLock is created, this one is linked to .NFSLock.init_lock.tmp.3635.21981.7756.8294996771 Thanks, Christoph Am 16.10.2012 17:18, schrieb Carson Holt: > Delete any file in the maker output directory that start with the name > .init or .NFS then rerun. > > Thanks, > Carson > > > > On 12-10-16 11:14 AM, "Christoph Hahn" wrote: > >> Hi Carson, >> >> It was a link. The path was not correct any more because of the >> migration to the new cluster. I could not change it so I just gave the >> path to the real file in the maker_opts file. That has resolved this >> error, but now I am getting the following: >> STATUS: Parsing control files... >> ERROR: Cannot get initialization lock. >> >> Probably not relevant any more, but: No trailing commas, no ':' >> characters in the path. >> >> Thanks for your help! >> Christoph >> >> Am 16.10.2012 16:19, schrieb Carson Holt: >>> No. that's just a recommendation for the maker2chado script to work. >>> >>> Is es.mod the actual HMM or a soft link to the HMM? >>> After the error message their should be a list of the actual files. >>> Does >>> it print that? >>> Are there trailing commas in you maker_opts.ctl file after the hmm file >>> name in the control files? >>> Are their ':' characters in the path name to the file? >>> >>> --Carson >>> >>> >>> >>> On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >>> >>>> Thanks for your help Carson! >>>> >>>> I tried maker2.26, but the error still remains. I also tried to change >>>> the TMP= option in the maker_opts file, but no change either.. >>>> >>>> STATUS: Parsing control files... >>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>> >>>> When building the maker2.26 I saw that it is complaining about one more >>>> (recommended) dependency (I must have overlooked that before): >>>> Checking prerequisites... >>>> recommends: >>>> * DBD::Pg is not installed >>>> >>>> I am having problems to get that one.. Is that causing the error? >>>> >>>> much obliged, >>>> Christoph >>>> >>>> Am 15.10.2012 20:05, schrieb Carson Holt: >>>>> It looks like either an NFS issue or perhaps an issue with the >>>>> location >>>>> provided for the TMP= in the maker_opts.ctl file. >>>>> >>>>> First I would suggest trying maker 2.26 to see if the issue still >>>>> happens >>>>> there (there are a few updates to how NFS locks are handled in 2.26). >>>>> >>>>> Second if you are setting TMP make sure your disk is not full. Or if >>>>> TMP >>>>> was set to a tmpfs location (in memory filesystem), it could be a full >>>>> memory issue in which case you could just set TMP to somewhere else. >>>>> >>>>> Let me know if you still see the issue in 2.26 or after changing the >>>>> location of TMP. >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>>>> >>>>>> Hi Carson, >>>>>> >>>>>> Thanks for your reply. I managed to install maker2.25 properly now, >>>>>> but >>>>>> when running it I am getting strange errors (two kinds): >>>>>> type one: >>>>>> STATUS: Parsing control files... >>>>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>>>> >>>>>> type two: >>>>>> STATUS: Parsing control files... >>>>>> ERROR: Cannot get initialization lock. >>>>>> >>>>>> I have divided the draft assembly to several files, each containing a >>>>>> subset of contigs on which I am running a maker instance. I am >>>>>> getting >>>>>> the above named error type one for the majority of the instances, >>>>>> although the HMM file it is complaining about (es.mod) is actually in >>>>>> the path it is naming (not shown above). In a small number of >>>>>> instances >>>>>> I am getting the type two error mentioned above. >>>>>> >>>>>> Can you help me? >>>>>> >>>>>> much obliged, >>>>>> Christoph >>>>>> >>>>>> >>>>>> >>>>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>>>> You may need to reconfigure your cpan preferences. >>>>>>> >>>>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm >>>>>>> file, >>>>>>> or >>>>>>> by starting cpan from your command line (command is cpan). Then run >>>>>>> 'o >>>>>>> conf init' inside the cpan ui. >>>>>>> >>>>>>> --Carson >>>>>>> >>>>>>> >>>>>>> On 12-10-09 7:07 AM, "Christoph Hahn" >>>>>>> wrote: >>>>>>> >>>>>>>> Hello maker-team, >>>>>>>> >>>>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>>>> privileges >>>>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>>>> seem >>>>>>>> to get it to work). All necessary prerequisite programs are >>>>>>>> installed >>>>>>>> and in the path (I followed the steps in the INSTALL file that >>>>>>>> comes >>>>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>>>> Checking prerequisites... >>>>>>>> requires: >>>>>>>> ! DBD::SQLite is not installed >>>>>>>> ! IO::All is not installed >>>>>>>> ! Inline::C is not installed >>>>>>>> ! Perl::Unsafe::Signals is not installed >>>>>>>> ! Proc::ProcessTable is not installed >>>>>>>> ! Want is not installed >>>>>>>> ! forks is not installed >>>>>>>> ! forks::shared is not installed >>>>>>>> build_requires: >>>>>>>> ! LWP::Simple is not installed >>>>>>>> recommends: >>>>>>>> * DBD::Pg is not installed >>>>>>>> >>>>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install >>>>>>>> the >>>>>>>> versions >>>>>>>> of the modules indicated above before proceeding with this >>>>>>>> installation >>>>>>>> >>>>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>>>> >>>>>>>> MAKER supports distributed parallelization via MPI. >>>>>>>> Would you like to configure MAKER for MPI (This >>>>>>>> requires that you have an MPI client installed)? [N ]n >>>>>>>> >>>>>>>> WARNING: Apache cannot be located. The optional web based >>>>>>>> interface to MAKER will not be available to you. >>>>>>>> >>>>>>>> Created MYMETA.yml and MYMETA.json >>>>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>>>> >>>>>>>> >>>>>>>> The file 'Build' has been created for you to finish installing >>>>>>>> MAKER. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> == >>>>>>>> == >>>>>>>> == >>>>>>>> ==== >>>>>>>> STATUS MAKER 2.25 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> == >>>>>>>> == >>>>>>>> == >>>>>>>> ==== >>>>>>>> PERL Dependencies: MISSING >>>>>>>> ! forks >>>>>>>> ! IO::All >>>>>>>> ! forks::shared >>>>>>>> ! Want >>>>>>>> ! DBD::SQLite >>>>>>>> ! Proc::ProcessTable >>>>>>>> ! Inline::C >>>>>>>> ! Perl::Unsafe::Signals >>>>>>>> >>>>>>>> External Programs: VERIFIED >>>>>>>> External C Libraries: VERIFIED >>>>>>>> MPI SUPPORT: DISABLED >>>>>>>> MWAS Web Interface: DISABLED >>>>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>>>> >>>>>>>> >>>>>>>> Important Commands: >>>>>>>> ./Build installdeps #installs missing PERL >>>>>>>> dependencies >>>>>>>> ./Build installexes #installs all missing external >>>>>>>> programs >>>>>>>> ./Build install #installs MAKER >>>>>>>> ./Build status #Shows this status menu >>>>>>>> >>>>>>>> Other Commands: >>>>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>>>> RepBase) >>>>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX >>>>>>>> / >>>>>>>> v1 >>>>>>>> on >>>>>>>> Mac OSX) >>>>>>>> ./Build snap #installs SNAP >>>>>>>> ./Build augustus #installs Augustus >>>>>>>> ./Build apollo #installs Apollo >>>>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, >>>>>>>> not >>>>>>>> web >>>>>>>> accecible) >>>>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>>>> install >>>>>>>> recommended) >>>>>>>> >>>>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>>>> INSTALL >>>>>>>> I followed the quick and dirty installation for Bioperl, so I am >>>>>>>> not >>>>>>>> sure what I did wrong/missed out. >>>>>>>> >>>>>>>> When I run ./Build installdeps, I get: >>>>>>>> You do not have write access to install missing Modules. >>>>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>>>> in the .../maker/perl/lib directory, or you can run >>>>>>>> './Build installdeps' as root or using sudo and try again. >>>>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>>>> mkdir /usit/titan: Permission denied at >>>>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>>>> line >>>>>>>> 59 >>>>>>>> >>>>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>>>> appreciated! >>>>>>>> >>>>>>>> much obliged, >>>>>>>> Christoph >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> maker-devel mailing list >>>>>>>> maker-devel at box290.bluehost.com >>>>>>>> >>>>>>>> >>>>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab. >>>>>>>> or >>>>>>>> g >> > From mikael.durling at slu.se Tue Oct 16 11:58:55 2012 From: mikael.durling at slu.se (=?iso-8859-1?Q?Mikael_Brandstr=F6m_Durling?=) Date: Tue, 16 Oct 2012 18:58:55 +0200 Subject: [maker-devel] Format of repeat_gff gff3 file Message-ID: Hi, I would like to mask my fungal genome from two different sources (ie. repbase and an inhouse repeat library). However, I suppose the that if I supply a library as rmlib in maker_opts, it will be mutually exclusive to the model_org option, in the same way as -spec and -lib options to RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the model_org option as fungi, and would like to provide maker with additional masking as a gff file. I tried by running RepeatMasker with my inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker package to obtain a gff3 file. This file was supplied to maker as rm_gff (see below for a sample from the file). The run fail with backtraces like the one paseted below. How should this gff file be formatted for maker to understand it? I see that in maker produced gff files, there are additional information found in the id of the hits. Is this required? Maybe it's easier to modify maker to make two rounds of RepeatMasker calls if both model_org and rmlib are specified? Thanks for any input, Mikael ------------- EXCEPTION: Bio::Root::Exception ------------- MSG: Must have defined a valid name for Hit STACK: Error::throw STACK: Bio::Root::Root::throw /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 STACK: Bio::Search::Hit::GenericHit::new /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:149 STACK: Bio::Search::Hit::PhatHit::Base::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm:127 STACK: Bio::Search::Hit::PhatHit::gff3::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/gff3.pm:23 STACK: GFFDB::_load_hits /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:1026 STACK: GFFDB::phathits_on_chunk /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:651 STACK: Process::MpiChunk::_go /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:752 STACK: Process::MpiChunk::run /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:331 STACK: main::node_thread /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 STACK: threads::new /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm:799 STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 ----------------------------------------------------------- Cannot restore overloading on HASH(0x2580810) (package Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at /opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm line 2256. Compilation failed in require at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. BEGIN failed--compilation aborted at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached ERROR: Could not open '/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.output/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fungi.rb.out' ERROR: Failed while doing repeat masking ERROR: Chunk failed at level:0, tier_type:1 FAILED CONTIG:scf_55419 The gff3-file looks like this: ##gff-version 3 ##sequence-region scf_42697 1 3949 scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 ##sequence-region scf_82496 1 2757 scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_family-1046 2478 4915 ##sequence-region scf_82727 1 4159 scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 23 scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_family-64 15 50 ##sequence-region scf_82785 1 4084 scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 24 scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_family-690 419 786 ##sequence-region scf_86740 1 4293 scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_family-262 1 25 scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_family-262 2 60 scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_family-262 13 60 ##sequence-region scf_86782 1 8564 scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_family-480 1027 1129 ##sequence-region scf_86808 1 4495 scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family-690 1 969 scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 74 ##sequence-region scf_86815 1 4139 scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-262 825 918 scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_family-262 526 4459 ##sequence-region scf_86823 1 2528 scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 178 scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 35 scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 45 scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 41 scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 96 scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 40 scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_family-262 881 920 ##sequence-region scf_86857 1 2778 From mike.thon at gmail.com Wed Oct 17 05:48:59 2012 From: mike.thon at gmail.com (Michael Thon) Date: Wed, 17 Oct 2012 12:48:59 +0200 Subject: [maker-devel] Format of repeat_gff gff3 file In-Reply-To: References: Message-ID: Hi - 2 ideas: 1) There is a utility included with RepeatMasker that enables you to extract sequences from it. its in the utils directory. You can run it like this: ./queryRepeatDatabase.pl -species Fungi -clade that will extract all repeat sequences belonging to Fungi or its descendants. Maybe the simplest thing for you to do is extract the sequences that you want and cat them together with your in house sequences to provide MAKER with a single file of reference sequences. 2) some of the MAKER parameters take a comma separated list of files. I don't know if this applied to rm_lib though. Mike On Oct 16, 2012, at 6:58 PM, Mikael Brandstr?m Durling wrote: > Hi, > > I would like to mask my fungal genome from two different sources (ie. repbase and an inhouse repeat library). However, I suppose the that if I supply a library as rmlib in maker_opts, it will be mutually exclusive to the model_org option, in the same way as -spec and -lib options to RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the model_org option as fungi, and would like to provide maker with additional masking as a gff file. I tried by running RepeatMasker with my inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker package to obtain a gff3 file. This file was supplied to maker as rm_gff (see below for a sample from the file). The run fail with backtraces like the one paseted below. How should this gff file be formatted for maker to understand it? I see that in maker produced gff files, there are additional information found in the id of the hits. Is this required? > > Maybe it's easier to modify maker to make two rounds of RepeatMasker calls if both model_org and rmlib are specified? > > Thanks for any input, > Mikael > > > ------------- EXCEPTION: Bio::Root::Exception ------------- > MSG: Must have defined a valid name for Hit > STACK: Error::throw > STACK: Bio::Root::Root::throw /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 > STACK: Bio::Search::Hit::GenericHit::new /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:149 > STACK: Bio::Search::Hit::PhatHit::Base::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm:127 > STACK: Bio::Search::Hit::PhatHit::gff3::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/gff3.pm:23 > STACK: GFFDB::_load_hits /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:1026 > STACK: GFFDB::phathits_on_chunk /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:651 > STACK: Process::MpiChunk::_go /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:752 > STACK: Process::MpiChunk::run /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:331 > STACK: main::node_thread /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 > STACK: threads::new /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm:799 > STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 > ----------------------------------------------------------- > Cannot restore overloading on HASH(0x2580810) (package Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at /opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm line 2256. > Compilation failed in require at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. > BEGIN failed--compilation aborted at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. > Perl exited with active threads: > 1 running and unjoined > 0 finished and unjoined > 0 running and detached > ERROR: Could not open '/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.output/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fungi.rb.out' > ERROR: Failed while doing repeat masking > ERROR: Chunk failed at level:0, tier_type:1 > FAILED CONTIG:scf_55419 > > > The gff3-file looks like this: > ##gff-version 3 > ##sequence-region scf_42697 1 3949 > scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 > scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 > scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 > ##sequence-region scf_82496 1 2757 > scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_family-1046 2478 4915 > ##sequence-region scf_82727 1 4159 > scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 > scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 23 > scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_family-64 15 50 > ##sequence-region scf_82785 1 4084 > scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 24 > scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_family-690 419 786 > ##sequence-region scf_86740 1 4293 > scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_family-262 1 25 > scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_family-262 2 60 > scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_family-262 13 60 > ##sequence-region scf_86782 1 8564 > scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_family-480 1027 1129 > ##sequence-region scf_86808 1 4495 > scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family-690 1 969 > scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 74 > ##sequence-region scf_86815 1 4139 > scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-262 825 918 > scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_family-262 526 4459 > ##sequence-region scf_86823 1 2528 > scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 > scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 > scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 178 > scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 35 > scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 45 > scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 41 > scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 96 > scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 40 > scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_family-262 881 920 > ##sequence-region scf_86857 1 2778 > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Fri Oct 19 09:00:12 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 19 Oct 2012 10:00:12 -0400 Subject: [maker-devel] Format of repeat_gff gff3 file In-Reply-To: Message-ID: This command line should add IDs to the end of the GFF3 to let it pass through without the error message. cat file.gff | perl -ane '$id; if(!/^\#/){@F = split(/\t/, $_); chomp $F[-1];$id++; $F[-1] .= "\;ID=$id"; $_ = join("\t", @F)."\n"} print $_' MAKER will use the GFF3 first if provided, then run the species specific library, and then any model organism specified. So if there is overlap and one must be excluded, then they will be kept in that same order of precedence. Thanks, Carson On 12-10-16 12:58 PM, "Mikael Brandstr?m Durling" wrote: >Hi, > >I would like to mask my fungal genome from two different sources (ie. >repbase and an inhouse repeat library). However, I suppose the that if I >supply a library as rmlib in maker_opts, it will be mutually exclusive to >the model_org option, in the same way as -spec and -lib options to >RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the >model_org option as fungi, and would like to provide maker with >additional masking as a gff file. I tried by running RepeatMasker with my >inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker >package to obtain a gff3 file. This file was supplied to maker as rm_gff >(see below for a sample from the file). The run fail with backtraces like >the one paseted below. How should this gff file be formatted for maker to >understand it? I see that in maker produced gff files, there are >additional information found in the id of the hits. Is this required? > >Maybe it's easier to modify maker to make two rounds of RepeatMasker >calls if both model_org and rmlib are specified? > >Thanks for any input, >Mikael > > >------------- EXCEPTION: Bio::Root::Exception ------------- >MSG: Must have defined a valid name for Hit >STACK: Error::throw >STACK: Bio::Root::Root::throw >/opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 >STACK: Bio::Search::Hit::GenericHit::new >/opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:14 >9 >STACK: Bio::Search::Hit::PhatHit::Base::new >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Bio/Search/Hit/PhatHit/Base.pm:127 >STACK: Bio::Search::Hit::PhatHit::gff3::new >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Bio/Search/Hit/PhatHit/gff3.pm:23 >STACK: GFFDB::_load_hits >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/GFFDB.pm:1026 >STACK: GFFDB::phathits_on_chunk >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/GFFDB.pm:651 >STACK: Process::MpiChunk::_go >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Process/MpiChunk.pm:752 >STACK: Process::MpiChunk::run >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Process/MpiChunk.pm:331 >STACK: main::node_thread >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 >STACK: threads::new >/proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16. >1/x86_64-linux/forks.pm:799 >STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 >----------------------------------------------------------- >Cannot restore overloading on HASH(0x2580810) (package >Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at >/opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at >/proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16. >1/x86_64-linux/forks.pm line 2256. >Compilation failed in require at >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. >BEGIN failed--compilation aborted at >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. >Perl exited with active threads: > 1 running and unjoined > 0 finished and unjoined > 0 running and detached >ERROR: Could not open >'/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.o >utput/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fung >i.rb.out' >ERROR: Failed while doing repeat masking >ERROR: Chunk failed at level:0, tier_type:1 >FAILED CONTIG:scf_55419 > > >The gff3-file looks like this: >##gff-version 3 >##sequence-region scf_42697 1 3949 >scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 >scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 >scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 >##sequence-region scf_82496 1 2757 >scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_fami >ly-1046 2478 4915 >##sequence-region scf_82727 1 4159 >scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 >scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 >23 >scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_fam >ily-64 15 50 >##sequence-region scf_82785 1 4084 >scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 >24 >scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_fam >ily-690 419 786 >##sequence-region scf_86740 1 4293 >scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_famil >y-262 1 25 >scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_famil >y-262 2 60 >scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_famil >y-262 13 60 >##sequence-region scf_86782 1 8564 >scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_fam >ily-480 1027 1129 >##sequence-region scf_86808 1 4495 >scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family >-690 1 969 >scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 >74 >##sequence-region scf_86815 1 4139 >scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-2 >62 825 918 >scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_fa >mily-262 526 4459 >##sequence-region scf_86823 1 2528 >scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 >scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 >scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 >178 >scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 >35 >scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 >45 >scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 >41 >scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 >96 >scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 >40 >scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_fam >ily-262 881 920 >##sequence-region scf_86857 1 2778 > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From parulk at caltech.edu Mon Oct 22 16:46:18 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 22 Oct 2012 14:46:18 -0700 (PDT) Subject: [maker-devel] gff3_preds2models script Message-ID: <3207.131.215.15.234.1350942378.squirrel@webmail.caltech.edu> Hello, I want to add gene structure(gene/mRNA/exon) to gff3 file. I am using gff3_preds2models for this purpose. However I get following error **WARNING: No top level feature found for ID WHL22.100252 I have attached the sample input gff3 file and the list of ids Thanks and regards, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: sample.gff3 Type: application/octet-stream Size: 1873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: list Type: application/octet-stream Size: 76 bytes Desc: not available URL: From fourie.joubert at up.ac.za Wed Oct 24 02:07:13 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 09:07:13 +0200 Subject: [maker-devel] Segfault during build_fasta_index Message-ID: <508793A1.1000704@up.ac.za> Hi I am getting a segfault in maker somewhere during the build_fasta_index step. I suspect it is happening after some updates I made to bioperl. Has anyone else experienced something similar, and is there a recommended bioperl version? Thanks and regards! Fourie -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 07:28:10 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 08:28:10 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <508793A1.1000704@up.ac.za> Message-ID: Yes. Don't use the live version of BioPerl. Use the CPAN version. The live version has a bug. Thanks, Carson On 12-10-24 3:07 AM, "Fourie Joubert" wrote: >Hi > >I am getting a segfault in maker somewhere during the build_fasta_index >step. > >I suspect it is happening after some updates I made to bioperl. > >Has anyone else experienced something similar, and is there a >recommended bioperl version? > >Thanks and regards! > >Fourie > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From fourie.joubert at up.ac.za Wed Oct 24 09:05:12 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:05:12 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087F598.20303@up.ac.za> Hi I have installed from CPAN (C/CJ/CJFIELDS/BioPerl-1.6.901.tar.gz), but the error persists... Does anyone know if the versions in CPAN may have been replaced recently? Best regards! Foorie On 10/24/2012 02:28 PM, Carson Holt wrote: > Yes. Don't use the live version of BioPerl. Use the CPAN version. The > live version has a bug. > > Thanks, > Carson > > > On 12-10-24 3:07 AM, "Fourie Joubert" wrote: > >> Hi >> >> I am getting a segfault in maker somewhere during the build_fasta_index >> step. >> >> I suspect it is happening after some updates I made to bioperl. >> >> Has anyone else experienced something similar, and is there a >> recommended bioperl version? >> >> Thanks and regards! >> >> Fourie >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 09:10:32 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:10:32 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087F598.20303@up.ac.za> Message-ID: Run this command --> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' What does it print? --Carson On 12-10-24 10:05 AM, "Fourie Joubert" wrote: >Hi > >I have installed from CPAN (C/CJ/CJFIELDS/BioPerl-1.6.901.tar.gz), but >the error persists... > >Does anyone know if the versions in CPAN may have been replaced recently? > >Best regards! > >Foorie > > > > > > >On 10/24/2012 02:28 PM, Carson Holt wrote: >> Yes. Don't use the live version of BioPerl. Use the CPAN version. The >> live version has a bug. >> >> Thanks, >> Carson >> >> >> On 12-10-24 3:07 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> I am getting a segfault in maker somewhere during the build_fasta_index >>> step. >>> >>> I suspect it is happening after some updates I made to bioperl. >>> >>> Has anyone else experienced something similar, and is there a >>> recommended bioperl version? >>> >>> Thanks and regards! >>> >>> Fourie >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 09:24:43 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:24:43 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FA2B.5020702@up.ac.za> Hi 1.006901 Regards! Fourie On 10/24/2012 04:10 PM, Carson Holt wrote: > perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 09:26:41 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:26:41 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FA2B.5020702@up.ac.za> Message-ID: Try completely deleting the mpi_blastdb directory under the MAKER output folder before restarting. Perhaps also try reinstalling DB_File via CPAN. --Carson On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >Hi > >1.006901 > >Regards! > >Fourie > > > >On 10/24/2012 04:10 PM, Carson Holt wrote: >> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 09:30:15 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:30:15 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FB77.7060802@up.ac.za> Hi Darn - did both, but unfortunately still segfaults... Best regards! Fourie On 10/24/2012 04:26 PM, Carson Holt wrote: > Try completely deleting the mpi_blastdb directory under the MAKER output > folder before restarting. Perhaps also try reinstalling DB_File via CPAN. > > --Carson > > On 12-10-24 10:24 AM, "Fourie Joubert" wrote: > >> Hi >> >> 1.006901 >> >> Regards! >> >> Fourie >> >> >> >> On 10/24/2012 04:10 PM, Carson Holt wrote: >>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 09:33:07 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:33:07 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FB77.7060802@up.ac.za> Message-ID: I'm going to write you a test script that just creates a fasta index to see if it seg faults. I'll send it to you later. For now could you run maker with the --debug flag set, apture the STDERR and send it to me. Thanks, Carson On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >Hi > >Darn - did both, but unfortunately still segfaults... > >Best regards! > >Fourie > > >On 10/24/2012 04:26 PM, Carson Holt wrote: >> Try completely deleting the mpi_blastdb directory under the MAKER output >> folder before restarting. Perhaps also try reinstalling DB_File via >>CPAN. >> >> --Carson >> >> On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> 1.006901 >>> >>> Regards! >>> >>> Fourie >>> >>> >>> >>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 09:41:15 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:41:15 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FE0B.7030206@up.ac.za> Hi Really weird - as soon as I add the debug flag - no more segfault... Best regards! Fourie On 10/24/2012 04:33 PM, Carson Holt wrote: > I'm going to write you a test script that just creates a fasta index to > see if it seg faults. I'll send it to you later. For now could you run > maker with the --debug flag set, apture the STDERR and send it to me. > > Thanks, > Carson > > > On 12-10-24 10:30 AM, "Fourie Joubert" wrote: > >> Hi >> >> Darn - did both, but unfortunately still segfaults... >> >> Best regards! >> >> Fourie >> >> >> On 10/24/2012 04:26 PM, Carson Holt wrote: >>> Try completely deleting the mpi_blastdb directory under the MAKER output >>> folder before restarting. Perhaps also try reinstalling DB_File via >>> CPAN. >>> >>> --Carson >>> >>> On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> 1.006901 >>>> >>>> Regards! >>>> >>>> Fourie >>>> >>>> >>>> >>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 09:56:51 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:56:51 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FE0B.7030206@up.ac.za> Message-ID: Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA variable. It's set by both BioPerl and a couple of MAKER modules. If it gets set twice, sometimes you get bugs. Try upgrading to MAKER 2.26 I have a statement in that that will make sure it won't get set wrong. Otherwise I can give you instructions on editing the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER modules. Thanks, Carson On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >Hi > >Really weird - as soon as I add the debug flag - no more segfault... > >Best regards! > >Fourie > > > > >On 10/24/2012 04:33 PM, Carson Holt wrote: >> I'm going to write you a test script that just creates a fasta index to >> see if it seg faults. I'll send it to you later. For now could you run >> maker with the --debug flag set, apture the STDERR and send it to me. >> >> Thanks, >> Carson >> >> >> On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> Darn - did both, but unfortunately still segfaults... >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>output >>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>> CPAN. >>>> >>>> --Carson >>>> >>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> 1.006901 >>>>> >>>>> Regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> >>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>> perl -MBio::Root::Version -e 'print >>>>>>"$Bio::Root::Version::VERSION\n"' >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 10:19:08 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 17:19:08 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <508806EC.6050600@up.ac.za> Hi Carson We are indeed running 2.26. Instructions on editing the statements would be great. Many thanks for all the help, and sorry for the bother! Best regards! Fourie On 10/24/2012 04:56 PM, Carson Holt wrote: > Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA > variable. It's set by both BioPerl and a couple of MAKER modules. If it > gets set twice, sometimes you get bugs. > > Try upgrading to MAKER 2.26 I have a statement in that that will make sure > it won't get set wrong. Otherwise I can give you instructions on editing > the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER > modules. > > Thanks, > Carson > > > On 12-10-24 10:41 AM, "Fourie Joubert" wrote: > >> Hi >> >> Really weird - as soon as I add the debug flag - no more segfault... >> >> Best regards! >> >> Fourie >> >> >> >> >> On 10/24/2012 04:33 PM, Carson Holt wrote: >>> I'm going to write you a test script that just creates a fasta index to >>> see if it seg faults. I'll send it to you later. For now could you run >>> maker with the --debug flag set, apture the STDERR and send it to me. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> Darn - did both, but unfortunately still segfaults... >>>> >>>> Best regards! >>>> >>>> Fourie >>>> >>>> >>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>> output >>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>> CPAN. >>>>> >>>>> --Carson >>>>> >>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> 1.006901 >>>>>> >>>>>> Regards! >>>>>> >>>>>> Fourie >>>>>> >>>>>> >>>>>> >>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>> perl -MBio::Root::Version -e 'print >>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>> -- >>>>>> -------------- >>>>>> Prof Fourie Joubert >>>>>> Bioinformatics and Computational Biology Unit >>>>>> Department of Biochemistry >>>>>> University of Pretoria >>>>>> fourie.joubert at up.ac.za >>>>>> http://www.bi.up.ac.za >>>>>> Tel. +27-12-420-5825 >>>>>> Fax. +27-12-420-5800 >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> -- >>>>>> - >>>>>> This message and attachments are subject to a disclaimer. Please >>>>>> refer >>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>> details. >>>>>> >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 10:50:03 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 11:50:03 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <508806EC.6050600@up.ac.za> Message-ID: Find these lines in these files in MAKER .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); And the Bio::DB::Fasta file in BioPerl. Use this command to identify the location of Bio::DB::Fasta --> perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); Use an editor like emacs or vi to make the changes. Thanks, Carson On 12-10-24 11:19 AM, "Fourie Joubert" wrote: >Hi Carson > >We are indeed running 2.26. > >Instructions on editing the statements would be great. > >Many thanks for all the help, and sorry for the bother! > >Best regards! > >Fourie > > > > >On 10/24/2012 04:56 PM, Carson Holt wrote: >> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >> variable. It's set by both BioPerl and a couple of MAKER modules. If it >> gets set twice, sometimes you get bugs. >> >> Try upgrading to MAKER 2.26 I have a statement in that that will make >>sure >> it won't get set wrong. Otherwise I can give you instructions on >>editing >> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >> modules. >> >> Thanks, >> Carson >> >> >> On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> Really weird - as soon as I add the debug flag - no more segfault... >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> >>> >>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>> I'm going to write you a test script that just creates a fasta index >>>>to >>>> see if it seg faults. I'll send it to you later. For now could you >>>>run >>>> maker with the --debug flag set, apture the STDERR and send it to me. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> Darn - did both, but unfortunately still segfaults... >>>>> >>>>> Best regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>> output >>>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>>> CPAN. >>>>>> >>>>>> --Carson >>>>>> >>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> 1.006901 >>>>>>> >>>>>>> Regards! >>>>>>> >>>>>>> Fourie >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>> -- >>>>>>> -------------- >>>>>>> Prof Fourie Joubert >>>>>>> Bioinformatics and Computational Biology Unit >>>>>>> Department of Biochemistry >>>>>>> University of Pretoria >>>>>>> fourie.joubert at up.ac.za >>>>>>> http://www.bi.up.ac.za >>>>>>> Tel. +27-12-420-5825 >>>>>>> Fax. +27-12-420-5800 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>> -- >>>>>>> - >>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>> refer >>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>> details. >>>>>>> >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Thu Oct 25 01:51:05 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Thu, 25 Oct 2012 08:51:05 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5088E159.3050502@up.ac.za> Hi Carson All happy now - much appreciated! Best regards! Fourie On 10/24/2012 05:50 PM, Carson Holt wrote: > Find these lines in these files in MAKER > > .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > > > And the Bio::DB::Fasta file in BioPerl. Use this command to identify the > location of Bio::DB::Fasta --> > perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' > > .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > > > Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); > > Use an editor like emacs or vi to make the changes. > > Thanks, > Carson > > > > On 12-10-24 11:19 AM, "Fourie Joubert" wrote: > >> Hi Carson >> >> We are indeed running 2.26. >> >> Instructions on editing the statements would be great. >> >> Many thanks for all the help, and sorry for the bother! >> >> Best regards! >> >> Fourie >> >> >> >> >> On 10/24/2012 04:56 PM, Carson Holt wrote: >>> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >>> variable. It's set by both BioPerl and a couple of MAKER modules. If it >>> gets set twice, sometimes you get bugs. >>> >>> Try upgrading to MAKER 2.26 I have a statement in that that will make >>> sure >>> it won't get set wrong. Otherwise I can give you instructions on >>> editing >>> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >>> modules. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> Really weird - as soon as I add the debug flag - no more segfault... >>>> >>>> Best regards! >>>> >>>> Fourie >>>> >>>> >>>> >>>> >>>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>>> I'm going to write you a test script that just creates a fasta index >>>>> to >>>>> see if it seg faults. I'll send it to you later. For now could you >>>>> run >>>>> maker with the --debug flag set, apture the STDERR and send it to me. >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> Darn - did both, but unfortunately still segfaults... >>>>>> >>>>>> Best regards! >>>>>> >>>>>> Fourie >>>>>> >>>>>> >>>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>>> output >>>>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>>>> CPAN. >>>>>>> >>>>>>> --Carson >>>>>>> >>>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>>> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> 1.006901 >>>>>>>> >>>>>>>> Regards! >>>>>>>> >>>>>>>> Fourie >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>>> -- >>>>>>>> -------------- >>>>>>>> Prof Fourie Joubert >>>>>>>> Bioinformatics and Computational Biology Unit >>>>>>>> Department of Biochemistry >>>>>>>> University of Pretoria >>>>>>>> fourie.joubert at up.ac.za >>>>>>>> http://www.bi.up.ac.za >>>>>>>> Tel. +27-12-420-5825 >>>>>>>> Fax. +27-12-420-5800 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -------------------------------------------------------------------- >>>>>>>> -- >>>>>>>> -- >>>>>>>> - >>>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>>> refer >>>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>>> details. >>>>>>>> >>>>>> -- >>>>>> -------------- >>>>>> Prof Fourie Joubert >>>>>> Bioinformatics and Computational Biology Unit >>>>>> Department of Biochemistry >>>>>> University of Pretoria >>>>>> fourie.joubert at up.ac.za >>>>>> http://www.bi.up.ac.za >>>>>> Tel. +27-12-420-5825 >>>>>> Fax. +27-12-420-5800 >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> -- >>>>>> - >>>>>> This message and attachments are subject to a disclaimer. Please >>>>>> refer >>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>> details. >>>>>> >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Thu Oct 25 07:57:31 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 25 Oct 2012 08:57:31 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5088E159.3050502@up.ac.za> Message-ID: Glad I could help. Thanks, Carson On 12-10-25 2:51 AM, "Fourie Joubert" wrote: >Hi Carson > >All happy now - much appreciated! > >Best regards! > >Fourie > >On 10/24/2012 05:50 PM, Carson Holt wrote: >> Find these lines in these files in MAKER >> >> .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> >> >> And the Bio::DB::Fasta file in BioPerl. Use this command to identify >>the >> location of Bio::DB::Fasta --> >> perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' >> >> .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> >> >> Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); >> >> Use an editor like emacs or vi to make the changes. >> >> Thanks, >> Carson >> >> >> >> On 12-10-24 11:19 AM, "Fourie Joubert" wrote: >> >>> Hi Carson >>> >>> We are indeed running 2.26. >>> >>> Instructions on editing the statements would be great. >>> >>> Many thanks for all the help, and sorry for the bother! >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> >>> >>> On 10/24/2012 04:56 PM, Carson Holt wrote: >>>> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >>>> variable. It's set by both BioPerl and a couple of MAKER modules. If >>>>it >>>> gets set twice, sometimes you get bugs. >>>> >>>> Try upgrading to MAKER 2.26 I have a statement in that that will make >>>> sure >>>> it won't get set wrong. Otherwise I can give you instructions on >>>> editing >>>> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >>>> modules. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-24 10:41 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> Really weird - as soon as I add the debug flag - no more segfault... >>>>> >>>>> Best regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> >>>>> >>>>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>>>> I'm going to write you a test script that just creates a fasta index >>>>>> to >>>>>> see if it seg faults. I'll send it to you later. For now could you >>>>>> run >>>>>> maker with the --debug flag set, apture the STDERR and send it to >>>>>>me. >>>>>> >>>>>> Thanks, >>>>>> Carson >>>>>> >>>>>> >>>>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> Darn - did both, but unfortunately still segfaults... >>>>>>> >>>>>>> Best regards! >>>>>>> >>>>>>> Fourie >>>>>>> >>>>>>> >>>>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>>>> output >>>>>>>> folder before restarting. Perhaps also try reinstalling DB_File >>>>>>>>via >>>>>>>> CPAN. >>>>>>>> >>>>>>>> --Carson >>>>>>>> >>>>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> 1.006901 >>>>>>>>> >>>>>>>>> Regards! >>>>>>>>> >>>>>>>>> Fourie >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>>>> -- >>>>>>>>> -------------- >>>>>>>>> Prof Fourie Joubert >>>>>>>>> Bioinformatics and Computational Biology Unit >>>>>>>>> Department of Biochemistry >>>>>>>>> University of Pretoria >>>>>>>>> fourie.joubert at up.ac.za >>>>>>>>> http://www.bi.up.ac.za >>>>>>>>> Tel. +27-12-420-5825 >>>>>>>>> Fax. +27-12-420-5800 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>------------------------------------------------------------------ >>>>>>>>>-- >>>>>>>>> -- >>>>>>>>> -- >>>>>>>>> - >>>>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>>>> refer >>>>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>>>> details. >>>>>>>>> >>>>>>> -- >>>>>>> -------------- >>>>>>> Prof Fourie Joubert >>>>>>> Bioinformatics and Computational Biology Unit >>>>>>> Department of Biochemistry >>>>>>> University of Pretoria >>>>>>> fourie.joubert at up.ac.za >>>>>>> http://www.bi.up.ac.za >>>>>>> Tel. +27-12-420-5825 >>>>>>> Fax. +27-12-420-5800 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>> -- >>>>>>> - >>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>> refer >>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>> details. >>>>>>> >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From daniel.standage at gmail.com Thu Oct 25 14:30:44 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 25 Oct 2012 15:30:44 -0400 Subject: [maker-devel] Strange error at blastn step Message-ID: Greetings! I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. #--------- command -------------# Widget::blastn: /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn #-------------------------------# deleted:0 hits ERROR: Could not obtain lock to format database FATAL ERROR ERROR: Failed while doing blastn of ESTs!! ERROR: Chunk failed at level 8 !! FAILED CONTIG:scaffold_866 Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:52:43 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 13:52:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx #-------------------------------# deleted:-10 hits formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx #-------------------------------# deleted:-6 hits WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. stop here:comp59088_c1_seq7 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:scaffold_0 --Next Contig-- #--------------------------------------------------------------------- Now starting the contig!! SeqID: scaffold_1 Length: 5805686 #--------------------------------------------------------------------- My first thought based on the message is that *blastdbcmd* could not find the sequence in the database. I verified this was the case--I could not extract sequence *comp59088_c1_seq7* from the database Maker had created under /tmp. However, after removing the index files and re-running * makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully extracted the sequence. I was initially happy with this finding, but upon closer inspection it looks like Maker does not use *blastdbcmd* to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a > pretty short contig (about 3kb). However, there seems to be a hiccup during > the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db > /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 > -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 > -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 > -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp > 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis > -out > /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff > old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > > Several blastn steps appeared to have completed successfully to this one > failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From myandell at genetics.utah.edu Fri Oct 26 13:02:29 2012 From: myandell at genetics.utah.edu (Mark Yandell) Date: Fri, 26 Oct 2012 18:02:29 +0000 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: , Message-ID: <7A60AB257EFF2B48B1F4C814817EA05331BA6EC8@mxb1.hg.genetics.utah.edu> Hi Daniel, I think its your fasta-file '> comp59088_c1_seq' note the space between the chevron and the id. This isn't allowed by the fasta format. cheers, --mark Mark Yandell Professor of Human Genetics H.A. & Edna Benning Presidential Endowed Chair Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 ph:801-587-7707 ________________________________________ From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Daniel Standage [daniel.standage at gmail.com] Sent: Friday, October 26, 2012 11:52 AM To: Maker Mailing List Subject: Re: [maker-devel] Strange error at blastn step I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx #-------------------------------# deleted:-10 hits formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx #-------------------------------# deleted:-6 hits WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. stop here:comp59088_c1_seq7 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:scaffold_0 --Next Contig-- #--------------------------------------------------------------------- Now starting the contig!! SeqID: scaffold_1 Length: 5805686 #--------------------------------------------------------------------- My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage > wrote: Greetings! I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. #--------- command -------------# Widget::blastn: /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn #-------------------------------# deleted:0 hits ERROR: Could not obtain lock to format database FATAL ERROR ERROR: Failed while doing blastn of ESTs!! ERROR: Chunk failed at level 8 !! FAILED CONTIG:scaffold_866 Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University From carsonhh at gmail.com Fri Oct 26 13:09:39 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:09:39 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Check to see where /tmp is located? Some clusters have it set up as a tmpfs directory and I have had problems with fasta indexes running from tmpfs mounts in the past. --Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:05 PM To: Carson Holt Subject: Re: [maker-devel] Strange error at blastn step The maker working directory is in a cluster environment with shared scratch space (I'm guessing NFS-mounted). I didn't change the temp directory setting, so it should be the local default (/tmp). I'll give the dev version a shot. Thanks. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > Could you try this development version and tell me if the error still happens? > > Use this command to download --> > <> > > Username: <> > Password: <> > > Are you running in an NFS mounted directory or are you resetting TMP to a > different location? > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 1:52 PM > To: Maker Mailing List > Subject: Re: [maker-devel] Strange error at blastn step > > I have since installed Maker on a different machine and tried it out. The test > run completed successfully, but as I commenced with the full genome > annotation, I have noticed the following error popping up frequently. > >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions >> 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes >> -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker >> .output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0 >> .0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi. >> 10.8.blastx >> #-------------------------------# >> deleted:-10 hits >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions >> 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes >> -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker >> .output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0 >> .0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi. >> 10.9.blastx >> #-------------------------------# >> deleted:-6 hits >> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >> stop here:comp59088_c1_seq7 >> ERROR: Fasta index error >> >> FATAL ERROR >> ERROR: Failed while polishig ESTs!! >> >> ERROR: Chunk failed at level 14 >> !! >> FAILED CONTIG:scaffold_0 >> >> >> >> >> --Next Contig-- >> >> #--------------------------------------------------------------------- >> Now starting the contig!! >> SeqID: scaffold_1 >> Length: 5805686 >> #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the > sequence in the database. I verified this was the case--I could not extract > sequence comp59088_c1_seq7 from the database Maker had created under /tmp. > However, after removing the index files and re-running makeblastdb with the > -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks > like Maker does not use blastdbcmd to extract sequences, but rather its own > internal code. Therefore I'm still not sure where the problem is and how I > might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage > wrote: >> Greetings! >> >> I am doing a test run of my Maker setup on a new machine, annotating a pretty >> short contig (about 3kb). However, there seems to be a hiccup during the >> blastn stage. This is the terminal message. >> >>> #--------- command -------------# >>> Widget::blastn: >>> /share/home/01854/standage/local/bin/blastn -db >>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta >>> .mpi.10.7 -query >>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments >>> 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 >>> -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 >>> -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out >>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.out >>> put/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.P >>> dom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmom >>> atic%2Efasta.mpi.10.7.blastn >>> #-------------------------------# >>> deleted:0 hits >>> ERROR: Could not obtain lock to format database >>> >>> >>> FATAL ERROR >>> ERROR: Failed while doing blastn of ESTs!! >>> >>> ERROR: Chunk failed at level 8 >>> !! >>> FAILED CONTIG:scaffold_866 >> >> Several blastn steps appeared to have completed successfully to this one >> failing. Any ideas what could be causing this? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/mak > er-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 13:12:38 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:12:38 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: It looks like /tmp is indeed being used: the files I played with were under */tmp/maker_1YQF9o*. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a > tmpfs directory and I have had problems with fasta indexes running from > tmpfs mounts in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared > scratch space (I'm guessing NFS-mounted). I didn't change the temp > directory setting, so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > >> Could you try this development version and tell me if the error still >> happens? >> >> Use this command to download --> >> <> >> >> Username: <> >> Password: <> >> >> Are you running in an NFS mounted directory or are you resetting TMP to a >> different location? >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 1:52 PM >> To: Maker Mailing List >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I have since installed Maker on a different machine and tried it out. The >> test run completed successfully, but as I commenced with the full genome >> annotation, I have noticed the following error popping up frequently. >> >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >> #-------------------------------# >> deleted:-10 hits >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >> #-------------------------------# >> deleted:-6 hits >> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >> stop here:comp59088_c1_seq7 >> ERROR: Fasta index error >> >> FATAL ERROR >> ERROR: Failed while polishig ESTs!! >> >> ERROR: Chunk failed at level 14 >> !! >> FAILED CONTIG:scaffold_0 >> >> >> >> >> --Next Contig-- >> >> #--------------------------------------------------------------------- >> Now starting the contig!! >> SeqID: scaffold_1 >> Length: 5805686 >> #--------------------------------------------------------------------- >> >> >> My first thought based on the message is that *blastdbcmd* could not >> find the sequence in the database. I verified this was the case--I could >> not extract sequence *comp59088_c1_seq7* from the database Maker had >> created under /tmp. However, after removing the index files and re-running >> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >> extracted the sequence. >> >> I was initially happy with this finding, but upon closer inspection it >> looks like Maker does not use *blastdbcmd* to extract sequences, but >> rather its own internal code. Therefore I'm still not sure where the >> problem is and how I might fix it. Any insights? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >> daniel.standage at gmail.com> wrote: >> >>> Greetings! >>> >>> I am doing a test run of my Maker setup on a new machine, annotating a >>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>> the blastn stage. This is the terminal message. >>> >>> #--------- command -------------# >>> Widget::blastn: >>> /share/home/01854/standage/local/bin/blastn -db >>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>> -out >>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>> #-------------------------------# >>> deleted:0 hits >>> ERROR: Could not obtain lock to format database >>> >>> >>> FATAL ERROR >>> ERROR: Failed while doing blastn of ESTs!! >>> >>> ERROR: Chunk failed at level 8 >>> !! >>> FAILED CONTIG:scaffold_866 >>> >>> >>> Several blastn steps appeared to have completed successfully to this one >>> failing. Any ideas what could be causing this? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 13:14:29 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:14:29 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: The command 'df /tmp' will tell you whether /tmp is a tmpfs mount Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:12 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step It looks like /tmp is indeed being used: the files I played with were under /tmp/maker_1YQF9o. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a tmpfs > directory and I have had problems with fasta indexes running from tmpfs mounts > in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared scratch > space (I'm guessing NFS-mounted). I didn't change the temp directory setting, > so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >> Could you try this development version and tell me if the error still >> happens? >> >> Use this command to download --> >> <> >> >> Username: <> >> Password: <> >> >> Are you running in an NFS mounted directory or are you resetting TMP to a >> different location? >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 1:52 PM >> To: Maker Mailing List >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I have since installed Maker on a different machine and tried it out. The >> test run completed successfully, but as I commenced with the full genome >> annotation, I have noticed the following error popping up frequently. >> >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.make >>> r.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold >>> _0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.m >>> pi.10.8.blastx >>> #-------------------------------# >>> deleted:-10 hits >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.make >>> r.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold >>> _0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.m >>> pi.10.9.blastx >>> #-------------------------------# >>> deleted:-6 hits >>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>> stop here:comp59088_c1_seq7 >>> ERROR: Fasta index error >>> >>> FATAL ERROR >>> ERROR: Failed while polishig ESTs!! >>> >>> ERROR: Chunk failed at level 14 >>> !! >>> FAILED CONTIG:scaffold_0 >>> >>> >>> >>> >>> --Next Contig-- >>> >>> #--------------------------------------------------------------------- >>> Now starting the contig!! >>> SeqID: scaffold_1 >>> Length: 5805686 >>> #--------------------------------------------------------------------- >> >> My first thought based on the message is that blastdbcmd could not find the >> sequence in the database. I verified this was the case--I could not extract >> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >> However, after removing the index files and re-running makeblastdb with the >> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >> >> I was initially happy with this finding, but upon closer inspection it looks >> like Maker does not use blastdbcmd to extract sequences, but rather its own >> internal code. Therefore I'm still not sure where the problem is and how I >> might fix it. Any insights? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >> wrote: >>> Greetings! >>> >>> I am doing a test run of my Maker setup on a new machine, annotating a >>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>> the blastn stage. This is the terminal message. >>> >>>> #--------- command -------------# >>>> Widget::blastn: >>>> /share/home/01854/standage/local/bin/blastn -db >>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efast >>>> a.mpi.10.7 -query >>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>> -show_gis -out >>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.ou >>>> tput/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0 >>>> .Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrim >>>> momatic%2Efasta.mpi.10.7.blastn >>>> #-------------------------------# >>>> deleted:0 hits >>>> ERROR: Could not obtain lock to format database >>>> >>>> >>>> FATAL ERROR >>>> ERROR: Failed while doing blastn of ESTs!! >>>> >>>> ERROR: Chunk failed at level 8 >>>> !! >>>> FAILED CONTIG:scaffold_866 >>> >>> Several blastn steps appeared to have completed successfully to this one >>> failing. Any ideas what could be causing this? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ma >> ker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry.moore at genetics.utah.edu Fri Oct 26 13:19:06 2012 From: barry.moore at genetics.utah.edu (Barry Moore) Date: Fri, 26 Oct 2012 12:19:06 -0600 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> Hi Daniel, What version or revision of MAKER are you running. This sounds like something we were seeing here last night. We traced it as far as what appeared to be soft links in /tmp being set incorrectly. The FastaDB objects had pointers to fasta files in the void for the correct fasta file, but their dir attribute pointed to a /tmp directory in where there were soft links to another (incorrect) fasta file with it's index. It would look appeared that it was looking in the /tmp index (which pointed to an incorrect fasta file) and when it failed to find what it was looking for it would re-index the fasta file in the void (the correct one) and then look again in the index in tmp. Don't know if that helps, but the errors look similar and that's as far as I got with our error here? This was on one of Mike's annotation projects, so I don't know for sure what revision he was running, but I think it was the latest. B On Oct 26, 2012, at 11:52 AM, Daniel Standage wrote: > I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Barry Moore Research Scientist Dept. of Human Genetics University of Utah Salt Lake City, UT 84112 -------------------------------------------- (801) 585-3543 -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 13:19:07 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:19:07 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Unfortunately, the job is no longer running and as a result I cannot connect to the compute nodes as I could while it was running. On the interactive node, it looks like it's real disk, although it looks like there are some tmpfs mounts. [dstandag at mason src] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 462824304 180235660 259078476 42% /tmp [dstandag at mason src] df Filesystem 1K-blocks Used Available Use% Mounted on login_x86_64 16497564 3077352 13420212 19% / tmpfs 16497564 0 16497564 0% /dev/shm tmpfs 10240 0 10240 0% /var/tmp /dev/sdb2 462824304 180235660 259078476 42% /tmp AFS 9000000 0 9000000 0% /afs bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs ... ... I'll see if I can launch another short job and verify this on the compute nodes. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > The command 'df /tmp' will tell you whether /tmp is a tmpfs mount > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:12 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > It looks like /tmp is indeed being used: the files I played with were > under */tmp/maker_1YQF9o*. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > >> Check to see where /tmp is located? Some clusters have it set up as a >> tmpfs directory and I have had problems with fasta indexes running from >> tmpfs mounts in the past. >> >> --Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:05 PM >> To: Carson Holt >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> The maker working directory is in a cluster environment with shared >> scratch space (I'm guessing NFS-mounted). I didn't change the temp >> directory setting, so it should be the local default (/tmp). >> >> I'll give the dev version a shot. Thanks. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >> >>> Could you try this development version and tell me if the error still >>> happens? >>> >>> Use this command to download --> >>> <> >>> >>> Username: <> >>> Password: <> >>> >>> Are you running in an NFS mounted directory or are you resetting TMP to >>> a different location? >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 1:52 PM >>> To: Maker Mailing List >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> I have since installed Maker on a different machine and tried it out. >>> The test run completed successfully, but as I commenced with the full >>> genome annotation, I have noticed the following error popping up frequently. >>> >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>> #-------------------------------# >>> deleted:-10 hits >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>> #-------------------------------# >>> deleted:-6 hits >>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>> stop here:comp59088_c1_seq7 >>> ERROR: Fasta index error >>> >>> FATAL ERROR >>> ERROR: Failed while polishig ESTs!! >>> >>> ERROR: Chunk failed at level 14 >>> !! >>> FAILED CONTIG:scaffold_0 >>> >>> >>> >>> >>> --Next Contig-- >>> >>> #--------------------------------------------------------------------- >>> Now starting the contig!! >>> SeqID: scaffold_1 >>> Length: 5805686 >>> #--------------------------------------------------------------------- >>> >>> >>> My first thought based on the message is that *blastdbcmd* could not >>> find the sequence in the database. I verified this was the case--I could >>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>> created under /tmp. However, after removing the index files and re-running >>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>> extracted the sequence. >>> >>> I was initially happy with this finding, but upon closer inspection it >>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>> rather its own internal code. Therefore I'm still not sure where the >>> problem is and how I might fix it. Any insights? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>> daniel.standage at gmail.com> wrote: >>> >>>> Greetings! >>>> >>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>> the blastn stage. This is the terminal message. >>>> >>>> #--------- command -------------# >>>> Widget::blastn: >>>> /share/home/01854/standage/local/bin/blastn -db >>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>>> -out >>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>> #-------------------------------# >>>> deleted:0 hits >>>> ERROR: Could not obtain lock to format database >>>> >>>> >>>> FATAL ERROR >>>> ERROR: Failed while doing blastn of ESTs!! >>>> >>>> ERROR: Chunk failed at level 8 >>>> !! >>>> FAILED CONTIG:scaffold_866 >>>> >>>> >>>> Several blastn steps appeared to have completed successfully to this >>>> one failing. Any ideas what could be causing this? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 13:20:55 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:20:55 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> References: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> Message-ID: Running 2.10. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:19 PM, Barry Moore wrote: > Hi Daniel, > > What version or revision of MAKER are you running. This sounds like > something we were seeing here last night. We traced it as far as what > appeared to be soft links in /tmp being set incorrectly. The FastaDB > objects had pointers to fasta files in the void for the correct fasta file, > but their dir attribute pointed to a /tmp directory in where there were > soft links to another (incorrect) fasta file with it's index. It would > look appeared that it was looking in the /tmp index (which pointed to an > incorrect fasta file) and when it failed to find what it was looking for it > would re-index the fasta file in the void (the correct one) and then look > again in the index in tmp. Don't know if that helps, but the errors look > similar and that's as far as I got with our error here? This was on one of > Mike's annotation projects, so I don't know for sure what revision he was > running, but I think it was the latest. > > B > > On Oct 26, 2012, at 11:52 AM, Daniel Standage wrote: > > I have since installed Maker on a different machine and tried it out. The > test run completed successfully, but as I commenced with the full genome > annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query > /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 > -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 > -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query > /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 > -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 > -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > > My first thought based on the message is that *blastdbcmd* could not find > the sequence in the database. I verified this was the case--I could not > extract sequence *comp59088_c1_seq7* from the database Maker had created > under /tmp. However, after removing the index files and re-running * > makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully > extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it > looks like Maker does not use *blastdbcmd* to extract sequences, but > rather its own internal code. Therefore I'm still not sure where the > problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < > daniel.standage at gmail.com> wrote: > >> Greetings! >> >> I am doing a test run of my Maker setup on a new machine, annotating a >> pretty short contig (about 3kb). However, there seems to be a hiccup during >> the blastn stage. This is the terminal message. >> >> #--------- command -------------# >> Widget::blastn: >> /share/home/01854/standage/local/bin/blastn -db >> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >> -out >> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >> #-------------------------------# >> deleted:0 hits >> ERROR: Could not obtain lock to format database >> >> >> FATAL ERROR >> ERROR: Failed while doing blastn of ESTs!! >> >> ERROR: Chunk failed at level 8 >> !! >> FAILED CONTIG:scaffold_866 >> >> >> Several blastn steps appeared to have completed successfully to this one >> failing. Any ideas what could be causing this? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > Barry Moore > Research Scientist > Dept. of Human Genetics > University of Utah > Salt Lake City, UT 84112 > -------------------------------------------- > (801) 585-3543 > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 13:24:43 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:24:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Ok. Try the developer release and see if it still happens. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:19 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Unfortunately, the job is no longer running and as a result I cannot connect to the compute nodes as I could while it was running. On the interactive node, it looks like it's real disk, although it looks like there are some tmpfs mounts. [dstandag at mason src] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 462824304 180235660 259078476 42% /tmp [dstandag at mason src] df Filesystem 1K-blocks Used Available Use% Mounted on login_x86_64 16497564 3077352 13420212 19% / tmpfs 16497564 0 16497564 0% /dev/shm tmpfs 10240 0 10240 0% /var/tmp /dev/sdb2 462824304 180235660 259078476 42% /tmp AFS 9000000 0 9000000 0% /afs bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs ... ... I'll see if I can launch another short job and verify this on the compute nodes. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > The command 'df /tmp' will tell you whether /tmp is a tmpfs mount > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:12 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > It looks like /tmp is indeed being used: the files I played with were under > /tmp/maker_1YQF9o. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >> directory and I have had problems with fasta indexes running from tmpfs >> mounts in the past. >> >> --Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:05 PM >> To: Carson Holt >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> The maker working directory is in a cluster environment with shared scratch >> space (I'm guessing NFS-mounted). I didn't change the temp directory setting, >> so it should be the local default (/tmp). >> >> I'll give the dev version a shot. Thanks. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>> Could you try this development version and tell me if the error still >>> happens? >>> >>> Use this command to download --> >>> <> >>> >>> Username: <> >>> Password: <> >>> >>> Are you running in an NFS mounted directory or are you resetting TMP to a >>> different location? >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 1:52 PM >>> To: Maker Mailing List >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> I have since installed Maker on a different machine and tried it out. The >>> test run completed successfully, but as I commenced with the full genome >>> annotation, I have noticed the following error popping up frequently. >>> >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.mak >>>> er.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffo >>>> ld_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efa >>>> a.mpi.10.8.blastx >>>> #-------------------------------# >>>> deleted:-10 hits >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.mak >>>> er.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffo >>>> ld_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efa >>>> a.mpi.10.9.blastx >>>> #-------------------------------# >>>> deleted:-6 hits >>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>> stop here:comp59088_c1_seq7 >>>> ERROR: Fasta index error >>>> >>>> FATAL ERROR >>>> ERROR: Failed while polishig ESTs!! >>>> >>>> ERROR: Chunk failed at level 14 >>>> !! >>>> FAILED CONTIG:scaffold_0 >>>> >>>> >>>> >>>> >>>> --Next Contig-- >>>> >>>> #--------------------------------------------------------------------- >>>> Now starting the contig!! >>>> SeqID: scaffold_1 >>>> Length: 5805686 >>>> #--------------------------------------------------------------------- >>> >>> My first thought based on the message is that blastdbcmd could not find the >>> sequence in the database. I verified this was the case--I could not extract >>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>> However, after removing the index files and re-running makeblastdb with the >>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>> >>> I was initially happy with this finding, but upon closer inspection it looks >>> like Maker does not use blastdbcmd to extract sequences, but rather its own >>> internal code. Therefore I'm still not sure where the problem is and how I >>> might fix it. Any insights? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>> wrote: >>>> Greetings! >>>> >>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>> the blastn stage. This is the terminal message. >>>> >>>>> #--------- command -------------# >>>>> Widget::blastn: >>>>> /share/home/01854/standage/local/bin/blastn -db >>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efas >>>>> ta.mpi.10.7 -query >>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>> -show_gis -out >>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.o >>>>> utput/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866 >>>>> .0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ET >>>>> rimmomatic%2Efasta.mpi.10.7.blastn >>>>> #-------------------------------# >>>>> deleted:0 hits >>>>> ERROR: Could not obtain lock to format database >>>>> >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while doing blastn of ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 8 >>>>> !! >>>>> FAILED CONTIG:scaffold_866 >>>> >>>> Several blastn steps appeared to have completed successfully to this one >>>> failing. Any ideas what could be causing this? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m >>> aker-devel_yandell-lab.org >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 13:29:18 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:29:18 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot > connect to the compute nodes as I could while it was running. On the > interactive node, it looks like it's real disk, although it looks like > there are some tmpfs mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were >> under */tmp/maker_1YQF9o*. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >> >>> Check to see where /tmp is located? Some clusters have it set up as a >>> tmpfs directory and I have had problems with fasta indexes running from >>> tmpfs mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared >>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>> directory setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>> >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to >>>> a different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. >>>> The test run completed successfully, but as I commenced with the full >>>> genome annotation, I have noticed the following error popping up frequently. >>>> >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>> #-------------------------------# >>>> deleted:-10 hits >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>> #-------------------------------# >>>> deleted:-6 hits >>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>> stop here:comp59088_c1_seq7 >>>> ERROR: Fasta index error >>>> >>>> FATAL ERROR >>>> ERROR: Failed while polishig ESTs!! >>>> >>>> ERROR: Chunk failed at level 14 >>>> !! >>>> FAILED CONTIG:scaffold_0 >>>> >>>> >>>> >>>> >>>> --Next Contig-- >>>> >>>> #--------------------------------------------------------------------- >>>> Now starting the contig!! >>>> SeqID: scaffold_1 >>>> Length: 5805686 >>>> #--------------------------------------------------------------------- >>>> >>>> >>>> My first thought based on the message is that *blastdbcmd* could not >>>> find the sequence in the database. I verified this was the case--I could >>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>> created under /tmp. However, after removing the index files and re-running >>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>> extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>> rather its own internal code. Therefore I'm still not sure where the >>>> problem is and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>> daniel.standage at gmail.com> wrote: >>>> >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>>> the blastn stage. This is the terminal message. >>>>> >>>>> #--------- command -------------# >>>>> Widget::blastn: >>>>> /share/home/01854/standage/local/bin/blastn -db >>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>>>> -out >>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>> #-------------------------------# >>>>> deleted:0 hits >>>>> ERROR: Could not obtain lock to format database >>>>> >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while doing blastn of ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 8 >>>>> !! >>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> >>>>> Several blastn steps appeared to have completed successfully to this >>>>> one failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>> _______________________________________________ maker-devel mailing >>>> list maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 13:32:35 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:32:35 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: If running './Build install_deps' to get the prereqs automatically, you can say yes to the local version question if you want those prereqs to be installed just for MAKER rather than globally to whatever perl you are using. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:29 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot connect > to the compute nodes as I could while it was running. On the interactive node, > it looks like it's real disk, although it looks like there are some tmpfs > mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were under >> /tmp/maker_1YQF9o. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >>> directory and I have had problems with fasta indexes running from tmpfs >>> mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared scratch >>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>> setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>> different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. The >>>> test run completed successfully, but as I commenced with the full genome >>>> annotation, I have noticed the following error popping up frequently. >>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>> >>>> My first thought based on the message is that blastdbcmd could not find the >>>> sequence in the database. I verified this was the case--I could not extract >>>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>>> However, after removing the index files and re-running makeblastdb with the >>>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>> its own internal code. Therefore I'm still not sure where the problem is >>>> and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>> wrote: >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>> during the blastn stage. This is the terminal message. >>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efa >>>>>> sta.mpi.10.7 -query >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>> -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker. >>>>>> output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_8 >>>>>> 66.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity% >>>>>> 2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> Several blastn steps appeared to have completed successfully to this one >>>>> failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>> >>>> _______________________________________________ maker-devel mailing list >>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ >>>> maker-devel_yandell-lab.org >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 13:47:43 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:47:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I've got a test run (with the dev version) waiting in the queue. But just to be clear, if the temp directory is indeed the problem, I'm assuming that would that be fixed by setting a different temp directory on the same disk as the scratch disk? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:32 PM, Carson Holt wrote: > If running './Build install_deps' to get the prereqs automatically, you > can say yes to the local version question if you want those prereqs to be > installed just for MAKER rather than globally to whatever perl you are > using. > > Thanks, > Carson > > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot >> connect to the compute nodes as I could while it was running. On the >> interactive node, it looks like it's real disk, although it looks like >> there are some tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were >>> under */tmp/maker_1YQF9o*. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared >>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>> directory setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>> to a different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. >>>>> The test run completed successfully, but as I commenced with the full >>>>> genome annotation, I have noticed the following error popping up frequently. >>>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>>> >>>>> >>>>> My first thought based on the message is that *blastdbcmd* could not >>>>> find the sequence in the database. I verified this was the case--I could >>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>> created under /tmp. However, after removing the index files and re-running >>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>>> rather its own internal code. Therefore I'm still not sure where the >>>>> problem is and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>> daniel.standage at gmail.com> wrote: >>>>> >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>> true -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this >>>>>> one failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>> _______________________________________________ maker-devel mailing >>>>> list maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 13:56:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:56:30 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Yes. That is correct. Also I've made one more update to the development release with respect to indexes for your test run. Could you run 'svn update' inside the devel maker directory. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:47 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step I've got a test run (with the dev version) waiting in the queue. But just to be clear, if the temp directory is indeed the problem, I'm assuming that would that be fixed by setting a different temp directory on the same disk as the scratch disk? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:32 PM, Carson Holt wrote: > If running './Build install_deps' to get the prereqs automatically, you can > say yes to the local version question if you want those prereqs to be > installed just for MAKER rather than globally to whatever perl you are using. > > Thanks, > Carson > > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot connect >> to the compute nodes as I could while it was running. On the interactive >> node, it looks like it's real disk, although it looks like there are some >> tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were under >>> /tmp/maker_1YQF9o. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared scratch >>>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>>> setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>>> different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. The >>>>> test run completed successfully, but as I commenced with the full genome >>>>> annotation, I have noticed the following error popping up frequently. >>>>> >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.m >>>>>> aker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/sc >>>>>> affold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E4 >>>>>> 7%2Efaa.mpi.10.8.blastx >>>>>> #-------------------------------# >>>>>> deleted:-10 hits >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.m >>>>>> aker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/sc >>>>>> affold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E4 >>>>>> 7%2Efaa.mpi.10.9.blastx >>>>>> #-------------------------------# >>>>>> deleted:-6 hits >>>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>>> stop here:comp59088_c1_seq7 >>>>>> ERROR: Fasta index error >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while polishig ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 14 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --Next Contig-- >>>>>> >>>>>> #--------------------------------------------------------------------- >>>>>> Now starting the contig!! >>>>>> SeqID: scaffold_1 >>>>>> Length: 5805686 >>>>>> #--------------------------------------------------------------------- >>>>> >>>>> My first thought based on the message is that blastdbcmd could not find >>>>> the sequence in the database. I verified this was the case--I could not >>>>> extract sequence comp59088_c1_seq7 from the database Maker had created >>>>> under /tmp. However, after removing the index files and re-running >>>>> makeblastdb with the -parse_seqids option set, blastdbcmd successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>>> its own internal code. Therefore I'm still not sure where the problem is >>>>> and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>>> wrote: >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>>> #--------- command -------------# >>>>>>> Widget::blastn: >>>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Ef >>>>>>> asta.mpi.10.7 -query >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>>> -show_gis -out >>>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker >>>>>>> .output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold >>>>>>> _866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrini >>>>>>> ty%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>>> #-------------------------------# >>>>>>> deleted:0 hits >>>>>>> ERROR: Could not obtain lock to format database >>>>>>> >>>>>>> >>>>>>> FATAL ERROR >>>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>>> >>>>>>> ERROR: Chunk failed at level 8 >>>>>>> !! >>>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this one >>>>>> failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>> >>>>> _______________________________________________ maker-devel mailing list >>>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo >>>>> /maker-devel_yandell-lab.org >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason.stajich at gmail.com Fri Oct 26 15:49:57 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Fri, 26 Oct 2012 13:49:57 -0700 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Right but is tmp on a disk or a tmpfs df -h /tmp would give you the first hint at whether it is a local drive or something else. On Oct 26, 2012, at 11:12 AM, Daniel Standage wrote: > It looks like /tmp is indeed being used: the files I played with were under /tmp/maker_1YQF9o. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a tmpfs directory and I have had problems with fasta indexes running from tmpfs mounts in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared scratch space (I'm guessing NFS-mounted). I didn't change the temp directory setting, so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > Could you try this development version and tell me if the error still happens? > > Use this command to download --> > <> > > Username: <> > Password: <> > > Are you running in an NFS mounted directory or are you resetting TMP to a different location? > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 1:52 PM > To: Maker Mailing List > Subject: Re: [maker-devel] Strange error at blastn step > > I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 17:59:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 18:59:30 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: From: Carson Holt Date: Friday, 26 October, 2012 6:10 PM To: Daniel Standage Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step I've been going over the indexing code using different scenarios and I may have isolated a candidate for what is causing this. Could you do one more 'svn update' inside the maker devel directory before running a test job? Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:29 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot connect > to the compute nodes as I could while it was running. On the interactive node, > it looks like it's real disk, although it looks like there are some tmpfs > mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were under >> /tmp/maker_1YQF9o. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >>> directory and I have had problems with fasta indexes running from tmpfs >>> mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared scratch >>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>> setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>> different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. The >>>> test run completed successfully, but as I commenced with the full genome >>>> annotation, I have noticed the following error popping up frequently. >>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>> >>>> My first thought based on the message is that blastdbcmd could not find the >>>> sequence in the database. I verified this was the case--I could not extract >>>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>>> However, after removing the index files and re-running makeblastdb with the >>>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>> its own internal code. Therefore I'm still not sure where the problem is >>>> and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>> wrote: >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>> during the blastn stage. This is the terminal message. >>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efa >>>>>> sta.mpi.10.7 -query >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>> -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker. >>>>>> output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_8 >>>>>> 66.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity% >>>>>> 2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> Several blastn steps appeared to have completed successfully to this one >>>>> failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>> >>>> _______________________________________________ maker-devel mailing list >>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ >>>> maker-devel_yandell-lab.org >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kokwei86 at gmail.com Sun Oct 28 13:50:19 2012 From: kokwei86 at gmail.com (kokwei) Date: Mon, 29 Oct 2012 02:50:19 +0800 Subject: [maker-devel] Query on genemark results Message-ID: <508D7E6B.3060001@gmail.com> Hi, I have the same problems as posted by Andr? Gomes on 10/4/10. I still have the same problems even though using the current available version of maker (maker 2.1 and maker 2.26 beta version). I have tried to do the gene prediction on eukaryotic genome using 3 ab-initio gene predictors (SNAP, Augustus and GeneMark-ES) using Maker. >From the fasta_merge output, I have 3 separate files of gene models ($prefix.all.maker.augustus_masked.proteins.fasta, $prefix.all.maker.snap_masked.proteins.fasta and $prefix.all.maker.genemark.proteins.fasta) and one $prefix.all.maker.proteins.fasta (I presume this should be the consensus gene models from 3 predictors' results, right?). From the consensus gene models file, I don't see even one result from genemark but all from snap/augustus only. Is that normal? Also from the file naming and gene model label in final gff file, it's showing that genemark is not masked like those of augustus and snap? Is that true? Why only augustus and snap masked but not genemark? Please assist and thanks for your helps. Kok Wei From daniel.standage at gmail.com Mon Oct 29 08:39:05 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Mon, 29 Oct 2012 09:39:05 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Carson, I was able to run the first svn update before the test job ran, but I didn't get the message about this second update until after it has already executed and failed. I got about 372 lines of such. STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_datastore To access files for individual sequences use the datastore index: /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_master_datastore_index.log STATUS: Now running MAKER... WARNING: Cannot find >scaffold_0, trying to re-index the fasta. stop here: scaffold_0 ERROR: Fasta index error at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 239. Process::MpiChunk::_prepare('Process::MpiChunk=HASH(0x3c8b300)', 'HASH(0x3c80820)', 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 73 Process::MpiTiers::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c808b0)', 'HASH(0x3c8ec40)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 79 Process::MpiTiers::_prepare('Process::MpiTiers=HASH(0x3c71f30)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 56 Process::MpiTiers::new('Process::MpiTiers', 'HASH(0x3c80640)', 0, 'Process::MpiChunk') called at /N/u/dstandag/Mason/local/src/maker-dev/bin/maker line 627 --> rank=NA, hostname=c4 ERROR: Failed in tier preparation WARNING: You must always set a rank before running MpiTiers FATAL: argument `seq_id` does not exist in MpiTier object at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 86. Process::MpiChunk::_initialize_vars('Process::MpiChunk=HASH(0x3cc93d0)', 'HASH(0x3cc9400)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 47 Process::MpiChunk::new('Process::MpiChunk', 'HASH(0x3c80820)', 0, 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 407 Process::MpiChunk::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c8f498)', 'HASH(0x3cc8f20)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 3811 Process::MpiChunk::_go('Process::MpiChunk=HASH(0x3c8b300)', 'load', 'HASH(0x3c80820)', 0, 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 310 Process::MpiChunk::_loader('Process::MpiChunk=HASH(0x3c8b300)', 'HASH(0x3c80820)', 0, 0, 'Process::MpiTiers=HASH(0x3c71f30)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 364 Process::MpiTiers::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c8f9c0)', 'HASH(0x3c8fb28)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 375 I'll try the next update and run the test again. I see a lot of references in there to MPI, and just thought I would make it clear that although I am running in a cluster environment, I am using the default serial version of Maker, not the parallel MPI version. Also, after this failed, I tried changing the TMP directory so that it was located on the same NFS mount as the scratch disk to which the output was written. This did not seem to have any affect, and I saw the same issues with the EST sequences unable to be found. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 6:59 PM, Carson Holt wrote: > > > From: Carson Holt > Date: Friday, 26 October, 2012 6:10 PM > To: Daniel Standage > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > I've been going over the indexing code using different scenarios and I may > have isolated a candidate for what is causing this. Could you do one more > 'svn update' inside the maker devel directory before running a test job? > > Thanks, > Carson > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot >> connect to the compute nodes as I could while it was running. On the >> interactive node, it looks like it's real disk, although it looks like >> there are some tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were >>> under */tmp/maker_1YQF9o*. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared >>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>> directory setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>> to a different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. >>>>> The test run completed successfully, but as I commenced with the full >>>>> genome annotation, I have noticed the following error popping up frequently. >>>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>>> >>>>> >>>>> My first thought based on the message is that *blastdbcmd* could not >>>>> find the sequence in the database. I verified this was the case--I could >>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>> created under /tmp. However, after removing the index files and re-running >>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>>> rather its own internal code. Therefore I'm still not sure where the >>>>> problem is and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>> daniel.standage at gmail.com> wrote: >>>>> >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>> true -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this >>>>>> one failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>> _______________________________________________ maker-devel mailing >>>>> list maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Mon Oct 29 08:43:46 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Mon, 29 Oct 2012 09:43:46 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I just ran svn update and tried again with the development version of Maker. Same long list of errors as in the last message. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Mon, Oct 29, 2012 at 9:39 AM, Daniel Standage wrote: > Carson, > > I was able to run the first svn update before the test job ran, but I > didn't get the message about this second update until after it has already > executed and failed. I got about 372 lines of such. > > STATUS: Parsing control files... > STATUS: Processing and indexing input FASTA files... > STATUS: Setting up database for any GFF3 input... > A data structure will be created for you at: > > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_datastore > > To access files for individual sequences use the datastore index: > > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_master_datastore_index.log > > STATUS: Now running MAKER... > WARNING: Cannot find >scaffold_0, trying to re-index the fasta. > stop here: scaffold_0 > ERROR: Fasta index error > at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 239. > Process::MpiChunk::_prepare('Process::MpiChunk=HASH(0x3c8b300)', > 'HASH(0x3c80820)', 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 73 > Process::MpiTiers::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c808b0)', 'HASH(0x3c8ec40)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 79 > Process::MpiTiers::_prepare('Process::MpiTiers=HASH(0x3c71f30)') > called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 56 > Process::MpiTiers::new('Process::MpiTiers', 'HASH(0x3c80640)', 0, > 'Process::MpiChunk') called at > /N/u/dstandag/Mason/local/src/maker-dev/bin/maker line 627 > --> rank=NA, hostname=c4 > ERROR: Failed in tier preparation > WARNING: You must always set a rank before running MpiTiers > FATAL: argument `seq_id` does not exist in MpiTier object > at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 86. > > Process::MpiChunk::_initialize_vars('Process::MpiChunk=HASH(0x3cc93d0)', > 'HASH(0x3cc9400)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 47 > Process::MpiChunk::new('Process::MpiChunk', 'HASH(0x3c80820)', 0, > 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 407 > Process::MpiChunk::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c8f498)', 'HASH(0x3cc8f20)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 3811 > Process::MpiChunk::_go('Process::MpiChunk=HASH(0x3c8b300)', > 'load', 'HASH(0x3c80820)', 0, 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 310 > Process::MpiChunk::_loader('Process::MpiChunk=HASH(0x3c8b300)', > 'HASH(0x3c80820)', 0, 0, 'Process::MpiTiers=HASH(0x3c71f30)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 364 > Process::MpiTiers::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c8f9c0)', 'HASH(0x3c8fb28)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 375 > > > I'll try the next update and run the test again. I see a lot of references > in there to MPI, and just thought I would make it clear that although I am > running in a cluster environment, I am using the default serial version of > Maker, not the parallel MPI version. > > Also, after this failed, I tried changing the TMP directory so that it was > located on the same NFS mount as the scratch disk to which the output was > written. This did not seem to have any affect, and I saw the same issues > with the EST sequences unable to be found. > > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 6:59 PM, Carson Holt wrote: > >> >> >> From: Carson Holt >> Date: Friday, 26 October, 2012 6:10 PM >> To: Daniel Standage >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I've been going over the indexing code using different scenarios and I >> may have isolated a candidate for what is causing this. Could you do one >> more 'svn update' inside the maker devel directory before running a test >> job? >> >> Thanks, >> Carson >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:29 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Got this from the compute node. Looks like native disk space to me. >> >> [dstandag at mason ~] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sda1 478573472 12319684 441943620 3% /tmp >> >> Installing a bundle of Perl prereqs for development version, will try >> that soon. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: >> >>> Ok. Try the developer release and see if it still happens. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:19 PM >>> >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> Unfortunately, the job is no longer running and as a result I cannot >>> connect to the compute nodes as I could while it was running. On the >>> interactive node, it looks like it's real disk, although it looks like >>> there are some tmpfs mounts. >>> >>> [dstandag at mason src] df /tmp >>> Filesystem 1K-blocks Used Available Use% Mounted on >>> /dev/sdb2 462824304 180235660 259078476 42% /tmp >>> [dstandag at mason src] df >>> Filesystem 1K-blocks Used Available Use% Mounted on >>> login_x86_64 16497564 3077352 13420212 19% / >>> tmpfs 16497564 0 16497564 0% /dev/shm >>> tmpfs 10240 0 10240 0% /var/tmp >>> /dev/sdb2 462824304 180235660 259078476 42% /tmp >>> AFS 9000000 0 9000000 0% /afs >>> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >>> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >>> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >>> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >>> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >>> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >>> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >>> ... >>> ... >>> >>> I'll see if I can launch another short job and verify this on the >>> compute nodes. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >>> >>>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:12 PM >>>> To: Carson Holt >>>> Cc: "maker-devel at yandell-lab.org" >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> It looks like /tmp is indeed being used: the files I played with were >>>> under */tmp/maker_1YQF9o*. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>>> >>>>> Check to see where /tmp is located? Some clusters have it set up as a >>>>> tmpfs directory and I have had problems with fasta indexes running from >>>>> tmpfs mounts in the past. >>>>> >>>>> --Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 2:05 PM >>>>> To: Carson Holt >>>>> >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> The maker working directory is in a cluster environment with shared >>>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>>> directory setting, so it should be the local default (/tmp). >>>>> >>>>> I'll give the dev version a shot. Thanks. >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>>> >>>>>> Could you try this development version and tell me if the error still >>>>>> happens? >>>>>> >>>>>> Use this command to download --> >>>>>> <> >>>>>> >>>>>> Username: <> >>>>>> Password: <> >>>>>> >>>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>>> to a different location? >>>>>> >>>>>> Thanks, >>>>>> Carson >>>>>> >>>>>> >>>>>> From: Daniel Standage >>>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>>> To: Maker Mailing List >>>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>>> >>>>>> I have since installed Maker on a different machine and tried it out. >>>>>> The test run completed successfully, but as I commenced with the full >>>>>> genome annotation, I have noticed the following error popping up frequently. >>>>>> >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>>> #-------------------------------# >>>>>> deleted:-10 hits >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>>> #-------------------------------# >>>>>> deleted:-6 hits >>>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>>> stop here:comp59088_c1_seq7 >>>>>> ERROR: Fasta index error >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while polishig ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 14 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --Next Contig-- >>>>>> >>>>>> #--------------------------------------------------------------------- >>>>>> Now starting the contig!! >>>>>> SeqID: scaffold_1 >>>>>> Length: 5805686 >>>>>> #--------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> My first thought based on the message is that *blastdbcmd* could not >>>>>> find the sequence in the database. I verified this was the case--I could >>>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>>> created under /tmp. However, after removing the index files and re-running >>>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>>> extracted the sequence. >>>>>> >>>>>> I was initially happy with this finding, but upon closer inspection >>>>>> it looks like Maker does not use *blastdbcmd* to extract sequences, >>>>>> but rather its own internal code. Therefore I'm still not sure where the >>>>>> problem is and how I might fix it. Any insights? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>>> daniel.standage at gmail.com> wrote: >>>>>> >>>>>>> Greetings! >>>>>>> >>>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>>> during the blastn stage. This is the terminal message. >>>>>>> >>>>>>> #--------- command -------------# >>>>>>> Widget::blastn: >>>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>>> true -show_gis -out >>>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>>> #-------------------------------# >>>>>>> deleted:0 hits >>>>>>> ERROR: Could not obtain lock to format database >>>>>>> >>>>>>> >>>>>>> FATAL ERROR >>>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>>> >>>>>>> ERROR: Chunk failed at level 8 >>>>>>> !! >>>>>>> FAILED CONTIG:scaffold_866 >>>>>>> >>>>>>> >>>>>>> Several blastn steps appeared to have completed successfully to this >>>>>>> one failing. Any ideas what could be causing this? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> -- >>>>>>> Daniel S. Standage >>>>>>> Ph.D. Candidate >>>>>>> Bioinformatics and Computational Biology Program >>>>>>> Department of Genetics, Development, and Cell Biology >>>>>>> Iowa State University >>>>>>> >>>>>>> >>>>>> _______________________________________________ maker-devel mailing >>>>>> list maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Tue Oct 30 16:18:06 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Tue, 30 Oct 2012 14:18:06 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: Message-ID: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Hello Carson and maker community, Thank you very much for your guidelines on using the maker-pipeline. Yes, green sea urchin genome that we are trying to annotate. We are running the on scaffolds and most of these scaffolds are small in size(very first genome assembly). We would typically expect 20,000 genes in this genome. So we are running maker using EST and proteins from the genome and out-groups to generate training dataset for SNAP and Augustus. Depending on the resulting predictions we may bootstrap the predicted genes once again using EST and proteins. Do you have any further suggestions? Also could you point how to convert training set generated for SNAP to be used as training set for Augustus as well? Would maker give equal weightage to SNAP and Augustus predictions for generating gene model? Thanks and regards, Parul Kudtarkar > One thing you seem to be missing is protein evidence. > > Is this a sea urchin (I looked up some of the ESTs)? If so, I would recommend adding all proteins from the Strongylocentrotus purpuratus genome, then throw in another Deuterstome of your choice. Perhaps you should also add a couple of outgroup organisms like Nematostella vectensis > (cnidaria) and a protostome of your choice. Be careful if adding adding to many protostome outgroups (i.e. C. elegans and Drosophila) because a big part of their evolution is gene loss (so distant cnidaria often match > deuterstomes better than most protostomes do). > > You could take the maker results when protein data is included and use it > to retrain SNAP again. > > Even a 22 kb contig is still really short. Is this genome primarily constituted by short contigs like this? I would recommend running CEGMA once on this genome to get an appropriate estimate of how recoverable the > genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). Cegma > will give you an estimate for genome completeness as well as estimates of > what percentage of genes will be found in their entirety and what percent > will be partial genes. This is important to do if your genome is fragmented as it will give you a reasonable expectation of what you can expected to recover (as short contigs don't annotate very well - you tend > to loose a lot). > > Thanks, > Carson > > > On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: > >>Hi Carson, >>Thanks. I have attached another contig which is 22 kb, with as many as 3 exons EST alignments. Could you please recommend additional training. We are currently running maker on the entire contig set and eventually merge >>all the gff3 contig predictions. The using suggested parameter/methods we >>would like to get a consensus gene-set with minimal false >>positives/negatives. >>Thanks, >>Parul >>> The contig in question is really too small to get much out of it (only 5 >>kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb >>> length or longer contigs to glean enough information for optimizing your >>parameters. >>> The general suggestions for any maker run are to use proteins from a >>closely related organism or a couple of closely related organisms for the >>> protein= option in maker. Also leave single_exon set to 0, except for >>certain eukaryotes that have a bias for single exon transcripts (i.e. some >>> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >>predictors tend to over-predict by a wide margin (lots of false >>> positives). >>> Additional training would really depend on what your other contigs look >>like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. >>> Thanks, >>> Carson >>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>Hello, >>>>Please advice on the aforementioned query? >>>>Thanks, >>>>Parul Kudtarkar >>>>---------------------------- Original Message >>>> ---------------------------- >>>>Subject: [maker-devel] Conensus gene model >>>>From: "Parul Kudtarkar" >>>>Date: Fri, October 12, 2012 2:46 pm >>>>To: maker-devel at yandell-lab.org >>>>------------------------------------------------------------------------ -- >>Hi, >>>>We are using snap(training set[hmm file] generated using est,protein >>>> and >>contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>> file >>>>as input). The output that we get is combination of various >>>>gene-predictors and evidences. I have attached sample result file. What >>would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? >>>>Thanks, >>>>Parul Kudtarkar >>>>-- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org_______________________________________________ >>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org_______________________________________________ >>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jason.stajich at gmail.com Tue Oct 30 19:52:43 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Tue, 30 Oct 2012 17:52:43 -0700 Subject: [maker-devel] Conensus gene model In-Reply-To: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> References: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Message-ID: Paul - I think I've posted on this before here if you are asking how to go from SNAP training to Augustus training. http://sourceforge.net/mailarchive/message.php?msg_id=29361270 I do this type of training a lot - here some pointers. I often train by generating models using cegma on the genome and get these 400 or so good models as my training set. when I have EST or RNA-Seq I use PASA to generate the best set of annotations. For CEGMA - then I run this script that comes with MAKER: cegma2zff output.cegma.gff genome.fa Then I follow the SNAP directions fathom genome.ann genome.dna -categorize 1000 fathom uni.ann uni.dna -export 1000 -plus mkdir MYGENOME cd MYGENOME forge ../export.ann ../export.dna --OPTIONS cd ../MYGENOME hmm-assembler.pl MYGENOME MYGENOME > MYGENOME.snap.hmm I then also make the augustus training data like this running in the directory that has the export.ann and export.dna files: perl gene_prediction/zff2augustus_gbk.pl > train.gb using this script: https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl I also make ZFF from GFF with this script if I got the RNA-Seq aligned and best models from PASA and incorporate all these data in to my SNAP training set, and then export again back to gbk for the augustus training. https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/pasatraining2zff.pl Then you just need to run the Augustus training (autoAugTrain.pl) on the train.gb file. Jason On Oct 30, 2012, at 2:18 PM, Parul Kudtarkar wrote: > Hello Carson and maker community, > > Thank you very much for your guidelines on using the maker-pipeline. Yes, > green sea urchin genome that we are trying to annotate. > We are running the on scaffolds and most of these scaffolds are small in > size(very first genome assembly). We would typically expect 20,000 genes > in this genome. So we are running maker using EST and proteins from the > genome and out-groups to generate training dataset for SNAP and Augustus. > Depending on the resulting predictions we may bootstrap the predicted > genes once again using EST and proteins. > > Do you have any further suggestions? Also could you point how to convert > training set generated for SNAP to be used as training set for Augustus as > well? Would maker give equal weightage to SNAP and Augustus predictions > for generating gene model? > > Thanks and regards, > Parul Kudtarkar > >> One thing you seem to be missing is protein evidence. >> >> Is this a sea urchin (I looked up some of the ESTs)? If so, I would > recommend adding all proteins from the Strongylocentrotus purpuratus > genome, then throw in another Deuterstome of your choice. Perhaps you > should also add a couple of outgroup organisms like Nematostella > vectensis >> (cnidaria) and a protostome of your choice. Be careful if adding adding > to many protostome outgroups (i.e. C. elegans and Drosophila) because a > big part of their evolution is gene loss (so distant cnidaria often > match >> deuterstomes better than most protostomes do). >> >> You could take the maker results when protein data is included and use > it >> to retrain SNAP again. >> >> Even a 22 kb contig is still really short. Is this genome primarily > constituted by short contigs like this? I would recommend running CEGMA > once on this genome to get an appropriate estimate of how recoverable > the >> genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). > Cegma >> will give you an estimate for genome completeness as well as estimates > of >> what percentage of genes will be found in their entirety and what > percent >> will be partial genes. This is important to do if your genome is > fragmented as it will give you a reasonable expectation of what you can > expected to recover (as short contigs don't annotate very well - you > tend >> to loose a lot). >> >> Thanks, >> Carson >> >> >> On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >> >>> Hi Carson, >>> Thanks. I have attached another contig which is 22 kb, with as many as 3 > exons EST alignments. Could you please recommend additional training. We > are currently running maker on the entire contig set and eventually > merge >>> all the gff3 contig predictions. The using suggested parameter/methods > we >>> would like to get a consensus gene-set with minimal false >>> positives/negatives. >>> Thanks, >>> Parul >>>> The contig in question is really too small to get much out of it (only 5 >>> kb). There was only one single exon EST alignments and a couple of > predictions with no evidence support. Anything smaller than 10 kb is > mostly useless for annotation purposes. You would really need a few > 100kb >>>> length or longer contigs to glean enough information for optimizing your >>> parameters. >>>> The general suggestions for any maker run are to use proteins from a >>> closely related organism or a couple of closely related organisms for the >>>> protein= option in maker. Also leave single_exon set to 0, except for >>> certain eukaryotes that have a bias for single exon transcripts (i.e. some >>>> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >>> predictors tend to over-predict by a wide margin (lots of false >>>> positives). >>>> Additional training would really depend on what your other contigs > look >>> like. Do you have any large contigs? I could look at one of those and > give suggestions but the provided contig is just too short to glean > much. >>>> Thanks, >>>> Carson >>>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>> Hello, >>>>> Please advice on the aforementioned query? >>>>> Thanks, >>>>> Parul Kudtarkar >>>>> ---------------------------- Original Message >>>>> ---------------------------- >>>>> Subject: [maker-devel] Conensus gene model >>>>> From: "Parul Kudtarkar" >>>>> Date: Fri, October 12, 2012 2:46 pm >>>>> To: maker-devel at yandell-lab.org >>>>> ------------------------------------------------------------------------ > -- >>> Hi, >>>>> We are using snap(training set[hmm file] generated using est,protein >>>>> and >>> contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>>> file >>>>> as input). The output that we get is combination of various >>>>> gene-predictors and evidences. I have attached sample result file. > What >>> would you recommend to get consensus result set? Bootstrapping the > resulting gff3 file (rerunning maker)? >>>>> Thanks, >>>>> Parul Kudtarkar >>>>> -- >>>>> Scientific Programmer >>>>> Center for Computational Regulatory Genomics >>>>> Beckman Institute, >>>>> California Institute of Technology >>>>> http://www.spbase.org_______________________________________________ >>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- >>>>> Scientific Programmer >>>>> Center for Computational Regulatory Genomics >>>>> Beckman Institute, >>>>> California Institute of Technology >>>>> http://www.spbase.org_______________________________________________ >>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> -- >>> Scientific Programmer >>> Center for Computational Regulatory Genomics >>> Beckman Institute, >>> California Institute of Technology >>> http://www.spbase.org >> >> >> > > > -- > Scientific Programmer > Center for Computational Regulatory Genomics > Beckman Institute, > California Institute of Technology > http://www.spbase.org > > > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org From parulk at caltech.edu Wed Oct 31 19:04:33 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Wed, 31 Oct 2012 17:04:33 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Message-ID: <1640.131.215.15.234.1351728273.squirrel@webmail.caltech.edu> Hi Jason, thanks for directions on generating training-set for augustus. Also as alignment evidence if we are providing protein sequences from the primary organism as well as other closely related species is there an option to give the primary protein file precedence over others? At the moment I have all the proteins(from primary organism as well as related species) into a single file as protein option in maker_opts.ctl Thanks and regards, Parul Kudtarkar > Paul - > > I think I've posted on this before here if you are asking how to go from > SNAP training to Augustus training. > http://sourceforge.net/mailarchive/message.php?msg_id=29361270 > > I do this type of training a lot - here some pointers. > > I often train by generating models using cegma on the genome and get these > 400 or so good models as my training set. when I have EST or RNA-Seq I > use PASA to generate the best set of annotations. > > For CEGMA - then I run this script that comes with MAKER: > cegma2zff output.cegma.gff genome.fa > > Then I follow the SNAP directions > > fathom genome.ann genome.dna -categorize 1000 > fathom uni.ann uni.dna -export 1000 -plus > mkdir MYGENOME > cd MYGENOME > forge ../export.ann ../export.dna --OPTIONS > cd ../MYGENOME > hmm-assembler.pl MYGENOME MYGENOME > MYGENOME.snap.hmm > > I then also make the augustus training data like this running in the > directory that has the export.ann and export.dna files: > perl gene_prediction/zff2augustus_gbk.pl > train.gb > > using this script: > https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl > > I also make ZFF from GFF with this script if I got the RNA-Seq aligned and > best models from PASA and incorporate all these data in to my SNAP > training set, and then export again back to gbk for the augustus training. > https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/pasatraining2zff.pl > > Then you just need to run the Augustus training (autoAugTrain.pl) on the > train.gb file. > > Jason > > On Oct 30, 2012, at 2:18 PM, Parul Kudtarkar wrote: > >> Hello Carson and maker community, >> >> Thank you very much for your guidelines on using the maker-pipeline. >> Yes, >> green sea urchin genome that we are trying to annotate. >> We are running the on scaffolds and most of these scaffolds are small in >> size(very first genome assembly). We would typically expect 20,000 genes >> in this genome. So we are running maker using EST and proteins from the >> genome and out-groups to generate training dataset for SNAP and >> Augustus. >> Depending on the resulting predictions we may bootstrap the predicted >> genes once again using EST and proteins. >> >> Do you have any further suggestions? Also could you point how to convert >> training set generated for SNAP to be used as training set for Augustus >> as >> well? Would maker give equal weightage to SNAP and Augustus predictions >> for generating gene model? >> >> Thanks and regards, >> Parul Kudtarkar >> >>> One thing you seem to be missing is protein evidence. >>> >>> Is this a sea urchin (I looked up some of the ESTs)? If so, I would >> recommend adding all proteins from the Strongylocentrotus purpuratus >> genome, then throw in another Deuterstome of your choice. Perhaps you >> should also add a couple of outgroup organisms like Nematostella >> vectensis >>> (cnidaria) and a protostome of your choice. Be careful if adding >>> adding >> to many protostome outgroups (i.e. C. elegans and Drosophila) because a >> big part of their evolution is gene loss (so distant cnidaria often >> match >>> deuterstomes better than most protostomes do). >>> >>> You could take the maker results when protein data is included and use >> it >>> to retrain SNAP again. >>> >>> Even a 22 kb contig is still really short. Is this genome primarily >> constituted by short contigs like this? I would recommend running CEGMA >> once on this genome to get an appropriate estimate of how recoverable >> the >>> genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). >> Cegma >>> will give you an estimate for genome completeness as well as estimates >> of >>> what percentage of genes will be found in their entirety and what >> percent >>> will be partial genes. This is important to do if your genome is >> fragmented as it will give you a reasonable expectation of what you can >> expected to recover (as short contigs don't annotate very well - you >> tend >>> to loose a lot). >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >>> >>>> Hi Carson, >>>> Thanks. I have attached another contig which is 22 kb, with as many as >>>> 3 >> exons EST alignments. Could you please recommend additional training. We >> are currently running maker on the entire contig set and eventually >> merge >>>> all the gff3 contig predictions. The using suggested parameter/methods >> we >>>> would like to get a consensus gene-set with minimal false >>>> positives/negatives. >>>> Thanks, >>>> Parul >>>>> The contig in question is really too small to get much out of it >>>>> (only 5 >>>> kb). There was only one single exon EST alignments and a couple of >> predictions with no evidence support. Anything smaller than 10 kb is >> mostly useless for annotation purposes. You would really need a few >> 100kb >>>>> length or longer contigs to glean enough information for optimizing >>>>> your >>>> parameters. >>>>> The general suggestions for any maker run are to use proteins from a >>>> closely related organism or a couple of closely related organisms for >>>> the >>>>> protein= option in maker. Also leave single_exon set to 0, except >>>>> for >>>> certain eukaryotes that have a bias for single exon transcripts (i.e. >>>> some >>>>> fungi and oomycetes). And leave keep_preds set to 0 because ab >>>>> initio >>>> predictors tend to over-predict by a wide margin (lots of false >>>>> positives). >>>>> Additional training would really depend on what your other contigs >> look >>>> like. Do you have any large contigs? I could look at one of those >>>> and >> give suggestions but the provided contig is just too short to glean >> much. >>>>> Thanks, >>>>> Carson >>>>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>>> Hello, >>>>>> Please advice on the aforementioned query? >>>>>> Thanks, >>>>>> Parul Kudtarkar >>>>>> ---------------------------- Original Message >>>>>> ---------------------------- >>>>>> Subject: [maker-devel] Conensus gene model >>>>>> From: "Parul Kudtarkar" >>>>>> Date: Fri, October 12, 2012 2:46 pm >>>>>> To: maker-devel at yandell-lab.org >>>>>> ------------------------------------------------------------------------ >> -- >>>> Hi, >>>>>> We are using snap(training set[hmm file] generated using est,protein >>>>>> and >>>> contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>>>> file >>>>>> as input). The output that we get is combination of various >>>>>> gene-predictors and evidences. I have attached sample result file. >> What >>>> would you recommend to get consensus result set? Bootstrapping the >> resulting gff3 file (rerunning maker)? >>>>>> Thanks, >>>>>> Parul Kudtarkar >>>>>> -- >>>>>> Scientific Programmer >>>>>> Center for Computational Regulatory Genomics >>>>>> Beckman Institute, >>>>>> California Institute of Technology >>>>>> http://www.spbase.org_______________________________________________ >>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> -- >>>>>> Scientific Programmer >>>>>> Center for Computational Regulatory Genomics >>>>>> Beckman Institute, >>>>>> California Institute of Technology >>>>>> http://www.spbase.org_______________________________________________ >>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> -- >>>> Scientific Programmer >>>> Center for Computational Regulatory Genomics >>>> Beckman Institute, >>>> California Institute of Technology >>>> http://www.spbase.org >>> >>> >>> >> >> >> -- >> Scientific Programmer >> Center for Computational Regulatory Genomics >> Beckman Institute, >> California Institute of Technology >> http://www.spbase.org >> >> >> >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > Jason Stajich > jason.stajich at gmail.com > jason at bioperl.org > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From carsonhh at gmail.com Mon Oct 1 08:01:01 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 01 Oct 2012 10:01:01 -0400 Subject: [maker-devel] model_gff question In-Reply-To: Message-ID: They can be replaced under two circumstances. 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. Thanks, Carson From: Michael Thon Date: Monday, 1 October, 2012 1:53 AM To: Subject: [maker-devel] model_gff question Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. -Mike _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.thon at gmail.com Mon Oct 1 23:01:09 2012 From: mike.thon at gmail.com (Michael Thon) Date: Tue, 2 Oct 2012 07:01:09 +0200 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: I looked at two cases in which the model_gff disappeared and they occurred in regions where there are multiple overlapping cufflinks features. One model that I'm looking at right now has overlapping protein2genome and a SNAP feature overlapping it but it was still not included in the output. it could be a problem in MAKER or it could be a problem with my RNA Seq data. I aligned the RNA Seq data using tophat/cufflinks and converted the transcripts.gtf file to gff using cufflinks2gff3 script. Is it better to use RNA Seq feature from tophat or cufflinks? On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > They can be replaced under two circumstances. > 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. > 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). > > MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. > > Thanks, > Carson > > > From: Michael Thon > Date: Monday, 1 October, 2012 1:53 AM > To: > Subject: [maker-devel] model_gff question > > Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: > https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion > > That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. > -Mike > > _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.thon at gmail.com Tue Oct 2 01:50:04 2012 From: mike.thon at gmail.com (Michael Thon) Date: Tue, 2 Oct 2012 09:50:04 +0200 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: <8E7B4629-B071-44A6-8E7F-2B1133A946D0@gmail.com> It seems to have disappeared completely. I'm running MAKER again now using the tophat alignments that I fed to cufflinks, instead of the cufflinks data. So far the two models visually checked as missing with the cufflinks data are present as they should be. I have to wait for the run to finish to get a whole genome count though. Maybe I need to look more closely at the cufflinks run that I did. The RNA-Seq data are from the NCBI SRA and I didn't do anything to clean them up before I ran tophat. On Oct 2, 2012, at 9:10 AM, Daniel Hughes wrote: > Did the whole model vanish or just the protein product - contaminated rnaseq that hasn't been cleaned up enough will regularly cause the later to become part of a bad utr. > > Dan > > On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: > I looked at two cases in which the model_gff disappeared and they occurred in regions where there are multiple overlapping cufflinks features. One model that I'm looking at right now has overlapping protein2genome and a SNAP feature overlapping it but it was still not included in the output. it could be a problem in MAKER or it could be a problem with my RNA Seq data. I aligned the RNA Seq data using tophat/cufflinks and converted the transcripts.gtf file to gff using cufflinks2gff3 script. > > Is it better to use RNA Seq feature from tophat or cufflinks? > > > On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > >> They can be replaced under two circumstances. >> 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. >> 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). >> >> MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. >> >> Thanks, >> Carson >> >> >> From: Michael Thon >> Date: Monday, 1 October, 2012 1:53 AM >> To: >> Subject: [maker-devel] model_gff question >> >> Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: >> https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion >> >> That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. >> -Mike >> >> _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Oct 2 12:05:04 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 02 Oct 2012 14:05:04 -0400 Subject: [maker-devel] model_gff question In-Reply-To: <8E7B4629-B071-44A6-8E7F-2B1133A946D0@gmail.com> Message-ID: If it overlaps the UTR of some other chosen model it might have been excluded for that reason as well. Sometimes this can happen when you have RNA-seq and high gene density (so models wander into each other). Try setting the the correct_est_fusion option to 1. This will take steps to trim UTR that might cause neighboring models to be left out because of UTR overlap (it also helps with false fusions caused by cufflinks results). I would not be surprised if the model being left out was because of UTR overlap. I recommend using the cufflinks data and leaving tophat out. Tophat results tend to be very noisy and can span large rather weird regions. Thanks, Carson From: Michael Thon Date: Tuesday, 2 October, 2012 3:50 AM To: Daniel Hughes Cc: Michael Thon , , Carson Holt Subject: Re: [maker-devel] model_gff question It seems to have disappeared completely. I'm running MAKER again now using the tophat alignments that I fed to cufflinks, instead of the cufflinks data. So far the two models visually checked as missing with the cufflinks data are present as they should be. I have to wait for the run to finish to get a whole genome count though. Maybe I need to look more closely at the cufflinks run that I did. The RNA-Seq data are from the NCBI SRA and I didn't do anything to clean them up before I ran tophat. On Oct 2, 2012, at 9:10 AM, Daniel Hughes wrote: > > Did the whole model vanish or just the protein product - contaminated rnaseq > that hasn't been cleaned up enough will regularly cause the later to become > part of a bad utr. > > Dan > > On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: >> I looked at two cases in which the model_gff disappeared and they occurred in >> regions where there are multiple overlapping cufflinks features. One model >> that I'm looking at right now has overlapping protein2genome and a SNAP >> feature overlapping it but it was still not included in the output. it could >> be a problem in MAKER or it could be a problem with my RNA Seq data. I >> aligned the RNA Seq data using tophat/cufflinks and converted the >> transcripts.gtf file to gff using cufflinks2gff3 script. >> >> Is it better to use RNA Seq feature from tophat or cufflinks? >> >> >> On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: >> >>> They can be replaced under two circumstances. >>> 1. If you provide two model_gff files (comma separated list), in which case >>> MAKER thinks it is merging legacy annotations and will only keep one or the >>> other if models overlap. >>> 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this >>> as a cue that if these other programs produce a better model, it can replace >>> the current model. If you set map_forward=1, MAKER will conserve the name >>> of the previous model (so models change structure but names are conserved); >>> otherwise, it gets a new name. Sometimes groups like to rename models every >>> time their is a structural change. I think you are supposed to get the >>> Alias attribute set when you don't get names mapped forward though (I can't >>> remember if I added this or just planned on adding the Alias mapping >>> though). >>> >>> MAKER should never drop a model_gff model. It can only replace it if >>> something better comes along, but it should not disappear. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Michael Thon >>> Date: Monday, 1 October, 2012 1:53 AM >>> To: >>> Subject: [maker-devel] model_gff question >>> >>> Under what circumstances will maker not include a gene model from the >>> model_gff file in its final output? It was my understanding from this post: >>> https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion >>> >>> That maker will keep or replace models in model_gff and never remove them. >>> I'm reannotating a fungal genome and in model_gff I'm providing the gene >>> models originally made by the sequencing center. I have 12006 models in the >>> file I specify in model_gff but maker's final annotation has only 10727 >>> models in it. >>> -Mike >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m >>> aker-devel_yandell-lab.org >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dsthughes at gmail.com Tue Oct 2 01:10:19 2012 From: dsthughes at gmail.com (Daniel Hughes) Date: Tue, 2 Oct 2012 08:10:19 +0100 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: Did the whole model vanish or just the protein product - contaminated rnaseq that hasn't been cleaned up enough will regularly cause the later to become part of a bad utr. Dan On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: > I looked at two cases in which the model_gff disappeared and they occurred > in regions where there are multiple overlapping cufflinks features. One > model that I'm looking at right now has overlapping protein2genome and a > SNAP feature overlapping it but it was still not included in the output. > it could be a problem in MAKER or it could be a problem with my RNA Seq > data. I aligned the RNA Seq data using tophat/cufflinks and converted the > transcripts.gtf file to gff using cufflinks2gff3 script. > > Is it better to use RNA Seq feature from tophat or cufflinks? > > > On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > > They can be replaced under two circumstances. > 1. If you provide two model_gff files (comma separated list), in which > case MAKER thinks it is merging legacy annotations and will only keep one > or the other if models overlap. > 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees > this as a cue that if these other programs produce a better model, it can > replace the current model. If you set map_forward=1, MAKER will conserve > the name of the previous model (so models change structure but names are > conserved); otherwise, it gets a new name. Sometimes groups like to rename > models every time their is a structural change. I think you are supposed > to get the Alias attribute set when you don't get names mapped forward > though (I can't remember if I added this or just planned on adding the > Alias mapping though). > > MAKER should never drop a model_gff model. It can only replace it if > something better comes along, but it should not disappear. > > Thanks, > Carson > > > From: Michael Thon > Date: Monday, 1 October, 2012 1:53 AM > To: > Subject: [maker-devel] model_gff question > > Under what circumstances will maker not include a gene model from the > model_gff file in its final output? It was my understanding from this post: > https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion > > That maker will keep or replace models in model_gff and never remove them. > I'm reannotating a fungal genome and in model_gff I'm providing the gene > models originally made by the sequencing center. I have 12006 models in > the file I specify in model_gff but maker's final annotation has only 10727 > models in it. > -Mike > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amelia.ireland at gmod.org Tue Oct 2 15:00:57 2012 From: amelia.ireland at gmod.org (Amelia Ireland) Date: Tue, 2 Oct 2012 14:00:57 -0700 Subject: [maker-devel] problem installing maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868AF3A@EXMBX05.austin.utexas.edu> References: <1B4E4BB25B711A4FBC93E4B3AFEB2A3218689DC6@EXMBX05.austin.utexas.edu> <1B4E4BB25B711A4FBC93E4B3AFEB2A321868AF3A@EXMBX05.austin.utexas.edu> Message-ID: Hello Oscar, I'm forwarding your email on to the MAKER mailing list and one of the developers; they should be able to answer your query. Thanks, Amelia. On Tue, Oct 2, 2012 at 1:55 PM, Dian "Oscar" Jiao wrote: > Hi, > > I was trying to install Maker. All the prerequisite seemed installed okay. > However, when I ran maker or mpimaker, I got the following error: > /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_File/DB_File.so: > undefined symbol: db_version". I have edited config.in before compiling to > make sure the paths are correct for db.h and libdb. Do you have any idea > what is causing this? > > It seems to have to do with the berkeleydb and DB_File. I have BerkeleyDB > 5.3.21 installed and set the paths correctly for db.h and libdb in config.in > when I was compiling DB_File, but it still failed. Do you know what is > causing this? > > Thanks > Oscar > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > -- > You received this message because you are subscribed to the Google Groups > "GMOD Help Desk" group. > To post to this group, send email to gmod-help-desk at googlegroups.com. > To unsubscribe from this group, send email to > gmod-help-desk+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > -- Amelia Ireland GMOD Community Support Specialist || http://gmod.org From carsonhh at gmail.com Wed Oct 3 08:13:20 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 03 Oct 2012 10:13:20 -0400 Subject: [maker-devel] problem installing maker In-Reply-To: Message-ID: You may need to completely delete this directory before reinstalling DB_File. --> /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_Fil e Also if that doesn't work you can skip DB_file completely, and use a different database backend During the MAKER install do the following: perl ./Build.PL --AnyDBM_ISA SDBM_File ./Build install Then test maker once on a new job. The --AnyDBM_ISA sets a different database than Berkley DB (All of which are slower than Berkley DB). --Carson On 12-10-02 5:00 PM, "Amelia Ireland" wrote: >Hello Oscar, > >I'm forwarding your email on to the MAKER mailing list and one of the >developers; they should be able to answer your query. > >Thanks, >Amelia. > >On Tue, Oct 2, 2012 at 1:55 PM, Dian "Oscar" Jiao >wrote: >> Hi, >> >> I was trying to install Maker. All the prerequisite seemed installed >>okay. >> However, when I ran maker or mpimaker, I got the following error: >> >>/work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_F >>ile/DB_File.so: >> undefined symbol: db_version". I have edited config.in before compiling >>to >> make sure the paths are correct for db.h and libdb. Do you have any idea >> what is causing this? >> >> It seems to have to do with the berkeleydb and DB_File. I have >>BerkeleyDB >> 5.3.21 installed and set the paths correctly for db.h and libdb in >>config.in >> when I was compiling DB_File, but it still failed. Do you know what is >> causing this? >> >> Thanks >> Oscar >> -- >> Dian (Oscar) Jiao, Ph.D., >> Research Associate, Life Sciences Computing Group >> Texas Advanced Computing Center >> University of Texas at Austin >> jiao at tacc.utexas.edu | (832) 303-1166 >> >> -- >> You received this message because you are subscribed to the Google >>Groups >> "GMOD Help Desk" group. >> To post to this group, send email to gmod-help-desk at googlegroups.com. >> To unsubscribe from this group, send email to >> gmod-help-desk+unsubscribe at googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > > >-- >Amelia Ireland >GMOD Community Support Specialist || http://gmod.org > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From parulk at caltech.edu Wed Oct 3 20:14:36 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Wed, 3 Oct 2012 19:14:36 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs Message-ID: <2553.131.215.15.234.1349316876.squirrel@webmail.caltech.edu> I am running maker on example data that comes along with installation and is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note Please advice on the aforementioned error. --------------------- Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_proteins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est%2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_protein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_proteins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est%2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_protein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ------------------------------------ Many thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jiao at tacc.utexas.edu Wed Oct 3 22:04:32 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 04:04:32 +0000 Subject: [maker-devel] problem running maker Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868CD39@EXMBX05.austin.utexas.edu> Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 06:27:05 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 08:27:05 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868CD39@EXMBX05.austin.utexas.edu> Message-ID: Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 06:34:38 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 08:34:38 -0400 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <2553.131.215.15.234.1349316876.squirrel@webmail.caltech.edu> Message-ID: You probably need to reinstall Bio-perl. Other things that can cause the same error are setting TMP in the maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or sometimes setting it to an NFS mount. A full drive can cause this as well or a broken Berkley DB. Use df -h to see if any of the drives used are full (either current working directory or TMP location). You can also swap Berkley DB for a different backend by setting AnyDBM_ISA during setup. Example: cd ../maker/src/ perl Build.PL --AnyDBM_ISA SDBM_File ./Build install --Carson On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >I am running maker on example data that comes along with installation and >is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note > >Please advice on the aforementioned error. > >--------------------- >Maker is now finished!!! > >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >eins%2Efasta.repeatrunner >deleted:0 hits > in cluster:shadow cluster... > i_size:0 j_size:0 > sorting hits in shadow cluster... >... finished. >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >%2Efasta.blastn >deleted:-1 hits >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >tein%2Efasta.blastx >WARNING: Multiple MAKER processes have been started in the >same directory. > >deleted:0 hits >WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >stop here:dpp-mRNA-4 >ERROR: Fasta index error > >FATAL ERROR >Maker is now finished!!! > >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >eins%2Efasta.repeatrunner >deleted:0 hits > in cluster:shadow cluster... > i_size:0 j_size:0 > sorting hits in shadow cluster... >... finished. >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >%2Efasta.blastn >deleted:-1 hits >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >tein%2Efasta.blastx >WARNING: Multiple MAKER processes have been started in the >same directory. > >deleted:0 hits >WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >stop here:dpp-mRNA-4 >ERROR: Fasta index error > >FATAL ERROR >ERROR: Failed while polishig ESTs!! > >ERROR: Chunk failed at level 14 >!! >FAILED CONTIG:contig-dpp-500-500 > > > >ERROR: Chunk failed at level 14 >!! >FAILED CONTIG:contig-dpp-500-500 >------------------------------------ > > >Many thanks, >Parul Kudtarkar > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From jiao at tacc.utexas.edu Thu Oct 4 08:02:14 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 14:02:14 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D305@EXMBX05.austin.utexas.edu> 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 08:03:01 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 10:03:01 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D305@EXMBX05.austin.utexas.edu> Message-ID: Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 09:36:22 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 15:36:22 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 11:17:08 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 13:17:08 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Message-ID: Seg faults really only happen from C and not Perl, so there may be a perl module that is broken on your machine that use C internally. Here are some modules that MAKER can use that I know use C and that you might want to reinstall from CPAN. DB_File Forks Proc::ProcessTable Inline::C Also during the maker install say no to the "use MPI" question. It's best to make the installation as simple as possible before getting into MPI configuration. Also after installing those and rerunning maker's 'perl Build.PL' and './Build install' steps do your first run with 'maker --debug' (redirect STDERR to a file). Then if it fails, you can send me the error log. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 11:36 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 11:23:52 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 17:23:52 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D36D@EXMBX05.austin.utexas.edu> I tried the debug trick and it worked (at least it finished). I however when I looked at the stderr, I again noticed the "can't locate GDBM_File for @AnyDBM_File::ISA" error. Although it did not affect the output, is it what is causing the segmentation error? Because this is the only error other than a bunch of "UNKOWN Bio" messages. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 12:17 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Seg faults really only happen from C and not Perl, so there may be a perl module that is broken on your machine that use C internally. Here are some modules that MAKER can use that I know use C and that you might want to reinstall from CPAN. DB_File Forks Proc::ProcessTable Inline::C Also during the maker install say no to the "use MPI" question. It's best to make the installation as simple as possible before getting into MPI configuration. Also after installing those and rerunning maker's 'perl Build.PL' and './Build install' steps do your first run with 'maker --debug' (redirect STDERR to a file). Then if it fails, you can send me the error log. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 11:36 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 12:52:51 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 14:52:51 -0400 Subject: [maker-devel] Expected format for EST GFF3 Message-ID: Greetings. I would like to use Maker's *est_gff* option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 12:58:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 14:58:56 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Match/match_part features are expected. MAKER expects these to be polished (I.e. correctly aligned around splice sites). For example exonerate, BLAT, and cufflinks results will be correct around splice sites and can be used; BLAST results on the other hand will not be correct and should not be used. The Gap attribute is used by maker if available, but is not required (Gap describes how to reconstruct an alignment for gaps and mismatches). Otherwise MAKER assumes all positions are matches to the reference. Thanks, Carson From: Daniel Standage Date: Thursday, 4 October, 2012 2:52 PM To: Maker Mailing List Subject: [maker-devel] Expected format for EST GFF3 Greetings. I would like to use Maker's est_gff option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:07:18 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:07:18 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the *transcript* features to *match* and the *exon* features to *match_part *and make sure the parent/child relationships are maintained with the *Parent* and *ID* attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > Match/match_part features are expected. MAKER expects these to be > polished (I.e. correctly aligned around splice sites). For example > exonerate, BLAT, and cufflinks results will be correct around splice sites > and can be used; BLAST results on the other hand will not be correct and > should not be used. > > The Gap attribute is used by maker if available, but is not required (Gap > describes how to reconstruct an alignment for gaps and mismatches). > Otherwise MAKER assumes all positions are matches to the reference. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM > To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 > > Greetings. > > I would like to use Maker's *est_gff* option to include EST evidence from > an external GFF3 file. In addition to being a valid, well-formed GFF3 file, > what expectations does Maker assume about the contents of this file? Which > feature types are expected and/or supported? Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 13:09:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 15:09:25 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Use the cufflinks2gff3 script that comes with MAKER. Thanks, Carson From: Daniel Standage Date: Thursday, 4 October, 2012 3:07 PM To: Carson Holt Cc: Maker Mailing List Subject: Re: [maker-devel] Expected format for EST GFF3 Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only transcript and exon features. So I'm assuming I can simply convert the transcript features to match and the exon features to match_part and make sure the parent/child relationships are maintained with the Parent and ID attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > Match/match_part features are expected. MAKER expects these to be polished > (I.e. correctly aligned around splice sites). For example exonerate, BLAT, > and cufflinks results will be correct around splice sites and can be used; > BLAST results on the other hand will not be correct and should not be used. > > The Gap attribute is used by maker if available, but is not required (Gap > describes how to reconstruct an alignment for gaps and mismatches). Otherwise > MAKER assumes all positions are matches to the reference. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM > To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 > > Greetings. > > I would like to use Maker's est_gff option to include EST evidence from an > external GFF3 file. In addition to being a valid, well-formed GFF3 file, what > expectations does Maker assume about the contents of this file? Which feature > types are expected and/or supported? Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/mak > er-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:16:53 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:16:53 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Does the cufflinks2gff3 script filter out single-exon transcripts? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt wrote: > Use the cufflinks2gff3 script that comes with MAKER. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 3:07 PM > To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 > > Great. I am using PE Illumina reads mapped by Tophat and assembled by > Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the > *transcript* features to *match* and the *exon* features to *match_part *and > make sure the parent/child relationships are maintained with the *Parent* and > *ID* attributes? > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > >> Match/match_part features are expected. MAKER expects these to be >> polished (I.e. correctly aligned around splice sites). For example >> exonerate, BLAT, and cufflinks results will be correct around splice sites >> and can be used; BLAST results on the other hand will not be correct and >> should not be used. >> >> The Gap attribute is used by maker if available, but is not required (Gap >> describes how to reconstruct an alignment for gaps and mismatches). >> Otherwise MAKER assumes all positions are matches to the reference. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Thursday, 4 October, 2012 2:52 PM >> To: Maker Mailing List >> Subject: [maker-devel] Expected format for EST GFF3 >> >> Greetings. >> >> I would like to use Maker's *est_gff* option to include EST evidence >> from an external GFF3 file. In addition to being a valid, well-formed GFF3 >> file, what expectations does Maker assume about the contents of this file? >> Which feature types are expected and/or supported? Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:21:52 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:21:52 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Good to know. Thanks for your help! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:19 PM, Carson Holt wrote: > Yes. You really have to because the single exon transcripts produced > from mRNA-seq assembly are strandless (you don't know where they belong). > They also tend to be heavily weighted to pseudogenes and transposons. > > Thanks, > Carson > > > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 3:16 PM > > To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 > > Does the cufflinks2gff3 script filter out single-exon transcripts? > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt wrote: > >> Use the cufflinks2gff3 script that comes with MAKER. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Thursday, 4 October, 2012 3:07 PM >> To: Carson Holt >> Cc: Maker Mailing List >> Subject: Re: [maker-devel] Expected format for EST GFF3 >> >> Great. I am using PE Illumina reads mapped by Tophat and assembled by >> Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the >> *transcript* features to *match* and the *exon* features to *match_part *and >> make sure the parent/child relationships are maintained with the *Parent* and >> *ID* attributes? >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: >> >>> Match/match_part features are expected. MAKER expects these to be >>> polished (I.e. correctly aligned around splice sites). For example >>> exonerate, BLAT, and cufflinks results will be correct around splice sites >>> and can be used; BLAST results on the other hand will not be correct and >>> should not be used. >>> >>> The Gap attribute is used by maker if available, but is not required >>> (Gap describes how to reconstruct an alignment for gaps and mismatches). >>> Otherwise MAKER assumes all positions are matches to the reference. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Thursday, 4 October, 2012 2:52 PM >>> To: Maker Mailing List >>> Subject: [maker-devel] Expected format for EST GFF3 >>> >>> Greetings. >>> >>> I would like to use Maker's *est_gff* option to include EST evidence >>> from an external GFF3 file. In addition to being a valid, well-formed GFF3 >>> file, what expectations does Maker assume about the contents of this file? >>> Which feature types are expected and/or supported? Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> _______________________________________________ maker-devel mailing >>> list maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy.semeiks at utsw.edu Thu Oct 4 13:39:25 2012 From: jeremy.semeiks at utsw.edu (Jeremy Semeiks) Date: Thu, 4 Oct 2012 14:39:25 -0500 Subject: [maker-devel] maker hung on a contig Message-ID: Hi, I ran maker 2.26 beta on a mammalian assembly containing ~200,000 scaffolds. I ran like this: $ /usr/bin/time mpiexec -n 30 maker -q < /dev/null > maker.oe 2>&1 & disown -h After 1--2 weeks, all but one contig had finished successfully, although maker needed a retry on ~80 others. The one contig that failed is small (1,146 nt) and not obviously weird. maker hung on this one contig, outputting the following type of stack-trace-looking thing to maker.oe, over and over: """ eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xdd9d480)', 'Error::Simple=HASH(0xa9a56f8)', undef, 'ARRAY(0x9ecc710)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xf1b6a40)', 'HASH(0xdd9d480)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0xab33f30)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0xab33f30)', 'SCALAR(0x17cd3a90)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xb3a58e0)', 'Error::Simple=HASH(0xab33f30)', undef, 'ARRAY(0x77ffae0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xbe47098)', 'HASH(0xb3a58e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0xc86fb88)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0xc86fb88)', 'SCALAR(0xb504490)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xc2d7d00)', 'Error::Simple=HASH(0xc86fb88)', undef, 'ARRAY(0x767a290)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0x5ab13f0)', 'HASH(0xc2d7d00)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0x18943010)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0x18943010)', 'SCALAR(0xa6e7cd0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 """ Every 100,000 lines or so, that sort of pattern is interrupted by this sort of pattern: """ Process::MpiTiers::__ANON__('Error::Simple=HASH(0xd9816c8)', 'SCALAR(0x2558c48)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0x1bd77ab8)', 'Error::Simple=HASH(0xd9816c8)', undef, 'ARRAY(0x2561948)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xf48b0d8)', 'HASH(0x1bd77ab8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 285 Process::MpiTiers::run_all('Process::MpiTiers=HASH(0xc10a7e0)', 0) calledERROR: Failed while builing masking tiers ERROR: Can not get next level ERROR: Can't open seq file: /export/home9/jrs/bmy/maker02/bmy.min1e3.maker.output/bmy.min1e3_datastore/E0/1F/C16738816//theVoid.C16738816/query.masked.gff.seq No such file or directory at /home/jrs/maker-2.26-beta/bin/../lib/Dumper/GFF/GFFV3.pm line 173. Dumper::GFF::GFFV3::finalize('Dumper::GFF::GFFV3=HASH(0x4c328c8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 700 Process::MpiChunk::__ANON__() called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 415 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0xfc1d208)', 'HASH(0xfc1fbd8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 3768 Process::MpiChunk::_go('Process::MpiChunk=HASH(0x47162a0)', 'flow', 'HASH(0x93ba680)', 2, 0) called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 369 """ The makers kept outputting these things until I killed them all, and also mpiexec, with kill -s SIGKILL. These strack traces go on for at least the last million lines of maker.oe, which is ~100 GB. Because the log is so big, without knowing what to grep for it's hard for me to suggest what originally went wrong. The contig's datastore directory does not contain anything obvious. Its run.log indicates only that it was the second try for this contig. None of the files in this directory (including theVoid) had been updated since 3 days ago. So this contig itself may even be a red herring. maker's protein output for all the other contigs seems sane, and this one contig is probably too small to contain any proteins, so it doesn't matter to me if maker finishes it successfully. I just want to figure out how I can keep maker from hanging. Suggestions? Thanks, Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 14:42:52 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 20:42:52 +0000 Subject: [maker-devel] maker mpi failed Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D692@EXMBX05.austin.utexas.edu> Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 14:53:08 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 16:53:08 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D692@EXMBX05.austin.utexas.edu> Message-ID: You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 15:20:48 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 21:20:48 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D7C8@EXMBX05.austin.utexas.edu> Hi Carson, I didn't know maker does not work with MVAPICH2. The cluster I was installing maker on only has mvapich2. So I guess I will have to install mpich2 under my directory. However, I installed perl threading based on mvapich2. Does that mean I need to reinstall perl with mpich2 before reinstalling maker? Oscar -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Thu Oct 4 17:06:13 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Thu, 4 Oct 2012 16:06:13 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: References: Message-ID: <1764.131.215.15.234.1349391973.squirrel@webmail.caltech.edu> Hi, Thanks, reinstalling BioPerl fixed the issue. Now we are training our genome to generate training dataset for SNAP and Augustus, as per the instructions provided at http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors However the training dataset would have multiple gff3 prediction files, one for each contig. Is it recommended to cat(command) all the gff3 files to generate a single file and finally generate hmm file with the steps mentioned in the tutorial. Would the same hmm file work as training dataset for AUGUSTUS? Many thanks, Parul Kudtarkar > You probably need to reinstall Bio-perl. > > Other things that can cause the same error are setting TMP in the > maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or > sometimes setting it to an NFS mount. > A full drive can cause this as well or a broken Berkley DB. Use df -h to > see if any of the drives used are full (either current working directory > or TMP location). You can also swap Berkley DB for a different backend by > setting AnyDBM_ISA during setup. > > Example: > cd ../maker/src/ > perl Build.PL --AnyDBM_ISA SDBM_File > ./Build install > > --Carson > > > > On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: > >>I am running maker on example data that comes along with installation and >>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >> >>Please advice on the aforementioned error. >> >>--------------------- >>Maker is now finished!!! >> >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >>eins%2Efasta.repeatrunner >>deleted:0 hits >> in cluster:shadow cluster... >> i_size:0 j_size:0 >> sorting hits in shadow cluster... >>... finished. >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >>%2Efasta.blastn >>deleted:-1 hits >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >>tein%2Efasta.blastx >>WARNING: Multiple MAKER processes have been started in the >>same directory. >> >>deleted:0 hits >>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>stop here:dpp-mRNA-4 >>ERROR: Fasta index error >> >>FATAL ERROR >>Maker is now finished!!! >> >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >>eins%2Efasta.repeatrunner >>deleted:0 hits >> in cluster:shadow cluster... >> i_size:0 j_size:0 >> sorting hits in shadow cluster... >>... finished. >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >>%2Efasta.blastn >>deleted:-1 hits >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >>tein%2Efasta.blastx >>WARNING: Multiple MAKER processes have been started in the >>same directory. >> >>deleted:0 hits >>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>stop here:dpp-mRNA-4 >>ERROR: Fasta index error >> >>FATAL ERROR >>ERROR: Failed while polishig ESTs!! >> >>ERROR: Chunk failed at level 14 >>!! >>FAILED CONTIG:contig-dpp-500-500 >> >> >> >>ERROR: Chunk failed at level 14 >>!! >>FAILED CONTIG:contig-dpp-500-500 >>------------------------------------ >> >> >>Many thanks, >>Parul Kudtarkar >> >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org >> >> >>_______________________________________________ >>maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From david.powell at monash.edu Thu Oct 4 17:44:32 2012 From: david.powell at monash.edu (David Powell) Date: Fri, 5 Oct 2012 09:44:32 +1000 Subject: [maker-devel] problem running maker In-Reply-To: References: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Message-ID: I've also had the problem with a segfault when running MAKER 2.26. I'm not sure it is the one experienced here, but in the end I was able to work around it. I tracked the problem to lib/FastA.pm in the _safe_new function. Specifically, in the local handler installed for warnings. By removing the "die" call, I was able to stop the perl process from segfaulting. I suspect this is not a great fix since it will ignore any warnings from Bio::DB::Fasta, but it worked for me (I did check the warnings generated were not important). Cheers, -- David Powell On 5 October 2012 03:17, Carson Holt wrote: > Seg faults really only happen from C and not Perl, so there may be a perl > module that is broken on your machine that use C internally. > > Here are some modules that MAKER can use that I know use C and that you > might want to reinstall from CPAN. > > DB_File > Forks > Proc::ProcessTable > Inline::C > > Also during the maker install say no to the "use MPI" question. It's best > to make the installation as simple as possible before getting into MPI > configuration. > > Also after installing those and rerunning maker's 'perl Build.PL' and > './Build install' steps do your first run with 'maker --debug' (redirect > STDERR to a file). Then if it fails, you can send me the error log. > > Thanks, > Carson > > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 11:36 AM > > To: Carson Holt , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Ok. So I installed 2.26 instead. When I ran maker again I got segmentation > fault with no additional message. I am using dpp_contig.fasta, > dpp_est.fasta and dpp_protein.fasta. > > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM > To: Oscar Jiao , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Try 2.26 and let me know. > > Thanks, > Carson > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM > To: Carson Holt , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > 2.10 > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM > To: Oscar Jiao , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Are you trying to install 2.26 or 2.10? > > --Carson > > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM > To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker > > Hi, > > I just installed maker and tried to run it with these dpp example fasta > files that came with the package. However when I do "maker maker_opts.ctl > maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like > there are three errors: (1) defined array deprecated; (2) unknown state in > Signal.pm and (3) GDBM_File (etc.) missing > > The first line seems to be just warning. I got it even when I do just > "maker" or "maker ctl". The control files did get generated. And what is > the relationship between the three perl modules, GDBM/NDBM/SDBM_File, > AnyDBM and DB_File? > > Oscar > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 20:24:16 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 22:24:16 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D7C8@EXMBX05.austin.utexas.edu> Message-ID: I don't think you will need to reinstall perl (you may but I don't think so). MVAPICH2 doesn't work because they have overridden malloc with their own version which causes weirdness with shared libraries in Perl. I've submitted some bug reports to the MVAPICH2 developers and made modifications to the development version of MAKER, and had some progress, but I am still not getting reliable behavior in MVAPICH2 when compiling for the OFA-IB-Nemesis or OFA-IB-CH3 interfaces. So for now using MPICH2 is your best choice. You lose some of the infiniband optimizations but IO rather than messages passing is MAKER's true bottleneck (so you won't get a performance hit by using MPICH2). --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 5:20 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Hi Carson, I didn't know maker does not work with MVAPICH2. The cluster I was installing maker on only has mvapich2. So I guess I will have to install mpich2 under my directory. However, I installed perl threading based on mvapich2. Does that mean I need to reinstall perl with mpich2 before reinstalling maker? Oscar -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 20:51:19 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 22:51:19 -0400 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <1764.131.215.15.234.1349391973.squirrel@webmail.caltech.edu> Message-ID: Just concatenating won't work. Use the gff3_merge tool to safely merge separate GFF3 files. In the tutorial dataset their should only be one contig to train with, but with real data you will have multiple contigs and will want to merge the GFF3 files. Augustus has a separate training procedure that is a little more complicated than SNAP's. You will need to read their documentation on training which is a little cryptic. I know Jason Stajich developed a tool for training augustus from snap results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be willing to share it with you ;-) Thanks, Carson On 12-10-04 7:06 PM, "Parul Kudtarkar" wrote: >Hi, > >Thanks, reinstalling BioPerl fixed the issue. Now we are training our >genome to generate training dataset for SNAP and Augustus, as per the >instructions provided at >http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors > >However the training dataset would have multiple gff3 prediction files, >one for each contig. Is it recommended to cat(command) all the gff3 files >to generate a single file and finally generate hmm file with the steps >mentioned in the tutorial. Would the same hmm file work as training >dataset for AUGUSTUS? > >Many thanks, >Parul Kudtarkar > >> You probably need to reinstall Bio-perl. >> >> Other things that can cause the same error are setting TMP in the >> maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or >> sometimes setting it to an NFS mount. >> A full drive can cause this as well or a broken Berkley DB. Use df -h >>to >> see if any of the drives used are full (either current working directory >> or TMP location). You can also swap Berkley DB for a different backend >>by >> setting AnyDBM_ISA during setup. >> >> Example: >> cd ../maker/src/ >> perl Build.PL --AnyDBM_ISA SDBM_File >> ./Build install >> >> --Carson >> >> >> >> On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >> >>>I am running maker on example data that comes along with installation >>>and >>>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >>> >>>Please advice on the aforementioned error. >>> >>>--------------------- >>>Maker is now finished!!! >>> >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>ot >>>eins%2Efasta.repeatrunner >>>deleted:0 hits >>> in cluster:shadow cluster... >>> i_size:0 j_size:0 >>> sorting hits in shadow cluster... >>>... finished. >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>st >>>%2Efasta.blastn >>>deleted:-1 hits >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>ro >>>tein%2Efasta.blastx >>>WARNING: Multiple MAKER processes have been started in the >>>same directory. >>> >>>deleted:0 hits >>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>stop here:dpp-mRNA-4 >>>ERROR: Fasta index error >>> >>>FATAL ERROR >>>Maker is now finished!!! >>> >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>ot >>>eins%2Efasta.repeatrunner >>>deleted:0 hits >>> in cluster:shadow cluster... >>> i_size:0 j_size:0 >>> sorting hits in shadow cluster... >>>... finished. >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>st >>>%2Efasta.blastn >>>deleted:-1 hits >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>ro >>>tein%2Efasta.blastx >>>WARNING: Multiple MAKER processes have been started in the >>>same directory. >>> >>>deleted:0 hits >>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>stop here:dpp-mRNA-4 >>>ERROR: Fasta index error >>> >>>FATAL ERROR >>>ERROR: Failed while polishig ESTs!! >>> >>>ERROR: Chunk failed at level 14 >>>!! >>>FAILED CONTIG:contig-dpp-500-500 >>> >>> >>> >>>ERROR: Chunk failed at level 14 >>>!! >>>FAILED CONTIG:contig-dpp-500-500 >>>------------------------------------ >>> >>> >>>Many thanks, >>>Parul Kudtarkar >>> >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org >>> >>> >>>_______________________________________________ >>>maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > From jiao at tacc.utexas.edu Thu Oct 4 23:06:48 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 05:06:48 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868DBE8@EXMBX05.austin.utexas.edu> I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Thu Oct 4 23:34:37 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Thu, 4 Oct 2012 22:34:37 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: References: Message-ID: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> Carson, thanks for very quick response. Jason, could you please elaborate/share document if any on using zff2augustus_gbk.pl Thanks, Parul Kudtarkar > Just concatenating won't work. Use the gff3_merge tool to safely merge > separate GFF3 files. > > In the tutorial dataset their should only be one contig to train with, but > with real data you will have multiple contigs and will want to merge the > GFF3 files. > > Augustus has a separate training procedure that is a little more > complicated than SNAP's. You will need to read their documentation on > training which is a little cryptic. > > I know Jason Stajich developed a tool for training augustus from snap > results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be > willing to share it with you ;-) > > Thanks, > Carson > > > > On 12-10-04 7:06 PM, "Parul Kudtarkar" wrote: > >>Hi, >> >>Thanks, reinstalling BioPerl fixed the issue. Now we are training our >>genome to generate training dataset for SNAP and Augustus, as per the >>instructions provided at >>http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors >> >>However the training dataset would have multiple gff3 prediction files, >>one for each contig. Is it recommended to cat(command) all the gff3 files >>to generate a single file and finally generate hmm file with the steps >>mentioned in the tutorial. Would the same hmm file work as training >>dataset for AUGUSTUS? >> >>Many thanks, >>Parul Kudtarkar >> >>> You probably need to reinstall Bio-perl. >>> >>> Other things that can cause the same error are setting TMP in the >>> maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) >>> or >>> sometimes setting it to an NFS mount. >>> A full drive can cause this as well or a broken Berkley DB. Use df -h >>>to >>> see if any of the drives used are full (either current working >>> directory >>> or TMP location). You can also swap Berkley DB for a different backend >>>by >>> setting AnyDBM_ISA during setup. >>> >>> Example: >>> cd ../maker/src/ >>> perl Build.PL --AnyDBM_ISA SDBM_File >>> ./Build install >>> >>> --Carson >>> >>> >>> >>> On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >>> >>>>I am running maker on example data that comes along with installation >>>>and >>>>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >>>> >>>>Please advice on the aforementioned error. >>>> >>>>--------------------- >>>>Maker is now finished!!! >>>> >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>>ot >>>>eins%2Efasta.repeatrunner >>>>deleted:0 hits >>>> in cluster:shadow cluster... >>>> i_size:0 j_size:0 >>>> sorting hits in shadow cluster... >>>>... finished. >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>>st >>>>%2Efasta.blastn >>>>deleted:-1 hits >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>>ro >>>>tein%2Efasta.blastx >>>>WARNING: Multiple MAKER processes have been started in the >>>>same directory. >>>> >>>>deleted:0 hits >>>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>>stop here:dpp-mRNA-4 >>>>ERROR: Fasta index error >>>> >>>>FATAL ERROR >>>>Maker is now finished!!! >>>> >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>>ot >>>>eins%2Efasta.repeatrunner >>>>deleted:0 hits >>>> in cluster:shadow cluster... >>>> i_size:0 j_size:0 >>>> sorting hits in shadow cluster... >>>>... finished. >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>>st >>>>%2Efasta.blastn >>>>deleted:-1 hits >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>>ro >>>>tein%2Efasta.blastx >>>>WARNING: Multiple MAKER processes have been started in the >>>>same directory. >>>> >>>>deleted:0 hits >>>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>>stop here:dpp-mRNA-4 >>>>ERROR: Fasta index error >>>> >>>>FATAL ERROR >>>>ERROR: Failed while polishig ESTs!! >>>> >>>>ERROR: Chunk failed at level 14 >>>>!! >>>>FAILED CONTIG:contig-dpp-500-500 >>>> >>>> >>>> >>>>ERROR: Chunk failed at level 14 >>>>!! >>>>FAILED CONTIG:contig-dpp-500-500 >>>>------------------------------------ >>>> >>>> >>>>Many thanks, >>>>Parul Kudtarkar >>>> >>>>-- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org >>>> >>>> >>>>_______________________________________________ >>>>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> >> >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org >> > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jason.stajich at ucr.edu Fri Oct 5 01:02:50 2012 From: jason.stajich at ucr.edu (Jason E Stajich) Date: Fri, 5 Oct 2012 07:02:50 +0000 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> References: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> Message-ID: <41A4465BD48EEE47A449FA03108C852A073651E2@EXCH-MBOX-3.exch.ucr.edu> I wrote up how to use it on this list msg - http://brie4.cshl.edu/pipermail/gmod-help/2012-June/001724.html Script is in this github repo - https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl Patches and documentation updates are always welcomed, I'll get around to that eventually. Jason On Oct 4, 2012, at 10:34 PM, Parul Kudtarkar > wrote: Carson, thanks for very quick response. Jason, could you please elaborate/share document if any on using zff2augustus_gbk.pl Thanks, Parul Kudtarkar Just concatenating won't work. Use the gff3_merge tool to safely merge separate GFF3 files. In the tutorial dataset their should only be one contig to train with, but with real data you will have multiple contigs and will want to merge the GFF3 files. Augustus has a separate training procedure that is a little more complicated than SNAP's. You will need to read their documentation on training which is a little cryptic. I know Jason Stajich developed a tool for training augustus from snap results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be willing to share it with you ;-) Thanks, Carson On 12-10-04 7:06 PM, "Parul Kudtarkar" > wrote: Hi, Thanks, reinstalling BioPerl fixed the issue. Now we are training our genome to generate training dataset for SNAP and Augustus, as per the instructions provided at http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors However the training dataset would have multiple gff3 prediction files, one for each contig. Is it recommended to cat(command) all the gff3 files to generate a single file and finally generate hmm file with the steps mentioned in the tutorial. Would the same hmm file work as training dataset for AUGUSTUS? Many thanks, Parul Kudtarkar You probably need to reinstall Bio-perl. Other things that can cause the same error are setting TMP in the maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or sometimes setting it to an NFS mount. A full drive can cause this as well or a broken Berkley DB. Use df -h to see if any of the drives used are full (either current working directory or TMP location). You can also swap Berkley DB for a different backend by setting AnyDBM_ISA during setup. Example: cd ../maker/src/ perl Build.PL --AnyDBM_ISA SDBM_File ./Build install --Carson On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: I am running maker on example data that comes along with installation and is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note Please advice on the aforementioned error. --------------------- Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr ot eins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e st %2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p ro tein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr ot eins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e st %2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p ro tein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ------------------------------------ Many thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -- Jason E Stajich, PhD Assistant Professor Plant Pathology & Microbiology University of California, Riverside 951.827.2363 http://lab.stajich.org http://fungalgenomes.org http://fungidb.org http://1000.fungalgenomes.org/ twitter @stajichlab @hyphaltip @fungalgenomes @fungidb http://plantpathology.ucr.edu http://genomics.ucr.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From Carson.Holt at oicr.on.ca Thu Oct 4 13:19:57 2012 From: Carson.Holt at oicr.on.ca (Carson Holt) Date: Thu, 4 Oct 2012 19:19:57 +0000 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Yes. You really have to because the single exon transcripts produced from mRNA-seq assembly are strandless (you don't know where they belong). They also tend to be heavily weighted to pseudogenes and transposons. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 3:16 PM To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 Does the cufflinks2gff3 script filter out single-exon transcripts? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt > wrote: Use the cufflinks2gff3 script that comes with MAKER. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 3:07 PM To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only transcript and exon features. So I'm assuming I can simply convert the transcript features to match and the exon features to match_part and make sure the parent/child relationships are maintained with the Parent and ID attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt > wrote: Match/match_part features are expected. MAKER expects these to be polished (I.e. correctly aligned around splice sites). For example exonerate, BLAT, and cufflinks results will be correct around splice sites and can be used; BLAST results on the other hand will not be correct and should not be used. The Gap attribute is used by maker if available, but is not required (Gap describes how to reconstruct an alignment for gaps and mismatches). Otherwise MAKER assumes all positions are matches to the reference. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 Greetings. I would like to use Maker's est_gff option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 10:05:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 12:05:56 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868DBE8@EXMBX05.austin.utexas.edu> Message-ID: Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 10:34:12 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 12:34:12 -0400 Subject: [maker-devel] editing MAKER mailing list options Message-ID: Just a small note to subscribers to the MAKER e-mail list. Being as there are spurts of high volume list activity that may be a nuisance to some, if you want to edit your mailing list options visit this link --> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org You can then change to digest mode if you don't want to receive every single message posted to the list, or you can set your options to be subscribed to but not receive messages from the list (so you can post and get help but won't get everyone else's posts). Enter your e-mail address in the box at the very very bottom of the page to edit option. Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 12:19:15 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 14:19:15 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E519@EXMBX05.austin.utexas.edu> Message-ID: Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 12:16:41 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 18:16:41 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E519@EXMBX05.austin.utexas.edu> See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: maker_opts.ctl Type: application/octet-stream Size: 4468 bytes Desc: maker_opts.ctl URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mpich-maker-debug.log Type: application/octet-stream Size: 315735 bytes Desc: mpich-maker-debug.log URL: From jiao at tacc.utexas.edu Fri Oct 5 13:55:39 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 19:55:39 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E7A3@EXMBX05.austin.utexas.edu> I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 14:03:09 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 16:03:09 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E7A3@EXMBX05.austin.utexas.edu> Message-ID: What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Ap plication/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.p m line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/M PI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 14:06:02 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 20:06:02 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E866@EXMBX05.austin.utexas.edu> Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 14:07:18 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 16:07:18 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E866@EXMBX05.austin.utexas.edu> Message-ID: Is /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2 a new install of mpich2 or is it the install you did with 2.26 just copied over? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 4:06 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Ap plication/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.p m line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/M PI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 14:08:35 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 20:08:35 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E8D6@EXMBX05.austin.utexas.edu> It is a new install. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:07 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Is /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2 a new install of mpich2 or is it the install you did with 2.26 just copied over? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 4:06 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisi.hahni at gmail.com Tue Oct 9 05:07:14 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 09 Oct 2012 13:07:14 +0200 Subject: [maker-devel] install maker on cluster Message-ID: <50740562.9010904@gmail.com> Hello maker-team, I am trying to install maker 2.25 on a cluster, without root privileges (I managed before, but now after migration to a new cluster I cant seem to get it to work). All necessary prerequisite programs are installed and in the path (I followed the steps in the INSTALL file that comes with maker2.25). When I do: perl Build.PL, I get: Checking prerequisites... requires: ! DBD::SQLite is not installed ! IO::All is not installed ! Inline::C is not installed ! Perl::Unsafe::Signals is not installed ! Proc::ProcessTable is not installed ! Want is not installed ! forks is not installed ! forks::shared is not installed build_requires: ! LWP::Simple is not installed recommends: * DBD::Pg is not installed ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the versions of the modules indicated above before proceeding with this installation Run 'Build installdeps' to install missing prerequisites. MAKER supports distributed parallelization via MPI. Would you like to configure MAKER for MPI (This requires that you have an MPI client installed)? [N ]n WARNING: Apache cannot be located. The optional web based interface to MAKER will not be available to you. Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'MAKER' version '2.25' The file 'Build' has been created for you to finish installing MAKER. ============================================================================== STATUS MAKER 2.25 ============================================================================== PERL Dependencies: MISSING ! forks ! IO::All ! forks::shared ! Want ! DBD::SQLite ! Proc::ProcessTable ! Inline::C ! Perl::Unsafe::Signals External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: DISABLED MWAS Web Interface: DISABLED MAKER PACKAGE: MISSING PREREQUISITES Important Commands: ./Build installdeps #installs missing PERL dependencies ./Build installexes #installs all missing external programs ./Build install #installs MAKER ./Build status #Shows this status menu Other Commands: ./Build repeatmasker #installs RepeatMasker (asks for RepBase) ./Build blast #installs BLAST (NCBI BLAST+) ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) ./Build snap #installs SNAP ./Build augustus #installs Augustus ./Build apollo #installs Apollo ./Build gbrowse #installs GBrowse (must be root) ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) ./Build mpich2 #installs MPICH2 (but manual install recommended) So it seems some Perl Dependencies are missing. As indicated in INSTALL I followed the quick and dirty installation for Bioperl, so I am not sure what I did wrong/missed out. When I run ./Build installdeps, I get: You do not have write access to install missing Modules. I can try and install these locally (i.e. only for MAKER) in the .../maker/perl/lib directory, or you can run './Build installdeps' as root or using sudo and try again. Do want MAKER to try and build a local installation? [N ]y mkdir /usit/titan: Permission denied at /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm line 59 Sorry to bother you with this minor things!! Any help would me much appreciated! much obliged, Christoph From carsonhh at gmail.com Tue Oct 9 10:27:36 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 09 Oct 2012 12:27:36 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <50740562.9010904@gmail.com> Message-ID: You may need to reconfigure your cpan preferences. You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, or by starting cpan from your command line (command is cpan). Then run 'o conf init' inside the cpan ui. --Carson On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >Hello maker-team, > >I am trying to install maker 2.25 on a cluster, without root privileges >(I managed before, but now after migration to a new cluster I cant seem >to get it to work). All necessary prerequisite programs are installed >and in the path (I followed the steps in the INSTALL file that comes >with maker2.25). When I do: perl Build.PL, I get: >Checking prerequisites... > requires: > ! DBD::SQLite is not installed > ! IO::All is not installed > ! Inline::C is not installed > ! Perl::Unsafe::Signals is not installed > ! Proc::ProcessTable is not installed > ! Want is not installed > ! forks is not installed > ! forks::shared is not installed > build_requires: > ! LWP::Simple is not installed > recommends: > * DBD::Pg is not installed > >ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >versions >of the modules indicated above before proceeding with this installation > >Run 'Build installdeps' to install missing prerequisites. > >MAKER supports distributed parallelization via MPI. >Would you like to configure MAKER for MPI (This >requires that you have an MPI client installed)? [N ]n > >WARNING: Apache cannot be located. The optional web based >interface to MAKER will not be available to you. > >Created MYMETA.yml and MYMETA.json >Creating new 'Build' script for 'MAKER' version '2.25' > > >The file 'Build' has been created for you to finish installing MAKER. > > >========================================================================== >==== >STATUS MAKER 2.25 >========================================================================== >==== >PERL Dependencies: MISSING > ! forks > ! IO::All > ! forks::shared > ! Want > ! DBD::SQLite > ! Proc::ProcessTable > ! Inline::C > ! Perl::Unsafe::Signals > >External Programs: VERIFIED >External C Libraries: VERIFIED >MPI SUPPORT: DISABLED >MWAS Web Interface: DISABLED >MAKER PACKAGE: MISSING PREREQUISITES > > >Important Commands: > ./Build installdeps #installs missing PERL dependencies > ./Build installexes #installs all missing external programs > ./Build install #installs MAKER > ./Build status #Shows this status menu > >Other Commands: > ./Build repeatmasker #installs RepeatMasker (asks for RepBase) > ./Build blast #installs BLAST (NCBI BLAST+) > ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on >Mac OSX) > ./Build snap #installs SNAP > ./Build augustus #installs Augustus > ./Build apollo #installs Apollo > ./Build gbrowse #installs GBrowse (must be root) > ./Build jbrowse #installs JBrowse (MAKER copy, not web >accecible) > ./Build mpich2 #installs MPICH2 (but manual install >recommended) > >So it seems some Perl Dependencies are missing. As indicated in INSTALL >I followed the quick and dirty installation for Bioperl, so I am not >sure what I did wrong/missed out. > >When I run ./Build installdeps, I get: >You do not have write access to install missing Modules. >I can try and install these locally (i.e. only for MAKER) >in the .../maker/perl/lib directory, or you can run >'./Build installdeps' as root or using sudo and try again. >Do want MAKER to try and build a local installation? [N ]y >mkdir /usit/titan: Permission denied at >/cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm line >59 > >Sorry to bother you with this minor things!! Any help would me much >appreciated! > >much obliged, >Christoph > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From kippjohnson at uchicago.edu Thu Oct 11 13:43:32 2012 From: kippjohnson at uchicago.edu (Kipp Johnson) Date: Thu, 11 Oct 2012 14:43:32 -0500 Subject: [maker-devel] Interpreting Maker Results Message-ID: Hi Carson, I'm trying to get my genetic annotation out of maker. My maker run on a non-model eukaryote finished, and I used your gff3_merge script to merge the resulting files. This file is enormous, because I used snap, augustus, genemark, repeatmasker, exonerate, and blast, and has a lot of entries from all of these different programs. I want to extract only the genetic regions predicted by maker, so I used the gff3_merge script with the "-g" option. However, when I do this, I get a maker file that only has about 12,000 genes, while I was expecting around 20,000 genes for our genome. However, when I use the fasta merge tool, however, I get output files (for example, "merged.fasta.all.maker.non_overlapping_ab_initio.proteins.fasta") with about 21,000 proteins, which is closer to the gene number that I was expecting. Does the "-g" option ignore evidence from blast/exonerate or similar? How should I extract the complete set of genetic regions to blast against, so that I can go about further working on the annotation? Also, what is maker using to find these 9,000 extra proteins? Are these these all alternately sliced or something along those lines? I can't find any documentation online for how to actually get the final annotations out of maker correctly. Thanks for your time! Best, Kipp Johnson kippjohnson at uchicago.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 11 16:21:49 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 11 Oct 2012 18:21:49 -0400 Subject: [maker-devel] Interpreting Maker Results In-Reply-To: Message-ID: The non_overlapping_ab_initio.proteins.fasta file contain models that were rejected for lack of homology evidence that do not overlap the maker models. So if you accept everything it is not 21,000 proteins rather it's 33,000 (12,000 + 21,000). This is because the contents of that file do not occupy the same regions as the gene models (I.e. non-overlapping). Ab initio predictors have a tendency to overpredict which is why there are so many models. If you think you are missing genes you can try and add additional evidence to the maker run (I.e all proteins from a few related species). Also try running Cegma (http://korflab.ucdavis.edu/datasets/cegma/) to estimate the completeness of you assembly. Sometime a lower number than expected can be attributed to an incomplete assembly. Also you can run something like InterProScan to identify models in the non_overlapping_ab_initio.proteins.fasta file that contain Uniprot protein domains (likely to be real genes), then add them to your results as a second step using maker's model_gff option. I've attached a couple of scripts that can help with that. gff3_preds2models will turn a set of match/match_part features to gene/mRNA/exon/CDS features. It doesn't check translation of CDS though, so only use gene predictions which should be all CDS. gff3_select is used to select some subset of features from a GFF3 file. Useful for slicing sections of data from a GFF3 file. Thanks, Carson From: Kipp Johnson Date: Thursday, 11 October, 2012 3:43 PM To: , Carson Holt Subject: Interpreting Maker Results Hi Carson, I'm trying to get my genetic annotation out of maker. My maker run on a non-model eukaryote finished, and I used your gff3_merge script to merge the resulting files. This file is enormous, because I used snap, augustus, genemark, repeatmasker, exonerate, and blast, and has a lot of entries from all of these different programs. I want to extract only the genetic regions predicted by maker, so I used the gff3_merge script with the "-g" option. However, when I do this, I get a maker file that only has about 12,000 genes, while I was expecting around 20,000 genes for our genome. However, when I use the fasta merge tool, however, I get output files (for example, "merged.fasta.all.maker.non_overlapping_ab_initio.proteins.fasta") with about 21,000 proteins, which is closer to the gene number that I was expecting. Does the "-g" option ignore evidence from blast/exonerate or similar? How should I extract the complete set of genetic regions to blast against, so that I can go about further working on the annotation? Also, what is maker using to find these 9,000 extra proteins? Are these these all alternately sliced or something along those lines? I can't find any documentation online for how to actually get the final annotations out of maker correctly. Thanks for your time! Best, Kipp Johnson kippjohnson at uchicago.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gff3_preds2models Type: application/octet-stream Size: 4777 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gff3_select Type: application/octet-stream Size: 3236 bytes Desc: not available URL: From parulk at caltech.edu Fri Oct 12 15:46:21 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Fri, 12 Oct 2012 14:46:21 -0700 (PDT) Subject: [maker-devel] Conensus gene model Message-ID: <4092.131.215.15.234.1350078381.squirrel@webmail.caltech.edu> Hi, We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 file as input). The output that we get is combination of various gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? Thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig2106.gff Type: application/octet-stream Size: 10562 bytes Desc: not available URL: From parulk at caltech.edu Mon Oct 15 11:41:54 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 15 Oct 2012 10:41:54 -0700 (PDT) Subject: [maker-devel] Conensus gene model Message-ID: <2992.131.215.15.234.1350322914.squirrel@webmail.caltech.edu> Hello, Please advice on the aforementioned query? Thanks, Parul Kudtarkar ---------------------------- Original Message ---------------------------- Subject: [maker-devel] Conensus gene model From: "Parul Kudtarkar" Date: Fri, October 12, 2012 2:46 pm To: maker-devel at yandell-lab.org -------------------------------------------------------------------------- Hi, We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 file as input). The output that we get is combination of various gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? Thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org_______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig2106.gff Type: application/octet-stream Size: 10562 bytes Desc: not available URL: From carsonhh at gmail.com Mon Oct 15 11:58:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 15 Oct 2012 13:58:25 -0400 Subject: [maker-devel] Conensus gene model In-Reply-To: <2992.131.215.15.234.1350322914.squirrel@webmail.caltech.edu> Message-ID: The contig in question is really too small to get much out of it (only 5 kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb length or longer contigs to glean enough information for optimizing your parameters. The general suggestions for any maker run are to use proteins from a closely related organism or a couple of closely related organisms for the protein= option in maker. Also leave single_exon set to 0, except for certain eukaryotes that have a bias for single exon transcripts (i.e. some fungi and oomycetes). And leave keep_preds set to 0 because ab initio predictors tend to over-predict by a wide margin (lots of false positives). Additional training would really depend on what your other contigs look like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. Thanks, Carson On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >Hello, > >Please advice on the aforementioned query? > >Thanks, >Parul Kudtarkar >---------------------------- Original Message ---------------------------- >Subject: [maker-devel] Conensus gene model >From: "Parul Kudtarkar" >Date: Fri, October 12, 2012 2:46 pm >To: maker-devel at yandell-lab.org >-------------------------------------------------------------------------- > >Hi, > >We are using snap(training set[hmm file] generated using est,protein and >contig file), agustus,genemarkE(we ran it outside maker and have gff3 file >as input). The output that we get is combination of various >gene-predictors and evidences. I have attached sample result file. What >would you recommend to get consensus result set? Bootstrapping the >resulting gff3 file (rerunning maker)? > >Thanks, >Parul Kudtarkar >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Mon Oct 15 14:10:00 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 15 Oct 2012 16:10:00 -0400 Subject: [maker-devel] Conensus gene model In-Reply-To: <4445.131.215.15.234.1350330328.squirrel@webmail.caltech.edu> Message-ID: One thing you seem to be missing is protein evidence. Is this a sea urchin (I looked up some of the ESTs)? If so, I would recommend adding all proteins from the Strongylocentrotus purpuratus genome, then throw in another Deuterstome of your choice. Perhaps you should also add a couple of outgroup organisms like Nematostella vectensis (cnidaria) and a protostome of your choice. Be careful if adding adding to many protostome outgroups (i.e. C. elegans and Drosophila) because a big part of their evolution is gene loss (so distant cnidaria often match deuterstomes better than most protostomes do). You could take the maker results when protein data is included and use it to retrain SNAP again. Even a 22 kb contig is still really short. Is this genome primarily constituted by short contigs like this? I would recommend running CEGMA once on this genome to get an appropriate estimate of how recoverable the genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). Cegma will give you an estimate for genome completeness as well as estimates of what percentage of genes will be found in their entirety and what percent will be partial genes. This is important to do if your genome is fragmented as it will give you a reasonable expectation of what you can expected to recover (as short contigs don't annotate very well - you tend to loose a lot). Thanks, Carson On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >Hi Carson, > >Thanks. I have attached another contig which is 22 kb, with as many as 3 >exons EST alignments. Could you please recommend additional training. We >are currently running maker on the entire contig set and eventually merge >all the gff3 contig predictions. The using suggested parameter/methods we >would like to get a consensus gene-set with minimal false >positives/negatives. > >Thanks, >Parul > >> The contig in question is really too small to get much out of it (only 5 >kb). There was only one single exon EST alignments and a couple of >predictions with no evidence support. Anything smaller than 10 kb is >mostly useless for annotation purposes. You would really need a few >100kb >> length or longer contigs to glean enough information for optimizing your >parameters. >> >> The general suggestions for any maker run are to use proteins from a >closely related organism or a couple of closely related organisms for >the >> protein= option in maker. Also leave single_exon set to 0, except for >certain eukaryotes that have a bias for single exon transcripts (i.e. >some >> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >predictors tend to over-predict by a wide margin (lots of false >> positives). >> >> Additional training would really depend on what your other contigs look >like. Do you have any large contigs? I could look at one of those and >give suggestions but the provided contig is just too short to glean >much. >> >> Thanks, >> Carson >> >> >> >> >> >> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >> >>>Hello, >>>Please advice on the aforementioned query? >>>Thanks, >>>Parul Kudtarkar >>>---------------------------- Original Message >>> ---------------------------- >>>Subject: [maker-devel] Conensus gene model >>>From: "Parul Kudtarkar" >>>Date: Fri, October 12, 2012 2:46 pm >>>To: maker-devel at yandell-lab.org >>>------------------------------------------------------------------------ >>>-- >Hi, >>>We are using snap(training set[hmm file] generated using est,protein and >contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>> file >>>as input). The output that we get is combination of various >>>gene-predictors and evidences. I have attached sample result file. What >would you recommend to get consensus result set? Bootstrapping the >resulting gff3 file (rerunning maker)? >>>Thanks, >>>Parul Kudtarkar >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org_______________________________________________ >maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org_______________________________________________ >maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > From chrisi.hahni at gmail.com Tue Oct 16 08:02:36 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 16:02:36 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D68FC.10201@gmail.com> Thanks for your help Carson! I tried maker2.26, but the error still remains. I also tried to change the TMP= option in the maker_opts file, but no change either.. STATUS: Parsing control files... ERROR: The HMM file[s] provided for gmhmm do not exist: When building the maker2.26 I saw that it is complaining about one more (recommended) dependency (I must have overlooked that before): Checking prerequisites... recommends: * DBD::Pg is not installed I am having problems to get that one.. Is that causing the error? much obliged, Christoph Am 15.10.2012 20:05, schrieb Carson Holt: > It looks like either an NFS issue or perhaps an issue with the location > provided for the TMP= in the maker_opts.ctl file. > > First I would suggest trying maker 2.26 to see if the issue still happens > there (there are a few updates to how NFS locks are handled in 2.26). > > Second if you are setting TMP make sure your disk is not full. Or if TMP > was set to a tmpfs location (in memory filesystem), it could be a full > memory issue in which case you could just set TMP to somewhere else. > > Let me know if you still see the issue in 2.26 or after changing the > location of TMP. > > Thanks, > Carson > > > On 12-10-12 1:09 PM, "Christoph Hahn" wrote: > >> Hi Carson, >> >> Thanks for your reply. I managed to install maker2.25 properly now, but >> when running it I am getting strange errors (two kinds): >> type one: >> STATUS: Parsing control files... >> ERROR: The HMM file[s] provided for gmhmm do not exist: >> >> type two: >> STATUS: Parsing control files... >> ERROR: Cannot get initialization lock. >> >> I have divided the draft assembly to several files, each containing a >> subset of contigs on which I am running a maker instance. I am getting >> the above named error type one for the majority of the instances, >> although the HMM file it is complaining about (es.mod) is actually in >> the path it is naming (not shown above). In a small number of instances >> I am getting the type two error mentioned above. >> >> Can you help me? >> >> much obliged, >> Christoph >> >> >> >> On 09.10.2012 18:27, Carson Holt wrote: >>> You may need to reconfigure your cpan preferences. >>> >>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>> or >>> by starting cpan from your command line (command is cpan). Then run 'o >>> conf init' inside the cpan ui. >>> >>> --Carson >>> >>> >>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>> >>>> Hello maker-team, >>>> >>>> I am trying to install maker 2.25 on a cluster, without root privileges >>>> (I managed before, but now after migration to a new cluster I cant seem >>>> to get it to work). All necessary prerequisite programs are installed >>>> and in the path (I followed the steps in the INSTALL file that comes >>>> with maker2.25). When I do: perl Build.PL, I get: >>>> Checking prerequisites... >>>> requires: >>>> ! DBD::SQLite is not installed >>>> ! IO::All is not installed >>>> ! Inline::C is not installed >>>> ! Perl::Unsafe::Signals is not installed >>>> ! Proc::ProcessTable is not installed >>>> ! Want is not installed >>>> ! forks is not installed >>>> ! forks::shared is not installed >>>> build_requires: >>>> ! LWP::Simple is not installed >>>> recommends: >>>> * DBD::Pg is not installed >>>> >>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>> versions >>>> of the modules indicated above before proceeding with this installation >>>> >>>> Run 'Build installdeps' to install missing prerequisites. >>>> >>>> MAKER supports distributed parallelization via MPI. >>>> Would you like to configure MAKER for MPI (This >>>> requires that you have an MPI client installed)? [N ]n >>>> >>>> WARNING: Apache cannot be located. The optional web based >>>> interface to MAKER will not be available to you. >>>> >>>> Created MYMETA.yml and MYMETA.json >>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>> >>>> >>>> The file 'Build' has been created for you to finish installing MAKER. >>>> >>>> >>>> >>>> ======================================================================== >>>> == >>>> ==== >>>> STATUS MAKER 2.25 >>>> >>>> ======================================================================== >>>> == >>>> ==== >>>> PERL Dependencies: MISSING >>>> ! forks >>>> ! IO::All >>>> ! forks::shared >>>> ! Want >>>> ! DBD::SQLite >>>> ! Proc::ProcessTable >>>> ! Inline::C >>>> ! Perl::Unsafe::Signals >>>> >>>> External Programs: VERIFIED >>>> External C Libraries: VERIFIED >>>> MPI SUPPORT: DISABLED >>>> MWAS Web Interface: DISABLED >>>> MAKER PACKAGE: MISSING PREREQUISITES >>>> >>>> >>>> Important Commands: >>>> ./Build installdeps #installs missing PERL dependencies >>>> ./Build installexes #installs all missing external >>>> programs >>>> ./Build install #installs MAKER >>>> ./Build status #Shows this status menu >>>> >>>> Other Commands: >>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>> RepBase) >>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>> ./Build exonerate #installs Exonerate (v2 on UNIX / v1 >>>> on >>>> Mac OSX) >>>> ./Build snap #installs SNAP >>>> ./Build augustus #installs Augustus >>>> ./Build apollo #installs Apollo >>>> ./Build gbrowse #installs GBrowse (must be root) >>>> ./Build jbrowse #installs JBrowse (MAKER copy, not web >>>> accecible) >>>> ./Build mpich2 #installs MPICH2 (but manual install >>>> recommended) >>>> >>>> So it seems some Perl Dependencies are missing. As indicated in INSTALL >>>> I followed the quick and dirty installation for Bioperl, so I am not >>>> sure what I did wrong/missed out. >>>> >>>> When I run ./Build installdeps, I get: >>>> You do not have write access to install missing Modules. >>>> I can try and install these locally (i.e. only for MAKER) >>>> in the .../maker/perl/lib directory, or you can run >>>> './Build installdeps' as root or using sudo and try again. >>>> Do want MAKER to try and build a local installation? [N ]y >>>> mkdir /usit/titan: Permission denied at >>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>> line >>>> 59 >>>> >>>> Sorry to bother you with this minor things!! Any help would me much >>>> appreciated! >>>> >>>> much obliged, >>>> Christoph >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> > From parulk at caltech.edu Mon Oct 15 13:45:28 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 15 Oct 2012 12:45:28 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: Message-ID: <4445.131.215.15.234.1350330328.squirrel@webmail.caltech.edu> Hi Carson, Thanks. I have attached another contig which is 22 kb, with as many as 3 exons EST alignments. Could you please recommend additional training. We are currently running maker on the entire contig set and eventually merge all the gff3 contig predictions. The using suggested parameter/methods we would like to get a consensus gene-set with minimal false positives/negatives. Thanks, Parul > The contig in question is really too small to get much out of it (only 5 kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb > length or longer contigs to glean enough information for optimizing your parameters. > > The general suggestions for any maker run are to use proteins from a closely related organism or a couple of closely related organisms for the > protein= option in maker. Also leave single_exon set to 0, except for certain eukaryotes that have a bias for single exon transcripts (i.e. some > fungi and oomycetes). And leave keep_preds set to 0 because ab initio predictors tend to over-predict by a wide margin (lots of false > positives). > > Additional training would really depend on what your other contigs look like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. > > Thanks, > Carson > > > > > > On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: > >>Hello, >>Please advice on the aforementioned query? >>Thanks, >>Parul Kudtarkar >>---------------------------- Original Message >> ---------------------------- >>Subject: [maker-devel] Conensus gene model >>From: "Parul Kudtarkar" >>Date: Fri, October 12, 2012 2:46 pm >>To: maker-devel at yandell-lab.org >>-------------------------------------------------------------------------- Hi, >>We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 >> file >>as input). The output that we get is combination of various >>gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? >>Thanks, >>Parul Kudtarkar >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org_______________________________________________ maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org_______________________________________________ maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig7.gff Type: application/octet-stream Size: 68447 bytes Desc: not available URL: From carsonhh at gmail.com Tue Oct 16 08:19:21 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 16 Oct 2012 10:19:21 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <507D68FC.10201@gmail.com> Message-ID: No. that's just a recommendation for the maker2chado script to work. Is es.mod the actual HMM or a soft link to the HMM? After the error message their should be a list of the actual files. Does it print that? Are there trailing commas in you maker_opts.ctl file after the hmm file name in the control files? Are their ':' characters in the path name to the file? --Carson On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >Thanks for your help Carson! > >I tried maker2.26, but the error still remains. I also tried to change >the TMP= option in the maker_opts file, but no change either.. > >STATUS: Parsing control files... >ERROR: The HMM file[s] provided for gmhmm do not exist: > >When building the maker2.26 I saw that it is complaining about one more >(recommended) dependency (I must have overlooked that before): > Checking prerequisites... > recommends: > * DBD::Pg is not installed > >I am having problems to get that one.. Is that causing the error? > >much obliged, >Christoph > >Am 15.10.2012 20:05, schrieb Carson Holt: >> It looks like either an NFS issue or perhaps an issue with the location >> provided for the TMP= in the maker_opts.ctl file. >> >> First I would suggest trying maker 2.26 to see if the issue still >>happens >> there (there are a few updates to how NFS locks are handled in 2.26). >> >> Second if you are setting TMP make sure your disk is not full. Or if >>TMP >> was set to a tmpfs location (in memory filesystem), it could be a full >> memory issue in which case you could just set TMP to somewhere else. >> >> Let me know if you still see the issue in 2.26 or after changing the >> location of TMP. >> >> Thanks, >> Carson >> >> >> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >> >>> Hi Carson, >>> >>> Thanks for your reply. I managed to install maker2.25 properly now, but >>> when running it I am getting strange errors (two kinds): >>> type one: >>> STATUS: Parsing control files... >>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>> >>> type two: >>> STATUS: Parsing control files... >>> ERROR: Cannot get initialization lock. >>> >>> I have divided the draft assembly to several files, each containing a >>> subset of contigs on which I am running a maker instance. I am getting >>> the above named error type one for the majority of the instances, >>> although the HMM file it is complaining about (es.mod) is actually in >>> the path it is naming (not shown above). In a small number of instances >>> I am getting the type two error mentioned above. >>> >>> Can you help me? >>> >>> much obliged, >>> Christoph >>> >>> >>> >>> On 09.10.2012 18:27, Carson Holt wrote: >>>> You may need to reconfigure your cpan preferences. >>>> >>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>>> or >>>> by starting cpan from your command line (command is cpan). Then run >>>>'o >>>> conf init' inside the cpan ui. >>>> >>>> --Carson >>>> >>>> >>>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>>> >>>>> Hello maker-team, >>>>> >>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>privileges >>>>> (I managed before, but now after migration to a new cluster I cant >>>>>seem >>>>> to get it to work). All necessary prerequisite programs are installed >>>>> and in the path (I followed the steps in the INSTALL file that comes >>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>> Checking prerequisites... >>>>> requires: >>>>> ! DBD::SQLite is not installed >>>>> ! IO::All is not installed >>>>> ! Inline::C is not installed >>>>> ! Perl::Unsafe::Signals is not installed >>>>> ! Proc::ProcessTable is not installed >>>>> ! Want is not installed >>>>> ! forks is not installed >>>>> ! forks::shared is not installed >>>>> build_requires: >>>>> ! LWP::Simple is not installed >>>>> recommends: >>>>> * DBD::Pg is not installed >>>>> >>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>>> versions >>>>> of the modules indicated above before proceeding with this >>>>>installation >>>>> >>>>> Run 'Build installdeps' to install missing prerequisites. >>>>> >>>>> MAKER supports distributed parallelization via MPI. >>>>> Would you like to configure MAKER for MPI (This >>>>> requires that you have an MPI client installed)? [N ]n >>>>> >>>>> WARNING: Apache cannot be located. The optional web based >>>>> interface to MAKER will not be available to you. >>>>> >>>>> Created MYMETA.yml and MYMETA.json >>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>> >>>>> >>>>> The file 'Build' has been created for you to finish installing MAKER. >>>>> >>>>> >>>>> >>>>> >>>>>====================================================================== >>>>>== >>>>> == >>>>> ==== >>>>> STATUS MAKER 2.25 >>>>> >>>>> >>>>>====================================================================== >>>>>== >>>>> == >>>>> ==== >>>>> PERL Dependencies: MISSING >>>>> ! forks >>>>> ! IO::All >>>>> ! forks::shared >>>>> ! Want >>>>> ! DBD::SQLite >>>>> ! Proc::ProcessTable >>>>> ! Inline::C >>>>> ! Perl::Unsafe::Signals >>>>> >>>>> External Programs: VERIFIED >>>>> External C Libraries: VERIFIED >>>>> MPI SUPPORT: DISABLED >>>>> MWAS Web Interface: DISABLED >>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>> >>>>> >>>>> Important Commands: >>>>> ./Build installdeps #installs missing PERL dependencies >>>>> ./Build installexes #installs all missing external >>>>> programs >>>>> ./Build install #installs MAKER >>>>> ./Build status #Shows this status menu >>>>> >>>>> Other Commands: >>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>> RepBase) >>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>> ./Build exonerate #installs Exonerate (v2 on UNIX / >>>>>v1 >>>>> on >>>>> Mac OSX) >>>>> ./Build snap #installs SNAP >>>>> ./Build augustus #installs Augustus >>>>> ./Build apollo #installs Apollo >>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>> ./Build jbrowse #installs JBrowse (MAKER copy, not >>>>>web >>>>> accecible) >>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>install >>>>> recommended) >>>>> >>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>INSTALL >>>>> I followed the quick and dirty installation for Bioperl, so I am not >>>>> sure what I did wrong/missed out. >>>>> >>>>> When I run ./Build installdeps, I get: >>>>> You do not have write access to install missing Modules. >>>>> I can try and install these locally (i.e. only for MAKER) >>>>> in the .../maker/perl/lib directory, or you can run >>>>> './Build installdeps' as root or using sudo and try again. >>>>> Do want MAKER to try and build a local installation? [N ]y >>>>> mkdir /usit/titan: Permission denied at >>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>> line >>>>> 59 >>>>> >>>>> Sorry to bother you with this minor things!! Any help would me much >>>>> appreciated! >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> >>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>g >>>> >> > > From chrisi.hahni at gmail.com Tue Oct 16 09:14:36 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 17:14:36 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D79DC.5090702@gmail.com> Hi Carson, It was a link. The path was not correct any more because of the migration to the new cluster. I could not change it so I just gave the path to the real file in the maker_opts file. That has resolved this error, but now I am getting the following: STATUS: Parsing control files... ERROR: Cannot get initialization lock. Probably not relevant any more, but: No trailing commas, no ':' characters in the path. Thanks for your help! Christoph Am 16.10.2012 16:19, schrieb Carson Holt: > No. that's just a recommendation for the maker2chado script to work. > > Is es.mod the actual HMM or a soft link to the HMM? > After the error message their should be a list of the actual files. Does > it print that? > Are there trailing commas in you maker_opts.ctl file after the hmm file > name in the control files? > Are their ':' characters in the path name to the file? > > --Carson > > > > On 12-10-16 10:02 AM, "Christoph Hahn" wrote: > >> Thanks for your help Carson! >> >> I tried maker2.26, but the error still remains. I also tried to change >> the TMP= option in the maker_opts file, but no change either.. >> >> STATUS: Parsing control files... >> ERROR: The HMM file[s] provided for gmhmm do not exist: >> >> When building the maker2.26 I saw that it is complaining about one more >> (recommended) dependency (I must have overlooked that before): >> Checking prerequisites... >> recommends: >> * DBD::Pg is not installed >> >> I am having problems to get that one.. Is that causing the error? >> >> much obliged, >> Christoph >> >> Am 15.10.2012 20:05, schrieb Carson Holt: >>> It looks like either an NFS issue or perhaps an issue with the location >>> provided for the TMP= in the maker_opts.ctl file. >>> >>> First I would suggest trying maker 2.26 to see if the issue still >>> happens >>> there (there are a few updates to how NFS locks are handled in 2.26). >>> >>> Second if you are setting TMP make sure your disk is not full. Or if >>> TMP >>> was set to a tmpfs location (in memory filesystem), it could be a full >>> memory issue in which case you could just set TMP to somewhere else. >>> >>> Let me know if you still see the issue in 2.26 or after changing the >>> location of TMP. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>> >>>> Hi Carson, >>>> >>>> Thanks for your reply. I managed to install maker2.25 properly now, but >>>> when running it I am getting strange errors (two kinds): >>>> type one: >>>> STATUS: Parsing control files... >>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>> >>>> type two: >>>> STATUS: Parsing control files... >>>> ERROR: Cannot get initialization lock. >>>> >>>> I have divided the draft assembly to several files, each containing a >>>> subset of contigs on which I am running a maker instance. I am getting >>>> the above named error type one for the majority of the instances, >>>> although the HMM file it is complaining about (es.mod) is actually in >>>> the path it is naming (not shown above). In a small number of instances >>>> I am getting the type two error mentioned above. >>>> >>>> Can you help me? >>>> >>>> much obliged, >>>> Christoph >>>> >>>> >>>> >>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>> You may need to reconfigure your cpan preferences. >>>>> >>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>>>> or >>>>> by starting cpan from your command line (command is cpan). Then run >>>>> 'o >>>>> conf init' inside the cpan ui. >>>>> >>>>> --Carson >>>>> >>>>> >>>>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>>>> >>>>>> Hello maker-team, >>>>>> >>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>> privileges >>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>> seem >>>>>> to get it to work). All necessary prerequisite programs are installed >>>>>> and in the path (I followed the steps in the INSTALL file that comes >>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>> Checking prerequisites... >>>>>> requires: >>>>>> ! DBD::SQLite is not installed >>>>>> ! IO::All is not installed >>>>>> ! Inline::C is not installed >>>>>> ! Perl::Unsafe::Signals is not installed >>>>>> ! Proc::ProcessTable is not installed >>>>>> ! Want is not installed >>>>>> ! forks is not installed >>>>>> ! forks::shared is not installed >>>>>> build_requires: >>>>>> ! LWP::Simple is not installed >>>>>> recommends: >>>>>> * DBD::Pg is not installed >>>>>> >>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>>>> versions >>>>>> of the modules indicated above before proceeding with this >>>>>> installation >>>>>> >>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>> >>>>>> MAKER supports distributed parallelization via MPI. >>>>>> Would you like to configure MAKER for MPI (This >>>>>> requires that you have an MPI client installed)? [N ]n >>>>>> >>>>>> WARNING: Apache cannot be located. The optional web based >>>>>> interface to MAKER will not be available to you. >>>>>> >>>>>> Created MYMETA.yml and MYMETA.json >>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>> >>>>>> >>>>>> The file 'Build' has been created for you to finish installing MAKER. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ====================================================================== >>>>>> == >>>>>> == >>>>>> ==== >>>>>> STATUS MAKER 2.25 >>>>>> >>>>>> >>>>>> ====================================================================== >>>>>> == >>>>>> == >>>>>> ==== >>>>>> PERL Dependencies: MISSING >>>>>> ! forks >>>>>> ! IO::All >>>>>> ! forks::shared >>>>>> ! Want >>>>>> ! DBD::SQLite >>>>>> ! Proc::ProcessTable >>>>>> ! Inline::C >>>>>> ! Perl::Unsafe::Signals >>>>>> >>>>>> External Programs: VERIFIED >>>>>> External C Libraries: VERIFIED >>>>>> MPI SUPPORT: DISABLED >>>>>> MWAS Web Interface: DISABLED >>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>> >>>>>> >>>>>> Important Commands: >>>>>> ./Build installdeps #installs missing PERL dependencies >>>>>> ./Build installexes #installs all missing external >>>>>> programs >>>>>> ./Build install #installs MAKER >>>>>> ./Build status #Shows this status menu >>>>>> >>>>>> Other Commands: >>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>> RepBase) >>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX / >>>>>> v1 >>>>>> on >>>>>> Mac OSX) >>>>>> ./Build snap #installs SNAP >>>>>> ./Build augustus #installs Augustus >>>>>> ./Build apollo #installs Apollo >>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, not >>>>>> web >>>>>> accecible) >>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>> install >>>>>> recommended) >>>>>> >>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>> INSTALL >>>>>> I followed the quick and dirty installation for Bioperl, so I am not >>>>>> sure what I did wrong/missed out. >>>>>> >>>>>> When I run ./Build installdeps, I get: >>>>>> You do not have write access to install missing Modules. >>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>> in the .../maker/perl/lib directory, or you can run >>>>>> './Build installdeps' as root or using sudo and try again. >>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>> mkdir /usit/titan: Permission denied at >>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>> line >>>>>> 59 >>>>>> >>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>> appreciated! >>>>>> >>>>>> much obliged, >>>>>> Christoph >>>>>> >>>>>> _______________________________________________ >>>>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>> g >> > From carsonhh at gmail.com Tue Oct 16 09:18:48 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 16 Oct 2012 11:18:48 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <507D79DC.5090702@gmail.com> Message-ID: Delete any file in the maker output directory that start with the name .init or .NFS then rerun. Thanks, Carson On 12-10-16 11:14 AM, "Christoph Hahn" wrote: >Hi Carson, > >It was a link. The path was not correct any more because of the >migration to the new cluster. I could not change it so I just gave the >path to the real file in the maker_opts file. That has resolved this >error, but now I am getting the following: >STATUS: Parsing control files... >ERROR: Cannot get initialization lock. > >Probably not relevant any more, but: No trailing commas, no ':' >characters in the path. > >Thanks for your help! >Christoph > >Am 16.10.2012 16:19, schrieb Carson Holt: >> No. that's just a recommendation for the maker2chado script to work. >> >> Is es.mod the actual HMM or a soft link to the HMM? >> After the error message their should be a list of the actual files. >>Does >> it print that? >> Are there trailing commas in you maker_opts.ctl file after the hmm file >> name in the control files? >> Are their ':' characters in the path name to the file? >> >> --Carson >> >> >> >> On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >> >>> Thanks for your help Carson! >>> >>> I tried maker2.26, but the error still remains. I also tried to change >>> the TMP= option in the maker_opts file, but no change either.. >>> >>> STATUS: Parsing control files... >>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>> >>> When building the maker2.26 I saw that it is complaining about one more >>> (recommended) dependency (I must have overlooked that before): >>> Checking prerequisites... >>> recommends: >>> * DBD::Pg is not installed >>> >>> I am having problems to get that one.. Is that causing the error? >>> >>> much obliged, >>> Christoph >>> >>> Am 15.10.2012 20:05, schrieb Carson Holt: >>>> It looks like either an NFS issue or perhaps an issue with the >>>>location >>>> provided for the TMP= in the maker_opts.ctl file. >>>> >>>> First I would suggest trying maker 2.26 to see if the issue still >>>> happens >>>> there (there are a few updates to how NFS locks are handled in 2.26). >>>> >>>> Second if you are setting TMP make sure your disk is not full. Or if >>>> TMP >>>> was set to a tmpfs location (in memory filesystem), it could be a full >>>> memory issue in which case you could just set TMP to somewhere else. >>>> >>>> Let me know if you still see the issue in 2.26 or after changing the >>>> location of TMP. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>>> >>>>> Hi Carson, >>>>> >>>>> Thanks for your reply. I managed to install maker2.25 properly now, >>>>>but >>>>> when running it I am getting strange errors (two kinds): >>>>> type one: >>>>> STATUS: Parsing control files... >>>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>>> >>>>> type two: >>>>> STATUS: Parsing control files... >>>>> ERROR: Cannot get initialization lock. >>>>> >>>>> I have divided the draft assembly to several files, each containing a >>>>> subset of contigs on which I am running a maker instance. I am >>>>>getting >>>>> the above named error type one for the majority of the instances, >>>>> although the HMM file it is complaining about (es.mod) is actually in >>>>> the path it is naming (not shown above). In a small number of >>>>>instances >>>>> I am getting the type two error mentioned above. >>>>> >>>>> Can you help me? >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> >>>>> >>>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>>> You may need to reconfigure your cpan preferences. >>>>>> >>>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm >>>>>>file, >>>>>> or >>>>>> by starting cpan from your command line (command is cpan). Then run >>>>>> 'o >>>>>> conf init' inside the cpan ui. >>>>>> >>>>>> --Carson >>>>>> >>>>>> >>>>>> On 12-10-09 7:07 AM, "Christoph Hahn" >>>>>>wrote: >>>>>> >>>>>>> Hello maker-team, >>>>>>> >>>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>>> privileges >>>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>>> seem >>>>>>> to get it to work). All necessary prerequisite programs are >>>>>>>installed >>>>>>> and in the path (I followed the steps in the INSTALL file that >>>>>>>comes >>>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>>> Checking prerequisites... >>>>>>> requires: >>>>>>> ! DBD::SQLite is not installed >>>>>>> ! IO::All is not installed >>>>>>> ! Inline::C is not installed >>>>>>> ! Perl::Unsafe::Signals is not installed >>>>>>> ! Proc::ProcessTable is not installed >>>>>>> ! Want is not installed >>>>>>> ! forks is not installed >>>>>>> ! forks::shared is not installed >>>>>>> build_requires: >>>>>>> ! LWP::Simple is not installed >>>>>>> recommends: >>>>>>> * DBD::Pg is not installed >>>>>>> >>>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install >>>>>>>the >>>>>>> versions >>>>>>> of the modules indicated above before proceeding with this >>>>>>> installation >>>>>>> >>>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>>> >>>>>>> MAKER supports distributed parallelization via MPI. >>>>>>> Would you like to configure MAKER for MPI (This >>>>>>> requires that you have an MPI client installed)? [N ]n >>>>>>> >>>>>>> WARNING: Apache cannot be located. The optional web based >>>>>>> interface to MAKER will not be available to you. >>>>>>> >>>>>>> Created MYMETA.yml and MYMETA.json >>>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>>> >>>>>>> >>>>>>> The file 'Build' has been created for you to finish installing >>>>>>>MAKER. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>== >>>>>>> == >>>>>>> == >>>>>>> ==== >>>>>>> STATUS MAKER 2.25 >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>== >>>>>>> == >>>>>>> == >>>>>>> ==== >>>>>>> PERL Dependencies: MISSING >>>>>>> ! forks >>>>>>> ! IO::All >>>>>>> ! forks::shared >>>>>>> ! Want >>>>>>> ! DBD::SQLite >>>>>>> ! Proc::ProcessTable >>>>>>> ! Inline::C >>>>>>> ! Perl::Unsafe::Signals >>>>>>> >>>>>>> External Programs: VERIFIED >>>>>>> External C Libraries: VERIFIED >>>>>>> MPI SUPPORT: DISABLED >>>>>>> MWAS Web Interface: DISABLED >>>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>>> >>>>>>> >>>>>>> Important Commands: >>>>>>> ./Build installdeps #installs missing PERL >>>>>>>dependencies >>>>>>> ./Build installexes #installs all missing external >>>>>>> programs >>>>>>> ./Build install #installs MAKER >>>>>>> ./Build status #Shows this status menu >>>>>>> >>>>>>> Other Commands: >>>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>>> RepBase) >>>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX >>>>>>>/ >>>>>>> v1 >>>>>>> on >>>>>>> Mac OSX) >>>>>>> ./Build snap #installs SNAP >>>>>>> ./Build augustus #installs Augustus >>>>>>> ./Build apollo #installs Apollo >>>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, >>>>>>>not >>>>>>> web >>>>>>> accecible) >>>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>>> install >>>>>>> recommended) >>>>>>> >>>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>>> INSTALL >>>>>>> I followed the quick and dirty installation for Bioperl, so I am >>>>>>>not >>>>>>> sure what I did wrong/missed out. >>>>>>> >>>>>>> When I run ./Build installdeps, I get: >>>>>>> You do not have write access to install missing Modules. >>>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>>> in the .../maker/perl/lib directory, or you can run >>>>>>> './Build installdeps' as root or using sudo and try again. >>>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>>> mkdir /usit/titan: Permission denied at >>>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>>> line >>>>>>> 59 >>>>>>> >>>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>>> appreciated! >>>>>>> >>>>>>> much obliged, >>>>>>> Christoph >>>>>>> >>>>>>> _______________________________________________ >>>>>>> maker-devel mailing list >>>>>>> maker-devel at box290.bluehost.com >>>>>>> >>>>>>> >>>>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab. >>>>>>>or >>>>>>> g >>> >> > > From chrisi.hahni at gmail.com Tue Oct 16 10:12:21 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 18:12:21 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D8765.8010909@gmail.com> The maker output directory contains only one thing: .NFSLock.init_lock.NFSLock, which is a link to .NFSLock.init_lock.tmp.3358.21603.1715.48578929183 in the same directory. When I delete it and rerun maker I get the same error and another .NFSLock.init_lock.NFSLock is created, this one is linked to .NFSLock.init_lock.tmp.3635.21981.7756.8294996771 Thanks, Christoph Am 16.10.2012 17:18, schrieb Carson Holt: > Delete any file in the maker output directory that start with the name > .init or .NFS then rerun. > > Thanks, > Carson > > > > On 12-10-16 11:14 AM, "Christoph Hahn" wrote: > >> Hi Carson, >> >> It was a link. The path was not correct any more because of the >> migration to the new cluster. I could not change it so I just gave the >> path to the real file in the maker_opts file. That has resolved this >> error, but now I am getting the following: >> STATUS: Parsing control files... >> ERROR: Cannot get initialization lock. >> >> Probably not relevant any more, but: No trailing commas, no ':' >> characters in the path. >> >> Thanks for your help! >> Christoph >> >> Am 16.10.2012 16:19, schrieb Carson Holt: >>> No. that's just a recommendation for the maker2chado script to work. >>> >>> Is es.mod the actual HMM or a soft link to the HMM? >>> After the error message their should be a list of the actual files. >>> Does >>> it print that? >>> Are there trailing commas in you maker_opts.ctl file after the hmm file >>> name in the control files? >>> Are their ':' characters in the path name to the file? >>> >>> --Carson >>> >>> >>> >>> On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >>> >>>> Thanks for your help Carson! >>>> >>>> I tried maker2.26, but the error still remains. I also tried to change >>>> the TMP= option in the maker_opts file, but no change either.. >>>> >>>> STATUS: Parsing control files... >>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>> >>>> When building the maker2.26 I saw that it is complaining about one more >>>> (recommended) dependency (I must have overlooked that before): >>>> Checking prerequisites... >>>> recommends: >>>> * DBD::Pg is not installed >>>> >>>> I am having problems to get that one.. Is that causing the error? >>>> >>>> much obliged, >>>> Christoph >>>> >>>> Am 15.10.2012 20:05, schrieb Carson Holt: >>>>> It looks like either an NFS issue or perhaps an issue with the >>>>> location >>>>> provided for the TMP= in the maker_opts.ctl file. >>>>> >>>>> First I would suggest trying maker 2.26 to see if the issue still >>>>> happens >>>>> there (there are a few updates to how NFS locks are handled in 2.26). >>>>> >>>>> Second if you are setting TMP make sure your disk is not full. Or if >>>>> TMP >>>>> was set to a tmpfs location (in memory filesystem), it could be a full >>>>> memory issue in which case you could just set TMP to somewhere else. >>>>> >>>>> Let me know if you still see the issue in 2.26 or after changing the >>>>> location of TMP. >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>>>> >>>>>> Hi Carson, >>>>>> >>>>>> Thanks for your reply. I managed to install maker2.25 properly now, >>>>>> but >>>>>> when running it I am getting strange errors (two kinds): >>>>>> type one: >>>>>> STATUS: Parsing control files... >>>>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>>>> >>>>>> type two: >>>>>> STATUS: Parsing control files... >>>>>> ERROR: Cannot get initialization lock. >>>>>> >>>>>> I have divided the draft assembly to several files, each containing a >>>>>> subset of contigs on which I am running a maker instance. I am >>>>>> getting >>>>>> the above named error type one for the majority of the instances, >>>>>> although the HMM file it is complaining about (es.mod) is actually in >>>>>> the path it is naming (not shown above). In a small number of >>>>>> instances >>>>>> I am getting the type two error mentioned above. >>>>>> >>>>>> Can you help me? >>>>>> >>>>>> much obliged, >>>>>> Christoph >>>>>> >>>>>> >>>>>> >>>>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>>>> You may need to reconfigure your cpan preferences. >>>>>>> >>>>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm >>>>>>> file, >>>>>>> or >>>>>>> by starting cpan from your command line (command is cpan). Then run >>>>>>> 'o >>>>>>> conf init' inside the cpan ui. >>>>>>> >>>>>>> --Carson >>>>>>> >>>>>>> >>>>>>> On 12-10-09 7:07 AM, "Christoph Hahn" >>>>>>> wrote: >>>>>>> >>>>>>>> Hello maker-team, >>>>>>>> >>>>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>>>> privileges >>>>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>>>> seem >>>>>>>> to get it to work). All necessary prerequisite programs are >>>>>>>> installed >>>>>>>> and in the path (I followed the steps in the INSTALL file that >>>>>>>> comes >>>>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>>>> Checking prerequisites... >>>>>>>> requires: >>>>>>>> ! DBD::SQLite is not installed >>>>>>>> ! IO::All is not installed >>>>>>>> ! Inline::C is not installed >>>>>>>> ! Perl::Unsafe::Signals is not installed >>>>>>>> ! Proc::ProcessTable is not installed >>>>>>>> ! Want is not installed >>>>>>>> ! forks is not installed >>>>>>>> ! forks::shared is not installed >>>>>>>> build_requires: >>>>>>>> ! LWP::Simple is not installed >>>>>>>> recommends: >>>>>>>> * DBD::Pg is not installed >>>>>>>> >>>>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install >>>>>>>> the >>>>>>>> versions >>>>>>>> of the modules indicated above before proceeding with this >>>>>>>> installation >>>>>>>> >>>>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>>>> >>>>>>>> MAKER supports distributed parallelization via MPI. >>>>>>>> Would you like to configure MAKER for MPI (This >>>>>>>> requires that you have an MPI client installed)? [N ]n >>>>>>>> >>>>>>>> WARNING: Apache cannot be located. The optional web based >>>>>>>> interface to MAKER will not be available to you. >>>>>>>> >>>>>>>> Created MYMETA.yml and MYMETA.json >>>>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>>>> >>>>>>>> >>>>>>>> The file 'Build' has been created for you to finish installing >>>>>>>> MAKER. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> == >>>>>>>> == >>>>>>>> == >>>>>>>> ==== >>>>>>>> STATUS MAKER 2.25 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> == >>>>>>>> == >>>>>>>> == >>>>>>>> ==== >>>>>>>> PERL Dependencies: MISSING >>>>>>>> ! forks >>>>>>>> ! IO::All >>>>>>>> ! forks::shared >>>>>>>> ! Want >>>>>>>> ! DBD::SQLite >>>>>>>> ! Proc::ProcessTable >>>>>>>> ! Inline::C >>>>>>>> ! Perl::Unsafe::Signals >>>>>>>> >>>>>>>> External Programs: VERIFIED >>>>>>>> External C Libraries: VERIFIED >>>>>>>> MPI SUPPORT: DISABLED >>>>>>>> MWAS Web Interface: DISABLED >>>>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>>>> >>>>>>>> >>>>>>>> Important Commands: >>>>>>>> ./Build installdeps #installs missing PERL >>>>>>>> dependencies >>>>>>>> ./Build installexes #installs all missing external >>>>>>>> programs >>>>>>>> ./Build install #installs MAKER >>>>>>>> ./Build status #Shows this status menu >>>>>>>> >>>>>>>> Other Commands: >>>>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>>>> RepBase) >>>>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX >>>>>>>> / >>>>>>>> v1 >>>>>>>> on >>>>>>>> Mac OSX) >>>>>>>> ./Build snap #installs SNAP >>>>>>>> ./Build augustus #installs Augustus >>>>>>>> ./Build apollo #installs Apollo >>>>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, >>>>>>>> not >>>>>>>> web >>>>>>>> accecible) >>>>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>>>> install >>>>>>>> recommended) >>>>>>>> >>>>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>>>> INSTALL >>>>>>>> I followed the quick and dirty installation for Bioperl, so I am >>>>>>>> not >>>>>>>> sure what I did wrong/missed out. >>>>>>>> >>>>>>>> When I run ./Build installdeps, I get: >>>>>>>> You do not have write access to install missing Modules. >>>>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>>>> in the .../maker/perl/lib directory, or you can run >>>>>>>> './Build installdeps' as root or using sudo and try again. >>>>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>>>> mkdir /usit/titan: Permission denied at >>>>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>>>> line >>>>>>>> 59 >>>>>>>> >>>>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>>>> appreciated! >>>>>>>> >>>>>>>> much obliged, >>>>>>>> Christoph >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> maker-devel mailing list >>>>>>>> maker-devel at box290.bluehost.com >>>>>>>> >>>>>>>> >>>>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab. >>>>>>>> or >>>>>>>> g >> > From mikael.durling at slu.se Tue Oct 16 10:58:55 2012 From: mikael.durling at slu.se (=?iso-8859-1?Q?Mikael_Brandstr=F6m_Durling?=) Date: Tue, 16 Oct 2012 18:58:55 +0200 Subject: [maker-devel] Format of repeat_gff gff3 file Message-ID: Hi, I would like to mask my fungal genome from two different sources (ie. repbase and an inhouse repeat library). However, I suppose the that if I supply a library as rmlib in maker_opts, it will be mutually exclusive to the model_org option, in the same way as -spec and -lib options to RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the model_org option as fungi, and would like to provide maker with additional masking as a gff file. I tried by running RepeatMasker with my inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker package to obtain a gff3 file. This file was supplied to maker as rm_gff (see below for a sample from the file). The run fail with backtraces like the one paseted below. How should this gff file be formatted for maker to understand it? I see that in maker produced gff files, there are additional information found in the id of the hits. Is this required? Maybe it's easier to modify maker to make two rounds of RepeatMasker calls if both model_org and rmlib are specified? Thanks for any input, Mikael ------------- EXCEPTION: Bio::Root::Exception ------------- MSG: Must have defined a valid name for Hit STACK: Error::throw STACK: Bio::Root::Root::throw /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 STACK: Bio::Search::Hit::GenericHit::new /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:149 STACK: Bio::Search::Hit::PhatHit::Base::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm:127 STACK: Bio::Search::Hit::PhatHit::gff3::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/gff3.pm:23 STACK: GFFDB::_load_hits /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:1026 STACK: GFFDB::phathits_on_chunk /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:651 STACK: Process::MpiChunk::_go /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:752 STACK: Process::MpiChunk::run /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:331 STACK: main::node_thread /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 STACK: threads::new /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm:799 STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 ----------------------------------------------------------- Cannot restore overloading on HASH(0x2580810) (package Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at /opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm line 2256. Compilation failed in require at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. BEGIN failed--compilation aborted at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached ERROR: Could not open '/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.output/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fungi.rb.out' ERROR: Failed while doing repeat masking ERROR: Chunk failed at level:0, tier_type:1 FAILED CONTIG:scf_55419 The gff3-file looks like this: ##gff-version 3 ##sequence-region scf_42697 1 3949 scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 ##sequence-region scf_82496 1 2757 scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_family-1046 2478 4915 ##sequence-region scf_82727 1 4159 scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 23 scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_family-64 15 50 ##sequence-region scf_82785 1 4084 scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 24 scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_family-690 419 786 ##sequence-region scf_86740 1 4293 scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_family-262 1 25 scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_family-262 2 60 scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_family-262 13 60 ##sequence-region scf_86782 1 8564 scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_family-480 1027 1129 ##sequence-region scf_86808 1 4495 scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family-690 1 969 scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 74 ##sequence-region scf_86815 1 4139 scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-262 825 918 scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_family-262 526 4459 ##sequence-region scf_86823 1 2528 scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 178 scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 35 scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 45 scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 41 scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 96 scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 40 scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_family-262 881 920 ##sequence-region scf_86857 1 2778 From mike.thon at gmail.com Wed Oct 17 04:48:59 2012 From: mike.thon at gmail.com (Michael Thon) Date: Wed, 17 Oct 2012 12:48:59 +0200 Subject: [maker-devel] Format of repeat_gff gff3 file In-Reply-To: References: Message-ID: Hi - 2 ideas: 1) There is a utility included with RepeatMasker that enables you to extract sequences from it. its in the utils directory. You can run it like this: ./queryRepeatDatabase.pl -species Fungi -clade that will extract all repeat sequences belonging to Fungi or its descendants. Maybe the simplest thing for you to do is extract the sequences that you want and cat them together with your in house sequences to provide MAKER with a single file of reference sequences. 2) some of the MAKER parameters take a comma separated list of files. I don't know if this applied to rm_lib though. Mike On Oct 16, 2012, at 6:58 PM, Mikael Brandstr?m Durling wrote: > Hi, > > I would like to mask my fungal genome from two different sources (ie. repbase and an inhouse repeat library). However, I suppose the that if I supply a library as rmlib in maker_opts, it will be mutually exclusive to the model_org option, in the same way as -spec and -lib options to RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the model_org option as fungi, and would like to provide maker with additional masking as a gff file. I tried by running RepeatMasker with my inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker package to obtain a gff3 file. This file was supplied to maker as rm_gff (see below for a sample from the file). The run fail with backtraces like the one paseted below. How should this gff file be formatted for maker to understand it? I see that in maker produced gff files, there are additional information found in the id of the hits. Is this required? > > Maybe it's easier to modify maker to make two rounds of RepeatMasker calls if both model_org and rmlib are specified? > > Thanks for any input, > Mikael > > > ------------- EXCEPTION: Bio::Root::Exception ------------- > MSG: Must have defined a valid name for Hit > STACK: Error::throw > STACK: Bio::Root::Root::throw /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 > STACK: Bio::Search::Hit::GenericHit::new /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:149 > STACK: Bio::Search::Hit::PhatHit::Base::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm:127 > STACK: Bio::Search::Hit::PhatHit::gff3::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/gff3.pm:23 > STACK: GFFDB::_load_hits /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:1026 > STACK: GFFDB::phathits_on_chunk /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:651 > STACK: Process::MpiChunk::_go /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:752 > STACK: Process::MpiChunk::run /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:331 > STACK: main::node_thread /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 > STACK: threads::new /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm:799 > STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 > ----------------------------------------------------------- > Cannot restore overloading on HASH(0x2580810) (package Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at /opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm line 2256. > Compilation failed in require at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. > BEGIN failed--compilation aborted at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. > Perl exited with active threads: > 1 running and unjoined > 0 finished and unjoined > 0 running and detached > ERROR: Could not open '/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.output/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fungi.rb.out' > ERROR: Failed while doing repeat masking > ERROR: Chunk failed at level:0, tier_type:1 > FAILED CONTIG:scf_55419 > > > The gff3-file looks like this: > ##gff-version 3 > ##sequence-region scf_42697 1 3949 > scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 > scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 > scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 > ##sequence-region scf_82496 1 2757 > scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_family-1046 2478 4915 > ##sequence-region scf_82727 1 4159 > scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 > scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 23 > scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_family-64 15 50 > ##sequence-region scf_82785 1 4084 > scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 24 > scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_family-690 419 786 > ##sequence-region scf_86740 1 4293 > scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_family-262 1 25 > scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_family-262 2 60 > scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_family-262 13 60 > ##sequence-region scf_86782 1 8564 > scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_family-480 1027 1129 > ##sequence-region scf_86808 1 4495 > scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family-690 1 969 > scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 74 > ##sequence-region scf_86815 1 4139 > scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-262 825 918 > scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_family-262 526 4459 > ##sequence-region scf_86823 1 2528 > scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 > scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 > scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 178 > scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 35 > scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 45 > scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 41 > scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 96 > scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 40 > scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_family-262 881 920 > ##sequence-region scf_86857 1 2778 > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Fri Oct 19 08:00:12 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 19 Oct 2012 10:00:12 -0400 Subject: [maker-devel] Format of repeat_gff gff3 file In-Reply-To: Message-ID: This command line should add IDs to the end of the GFF3 to let it pass through without the error message. cat file.gff | perl -ane '$id; if(!/^\#/){@F = split(/\t/, $_); chomp $F[-1];$id++; $F[-1] .= "\;ID=$id"; $_ = join("\t", @F)."\n"} print $_' MAKER will use the GFF3 first if provided, then run the species specific library, and then any model organism specified. So if there is overlap and one must be excluded, then they will be kept in that same order of precedence. Thanks, Carson On 12-10-16 12:58 PM, "Mikael Brandstr?m Durling" wrote: >Hi, > >I would like to mask my fungal genome from two different sources (ie. >repbase and an inhouse repeat library). However, I suppose the that if I >supply a library as rmlib in maker_opts, it will be mutually exclusive to >the model_org option, in the same way as -spec and -lib options to >RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the >model_org option as fungi, and would like to provide maker with >additional masking as a gff file. I tried by running RepeatMasker with my >inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker >package to obtain a gff3 file. This file was supplied to maker as rm_gff >(see below for a sample from the file). The run fail with backtraces like >the one paseted below. How should this gff file be formatted for maker to >understand it? I see that in maker produced gff files, there are >additional information found in the id of the hits. Is this required? > >Maybe it's easier to modify maker to make two rounds of RepeatMasker >calls if both model_org and rmlib are specified? > >Thanks for any input, >Mikael > > >------------- EXCEPTION: Bio::Root::Exception ------------- >MSG: Must have defined a valid name for Hit >STACK: Error::throw >STACK: Bio::Root::Root::throw >/opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 >STACK: Bio::Search::Hit::GenericHit::new >/opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:14 >9 >STACK: Bio::Search::Hit::PhatHit::Base::new >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Bio/Search/Hit/PhatHit/Base.pm:127 >STACK: Bio::Search::Hit::PhatHit::gff3::new >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Bio/Search/Hit/PhatHit/gff3.pm:23 >STACK: GFFDB::_load_hits >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/GFFDB.pm:1026 >STACK: GFFDB::phathits_on_chunk >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/GFFDB.pm:651 >STACK: Process::MpiChunk::_go >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Process/MpiChunk.pm:752 >STACK: Process::MpiChunk::run >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Process/MpiChunk.pm:331 >STACK: main::node_thread >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 >STACK: threads::new >/proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16. >1/x86_64-linux/forks.pm:799 >STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 >----------------------------------------------------------- >Cannot restore overloading on HASH(0x2580810) (package >Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at >/opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at >/proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16. >1/x86_64-linux/forks.pm line 2256. >Compilation failed in require at >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. >BEGIN failed--compilation aborted at >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. >Perl exited with active threads: > 1 running and unjoined > 0 finished and unjoined > 0 running and detached >ERROR: Could not open >'/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.o >utput/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fung >i.rb.out' >ERROR: Failed while doing repeat masking >ERROR: Chunk failed at level:0, tier_type:1 >FAILED CONTIG:scf_55419 > > >The gff3-file looks like this: >##gff-version 3 >##sequence-region scf_42697 1 3949 >scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 >scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 >scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 >##sequence-region scf_82496 1 2757 >scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_fami >ly-1046 2478 4915 >##sequence-region scf_82727 1 4159 >scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 >scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 >23 >scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_fam >ily-64 15 50 >##sequence-region scf_82785 1 4084 >scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 >24 >scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_fam >ily-690 419 786 >##sequence-region scf_86740 1 4293 >scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_famil >y-262 1 25 >scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_famil >y-262 2 60 >scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_famil >y-262 13 60 >##sequence-region scf_86782 1 8564 >scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_fam >ily-480 1027 1129 >##sequence-region scf_86808 1 4495 >scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family >-690 1 969 >scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 >74 >##sequence-region scf_86815 1 4139 >scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-2 >62 825 918 >scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_fa >mily-262 526 4459 >##sequence-region scf_86823 1 2528 >scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 >scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 >scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 >178 >scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 >35 >scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 >45 >scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 >41 >scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 >96 >scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 >40 >scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_fam >ily-262 881 920 >##sequence-region scf_86857 1 2778 > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From parulk at caltech.edu Mon Oct 22 15:46:18 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 22 Oct 2012 14:46:18 -0700 (PDT) Subject: [maker-devel] gff3_preds2models script Message-ID: <3207.131.215.15.234.1350942378.squirrel@webmail.caltech.edu> Hello, I want to add gene structure(gene/mRNA/exon) to gff3 file. I am using gff3_preds2models for this purpose. However I get following error **WARNING: No top level feature found for ID WHL22.100252 I have attached the sample input gff3 file and the list of ids Thanks and regards, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: sample.gff3 Type: application/octet-stream Size: 1873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: list Type: application/octet-stream Size: 76 bytes Desc: not available URL: From fourie.joubert at up.ac.za Wed Oct 24 01:07:13 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 09:07:13 +0200 Subject: [maker-devel] Segfault during build_fasta_index Message-ID: <508793A1.1000704@up.ac.za> Hi I am getting a segfault in maker somewhere during the build_fasta_index step. I suspect it is happening after some updates I made to bioperl. Has anyone else experienced something similar, and is there a recommended bioperl version? Thanks and regards! Fourie -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 06:28:10 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 08:28:10 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <508793A1.1000704@up.ac.za> Message-ID: Yes. Don't use the live version of BioPerl. Use the CPAN version. The live version has a bug. Thanks, Carson On 12-10-24 3:07 AM, "Fourie Joubert" wrote: >Hi > >I am getting a segfault in maker somewhere during the build_fasta_index >step. > >I suspect it is happening after some updates I made to bioperl. > >Has anyone else experienced something similar, and is there a >recommended bioperl version? > >Thanks and regards! > >Fourie > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From fourie.joubert at up.ac.za Wed Oct 24 08:05:12 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:05:12 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087F598.20303@up.ac.za> Hi I have installed from CPAN (C/CJ/CJFIELDS/BioPerl-1.6.901.tar.gz), but the error persists... Does anyone know if the versions in CPAN may have been replaced recently? Best regards! Foorie On 10/24/2012 02:28 PM, Carson Holt wrote: > Yes. Don't use the live version of BioPerl. Use the CPAN version. The > live version has a bug. > > Thanks, > Carson > > > On 12-10-24 3:07 AM, "Fourie Joubert" wrote: > >> Hi >> >> I am getting a segfault in maker somewhere during the build_fasta_index >> step. >> >> I suspect it is happening after some updates I made to bioperl. >> >> Has anyone else experienced something similar, and is there a >> recommended bioperl version? >> >> Thanks and regards! >> >> Fourie >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:10:32 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:10:32 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087F598.20303@up.ac.za> Message-ID: Run this command --> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' What does it print? --Carson On 12-10-24 10:05 AM, "Fourie Joubert" wrote: >Hi > >I have installed from CPAN (C/CJ/CJFIELDS/BioPerl-1.6.901.tar.gz), but >the error persists... > >Does anyone know if the versions in CPAN may have been replaced recently? > >Best regards! > >Foorie > > > > > > >On 10/24/2012 02:28 PM, Carson Holt wrote: >> Yes. Don't use the live version of BioPerl. Use the CPAN version. The >> live version has a bug. >> >> Thanks, >> Carson >> >> >> On 12-10-24 3:07 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> I am getting a segfault in maker somewhere during the build_fasta_index >>> step. >>> >>> I suspect it is happening after some updates I made to bioperl. >>> >>> Has anyone else experienced something similar, and is there a >>> recommended bioperl version? >>> >>> Thanks and regards! >>> >>> Fourie >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:24:43 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:24:43 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FA2B.5020702@up.ac.za> Hi 1.006901 Regards! Fourie On 10/24/2012 04:10 PM, Carson Holt wrote: > perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:26:41 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:26:41 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FA2B.5020702@up.ac.za> Message-ID: Try completely deleting the mpi_blastdb directory under the MAKER output folder before restarting. Perhaps also try reinstalling DB_File via CPAN. --Carson On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >Hi > >1.006901 > >Regards! > >Fourie > > > >On 10/24/2012 04:10 PM, Carson Holt wrote: >> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:30:15 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:30:15 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FB77.7060802@up.ac.za> Hi Darn - did both, but unfortunately still segfaults... Best regards! Fourie On 10/24/2012 04:26 PM, Carson Holt wrote: > Try completely deleting the mpi_blastdb directory under the MAKER output > folder before restarting. Perhaps also try reinstalling DB_File via CPAN. > > --Carson > > On 12-10-24 10:24 AM, "Fourie Joubert" wrote: > >> Hi >> >> 1.006901 >> >> Regards! >> >> Fourie >> >> >> >> On 10/24/2012 04:10 PM, Carson Holt wrote: >>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:33:07 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:33:07 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FB77.7060802@up.ac.za> Message-ID: I'm going to write you a test script that just creates a fasta index to see if it seg faults. I'll send it to you later. For now could you run maker with the --debug flag set, apture the STDERR and send it to me. Thanks, Carson On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >Hi > >Darn - did both, but unfortunately still segfaults... > >Best regards! > >Fourie > > >On 10/24/2012 04:26 PM, Carson Holt wrote: >> Try completely deleting the mpi_blastdb directory under the MAKER output >> folder before restarting. Perhaps also try reinstalling DB_File via >>CPAN. >> >> --Carson >> >> On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> 1.006901 >>> >>> Regards! >>> >>> Fourie >>> >>> >>> >>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:41:15 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:41:15 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FE0B.7030206@up.ac.za> Hi Really weird - as soon as I add the debug flag - no more segfault... Best regards! Fourie On 10/24/2012 04:33 PM, Carson Holt wrote: > I'm going to write you a test script that just creates a fasta index to > see if it seg faults. I'll send it to you later. For now could you run > maker with the --debug flag set, apture the STDERR and send it to me. > > Thanks, > Carson > > > On 12-10-24 10:30 AM, "Fourie Joubert" wrote: > >> Hi >> >> Darn - did both, but unfortunately still segfaults... >> >> Best regards! >> >> Fourie >> >> >> On 10/24/2012 04:26 PM, Carson Holt wrote: >>> Try completely deleting the mpi_blastdb directory under the MAKER output >>> folder before restarting. Perhaps also try reinstalling DB_File via >>> CPAN. >>> >>> --Carson >>> >>> On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> 1.006901 >>>> >>>> Regards! >>>> >>>> Fourie >>>> >>>> >>>> >>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:56:51 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:56:51 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FE0B.7030206@up.ac.za> Message-ID: Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA variable. It's set by both BioPerl and a couple of MAKER modules. If it gets set twice, sometimes you get bugs. Try upgrading to MAKER 2.26 I have a statement in that that will make sure it won't get set wrong. Otherwise I can give you instructions on editing the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER modules. Thanks, Carson On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >Hi > >Really weird - as soon as I add the debug flag - no more segfault... > >Best regards! > >Fourie > > > > >On 10/24/2012 04:33 PM, Carson Holt wrote: >> I'm going to write you a test script that just creates a fasta index to >> see if it seg faults. I'll send it to you later. For now could you run >> maker with the --debug flag set, apture the STDERR and send it to me. >> >> Thanks, >> Carson >> >> >> On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> Darn - did both, but unfortunately still segfaults... >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>output >>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>> CPAN. >>>> >>>> --Carson >>>> >>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> 1.006901 >>>>> >>>>> Regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> >>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>> perl -MBio::Root::Version -e 'print >>>>>>"$Bio::Root::Version::VERSION\n"' >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 09:19:08 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 17:19:08 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <508806EC.6050600@up.ac.za> Hi Carson We are indeed running 2.26. Instructions on editing the statements would be great. Many thanks for all the help, and sorry for the bother! Best regards! Fourie On 10/24/2012 04:56 PM, Carson Holt wrote: > Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA > variable. It's set by both BioPerl and a couple of MAKER modules. If it > gets set twice, sometimes you get bugs. > > Try upgrading to MAKER 2.26 I have a statement in that that will make sure > it won't get set wrong. Otherwise I can give you instructions on editing > the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER > modules. > > Thanks, > Carson > > > On 12-10-24 10:41 AM, "Fourie Joubert" wrote: > >> Hi >> >> Really weird - as soon as I add the debug flag - no more segfault... >> >> Best regards! >> >> Fourie >> >> >> >> >> On 10/24/2012 04:33 PM, Carson Holt wrote: >>> I'm going to write you a test script that just creates a fasta index to >>> see if it seg faults. I'll send it to you later. For now could you run >>> maker with the --debug flag set, apture the STDERR and send it to me. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> Darn - did both, but unfortunately still segfaults... >>>> >>>> Best regards! >>>> >>>> Fourie >>>> >>>> >>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>> output >>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>> CPAN. >>>>> >>>>> --Carson >>>>> >>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> 1.006901 >>>>>> >>>>>> Regards! >>>>>> >>>>>> Fourie >>>>>> >>>>>> >>>>>> >>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>> perl -MBio::Root::Version -e 'print >>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>> -- >>>>>> -------------- >>>>>> Prof Fourie Joubert >>>>>> Bioinformatics and Computational Biology Unit >>>>>> Department of Biochemistry >>>>>> University of Pretoria >>>>>> fourie.joubert at up.ac.za >>>>>> http://www.bi.up.ac.za >>>>>> Tel. +27-12-420-5825 >>>>>> Fax. +27-12-420-5800 >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> -- >>>>>> - >>>>>> This message and attachments are subject to a disclaimer. Please >>>>>> refer >>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>> details. >>>>>> >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 09:50:03 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 11:50:03 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <508806EC.6050600@up.ac.za> Message-ID: Find these lines in these files in MAKER .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); And the Bio::DB::Fasta file in BioPerl. Use this command to identify the location of Bio::DB::Fasta --> perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); Use an editor like emacs or vi to make the changes. Thanks, Carson On 12-10-24 11:19 AM, "Fourie Joubert" wrote: >Hi Carson > >We are indeed running 2.26. > >Instructions on editing the statements would be great. > >Many thanks for all the help, and sorry for the bother! > >Best regards! > >Fourie > > > > >On 10/24/2012 04:56 PM, Carson Holt wrote: >> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >> variable. It's set by both BioPerl and a couple of MAKER modules. If it >> gets set twice, sometimes you get bugs. >> >> Try upgrading to MAKER 2.26 I have a statement in that that will make >>sure >> it won't get set wrong. Otherwise I can give you instructions on >>editing >> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >> modules. >> >> Thanks, >> Carson >> >> >> On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> Really weird - as soon as I add the debug flag - no more segfault... >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> >>> >>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>> I'm going to write you a test script that just creates a fasta index >>>>to >>>> see if it seg faults. I'll send it to you later. For now could you >>>>run >>>> maker with the --debug flag set, apture the STDERR and send it to me. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> Darn - did both, but unfortunately still segfaults... >>>>> >>>>> Best regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>> output >>>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>>> CPAN. >>>>>> >>>>>> --Carson >>>>>> >>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> 1.006901 >>>>>>> >>>>>>> Regards! >>>>>>> >>>>>>> Fourie >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>> -- >>>>>>> -------------- >>>>>>> Prof Fourie Joubert >>>>>>> Bioinformatics and Computational Biology Unit >>>>>>> Department of Biochemistry >>>>>>> University of Pretoria >>>>>>> fourie.joubert at up.ac.za >>>>>>> http://www.bi.up.ac.za >>>>>>> Tel. +27-12-420-5825 >>>>>>> Fax. +27-12-420-5800 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>> -- >>>>>>> - >>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>> refer >>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>> details. >>>>>>> >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Thu Oct 25 00:51:05 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Thu, 25 Oct 2012 08:51:05 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5088E159.3050502@up.ac.za> Hi Carson All happy now - much appreciated! Best regards! Fourie On 10/24/2012 05:50 PM, Carson Holt wrote: > Find these lines in these files in MAKER > > .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > > > And the Bio::DB::Fasta file in BioPerl. Use this command to identify the > location of Bio::DB::Fasta --> > perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' > > .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > > > Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); > > Use an editor like emacs or vi to make the changes. > > Thanks, > Carson > > > > On 12-10-24 11:19 AM, "Fourie Joubert" wrote: > >> Hi Carson >> >> We are indeed running 2.26. >> >> Instructions on editing the statements would be great. >> >> Many thanks for all the help, and sorry for the bother! >> >> Best regards! >> >> Fourie >> >> >> >> >> On 10/24/2012 04:56 PM, Carson Holt wrote: >>> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >>> variable. It's set by both BioPerl and a couple of MAKER modules. If it >>> gets set twice, sometimes you get bugs. >>> >>> Try upgrading to MAKER 2.26 I have a statement in that that will make >>> sure >>> it won't get set wrong. Otherwise I can give you instructions on >>> editing >>> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >>> modules. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> Really weird - as soon as I add the debug flag - no more segfault... >>>> >>>> Best regards! >>>> >>>> Fourie >>>> >>>> >>>> >>>> >>>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>>> I'm going to write you a test script that just creates a fasta index >>>>> to >>>>> see if it seg faults. I'll send it to you later. For now could you >>>>> run >>>>> maker with the --debug flag set, apture the STDERR and send it to me. >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> Darn - did both, but unfortunately still segfaults... >>>>>> >>>>>> Best regards! >>>>>> >>>>>> Fourie >>>>>> >>>>>> >>>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>>> output >>>>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>>>> CPAN. >>>>>>> >>>>>>> --Carson >>>>>>> >>>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>>> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> 1.006901 >>>>>>>> >>>>>>>> Regards! >>>>>>>> >>>>>>>> Fourie >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>>> -- >>>>>>>> -------------- >>>>>>>> Prof Fourie Joubert >>>>>>>> Bioinformatics and Computational Biology Unit >>>>>>>> Department of Biochemistry >>>>>>>> University of Pretoria >>>>>>>> fourie.joubert at up.ac.za >>>>>>>> http://www.bi.up.ac.za >>>>>>>> Tel. +27-12-420-5825 >>>>>>>> Fax. +27-12-420-5800 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -------------------------------------------------------------------- >>>>>>>> -- >>>>>>>> -- >>>>>>>> - >>>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>>> refer >>>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>>> details. >>>>>>>> >>>>>> -- >>>>>> -------------- >>>>>> Prof Fourie Joubert >>>>>> Bioinformatics and Computational Biology Unit >>>>>> Department of Biochemistry >>>>>> University of Pretoria >>>>>> fourie.joubert at up.ac.za >>>>>> http://www.bi.up.ac.za >>>>>> Tel. +27-12-420-5825 >>>>>> Fax. +27-12-420-5800 >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> -- >>>>>> - >>>>>> This message and attachments are subject to a disclaimer. Please >>>>>> refer >>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>> details. >>>>>> >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Thu Oct 25 06:57:31 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 25 Oct 2012 08:57:31 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5088E159.3050502@up.ac.za> Message-ID: Glad I could help. Thanks, Carson On 12-10-25 2:51 AM, "Fourie Joubert" wrote: >Hi Carson > >All happy now - much appreciated! > >Best regards! > >Fourie > >On 10/24/2012 05:50 PM, Carson Holt wrote: >> Find these lines in these files in MAKER >> >> .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> >> >> And the Bio::DB::Fasta file in BioPerl. Use this command to identify >>the >> location of Bio::DB::Fasta --> >> perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' >> >> .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> >> >> Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); >> >> Use an editor like emacs or vi to make the changes. >> >> Thanks, >> Carson >> >> >> >> On 12-10-24 11:19 AM, "Fourie Joubert" wrote: >> >>> Hi Carson >>> >>> We are indeed running 2.26. >>> >>> Instructions on editing the statements would be great. >>> >>> Many thanks for all the help, and sorry for the bother! >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> >>> >>> On 10/24/2012 04:56 PM, Carson Holt wrote: >>>> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >>>> variable. It's set by both BioPerl and a couple of MAKER modules. If >>>>it >>>> gets set twice, sometimes you get bugs. >>>> >>>> Try upgrading to MAKER 2.26 I have a statement in that that will make >>>> sure >>>> it won't get set wrong. Otherwise I can give you instructions on >>>> editing >>>> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >>>> modules. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-24 10:41 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> Really weird - as soon as I add the debug flag - no more segfault... >>>>> >>>>> Best regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> >>>>> >>>>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>>>> I'm going to write you a test script that just creates a fasta index >>>>>> to >>>>>> see if it seg faults. I'll send it to you later. For now could you >>>>>> run >>>>>> maker with the --debug flag set, apture the STDERR and send it to >>>>>>me. >>>>>> >>>>>> Thanks, >>>>>> Carson >>>>>> >>>>>> >>>>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> Darn - did both, but unfortunately still segfaults... >>>>>>> >>>>>>> Best regards! >>>>>>> >>>>>>> Fourie >>>>>>> >>>>>>> >>>>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>>>> output >>>>>>>> folder before restarting. Perhaps also try reinstalling DB_File >>>>>>>>via >>>>>>>> CPAN. >>>>>>>> >>>>>>>> --Carson >>>>>>>> >>>>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> 1.006901 >>>>>>>>> >>>>>>>>> Regards! >>>>>>>>> >>>>>>>>> Fourie >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>>>> -- >>>>>>>>> -------------- >>>>>>>>> Prof Fourie Joubert >>>>>>>>> Bioinformatics and Computational Biology Unit >>>>>>>>> Department of Biochemistry >>>>>>>>> University of Pretoria >>>>>>>>> fourie.joubert at up.ac.za >>>>>>>>> http://www.bi.up.ac.za >>>>>>>>> Tel. +27-12-420-5825 >>>>>>>>> Fax. +27-12-420-5800 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>------------------------------------------------------------------ >>>>>>>>>-- >>>>>>>>> -- >>>>>>>>> -- >>>>>>>>> - >>>>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>>>> refer >>>>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>>>> details. >>>>>>>>> >>>>>>> -- >>>>>>> -------------- >>>>>>> Prof Fourie Joubert >>>>>>> Bioinformatics and Computational Biology Unit >>>>>>> Department of Biochemistry >>>>>>> University of Pretoria >>>>>>> fourie.joubert at up.ac.za >>>>>>> http://www.bi.up.ac.za >>>>>>> Tel. +27-12-420-5825 >>>>>>> Fax. +27-12-420-5800 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>> -- >>>>>>> - >>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>> refer >>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>> details. >>>>>>> >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From daniel.standage at gmail.com Thu Oct 25 13:30:44 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 25 Oct 2012 15:30:44 -0400 Subject: [maker-devel] Strange error at blastn step Message-ID: Greetings! I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. #--------- command -------------# Widget::blastn: /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn #-------------------------------# deleted:0 hits ERROR: Could not obtain lock to format database FATAL ERROR ERROR: Failed while doing blastn of ESTs!! ERROR: Chunk failed at level 8 !! FAILED CONTIG:scaffold_866 Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 11:52:43 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 13:52:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx #-------------------------------# deleted:-10 hits formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx #-------------------------------# deleted:-6 hits WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. stop here:comp59088_c1_seq7 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:scaffold_0 --Next Contig-- #--------------------------------------------------------------------- Now starting the contig!! SeqID: scaffold_1 Length: 5805686 #--------------------------------------------------------------------- My first thought based on the message is that *blastdbcmd* could not find the sequence in the database. I verified this was the case--I could not extract sequence *comp59088_c1_seq7* from the database Maker had created under /tmp. However, after removing the index files and re-running * makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully extracted the sequence. I was initially happy with this finding, but upon closer inspection it looks like Maker does not use *blastdbcmd* to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a > pretty short contig (about 3kb). However, there seems to be a hiccup during > the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db > /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 > -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 > -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 > -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp > 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis > -out > /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff > old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > > Several blastn steps appeared to have completed successfully to this one > failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From myandell at genetics.utah.edu Fri Oct 26 12:02:29 2012 From: myandell at genetics.utah.edu (Mark Yandell) Date: Fri, 26 Oct 2012 18:02:29 +0000 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: , Message-ID: <7A60AB257EFF2B48B1F4C814817EA05331BA6EC8@mxb1.hg.genetics.utah.edu> Hi Daniel, I think its your fasta-file '> comp59088_c1_seq' note the space between the chevron and the id. This isn't allowed by the fasta format. cheers, --mark Mark Yandell Professor of Human Genetics H.A. & Edna Benning Presidential Endowed Chair Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 ph:801-587-7707 ________________________________________ From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Daniel Standage [daniel.standage at gmail.com] Sent: Friday, October 26, 2012 11:52 AM To: Maker Mailing List Subject: Re: [maker-devel] Strange error at blastn step I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx #-------------------------------# deleted:-10 hits formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx #-------------------------------# deleted:-6 hits WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. stop here:comp59088_c1_seq7 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:scaffold_0 --Next Contig-- #--------------------------------------------------------------------- Now starting the contig!! SeqID: scaffold_1 Length: 5805686 #--------------------------------------------------------------------- My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage > wrote: Greetings! I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. #--------- command -------------# Widget::blastn: /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn #-------------------------------# deleted:0 hits ERROR: Could not obtain lock to format database FATAL ERROR ERROR: Failed while doing blastn of ESTs!! ERROR: Chunk failed at level 8 !! FAILED CONTIG:scaffold_866 Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University From carsonhh at gmail.com Fri Oct 26 12:09:39 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:09:39 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Check to see where /tmp is located? Some clusters have it set up as a tmpfs directory and I have had problems with fasta indexes running from tmpfs mounts in the past. --Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:05 PM To: Carson Holt Subject: Re: [maker-devel] Strange error at blastn step The maker working directory is in a cluster environment with shared scratch space (I'm guessing NFS-mounted). I didn't change the temp directory setting, so it should be the local default (/tmp). I'll give the dev version a shot. Thanks. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > Could you try this development version and tell me if the error still happens? > > Use this command to download --> > <> > > Username: <> > Password: <> > > Are you running in an NFS mounted directory or are you resetting TMP to a > different location? > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 1:52 PM > To: Maker Mailing List > Subject: Re: [maker-devel] Strange error at blastn step > > I have since installed Maker on a different machine and tried it out. The test > run completed successfully, but as I commenced with the full genome > annotation, I have noticed the following error popping up frequently. > >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions >> 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes >> -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker >> .output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0 >> .0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi. >> 10.8.blastx >> #-------------------------------# >> deleted:-10 hits >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions >> 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes >> -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker >> .output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0 >> .0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi. >> 10.9.blastx >> #-------------------------------# >> deleted:-6 hits >> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >> stop here:comp59088_c1_seq7 >> ERROR: Fasta index error >> >> FATAL ERROR >> ERROR: Failed while polishig ESTs!! >> >> ERROR: Chunk failed at level 14 >> !! >> FAILED CONTIG:scaffold_0 >> >> >> >> >> --Next Contig-- >> >> #--------------------------------------------------------------------- >> Now starting the contig!! >> SeqID: scaffold_1 >> Length: 5805686 >> #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the > sequence in the database. I verified this was the case--I could not extract > sequence comp59088_c1_seq7 from the database Maker had created under /tmp. > However, after removing the index files and re-running makeblastdb with the > -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks > like Maker does not use blastdbcmd to extract sequences, but rather its own > internal code. Therefore I'm still not sure where the problem is and how I > might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage > wrote: >> Greetings! >> >> I am doing a test run of my Maker setup on a new machine, annotating a pretty >> short contig (about 3kb). However, there seems to be a hiccup during the >> blastn stage. This is the terminal message. >> >>> #--------- command -------------# >>> Widget::blastn: >>> /share/home/01854/standage/local/bin/blastn -db >>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta >>> .mpi.10.7 -query >>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments >>> 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 >>> -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 >>> -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out >>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.out >>> put/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.P >>> dom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmom >>> atic%2Efasta.mpi.10.7.blastn >>> #-------------------------------# >>> deleted:0 hits >>> ERROR: Could not obtain lock to format database >>> >>> >>> FATAL ERROR >>> ERROR: Failed while doing blastn of ESTs!! >>> >>> ERROR: Chunk failed at level 8 >>> !! >>> FAILED CONTIG:scaffold_866 >> >> Several blastn steps appeared to have completed successfully to this one >> failing. Any ideas what could be causing this? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/mak > er-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:12:38 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:12:38 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: It looks like /tmp is indeed being used: the files I played with were under */tmp/maker_1YQF9o*. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a > tmpfs directory and I have had problems with fasta indexes running from > tmpfs mounts in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared > scratch space (I'm guessing NFS-mounted). I didn't change the temp > directory setting, so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > >> Could you try this development version and tell me if the error still >> happens? >> >> Use this command to download --> >> <> >> >> Username: <> >> Password: <> >> >> Are you running in an NFS mounted directory or are you resetting TMP to a >> different location? >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 1:52 PM >> To: Maker Mailing List >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I have since installed Maker on a different machine and tried it out. The >> test run completed successfully, but as I commenced with the full genome >> annotation, I have noticed the following error popping up frequently. >> >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >> #-------------------------------# >> deleted:-10 hits >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >> #-------------------------------# >> deleted:-6 hits >> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >> stop here:comp59088_c1_seq7 >> ERROR: Fasta index error >> >> FATAL ERROR >> ERROR: Failed while polishig ESTs!! >> >> ERROR: Chunk failed at level 14 >> !! >> FAILED CONTIG:scaffold_0 >> >> >> >> >> --Next Contig-- >> >> #--------------------------------------------------------------------- >> Now starting the contig!! >> SeqID: scaffold_1 >> Length: 5805686 >> #--------------------------------------------------------------------- >> >> >> My first thought based on the message is that *blastdbcmd* could not >> find the sequence in the database. I verified this was the case--I could >> not extract sequence *comp59088_c1_seq7* from the database Maker had >> created under /tmp. However, after removing the index files and re-running >> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >> extracted the sequence. >> >> I was initially happy with this finding, but upon closer inspection it >> looks like Maker does not use *blastdbcmd* to extract sequences, but >> rather its own internal code. Therefore I'm still not sure where the >> problem is and how I might fix it. Any insights? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >> daniel.standage at gmail.com> wrote: >> >>> Greetings! >>> >>> I am doing a test run of my Maker setup on a new machine, annotating a >>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>> the blastn stage. This is the terminal message. >>> >>> #--------- command -------------# >>> Widget::blastn: >>> /share/home/01854/standage/local/bin/blastn -db >>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>> -out >>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>> #-------------------------------# >>> deleted:0 hits >>> ERROR: Could not obtain lock to format database >>> >>> >>> FATAL ERROR >>> ERROR: Failed while doing blastn of ESTs!! >>> >>> ERROR: Chunk failed at level 8 >>> !! >>> FAILED CONTIG:scaffold_866 >>> >>> >>> Several blastn steps appeared to have completed successfully to this one >>> failing. Any ideas what could be causing this? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:14:29 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:14:29 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: The command 'df /tmp' will tell you whether /tmp is a tmpfs mount Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:12 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step It looks like /tmp is indeed being used: the files I played with were under /tmp/maker_1YQF9o. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a tmpfs > directory and I have had problems with fasta indexes running from tmpfs mounts > in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared scratch > space (I'm guessing NFS-mounted). I didn't change the temp directory setting, > so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >> Could you try this development version and tell me if the error still >> happens? >> >> Use this command to download --> >> <> >> >> Username: <> >> Password: <> >> >> Are you running in an NFS mounted directory or are you resetting TMP to a >> different location? >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 1:52 PM >> To: Maker Mailing List >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I have since installed Maker on a different machine and tried it out. The >> test run completed successfully, but as I commenced with the full genome >> annotation, I have noticed the following error popping up frequently. >> >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.make >>> r.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold >>> _0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.m >>> pi.10.8.blastx >>> #-------------------------------# >>> deleted:-10 hits >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.make >>> r.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold >>> _0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.m >>> pi.10.9.blastx >>> #-------------------------------# >>> deleted:-6 hits >>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>> stop here:comp59088_c1_seq7 >>> ERROR: Fasta index error >>> >>> FATAL ERROR >>> ERROR: Failed while polishig ESTs!! >>> >>> ERROR: Chunk failed at level 14 >>> !! >>> FAILED CONTIG:scaffold_0 >>> >>> >>> >>> >>> --Next Contig-- >>> >>> #--------------------------------------------------------------------- >>> Now starting the contig!! >>> SeqID: scaffold_1 >>> Length: 5805686 >>> #--------------------------------------------------------------------- >> >> My first thought based on the message is that blastdbcmd could not find the >> sequence in the database. I verified this was the case--I could not extract >> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >> However, after removing the index files and re-running makeblastdb with the >> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >> >> I was initially happy with this finding, but upon closer inspection it looks >> like Maker does not use blastdbcmd to extract sequences, but rather its own >> internal code. Therefore I'm still not sure where the problem is and how I >> might fix it. Any insights? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >> wrote: >>> Greetings! >>> >>> I am doing a test run of my Maker setup on a new machine, annotating a >>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>> the blastn stage. This is the terminal message. >>> >>>> #--------- command -------------# >>>> Widget::blastn: >>>> /share/home/01854/standage/local/bin/blastn -db >>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efast >>>> a.mpi.10.7 -query >>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>> -show_gis -out >>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.ou >>>> tput/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0 >>>> .Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrim >>>> momatic%2Efasta.mpi.10.7.blastn >>>> #-------------------------------# >>>> deleted:0 hits >>>> ERROR: Could not obtain lock to format database >>>> >>>> >>>> FATAL ERROR >>>> ERROR: Failed while doing blastn of ESTs!! >>>> >>>> ERROR: Chunk failed at level 8 >>>> !! >>>> FAILED CONTIG:scaffold_866 >>> >>> Several blastn steps appeared to have completed successfully to this one >>> failing. Any ideas what could be causing this? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ma >> ker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry.moore at genetics.utah.edu Fri Oct 26 12:19:06 2012 From: barry.moore at genetics.utah.edu (Barry Moore) Date: Fri, 26 Oct 2012 12:19:06 -0600 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> Hi Daniel, What version or revision of MAKER are you running. This sounds like something we were seeing here last night. We traced it as far as what appeared to be soft links in /tmp being set incorrectly. The FastaDB objects had pointers to fasta files in the void for the correct fasta file, but their dir attribute pointed to a /tmp directory in where there were soft links to another (incorrect) fasta file with it's index. It would look appeared that it was looking in the /tmp index (which pointed to an incorrect fasta file) and when it failed to find what it was looking for it would re-index the fasta file in the void (the correct one) and then look again in the index in tmp. Don't know if that helps, but the errors look similar and that's as far as I got with our error here? This was on one of Mike's annotation projects, so I don't know for sure what revision he was running, but I think it was the latest. B On Oct 26, 2012, at 11:52 AM, Daniel Standage wrote: > I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Barry Moore Research Scientist Dept. of Human Genetics University of Utah Salt Lake City, UT 84112 -------------------------------------------- (801) 585-3543 -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:19:07 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:19:07 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Unfortunately, the job is no longer running and as a result I cannot connect to the compute nodes as I could while it was running. On the interactive node, it looks like it's real disk, although it looks like there are some tmpfs mounts. [dstandag at mason src] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 462824304 180235660 259078476 42% /tmp [dstandag at mason src] df Filesystem 1K-blocks Used Available Use% Mounted on login_x86_64 16497564 3077352 13420212 19% / tmpfs 16497564 0 16497564 0% /dev/shm tmpfs 10240 0 10240 0% /var/tmp /dev/sdb2 462824304 180235660 259078476 42% /tmp AFS 9000000 0 9000000 0% /afs bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs ... ... I'll see if I can launch another short job and verify this on the compute nodes. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > The command 'df /tmp' will tell you whether /tmp is a tmpfs mount > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:12 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > It looks like /tmp is indeed being used: the files I played with were > under */tmp/maker_1YQF9o*. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > >> Check to see where /tmp is located? Some clusters have it set up as a >> tmpfs directory and I have had problems with fasta indexes running from >> tmpfs mounts in the past. >> >> --Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:05 PM >> To: Carson Holt >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> The maker working directory is in a cluster environment with shared >> scratch space (I'm guessing NFS-mounted). I didn't change the temp >> directory setting, so it should be the local default (/tmp). >> >> I'll give the dev version a shot. Thanks. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >> >>> Could you try this development version and tell me if the error still >>> happens? >>> >>> Use this command to download --> >>> <> >>> >>> Username: <> >>> Password: <> >>> >>> Are you running in an NFS mounted directory or are you resetting TMP to >>> a different location? >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 1:52 PM >>> To: Maker Mailing List >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> I have since installed Maker on a different machine and tried it out. >>> The test run completed successfully, but as I commenced with the full >>> genome annotation, I have noticed the following error popping up frequently. >>> >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>> #-------------------------------# >>> deleted:-10 hits >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>> #-------------------------------# >>> deleted:-6 hits >>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>> stop here:comp59088_c1_seq7 >>> ERROR: Fasta index error >>> >>> FATAL ERROR >>> ERROR: Failed while polishig ESTs!! >>> >>> ERROR: Chunk failed at level 14 >>> !! >>> FAILED CONTIG:scaffold_0 >>> >>> >>> >>> >>> --Next Contig-- >>> >>> #--------------------------------------------------------------------- >>> Now starting the contig!! >>> SeqID: scaffold_1 >>> Length: 5805686 >>> #--------------------------------------------------------------------- >>> >>> >>> My first thought based on the message is that *blastdbcmd* could not >>> find the sequence in the database. I verified this was the case--I could >>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>> created under /tmp. However, after removing the index files and re-running >>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>> extracted the sequence. >>> >>> I was initially happy with this finding, but upon closer inspection it >>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>> rather its own internal code. Therefore I'm still not sure where the >>> problem is and how I might fix it. Any insights? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>> daniel.standage at gmail.com> wrote: >>> >>>> Greetings! >>>> >>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>> the blastn stage. This is the terminal message. >>>> >>>> #--------- command -------------# >>>> Widget::blastn: >>>> /share/home/01854/standage/local/bin/blastn -db >>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>>> -out >>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>> #-------------------------------# >>>> deleted:0 hits >>>> ERROR: Could not obtain lock to format database >>>> >>>> >>>> FATAL ERROR >>>> ERROR: Failed while doing blastn of ESTs!! >>>> >>>> ERROR: Chunk failed at level 8 >>>> !! >>>> FAILED CONTIG:scaffold_866 >>>> >>>> >>>> Several blastn steps appeared to have completed successfully to this >>>> one failing. Any ideas what could be causing this? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:20:55 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:20:55 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> References: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> Message-ID: Running 2.10. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:19 PM, Barry Moore wrote: > Hi Daniel, > > What version or revision of MAKER are you running. This sounds like > something we were seeing here last night. We traced it as far as what > appeared to be soft links in /tmp being set incorrectly. The FastaDB > objects had pointers to fasta files in the void for the correct fasta file, > but their dir attribute pointed to a /tmp directory in where there were > soft links to another (incorrect) fasta file with it's index. It would > look appeared that it was looking in the /tmp index (which pointed to an > incorrect fasta file) and when it failed to find what it was looking for it > would re-index the fasta file in the void (the correct one) and then look > again in the index in tmp. Don't know if that helps, but the errors look > similar and that's as far as I got with our error here? This was on one of > Mike's annotation projects, so I don't know for sure what revision he was > running, but I think it was the latest. > > B > > On Oct 26, 2012, at 11:52 AM, Daniel Standage wrote: > > I have since installed Maker on a different machine and tried it out. The > test run completed successfully, but as I commenced with the full genome > annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query > /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 > -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 > -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query > /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 > -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 > -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > > My first thought based on the message is that *blastdbcmd* could not find > the sequence in the database. I verified this was the case--I could not > extract sequence *comp59088_c1_seq7* from the database Maker had created > under /tmp. However, after removing the index files and re-running * > makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully > extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it > looks like Maker does not use *blastdbcmd* to extract sequences, but > rather its own internal code. Therefore I'm still not sure where the > problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < > daniel.standage at gmail.com> wrote: > >> Greetings! >> >> I am doing a test run of my Maker setup on a new machine, annotating a >> pretty short contig (about 3kb). However, there seems to be a hiccup during >> the blastn stage. This is the terminal message. >> >> #--------- command -------------# >> Widget::blastn: >> /share/home/01854/standage/local/bin/blastn -db >> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >> -out >> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >> #-------------------------------# >> deleted:0 hits >> ERROR: Could not obtain lock to format database >> >> >> FATAL ERROR >> ERROR: Failed while doing blastn of ESTs!! >> >> ERROR: Chunk failed at level 8 >> !! >> FAILED CONTIG:scaffold_866 >> >> >> Several blastn steps appeared to have completed successfully to this one >> failing. Any ideas what could be causing this? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > Barry Moore > Research Scientist > Dept. of Human Genetics > University of Utah > Salt Lake City, UT 84112 > -------------------------------------------- > (801) 585-3543 > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:24:43 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:24:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Ok. Try the developer release and see if it still happens. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:19 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Unfortunately, the job is no longer running and as a result I cannot connect to the compute nodes as I could while it was running. On the interactive node, it looks like it's real disk, although it looks like there are some tmpfs mounts. [dstandag at mason src] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 462824304 180235660 259078476 42% /tmp [dstandag at mason src] df Filesystem 1K-blocks Used Available Use% Mounted on login_x86_64 16497564 3077352 13420212 19% / tmpfs 16497564 0 16497564 0% /dev/shm tmpfs 10240 0 10240 0% /var/tmp /dev/sdb2 462824304 180235660 259078476 42% /tmp AFS 9000000 0 9000000 0% /afs bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs ... ... I'll see if I can launch another short job and verify this on the compute nodes. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > The command 'df /tmp' will tell you whether /tmp is a tmpfs mount > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:12 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > It looks like /tmp is indeed being used: the files I played with were under > /tmp/maker_1YQF9o. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >> directory and I have had problems with fasta indexes running from tmpfs >> mounts in the past. >> >> --Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:05 PM >> To: Carson Holt >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> The maker working directory is in a cluster environment with shared scratch >> space (I'm guessing NFS-mounted). I didn't change the temp directory setting, >> so it should be the local default (/tmp). >> >> I'll give the dev version a shot. Thanks. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>> Could you try this development version and tell me if the error still >>> happens? >>> >>> Use this command to download --> >>> <> >>> >>> Username: <> >>> Password: <> >>> >>> Are you running in an NFS mounted directory or are you resetting TMP to a >>> different location? >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 1:52 PM >>> To: Maker Mailing List >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> I have since installed Maker on a different machine and tried it out. The >>> test run completed successfully, but as I commenced with the full genome >>> annotation, I have noticed the following error popping up frequently. >>> >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.mak >>>> er.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffo >>>> ld_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efa >>>> a.mpi.10.8.blastx >>>> #-------------------------------# >>>> deleted:-10 hits >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.mak >>>> er.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffo >>>> ld_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efa >>>> a.mpi.10.9.blastx >>>> #-------------------------------# >>>> deleted:-6 hits >>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>> stop here:comp59088_c1_seq7 >>>> ERROR: Fasta index error >>>> >>>> FATAL ERROR >>>> ERROR: Failed while polishig ESTs!! >>>> >>>> ERROR: Chunk failed at level 14 >>>> !! >>>> FAILED CONTIG:scaffold_0 >>>> >>>> >>>> >>>> >>>> --Next Contig-- >>>> >>>> #--------------------------------------------------------------------- >>>> Now starting the contig!! >>>> SeqID: scaffold_1 >>>> Length: 5805686 >>>> #--------------------------------------------------------------------- >>> >>> My first thought based on the message is that blastdbcmd could not find the >>> sequence in the database. I verified this was the case--I could not extract >>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>> However, after removing the index files and re-running makeblastdb with the >>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>> >>> I was initially happy with this finding, but upon closer inspection it looks >>> like Maker does not use blastdbcmd to extract sequences, but rather its own >>> internal code. Therefore I'm still not sure where the problem is and how I >>> might fix it. Any insights? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>> wrote: >>>> Greetings! >>>> >>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>> the blastn stage. This is the terminal message. >>>> >>>>> #--------- command -------------# >>>>> Widget::blastn: >>>>> /share/home/01854/standage/local/bin/blastn -db >>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efas >>>>> ta.mpi.10.7 -query >>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>> -show_gis -out >>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.o >>>>> utput/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866 >>>>> .0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ET >>>>> rimmomatic%2Efasta.mpi.10.7.blastn >>>>> #-------------------------------# >>>>> deleted:0 hits >>>>> ERROR: Could not obtain lock to format database >>>>> >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while doing blastn of ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 8 >>>>> !! >>>>> FAILED CONTIG:scaffold_866 >>>> >>>> Several blastn steps appeared to have completed successfully to this one >>>> failing. Any ideas what could be causing this? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m >>> aker-devel_yandell-lab.org >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:29:18 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:29:18 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot > connect to the compute nodes as I could while it was running. On the > interactive node, it looks like it's real disk, although it looks like > there are some tmpfs mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were >> under */tmp/maker_1YQF9o*. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >> >>> Check to see where /tmp is located? Some clusters have it set up as a >>> tmpfs directory and I have had problems with fasta indexes running from >>> tmpfs mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared >>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>> directory setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>> >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to >>>> a different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. >>>> The test run completed successfully, but as I commenced with the full >>>> genome annotation, I have noticed the following error popping up frequently. >>>> >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>> #-------------------------------# >>>> deleted:-10 hits >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>> #-------------------------------# >>>> deleted:-6 hits >>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>> stop here:comp59088_c1_seq7 >>>> ERROR: Fasta index error >>>> >>>> FATAL ERROR >>>> ERROR: Failed while polishig ESTs!! >>>> >>>> ERROR: Chunk failed at level 14 >>>> !! >>>> FAILED CONTIG:scaffold_0 >>>> >>>> >>>> >>>> >>>> --Next Contig-- >>>> >>>> #--------------------------------------------------------------------- >>>> Now starting the contig!! >>>> SeqID: scaffold_1 >>>> Length: 5805686 >>>> #--------------------------------------------------------------------- >>>> >>>> >>>> My first thought based on the message is that *blastdbcmd* could not >>>> find the sequence in the database. I verified this was the case--I could >>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>> created under /tmp. However, after removing the index files and re-running >>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>> extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>> rather its own internal code. Therefore I'm still not sure where the >>>> problem is and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>> daniel.standage at gmail.com> wrote: >>>> >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>>> the blastn stage. This is the terminal message. >>>>> >>>>> #--------- command -------------# >>>>> Widget::blastn: >>>>> /share/home/01854/standage/local/bin/blastn -db >>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>>>> -out >>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>> #-------------------------------# >>>>> deleted:0 hits >>>>> ERROR: Could not obtain lock to format database >>>>> >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while doing blastn of ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 8 >>>>> !! >>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> >>>>> Several blastn steps appeared to have completed successfully to this >>>>> one failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>> _______________________________________________ maker-devel mailing >>>> list maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:32:35 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:32:35 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: If running './Build install_deps' to get the prereqs automatically, you can say yes to the local version question if you want those prereqs to be installed just for MAKER rather than globally to whatever perl you are using. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:29 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot connect > to the compute nodes as I could while it was running. On the interactive node, > it looks like it's real disk, although it looks like there are some tmpfs > mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were under >> /tmp/maker_1YQF9o. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >>> directory and I have had problems with fasta indexes running from tmpfs >>> mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared scratch >>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>> setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>> different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. The >>>> test run completed successfully, but as I commenced with the full genome >>>> annotation, I have noticed the following error popping up frequently. >>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>> >>>> My first thought based on the message is that blastdbcmd could not find the >>>> sequence in the database. I verified this was the case--I could not extract >>>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>>> However, after removing the index files and re-running makeblastdb with the >>>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>> its own internal code. Therefore I'm still not sure where the problem is >>>> and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>> wrote: >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>> during the blastn stage. This is the terminal message. >>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efa >>>>>> sta.mpi.10.7 -query >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>> -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker. >>>>>> output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_8 >>>>>> 66.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity% >>>>>> 2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> Several blastn steps appeared to have completed successfully to this one >>>>> failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>> >>>> _______________________________________________ maker-devel mailing list >>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ >>>> maker-devel_yandell-lab.org >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:47:43 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:47:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I've got a test run (with the dev version) waiting in the queue. But just to be clear, if the temp directory is indeed the problem, I'm assuming that would that be fixed by setting a different temp directory on the same disk as the scratch disk? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:32 PM, Carson Holt wrote: > If running './Build install_deps' to get the prereqs automatically, you > can say yes to the local version question if you want those prereqs to be > installed just for MAKER rather than globally to whatever perl you are > using. > > Thanks, > Carson > > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot >> connect to the compute nodes as I could while it was running. On the >> interactive node, it looks like it's real disk, although it looks like >> there are some tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were >>> under */tmp/maker_1YQF9o*. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared >>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>> directory setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>> to a different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. >>>>> The test run completed successfully, but as I commenced with the full >>>>> genome annotation, I have noticed the following error popping up frequently. >>>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>>> >>>>> >>>>> My first thought based on the message is that *blastdbcmd* could not >>>>> find the sequence in the database. I verified this was the case--I could >>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>> created under /tmp. However, after removing the index files and re-running >>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>>> rather its own internal code. Therefore I'm still not sure where the >>>>> problem is and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>> daniel.standage at gmail.com> wrote: >>>>> >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>> true -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this >>>>>> one failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>> _______________________________________________ maker-devel mailing >>>>> list maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:56:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:56:30 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Yes. That is correct. Also I've made one more update to the development release with respect to indexes for your test run. Could you run 'svn update' inside the devel maker directory. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:47 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step I've got a test run (with the dev version) waiting in the queue. But just to be clear, if the temp directory is indeed the problem, I'm assuming that would that be fixed by setting a different temp directory on the same disk as the scratch disk? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:32 PM, Carson Holt wrote: > If running './Build install_deps' to get the prereqs automatically, you can > say yes to the local version question if you want those prereqs to be > installed just for MAKER rather than globally to whatever perl you are using. > > Thanks, > Carson > > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot connect >> to the compute nodes as I could while it was running. On the interactive >> node, it looks like it's real disk, although it looks like there are some >> tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were under >>> /tmp/maker_1YQF9o. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared scratch >>>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>>> setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>>> different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. The >>>>> test run completed successfully, but as I commenced with the full genome >>>>> annotation, I have noticed the following error popping up frequently. >>>>> >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.m >>>>>> aker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/sc >>>>>> affold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E4 >>>>>> 7%2Efaa.mpi.10.8.blastx >>>>>> #-------------------------------# >>>>>> deleted:-10 hits >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.m >>>>>> aker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/sc >>>>>> affold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E4 >>>>>> 7%2Efaa.mpi.10.9.blastx >>>>>> #-------------------------------# >>>>>> deleted:-6 hits >>>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>>> stop here:comp59088_c1_seq7 >>>>>> ERROR: Fasta index error >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while polishig ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 14 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --Next Contig-- >>>>>> >>>>>> #--------------------------------------------------------------------- >>>>>> Now starting the contig!! >>>>>> SeqID: scaffold_1 >>>>>> Length: 5805686 >>>>>> #--------------------------------------------------------------------- >>>>> >>>>> My first thought based on the message is that blastdbcmd could not find >>>>> the sequence in the database. I verified this was the case--I could not >>>>> extract sequence comp59088_c1_seq7 from the database Maker had created >>>>> under /tmp. However, after removing the index files and re-running >>>>> makeblastdb with the -parse_seqids option set, blastdbcmd successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>>> its own internal code. Therefore I'm still not sure where the problem is >>>>> and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>>> wrote: >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>>> #--------- command -------------# >>>>>>> Widget::blastn: >>>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Ef >>>>>>> asta.mpi.10.7 -query >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>>> -show_gis -out >>>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker >>>>>>> .output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold >>>>>>> _866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrini >>>>>>> ty%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>>> #-------------------------------# >>>>>>> deleted:0 hits >>>>>>> ERROR: Could not obtain lock to format database >>>>>>> >>>>>>> >>>>>>> FATAL ERROR >>>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>>> >>>>>>> ERROR: Chunk failed at level 8 >>>>>>> !! >>>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this one >>>>>> failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>> >>>>> _______________________________________________ maker-devel mailing list >>>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo >>>>> /maker-devel_yandell-lab.org >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason.stajich at gmail.com Fri Oct 26 14:49:57 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Fri, 26 Oct 2012 13:49:57 -0700 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Right but is tmp on a disk or a tmpfs df -h /tmp would give you the first hint at whether it is a local drive or something else. On Oct 26, 2012, at 11:12 AM, Daniel Standage wrote: > It looks like /tmp is indeed being used: the files I played with were under /tmp/maker_1YQF9o. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a tmpfs directory and I have had problems with fasta indexes running from tmpfs mounts in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared scratch space (I'm guessing NFS-mounted). I didn't change the temp directory setting, so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > Could you try this development version and tell me if the error still happens? > > Use this command to download --> > <> > > Username: <> > Password: <> > > Are you running in an NFS mounted directory or are you resetting TMP to a different location? > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 1:52 PM > To: Maker Mailing List > Subject: Re: [maker-devel] Strange error at blastn step > > I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 16:59:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 18:59:30 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: From: Carson Holt Date: Friday, 26 October, 2012 6:10 PM To: Daniel Standage Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step I've been going over the indexing code using different scenarios and I may have isolated a candidate for what is causing this. Could you do one more 'svn update' inside the maker devel directory before running a test job? Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:29 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot connect > to the compute nodes as I could while it was running. On the interactive node, > it looks like it's real disk, although it looks like there are some tmpfs > mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were under >> /tmp/maker_1YQF9o. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >>> directory and I have had problems with fasta indexes running from tmpfs >>> mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared scratch >>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>> setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>> different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. The >>>> test run completed successfully, but as I commenced with the full genome >>>> annotation, I have noticed the following error popping up frequently. >>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>> >>>> My first thought based on the message is that blastdbcmd could not find the >>>> sequence in the database. I verified this was the case--I could not extract >>>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>>> However, after removing the index files and re-running makeblastdb with the >>>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>> its own internal code. Therefore I'm still not sure where the problem is >>>> and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>> wrote: >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>> during the blastn stage. This is the terminal message. >>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efa >>>>>> sta.mpi.10.7 -query >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>> -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker. >>>>>> output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_8 >>>>>> 66.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity% >>>>>> 2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> Several blastn steps appeared to have completed successfully to this one >>>>> failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>> >>>> _______________________________________________ maker-devel mailing list >>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ >>>> maker-devel_yandell-lab.org >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kokwei86 at gmail.com Sun Oct 28 12:50:19 2012 From: kokwei86 at gmail.com (kokwei) Date: Mon, 29 Oct 2012 02:50:19 +0800 Subject: [maker-devel] Query on genemark results Message-ID: <508D7E6B.3060001@gmail.com> Hi, I have the same problems as posted by Andr? Gomes on 10/4/10. I still have the same problems even though using the current available version of maker (maker 2.1 and maker 2.26 beta version). I have tried to do the gene prediction on eukaryotic genome using 3 ab-initio gene predictors (SNAP, Augustus and GeneMark-ES) using Maker. >From the fasta_merge output, I have 3 separate files of gene models ($prefix.all.maker.augustus_masked.proteins.fasta, $prefix.all.maker.snap_masked.proteins.fasta and $prefix.all.maker.genemark.proteins.fasta) and one $prefix.all.maker.proteins.fasta (I presume this should be the consensus gene models from 3 predictors' results, right?). From the consensus gene models file, I don't see even one result from genemark but all from snap/augustus only. Is that normal? Also from the file naming and gene model label in final gff file, it's showing that genemark is not masked like those of augustus and snap? Is that true? Why only augustus and snap masked but not genemark? Please assist and thanks for your helps. Kok Wei From daniel.standage at gmail.com Mon Oct 29 07:39:05 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Mon, 29 Oct 2012 09:39:05 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Carson, I was able to run the first svn update before the test job ran, but I didn't get the message about this second update until after it has already executed and failed. I got about 372 lines of such. STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_datastore To access files for individual sequences use the datastore index: /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_master_datastore_index.log STATUS: Now running MAKER... WARNING: Cannot find >scaffold_0, trying to re-index the fasta. stop here: scaffold_0 ERROR: Fasta index error at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 239. Process::MpiChunk::_prepare('Process::MpiChunk=HASH(0x3c8b300)', 'HASH(0x3c80820)', 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 73 Process::MpiTiers::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c808b0)', 'HASH(0x3c8ec40)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 79 Process::MpiTiers::_prepare('Process::MpiTiers=HASH(0x3c71f30)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 56 Process::MpiTiers::new('Process::MpiTiers', 'HASH(0x3c80640)', 0, 'Process::MpiChunk') called at /N/u/dstandag/Mason/local/src/maker-dev/bin/maker line 627 --> rank=NA, hostname=c4 ERROR: Failed in tier preparation WARNING: You must always set a rank before running MpiTiers FATAL: argument `seq_id` does not exist in MpiTier object at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 86. Process::MpiChunk::_initialize_vars('Process::MpiChunk=HASH(0x3cc93d0)', 'HASH(0x3cc9400)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 47 Process::MpiChunk::new('Process::MpiChunk', 'HASH(0x3c80820)', 0, 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 407 Process::MpiChunk::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c8f498)', 'HASH(0x3cc8f20)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 3811 Process::MpiChunk::_go('Process::MpiChunk=HASH(0x3c8b300)', 'load', 'HASH(0x3c80820)', 0, 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 310 Process::MpiChunk::_loader('Process::MpiChunk=HASH(0x3c8b300)', 'HASH(0x3c80820)', 0, 0, 'Process::MpiTiers=HASH(0x3c71f30)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 364 Process::MpiTiers::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c8f9c0)', 'HASH(0x3c8fb28)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 375 I'll try the next update and run the test again. I see a lot of references in there to MPI, and just thought I would make it clear that although I am running in a cluster environment, I am using the default serial version of Maker, not the parallel MPI version. Also, after this failed, I tried changing the TMP directory so that it was located on the same NFS mount as the scratch disk to which the output was written. This did not seem to have any affect, and I saw the same issues with the EST sequences unable to be found. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 6:59 PM, Carson Holt wrote: > > > From: Carson Holt > Date: Friday, 26 October, 2012 6:10 PM > To: Daniel Standage > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > I've been going over the indexing code using different scenarios and I may > have isolated a candidate for what is causing this. Could you do one more > 'svn update' inside the maker devel directory before running a test job? > > Thanks, > Carson > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot >> connect to the compute nodes as I could while it was running. On the >> interactive node, it looks like it's real disk, although it looks like >> there are some tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were >>> under */tmp/maker_1YQF9o*. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared >>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>> directory setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>> to a different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. >>>>> The test run completed successfully, but as I commenced with the full >>>>> genome annotation, I have noticed the following error popping up frequently. >>>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>>> >>>>> >>>>> My first thought based on the message is that *blastdbcmd* could not >>>>> find the sequence in the database. I verified this was the case--I could >>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>> created under /tmp. However, after removing the index files and re-running >>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>>> rather its own internal code. Therefore I'm still not sure where the >>>>> problem is and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>> daniel.standage at gmail.com> wrote: >>>>> >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>> true -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this >>>>>> one failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>> _______________________________________________ maker-devel mailing >>>>> list maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Mon Oct 29 07:43:46 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Mon, 29 Oct 2012 09:43:46 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I just ran svn update and tried again with the development version of Maker. Same long list of errors as in the last message. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Mon, Oct 29, 2012 at 9:39 AM, Daniel Standage wrote: > Carson, > > I was able to run the first svn update before the test job ran, but I > didn't get the message about this second update until after it has already > executed and failed. I got about 372 lines of such. > > STATUS: Parsing control files... > STATUS: Processing and indexing input FASTA files... > STATUS: Setting up database for any GFF3 input... > A data structure will be created for you at: > > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_datastore > > To access files for individual sequences use the datastore index: > > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_master_datastore_index.log > > STATUS: Now running MAKER... > WARNING: Cannot find >scaffold_0, trying to re-index the fasta. > stop here: scaffold_0 > ERROR: Fasta index error > at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 239. > Process::MpiChunk::_prepare('Process::MpiChunk=HASH(0x3c8b300)', > 'HASH(0x3c80820)', 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 73 > Process::MpiTiers::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c808b0)', 'HASH(0x3c8ec40)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 79 > Process::MpiTiers::_prepare('Process::MpiTiers=HASH(0x3c71f30)') > called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 56 > Process::MpiTiers::new('Process::MpiTiers', 'HASH(0x3c80640)', 0, > 'Process::MpiChunk') called at > /N/u/dstandag/Mason/local/src/maker-dev/bin/maker line 627 > --> rank=NA, hostname=c4 > ERROR: Failed in tier preparation > WARNING: You must always set a rank before running MpiTiers > FATAL: argument `seq_id` does not exist in MpiTier object > at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 86. > > Process::MpiChunk::_initialize_vars('Process::MpiChunk=HASH(0x3cc93d0)', > 'HASH(0x3cc9400)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 47 > Process::MpiChunk::new('Process::MpiChunk', 'HASH(0x3c80820)', 0, > 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 407 > Process::MpiChunk::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c8f498)', 'HASH(0x3cc8f20)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 3811 > Process::MpiChunk::_go('Process::MpiChunk=HASH(0x3c8b300)', > 'load', 'HASH(0x3c80820)', 0, 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 310 > Process::MpiChunk::_loader('Process::MpiChunk=HASH(0x3c8b300)', > 'HASH(0x3c80820)', 0, 0, 'Process::MpiTiers=HASH(0x3c71f30)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 364 > Process::MpiTiers::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c8f9c0)', 'HASH(0x3c8fb28)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 375 > > > I'll try the next update and run the test again. I see a lot of references > in there to MPI, and just thought I would make it clear that although I am > running in a cluster environment, I am using the default serial version of > Maker, not the parallel MPI version. > > Also, after this failed, I tried changing the TMP directory so that it was > located on the same NFS mount as the scratch disk to which the output was > written. This did not seem to have any affect, and I saw the same issues > with the EST sequences unable to be found. > > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 6:59 PM, Carson Holt wrote: > >> >> >> From: Carson Holt >> Date: Friday, 26 October, 2012 6:10 PM >> To: Daniel Standage >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I've been going over the indexing code using different scenarios and I >> may have isolated a candidate for what is causing this. Could you do one >> more 'svn update' inside the maker devel directory before running a test >> job? >> >> Thanks, >> Carson >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:29 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Got this from the compute node. Looks like native disk space to me. >> >> [dstandag at mason ~] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sda1 478573472 12319684 441943620 3% /tmp >> >> Installing a bundle of Perl prereqs for development version, will try >> that soon. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: >> >>> Ok. Try the developer release and see if it still happens. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:19 PM >>> >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> Unfortunately, the job is no longer running and as a result I cannot >>> connect to the compute nodes as I could while it was running. On the >>> interactive node, it looks like it's real disk, although it looks like >>> there are some tmpfs mounts. >>> >>> [dstandag at mason src] df /tmp >>> Filesystem 1K-blocks Used Available Use% Mounted on >>> /dev/sdb2 462824304 180235660 259078476 42% /tmp >>> [dstandag at mason src] df >>> Filesystem 1K-blocks Used Available Use% Mounted on >>> login_x86_64 16497564 3077352 13420212 19% / >>> tmpfs 16497564 0 16497564 0% /dev/shm >>> tmpfs 10240 0 10240 0% /var/tmp >>> /dev/sdb2 462824304 180235660 259078476 42% /tmp >>> AFS 9000000 0 9000000 0% /afs >>> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >>> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >>> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >>> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >>> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >>> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >>> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >>> ... >>> ... >>> >>> I'll see if I can launch another short job and verify this on the >>> compute nodes. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >>> >>>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:12 PM >>>> To: Carson Holt >>>> Cc: "maker-devel at yandell-lab.org" >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> It looks like /tmp is indeed being used: the files I played with were >>>> under */tmp/maker_1YQF9o*. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>>> >>>>> Check to see where /tmp is located? Some clusters have it set up as a >>>>> tmpfs directory and I have had problems with fasta indexes running from >>>>> tmpfs mounts in the past. >>>>> >>>>> --Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 2:05 PM >>>>> To: Carson Holt >>>>> >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> The maker working directory is in a cluster environment with shared >>>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>>> directory setting, so it should be the local default (/tmp). >>>>> >>>>> I'll give the dev version a shot. Thanks. >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>>> >>>>>> Could you try this development version and tell me if the error still >>>>>> happens? >>>>>> >>>>>> Use this command to download --> >>>>>> <> >>>>>> >>>>>> Username: <> >>>>>> Password: <> >>>>>> >>>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>>> to a different location? >>>>>> >>>>>> Thanks, >>>>>> Carson >>>>>> >>>>>> >>>>>> From: Daniel Standage >>>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>>> To: Maker Mailing List >>>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>>> >>>>>> I have since installed Maker on a different machine and tried it out. >>>>>> The test run completed successfully, but as I commenced with the full >>>>>> genome annotation, I have noticed the following error popping up frequently. >>>>>> >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>>> #-------------------------------# >>>>>> deleted:-10 hits >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>>> #-------------------------------# >>>>>> deleted:-6 hits >>>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>>> stop here:comp59088_c1_seq7 >>>>>> ERROR: Fasta index error >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while polishig ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 14 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --Next Contig-- >>>>>> >>>>>> #--------------------------------------------------------------------- >>>>>> Now starting the contig!! >>>>>> SeqID: scaffold_1 >>>>>> Length: 5805686 >>>>>> #--------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> My first thought based on the message is that *blastdbcmd* could not >>>>>> find the sequence in the database. I verified this was the case--I could >>>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>>> created under /tmp. However, after removing the index files and re-running >>>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>>> extracted the sequence. >>>>>> >>>>>> I was initially happy with this finding, but upon closer inspection >>>>>> it looks like Maker does not use *blastdbcmd* to extract sequences, >>>>>> but rather its own internal code. Therefore I'm still not sure where the >>>>>> problem is and how I might fix it. Any insights? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>>> daniel.standage at gmail.com> wrote: >>>>>> >>>>>>> Greetings! >>>>>>> >>>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>>> during the blastn stage. This is the terminal message. >>>>>>> >>>>>>> #--------- command -------------# >>>>>>> Widget::blastn: >>>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>>> true -show_gis -out >>>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>>> #-------------------------------# >>>>>>> deleted:0 hits >>>>>>> ERROR: Could not obtain lock to format database >>>>>>> >>>>>>> >>>>>>> FATAL ERROR >>>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>>> >>>>>>> ERROR: Chunk failed at level 8 >>>>>>> !! >>>>>>> FAILED CONTIG:scaffold_866 >>>>>>> >>>>>>> >>>>>>> Several blastn steps appeared to have completed successfully to this >>>>>>> one failing. Any ideas what could be causing this? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> -- >>>>>>> Daniel S. Standage >>>>>>> Ph.D. Candidate >>>>>>> Bioinformatics and Computational Biology Program >>>>>>> Department of Genetics, Development, and Cell Biology >>>>>>> Iowa State University >>>>>>> >>>>>>> >>>>>> _______________________________________________ maker-devel mailing >>>>>> list maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Tue Oct 30 15:18:06 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Tue, 30 Oct 2012 14:18:06 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: Message-ID: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Hello Carson and maker community, Thank you very much for your guidelines on using the maker-pipeline. Yes, green sea urchin genome that we are trying to annotate. We are running the on scaffolds and most of these scaffolds are small in size(very first genome assembly). We would typically expect 20,000 genes in this genome. So we are running maker using EST and proteins from the genome and out-groups to generate training dataset for SNAP and Augustus. Depending on the resulting predictions we may bootstrap the predicted genes once again using EST and proteins. Do you have any further suggestions? Also could you point how to convert training set generated for SNAP to be used as training set for Augustus as well? Would maker give equal weightage to SNAP and Augustus predictions for generating gene model? Thanks and regards, Parul Kudtarkar > One thing you seem to be missing is protein evidence. > > Is this a sea urchin (I looked up some of the ESTs)? If so, I would recommend adding all proteins from the Strongylocentrotus purpuratus genome, then throw in another Deuterstome of your choice. Perhaps you should also add a couple of outgroup organisms like Nematostella vectensis > (cnidaria) and a protostome of your choice. Be careful if adding adding to many protostome outgroups (i.e. C. elegans and Drosophila) because a big part of their evolution is gene loss (so distant cnidaria often match > deuterstomes better than most protostomes do). > > You could take the maker results when protein data is included and use it > to retrain SNAP again. > > Even a 22 kb contig is still really short. Is this genome primarily constituted by short contigs like this? I would recommend running CEGMA once on this genome to get an appropriate estimate of how recoverable the > genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). Cegma > will give you an estimate for genome completeness as well as estimates of > what percentage of genes will be found in their entirety and what percent > will be partial genes. This is important to do if your genome is fragmented as it will give you a reasonable expectation of what you can expected to recover (as short contigs don't annotate very well - you tend > to loose a lot). > > Thanks, > Carson > > > On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: > >>Hi Carson, >>Thanks. I have attached another contig which is 22 kb, with as many as 3 exons EST alignments. Could you please recommend additional training. We are currently running maker on the entire contig set and eventually merge >>all the gff3 contig predictions. The using suggested parameter/methods we >>would like to get a consensus gene-set with minimal false >>positives/negatives. >>Thanks, >>Parul >>> The contig in question is really too small to get much out of it (only 5 >>kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb >>> length or longer contigs to glean enough information for optimizing your >>parameters. >>> The general suggestions for any maker run are to use proteins from a >>closely related organism or a couple of closely related organisms for the >>> protein= option in maker. Also leave single_exon set to 0, except for >>certain eukaryotes that have a bias for single exon transcripts (i.e. some >>> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >>predictors tend to over-predict by a wide margin (lots of false >>> positives). >>> Additional training would really depend on what your other contigs look >>like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. >>> Thanks, >>> Carson >>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>Hello, >>>>Please advice on the aforementioned query? >>>>Thanks, >>>>Parul Kudtarkar >>>>---------------------------- Original Message >>>> ---------------------------- >>>>Subject: [maker-devel] Conensus gene model >>>>From: "Parul Kudtarkar" >>>>Date: Fri, October 12, 2012 2:46 pm >>>>To: maker-devel at yandell-lab.org >>>>------------------------------------------------------------------------ -- >>Hi, >>>>We are using snap(training set[hmm file] generated using est,protein >>>> and >>contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>> file >>>>as input). The output that we get is combination of various >>>>gene-predictors and evidences. I have attached sample result file. What >>would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? >>>>Thanks, >>>>Parul Kudtarkar >>>>-- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org_______________________________________________ >>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org_______________________________________________ >>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jason.stajich at gmail.com Tue Oct 30 18:52:43 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Tue, 30 Oct 2012 17:52:43 -0700 Subject: [maker-devel] Conensus gene model In-Reply-To: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> References: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Message-ID: Paul - I think I've posted on this before here if you are asking how to go from SNAP training to Augustus training. http://sourceforge.net/mailarchive/message.php?msg_id=29361270 I do this type of training a lot - here some pointers. I often train by generating models using cegma on the genome and get these 400 or so good models as my training set. when I have EST or RNA-Seq I use PASA to generate the best set of annotations. For CEGMA - then I run this script that comes with MAKER: cegma2zff output.cegma.gff genome.fa Then I follow the SNAP directions fathom genome.ann genome.dna -categorize 1000 fathom uni.ann uni.dna -export 1000 -plus mkdir MYGENOME cd MYGENOME forge ../export.ann ../export.dna --OPTIONS cd ../MYGENOME hmm-assembler.pl MYGENOME MYGENOME > MYGENOME.snap.hmm I then also make the augustus training data like this running in the directory that has the export.ann and export.dna files: perl gene_prediction/zff2augustus_gbk.pl > train.gb using this script: https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl I also make ZFF from GFF with this script if I got the RNA-Seq aligned and best models from PASA and incorporate all these data in to my SNAP training set, and then export again back to gbk for the augustus training. https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/pasatraining2zff.pl Then you just need to run the Augustus training (autoAugTrain.pl) on the train.gb file. Jason On Oct 30, 2012, at 2:18 PM, Parul Kudtarkar wrote: > Hello Carson and maker community, > > Thank you very much for your guidelines on using the maker-pipeline. Yes, > green sea urchin genome that we are trying to annotate. > We are running the on scaffolds and most of these scaffolds are small in > size(very first genome assembly). We would typically expect 20,000 genes > in this genome. So we are running maker using EST and proteins from the > genome and out-groups to generate training dataset for SNAP and Augustus. > Depending on the resulting predictions we may bootstrap the predicted > genes once again using EST and proteins. > > Do you have any further suggestions? Also could you point how to convert > training set generated for SNAP to be used as training set for Augustus as > well? Would maker give equal weightage to SNAP and Augustus predictions > for generating gene model? > > Thanks and regards, > Parul Kudtarkar > >> One thing you seem to be missing is protein evidence. >> >> Is this a sea urchin (I looked up some of the ESTs)? If so, I would > recommend adding all proteins from the Strongylocentrotus purpuratus > genome, then throw in another Deuterstome of your choice. Perhaps you > should also add a couple of outgroup organisms like Nematostella > vectensis >> (cnidaria) and a protostome of your choice. Be careful if adding adding > to many protostome outgroups (i.e. C. elegans and Drosophila) because a > big part of their evolution is gene loss (so distant cnidaria often > match >> deuterstomes better than most protostomes do). >> >> You could take the maker results when protein data is included and use > it >> to retrain SNAP again. >> >> Even a 22 kb contig is still really short. Is this genome primarily > constituted by short contigs like this? I would recommend running CEGMA > once on this genome to get an appropriate estimate of how recoverable > the >> genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). > Cegma >> will give you an estimate for genome completeness as well as estimates > of >> what percentage of genes will be found in their entirety and what > percent >> will be partial genes. This is important to do if your genome is > fragmented as it will give you a reasonable expectation of what you can > expected to recover (as short contigs don't annotate very well - you > tend >> to loose a lot). >> >> Thanks, >> Carson >> >> >> On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >> >>> Hi Carson, >>> Thanks. I have attached another contig which is 22 kb, with as many as 3 > exons EST alignments. Could you please recommend additional training. We > are currently running maker on the entire contig set and eventually > merge >>> all the gff3 contig predictions. The using suggested parameter/methods > we >>> would like to get a consensus gene-set with minimal false >>> positives/negatives. >>> Thanks, >>> Parul >>>> The contig in question is really too small to get much out of it (only 5 >>> kb). There was only one single exon EST alignments and a couple of > predictions with no evidence support. Anything smaller than 10 kb is > mostly useless for annotation purposes. You would really need a few > 100kb >>>> length or longer contigs to glean enough information for optimizing your >>> parameters. >>>> The general suggestions for any maker run are to use proteins from a >>> closely related organism or a couple of closely related organisms for the >>>> protein= option in maker. Also leave single_exon set to 0, except for >>> certain eukaryotes that have a bias for single exon transcripts (i.e. some >>>> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >>> predictors tend to over-predict by a wide margin (lots of false >>>> positives). >>>> Additional training would really depend on what your other contigs > look >>> like. Do you have any large contigs? I could look at one of those and > give suggestions but the provided contig is just too short to glean > much. >>>> Thanks, >>>> Carson >>>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>> Hello, >>>>> Please advice on the aforementioned query? >>>>> Thanks, >>>>> Parul Kudtarkar >>>>> ---------------------------- Original Message >>>>> ---------------------------- >>>>> Subject: [maker-devel] Conensus gene model >>>>> From: "Parul Kudtarkar" >>>>> Date: Fri, October 12, 2012 2:46 pm >>>>> To: maker-devel at yandell-lab.org >>>>> ------------------------------------------------------------------------ > -- >>> Hi, >>>>> We are using snap(training set[hmm file] generated using est,protein >>>>> and >>> contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>>> file >>>>> as input). The output that we get is combination of various >>>>> gene-predictors and evidences. I have attached sample result file. > What >>> would you recommend to get consensus result set? Bootstrapping the > resulting gff3 file (rerunning maker)? >>>>> Thanks, >>>>> Parul Kudtarkar >>>>> -- >>>>> Scientific Programmer >>>>> Center for Computational Regulatory Genomics >>>>> Beckman Institute, >>>>> California Institute of Technology >>>>> http://www.spbase.org_______________________________________________ >>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- >>>>> Scientific Programmer >>>>> Center for Computational Regulatory Genomics >>>>> Beckman Institute, >>>>> California Institute of Technology >>>>> http://www.spbase.org_______________________________________________ >>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> -- >>> Scientific Programmer >>> Center for Computational Regulatory Genomics >>> Beckman Institute, >>> California Institute of Technology >>> http://www.spbase.org >> >> >> > > > -- > Scientific Programmer > Center for Computational Regulatory Genomics > Beckman Institute, > California Institute of Technology > http://www.spbase.org > > > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org From parulk at caltech.edu Wed Oct 31 18:04:33 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Wed, 31 Oct 2012 17:04:33 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Message-ID: <1640.131.215.15.234.1351728273.squirrel@webmail.caltech.edu> Hi Jason, thanks for directions on generating training-set for augustus. Also as alignment evidence if we are providing protein sequences from the primary organism as well as other closely related species is there an option to give the primary protein file precedence over others? At the moment I have all the proteins(from primary organism as well as related species) into a single file as protein option in maker_opts.ctl Thanks and regards, Parul Kudtarkar > Paul - > > I think I've posted on this before here if you are asking how to go from > SNAP training to Augustus training. > http://sourceforge.net/mailarchive/message.php?msg_id=29361270 > > I do this type of training a lot - here some pointers. > > I often train by generating models using cegma on the genome and get these > 400 or so good models as my training set. when I have EST or RNA-Seq I > use PASA to generate the best set of annotations. > > For CEGMA - then I run this script that comes with MAKER: > cegma2zff output.cegma.gff genome.fa > > Then I follow the SNAP directions > > fathom genome.ann genome.dna -categorize 1000 > fathom uni.ann uni.dna -export 1000 -plus > mkdir MYGENOME > cd MYGENOME > forge ../export.ann ../export.dna --OPTIONS > cd ../MYGENOME > hmm-assembler.pl MYGENOME MYGENOME > MYGENOME.snap.hmm > > I then also make the augustus training data like this running in the > directory that has the export.ann and export.dna files: > perl gene_prediction/zff2augustus_gbk.pl > train.gb > > using this script: > https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl > > I also make ZFF from GFF with this script if I got the RNA-Seq aligned and > best models from PASA and incorporate all these data in to my SNAP > training set, and then export again back to gbk for the augustus training. > https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/pasatraining2zff.pl > > Then you just need to run the Augustus training (autoAugTrain.pl) on the > train.gb file. > > Jason > > On Oct 30, 2012, at 2:18 PM, Parul Kudtarkar wrote: > >> Hello Carson and maker community, >> >> Thank you very much for your guidelines on using the maker-pipeline. >> Yes, >> green sea urchin genome that we are trying to annotate. >> We are running the on scaffolds and most of these scaffolds are small in >> size(very first genome assembly). We would typically expect 20,000 genes >> in this genome. So we are running maker using EST and proteins from the >> genome and out-groups to generate training dataset for SNAP and >> Augustus. >> Depending on the resulting predictions we may bootstrap the predicted >> genes once again using EST and proteins. >> >> Do you have any further suggestions? Also could you point how to convert >> training set generated for SNAP to be used as training set for Augustus >> as >> well? Would maker give equal weightage to SNAP and Augustus predictions >> for generating gene model? >> >> Thanks and regards, >> Parul Kudtarkar >> >>> One thing you seem to be missing is protein evidence. >>> >>> Is this a sea urchin (I looked up some of the ESTs)? If so, I would >> recommend adding all proteins from the Strongylocentrotus purpuratus >> genome, then throw in another Deuterstome of your choice. Perhaps you >> should also add a couple of outgroup organisms like Nematostella >> vectensis >>> (cnidaria) and a protostome of your choice. Be careful if adding >>> adding >> to many protostome outgroups (i.e. C. elegans and Drosophila) because a >> big part of their evolution is gene loss (so distant cnidaria often >> match >>> deuterstomes better than most protostomes do). >>> >>> You could take the maker results when protein data is included and use >> it >>> to retrain SNAP again. >>> >>> Even a 22 kb contig is still really short. Is this genome primarily >> constituted by short contigs like this? I would recommend running CEGMA >> once on this genome to get an appropriate estimate of how recoverable >> the >>> genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). >> Cegma >>> will give you an estimate for genome completeness as well as estimates >> of >>> what percentage of genes will be found in their entirety and what >> percent >>> will be partial genes. This is important to do if your genome is >> fragmented as it will give you a reasonable expectation of what you can >> expected to recover (as short contigs don't annotate very well - you >> tend >>> to loose a lot). >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >>> >>>> Hi Carson, >>>> Thanks. I have attached another contig which is 22 kb, with as many as >>>> 3 >> exons EST alignments. Could you please recommend additional training. We >> are currently running maker on the entire contig set and eventually >> merge >>>> all the gff3 contig predictions. The using suggested parameter/methods >> we >>>> would like to get a consensus gene-set with minimal false >>>> positives/negatives. >>>> Thanks, >>>> Parul >>>>> The contig in question is really too small to get much out of it >>>>> (only 5 >>>> kb). There was only one single exon EST alignments and a couple of >> predictions with no evidence support. Anything smaller than 10 kb is >> mostly useless for annotation purposes. You would really need a few >> 100kb >>>>> length or longer contigs to glean enough information for optimizing >>>>> your >>>> parameters. >>>>> The general suggestions for any maker run are to use proteins from a >>>> closely related organism or a couple of closely related organisms for >>>> the >>>>> protein= option in maker. Also leave single_exon set to 0, except >>>>> for >>>> certain eukaryotes that have a bias for single exon transcripts (i.e. >>>> some >>>>> fungi and oomycetes). And leave keep_preds set to 0 because ab >>>>> initio >>>> predictors tend to over-predict by a wide margin (lots of false >>>>> positives). >>>>> Additional training would really depend on what your other contigs >> look >>>> like. Do you have any large contigs? I could look at one of those >>>> and >> give suggestions but the provided contig is just too short to glean >> much. >>>>> Thanks, >>>>> Carson >>>>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>>> Hello, >>>>>> Please advice on the aforementioned query? >>>>>> Thanks, >>>>>> Parul Kudtarkar >>>>>> ---------------------------- Original Message >>>>>> ---------------------------- >>>>>> Subject: [maker-devel] Conensus gene model >>>>>> From: "Parul Kudtarkar" >>>>>> Date: Fri, October 12, 2012 2:46 pm >>>>>> To: maker-devel at yandell-lab.org >>>>>> ------------------------------------------------------------------------ >> -- >>>> Hi, >>>>>> We are using snap(training set[hmm file] generated using est,protein >>>>>> and >>>> contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>>>> file >>>>>> as input). The output that we get is combination of various >>>>>> gene-predictors and evidences. I have attached sample result file. >> What >>>> would you recommend to get consensus result set? Bootstrapping the >> resulting gff3 file (rerunning maker)? >>>>>> Thanks, >>>>>> Parul Kudtarkar >>>>>> -- >>>>>> Scientific Programmer >>>>>> Center for Computational Regulatory Genomics >>>>>> Beckman Institute, >>>>>> California Institute of Technology >>>>>> http://www.spbase.org_______________________________________________ >>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> -- >>>>>> Scientific Programmer >>>>>> Center for Computational Regulatory Genomics >>>>>> Beckman Institute, >>>>>> California Institute of Technology >>>>>> http://www.spbase.org_______________________________________________ >>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> -- >>>> Scientific Programmer >>>> Center for Computational Regulatory Genomics >>>> Beckman Institute, >>>> California Institute of Technology >>>> http://www.spbase.org >>> >>> >>> >> >> >> -- >> Scientific Programmer >> Center for Computational Regulatory Genomics >> Beckman Institute, >> California Institute of Technology >> http://www.spbase.org >> >> >> >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > Jason Stajich > jason.stajich at gmail.com > jason at bioperl.org > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From carsonhh at gmail.com Mon Oct 1 08:01:01 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 01 Oct 2012 10:01:01 -0400 Subject: [maker-devel] model_gff question In-Reply-To: Message-ID: They can be replaced under two circumstances. 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. Thanks, Carson From: Michael Thon Date: Monday, 1 October, 2012 1:53 AM To: Subject: [maker-devel] model_gff question Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. -Mike _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.thon at gmail.com Mon Oct 1 23:01:09 2012 From: mike.thon at gmail.com (Michael Thon) Date: Tue, 2 Oct 2012 07:01:09 +0200 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: I looked at two cases in which the model_gff disappeared and they occurred in regions where there are multiple overlapping cufflinks features. One model that I'm looking at right now has overlapping protein2genome and a SNAP feature overlapping it but it was still not included in the output. it could be a problem in MAKER or it could be a problem with my RNA Seq data. I aligned the RNA Seq data using tophat/cufflinks and converted the transcripts.gtf file to gff using cufflinks2gff3 script. Is it better to use RNA Seq feature from tophat or cufflinks? On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > They can be replaced under two circumstances. > 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. > 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). > > MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. > > Thanks, > Carson > > > From: Michael Thon > Date: Monday, 1 October, 2012 1:53 AM > To: > Subject: [maker-devel] model_gff question > > Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: > https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion > > That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. > -Mike > > _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.thon at gmail.com Tue Oct 2 01:50:04 2012 From: mike.thon at gmail.com (Michael Thon) Date: Tue, 2 Oct 2012 09:50:04 +0200 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: <8E7B4629-B071-44A6-8E7F-2B1133A946D0@gmail.com> It seems to have disappeared completely. I'm running MAKER again now using the tophat alignments that I fed to cufflinks, instead of the cufflinks data. So far the two models visually checked as missing with the cufflinks data are present as they should be. I have to wait for the run to finish to get a whole genome count though. Maybe I need to look more closely at the cufflinks run that I did. The RNA-Seq data are from the NCBI SRA and I didn't do anything to clean them up before I ran tophat. On Oct 2, 2012, at 9:10 AM, Daniel Hughes wrote: > Did the whole model vanish or just the protein product - contaminated rnaseq that hasn't been cleaned up enough will regularly cause the later to become part of a bad utr. > > Dan > > On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: > I looked at two cases in which the model_gff disappeared and they occurred in regions where there are multiple overlapping cufflinks features. One model that I'm looking at right now has overlapping protein2genome and a SNAP feature overlapping it but it was still not included in the output. it could be a problem in MAKER or it could be a problem with my RNA Seq data. I aligned the RNA Seq data using tophat/cufflinks and converted the transcripts.gtf file to gff using cufflinks2gff3 script. > > Is it better to use RNA Seq feature from tophat or cufflinks? > > > On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > >> They can be replaced under two circumstances. >> 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. >> 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). >> >> MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. >> >> Thanks, >> Carson >> >> >> From: Michael Thon >> Date: Monday, 1 October, 2012 1:53 AM >> To: >> Subject: [maker-devel] model_gff question >> >> Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: >> https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion >> >> That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. >> -Mike >> >> _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Oct 2 12:05:04 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 02 Oct 2012 14:05:04 -0400 Subject: [maker-devel] model_gff question In-Reply-To: <8E7B4629-B071-44A6-8E7F-2B1133A946D0@gmail.com> Message-ID: If it overlaps the UTR of some other chosen model it might have been excluded for that reason as well. Sometimes this can happen when you have RNA-seq and high gene density (so models wander into each other). Try setting the the correct_est_fusion option to 1. This will take steps to trim UTR that might cause neighboring models to be left out because of UTR overlap (it also helps with false fusions caused by cufflinks results). I would not be surprised if the model being left out was because of UTR overlap. I recommend using the cufflinks data and leaving tophat out. Tophat results tend to be very noisy and can span large rather weird regions. Thanks, Carson From: Michael Thon Date: Tuesday, 2 October, 2012 3:50 AM To: Daniel Hughes Cc: Michael Thon , , Carson Holt Subject: Re: [maker-devel] model_gff question It seems to have disappeared completely. I'm running MAKER again now using the tophat alignments that I fed to cufflinks, instead of the cufflinks data. So far the two models visually checked as missing with the cufflinks data are present as they should be. I have to wait for the run to finish to get a whole genome count though. Maybe I need to look more closely at the cufflinks run that I did. The RNA-Seq data are from the NCBI SRA and I didn't do anything to clean them up before I ran tophat. On Oct 2, 2012, at 9:10 AM, Daniel Hughes wrote: > > Did the whole model vanish or just the protein product - contaminated rnaseq > that hasn't been cleaned up enough will regularly cause the later to become > part of a bad utr. > > Dan > > On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: >> I looked at two cases in which the model_gff disappeared and they occurred in >> regions where there are multiple overlapping cufflinks features. One model >> that I'm looking at right now has overlapping protein2genome and a SNAP >> feature overlapping it but it was still not included in the output. it could >> be a problem in MAKER or it could be a problem with my RNA Seq data. I >> aligned the RNA Seq data using tophat/cufflinks and converted the >> transcripts.gtf file to gff using cufflinks2gff3 script. >> >> Is it better to use RNA Seq feature from tophat or cufflinks? >> >> >> On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: >> >>> They can be replaced under two circumstances. >>> 1. If you provide two model_gff files (comma separated list), in which case >>> MAKER thinks it is merging legacy annotations and will only keep one or the >>> other if models overlap. >>> 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this >>> as a cue that if these other programs produce a better model, it can replace >>> the current model. If you set map_forward=1, MAKER will conserve the name >>> of the previous model (so models change structure but names are conserved); >>> otherwise, it gets a new name. Sometimes groups like to rename models every >>> time their is a structural change. I think you are supposed to get the >>> Alias attribute set when you don't get names mapped forward though (I can't >>> remember if I added this or just planned on adding the Alias mapping >>> though). >>> >>> MAKER should never drop a model_gff model. It can only replace it if >>> something better comes along, but it should not disappear. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Michael Thon >>> Date: Monday, 1 October, 2012 1:53 AM >>> To: >>> Subject: [maker-devel] model_gff question >>> >>> Under what circumstances will maker not include a gene model from the >>> model_gff file in its final output? It was my understanding from this post: >>> https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion >>> >>> That maker will keep or replace models in model_gff and never remove them. >>> I'm reannotating a fungal genome and in model_gff I'm providing the gene >>> models originally made by the sequencing center. I have 12006 models in the >>> file I specify in model_gff but maker's final annotation has only 10727 >>> models in it. >>> -Mike >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m >>> aker-devel_yandell-lab.org >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dsthughes at gmail.com Tue Oct 2 01:10:19 2012 From: dsthughes at gmail.com (Daniel Hughes) Date: Tue, 2 Oct 2012 08:10:19 +0100 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: Did the whole model vanish or just the protein product - contaminated rnaseq that hasn't been cleaned up enough will regularly cause the later to become part of a bad utr. Dan On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: > I looked at two cases in which the model_gff disappeared and they occurred > in regions where there are multiple overlapping cufflinks features. One > model that I'm looking at right now has overlapping protein2genome and a > SNAP feature overlapping it but it was still not included in the output. > it could be a problem in MAKER or it could be a problem with my RNA Seq > data. I aligned the RNA Seq data using tophat/cufflinks and converted the > transcripts.gtf file to gff using cufflinks2gff3 script. > > Is it better to use RNA Seq feature from tophat or cufflinks? > > > On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > > They can be replaced under two circumstances. > 1. If you provide two model_gff files (comma separated list), in which > case MAKER thinks it is merging legacy annotations and will only keep one > or the other if models overlap. > 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees > this as a cue that if these other programs produce a better model, it can > replace the current model. If you set map_forward=1, MAKER will conserve > the name of the previous model (so models change structure but names are > conserved); otherwise, it gets a new name. Sometimes groups like to rename > models every time their is a structural change. I think you are supposed > to get the Alias attribute set when you don't get names mapped forward > though (I can't remember if I added this or just planned on adding the > Alias mapping though). > > MAKER should never drop a model_gff model. It can only replace it if > something better comes along, but it should not disappear. > > Thanks, > Carson > > > From: Michael Thon > Date: Monday, 1 October, 2012 1:53 AM > To: > Subject: [maker-devel] model_gff question > > Under what circumstances will maker not include a gene model from the > model_gff file in its final output? It was my understanding from this post: > https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion > > That maker will keep or replace models in model_gff and never remove them. > I'm reannotating a fungal genome and in model_gff I'm providing the gene > models originally made by the sequencing center. I have 12006 models in > the file I specify in model_gff but maker's final annotation has only 10727 > models in it. > -Mike > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amelia.ireland at gmod.org Tue Oct 2 15:00:57 2012 From: amelia.ireland at gmod.org (Amelia Ireland) Date: Tue, 2 Oct 2012 14:00:57 -0700 Subject: [maker-devel] problem installing maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868AF3A@EXMBX05.austin.utexas.edu> References: <1B4E4BB25B711A4FBC93E4B3AFEB2A3218689DC6@EXMBX05.austin.utexas.edu> <1B4E4BB25B711A4FBC93E4B3AFEB2A321868AF3A@EXMBX05.austin.utexas.edu> Message-ID: Hello Oscar, I'm forwarding your email on to the MAKER mailing list and one of the developers; they should be able to answer your query. Thanks, Amelia. On Tue, Oct 2, 2012 at 1:55 PM, Dian "Oscar" Jiao wrote: > Hi, > > I was trying to install Maker. All the prerequisite seemed installed okay. > However, when I ran maker or mpimaker, I got the following error: > /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_File/DB_File.so: > undefined symbol: db_version". I have edited config.in before compiling to > make sure the paths are correct for db.h and libdb. Do you have any idea > what is causing this? > > It seems to have to do with the berkeleydb and DB_File. I have BerkeleyDB > 5.3.21 installed and set the paths correctly for db.h and libdb in config.in > when I was compiling DB_File, but it still failed. Do you know what is > causing this? > > Thanks > Oscar > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > -- > You received this message because you are subscribed to the Google Groups > "GMOD Help Desk" group. > To post to this group, send email to gmod-help-desk at googlegroups.com. > To unsubscribe from this group, send email to > gmod-help-desk+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > -- Amelia Ireland GMOD Community Support Specialist || http://gmod.org From carsonhh at gmail.com Wed Oct 3 08:13:20 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 03 Oct 2012 10:13:20 -0400 Subject: [maker-devel] problem installing maker In-Reply-To: Message-ID: You may need to completely delete this directory before reinstalling DB_File. --> /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_Fil e Also if that doesn't work you can skip DB_file completely, and use a different database backend During the MAKER install do the following: perl ./Build.PL --AnyDBM_ISA SDBM_File ./Build install Then test maker once on a new job. The --AnyDBM_ISA sets a different database than Berkley DB (All of which are slower than Berkley DB). --Carson On 12-10-02 5:00 PM, "Amelia Ireland" wrote: >Hello Oscar, > >I'm forwarding your email on to the MAKER mailing list and one of the >developers; they should be able to answer your query. > >Thanks, >Amelia. > >On Tue, Oct 2, 2012 at 1:55 PM, Dian "Oscar" Jiao >wrote: >> Hi, >> >> I was trying to install Maker. All the prerequisite seemed installed >>okay. >> However, when I ran maker or mpimaker, I got the following error: >> >>/work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_F >>ile/DB_File.so: >> undefined symbol: db_version". I have edited config.in before compiling >>to >> make sure the paths are correct for db.h and libdb. Do you have any idea >> what is causing this? >> >> It seems to have to do with the berkeleydb and DB_File. I have >>BerkeleyDB >> 5.3.21 installed and set the paths correctly for db.h and libdb in >>config.in >> when I was compiling DB_File, but it still failed. Do you know what is >> causing this? >> >> Thanks >> Oscar >> -- >> Dian (Oscar) Jiao, Ph.D., >> Research Associate, Life Sciences Computing Group >> Texas Advanced Computing Center >> University of Texas at Austin >> jiao at tacc.utexas.edu | (832) 303-1166 >> >> -- >> You received this message because you are subscribed to the Google >>Groups >> "GMOD Help Desk" group. >> To post to this group, send email to gmod-help-desk at googlegroups.com. >> To unsubscribe from this group, send email to >> gmod-help-desk+unsubscribe at googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > > >-- >Amelia Ireland >GMOD Community Support Specialist || http://gmod.org > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From parulk at caltech.edu Wed Oct 3 20:14:36 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Wed, 3 Oct 2012 19:14:36 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs Message-ID: <2553.131.215.15.234.1349316876.squirrel@webmail.caltech.edu> I am running maker on example data that comes along with installation and is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note Please advice on the aforementioned error. --------------------- Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_proteins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est%2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_protein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_proteins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est%2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_protein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ------------------------------------ Many thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jiao at tacc.utexas.edu Wed Oct 3 22:04:32 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 04:04:32 +0000 Subject: [maker-devel] problem running maker Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868CD39@EXMBX05.austin.utexas.edu> Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 06:27:05 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 08:27:05 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868CD39@EXMBX05.austin.utexas.edu> Message-ID: Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 06:34:38 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 08:34:38 -0400 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <2553.131.215.15.234.1349316876.squirrel@webmail.caltech.edu> Message-ID: You probably need to reinstall Bio-perl. Other things that can cause the same error are setting TMP in the maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or sometimes setting it to an NFS mount. A full drive can cause this as well or a broken Berkley DB. Use df -h to see if any of the drives used are full (either current working directory or TMP location). You can also swap Berkley DB for a different backend by setting AnyDBM_ISA during setup. Example: cd ../maker/src/ perl Build.PL --AnyDBM_ISA SDBM_File ./Build install --Carson On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >I am running maker on example data that comes along with installation and >is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note > >Please advice on the aforementioned error. > >--------------------- >Maker is now finished!!! > >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >eins%2Efasta.repeatrunner >deleted:0 hits > in cluster:shadow cluster... > i_size:0 j_size:0 > sorting hits in shadow cluster... >... finished. >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >%2Efasta.blastn >deleted:-1 hits >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >tein%2Efasta.blastx >WARNING: Multiple MAKER processes have been started in the >same directory. > >deleted:0 hits >WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >stop here:dpp-mRNA-4 >ERROR: Fasta index error > >FATAL ERROR >Maker is now finished!!! > >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >eins%2Efasta.repeatrunner >deleted:0 hits > in cluster:shadow cluster... > i_size:0 j_size:0 > sorting hits in shadow cluster... >... finished. >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >%2Efasta.blastn >deleted:-1 hits >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >tein%2Efasta.blastx >WARNING: Multiple MAKER processes have been started in the >same directory. > >deleted:0 hits >WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >stop here:dpp-mRNA-4 >ERROR: Fasta index error > >FATAL ERROR >ERROR: Failed while polishig ESTs!! > >ERROR: Chunk failed at level 14 >!! >FAILED CONTIG:contig-dpp-500-500 > > > >ERROR: Chunk failed at level 14 >!! >FAILED CONTIG:contig-dpp-500-500 >------------------------------------ > > >Many thanks, >Parul Kudtarkar > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From jiao at tacc.utexas.edu Thu Oct 4 08:02:14 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 14:02:14 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D305@EXMBX05.austin.utexas.edu> 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 08:03:01 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 10:03:01 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D305@EXMBX05.austin.utexas.edu> Message-ID: Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 09:36:22 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 15:36:22 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 11:17:08 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 13:17:08 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Message-ID: Seg faults really only happen from C and not Perl, so there may be a perl module that is broken on your machine that use C internally. Here are some modules that MAKER can use that I know use C and that you might want to reinstall from CPAN. DB_File Forks Proc::ProcessTable Inline::C Also during the maker install say no to the "use MPI" question. It's best to make the installation as simple as possible before getting into MPI configuration. Also after installing those and rerunning maker's 'perl Build.PL' and './Build install' steps do your first run with 'maker --debug' (redirect STDERR to a file). Then if it fails, you can send me the error log. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 11:36 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 11:23:52 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 17:23:52 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D36D@EXMBX05.austin.utexas.edu> I tried the debug trick and it worked (at least it finished). I however when I looked at the stderr, I again noticed the "can't locate GDBM_File for @AnyDBM_File::ISA" error. Although it did not affect the output, is it what is causing the segmentation error? Because this is the only error other than a bunch of "UNKOWN Bio" messages. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 12:17 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Seg faults really only happen from C and not Perl, so there may be a perl module that is broken on your machine that use C internally. Here are some modules that MAKER can use that I know use C and that you might want to reinstall from CPAN. DB_File Forks Proc::ProcessTable Inline::C Also during the maker install say no to the "use MPI" question. It's best to make the installation as simple as possible before getting into MPI configuration. Also after installing those and rerunning maker's 'perl Build.PL' and './Build install' steps do your first run with 'maker --debug' (redirect STDERR to a file). Then if it fails, you can send me the error log. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 11:36 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 12:52:51 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 14:52:51 -0400 Subject: [maker-devel] Expected format for EST GFF3 Message-ID: Greetings. I would like to use Maker's *est_gff* option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 12:58:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 14:58:56 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Match/match_part features are expected. MAKER expects these to be polished (I.e. correctly aligned around splice sites). For example exonerate, BLAT, and cufflinks results will be correct around splice sites and can be used; BLAST results on the other hand will not be correct and should not be used. The Gap attribute is used by maker if available, but is not required (Gap describes how to reconstruct an alignment for gaps and mismatches). Otherwise MAKER assumes all positions are matches to the reference. Thanks, Carson From: Daniel Standage Date: Thursday, 4 October, 2012 2:52 PM To: Maker Mailing List Subject: [maker-devel] Expected format for EST GFF3 Greetings. I would like to use Maker's est_gff option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:07:18 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:07:18 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the *transcript* features to *match* and the *exon* features to *match_part *and make sure the parent/child relationships are maintained with the *Parent* and *ID* attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > Match/match_part features are expected. MAKER expects these to be > polished (I.e. correctly aligned around splice sites). For example > exonerate, BLAT, and cufflinks results will be correct around splice sites > and can be used; BLAST results on the other hand will not be correct and > should not be used. > > The Gap attribute is used by maker if available, but is not required (Gap > describes how to reconstruct an alignment for gaps and mismatches). > Otherwise MAKER assumes all positions are matches to the reference. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM > To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 > > Greetings. > > I would like to use Maker's *est_gff* option to include EST evidence from > an external GFF3 file. In addition to being a valid, well-formed GFF3 file, > what expectations does Maker assume about the contents of this file? Which > feature types are expected and/or supported? Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 13:09:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 15:09:25 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Use the cufflinks2gff3 script that comes with MAKER. Thanks, Carson From: Daniel Standage Date: Thursday, 4 October, 2012 3:07 PM To: Carson Holt Cc: Maker Mailing List Subject: Re: [maker-devel] Expected format for EST GFF3 Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only transcript and exon features. So I'm assuming I can simply convert the transcript features to match and the exon features to match_part and make sure the parent/child relationships are maintained with the Parent and ID attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > Match/match_part features are expected. MAKER expects these to be polished > (I.e. correctly aligned around splice sites). For example exonerate, BLAT, > and cufflinks results will be correct around splice sites and can be used; > BLAST results on the other hand will not be correct and should not be used. > > The Gap attribute is used by maker if available, but is not required (Gap > describes how to reconstruct an alignment for gaps and mismatches). Otherwise > MAKER assumes all positions are matches to the reference. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM > To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 > > Greetings. > > I would like to use Maker's est_gff option to include EST evidence from an > external GFF3 file. In addition to being a valid, well-formed GFF3 file, what > expectations does Maker assume about the contents of this file? Which feature > types are expected and/or supported? Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/mak > er-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:16:53 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:16:53 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Does the cufflinks2gff3 script filter out single-exon transcripts? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt wrote: > Use the cufflinks2gff3 script that comes with MAKER. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 3:07 PM > To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 > > Great. I am using PE Illumina reads mapped by Tophat and assembled by > Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the > *transcript* features to *match* and the *exon* features to *match_part *and > make sure the parent/child relationships are maintained with the *Parent* and > *ID* attributes? > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > >> Match/match_part features are expected. MAKER expects these to be >> polished (I.e. correctly aligned around splice sites). For example >> exonerate, BLAT, and cufflinks results will be correct around splice sites >> and can be used; BLAST results on the other hand will not be correct and >> should not be used. >> >> The Gap attribute is used by maker if available, but is not required (Gap >> describes how to reconstruct an alignment for gaps and mismatches). >> Otherwise MAKER assumes all positions are matches to the reference. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Thursday, 4 October, 2012 2:52 PM >> To: Maker Mailing List >> Subject: [maker-devel] Expected format for EST GFF3 >> >> Greetings. >> >> I would like to use Maker's *est_gff* option to include EST evidence >> from an external GFF3 file. In addition to being a valid, well-formed GFF3 >> file, what expectations does Maker assume about the contents of this file? >> Which feature types are expected and/or supported? Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:21:52 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:21:52 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Good to know. Thanks for your help! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:19 PM, Carson Holt wrote: > Yes. You really have to because the single exon transcripts produced > from mRNA-seq assembly are strandless (you don't know where they belong). > They also tend to be heavily weighted to pseudogenes and transposons. > > Thanks, > Carson > > > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 3:16 PM > > To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 > > Does the cufflinks2gff3 script filter out single-exon transcripts? > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt wrote: > >> Use the cufflinks2gff3 script that comes with MAKER. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Thursday, 4 October, 2012 3:07 PM >> To: Carson Holt >> Cc: Maker Mailing List >> Subject: Re: [maker-devel] Expected format for EST GFF3 >> >> Great. I am using PE Illumina reads mapped by Tophat and assembled by >> Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the >> *transcript* features to *match* and the *exon* features to *match_part *and >> make sure the parent/child relationships are maintained with the *Parent* and >> *ID* attributes? >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: >> >>> Match/match_part features are expected. MAKER expects these to be >>> polished (I.e. correctly aligned around splice sites). For example >>> exonerate, BLAT, and cufflinks results will be correct around splice sites >>> and can be used; BLAST results on the other hand will not be correct and >>> should not be used. >>> >>> The Gap attribute is used by maker if available, but is not required >>> (Gap describes how to reconstruct an alignment for gaps and mismatches). >>> Otherwise MAKER assumes all positions are matches to the reference. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Thursday, 4 October, 2012 2:52 PM >>> To: Maker Mailing List >>> Subject: [maker-devel] Expected format for EST GFF3 >>> >>> Greetings. >>> >>> I would like to use Maker's *est_gff* option to include EST evidence >>> from an external GFF3 file. In addition to being a valid, well-formed GFF3 >>> file, what expectations does Maker assume about the contents of this file? >>> Which feature types are expected and/or supported? Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> _______________________________________________ maker-devel mailing >>> list maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy.semeiks at utsw.edu Thu Oct 4 13:39:25 2012 From: jeremy.semeiks at utsw.edu (Jeremy Semeiks) Date: Thu, 4 Oct 2012 14:39:25 -0500 Subject: [maker-devel] maker hung on a contig Message-ID: Hi, I ran maker 2.26 beta on a mammalian assembly containing ~200,000 scaffolds. I ran like this: $ /usr/bin/time mpiexec -n 30 maker -q < /dev/null > maker.oe 2>&1 & disown -h After 1--2 weeks, all but one contig had finished successfully, although maker needed a retry on ~80 others. The one contig that failed is small (1,146 nt) and not obviously weird. maker hung on this one contig, outputting the following type of stack-trace-looking thing to maker.oe, over and over: """ eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xdd9d480)', 'Error::Simple=HASH(0xa9a56f8)', undef, 'ARRAY(0x9ecc710)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xf1b6a40)', 'HASH(0xdd9d480)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0xab33f30)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0xab33f30)', 'SCALAR(0x17cd3a90)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xb3a58e0)', 'Error::Simple=HASH(0xab33f30)', undef, 'ARRAY(0x77ffae0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xbe47098)', 'HASH(0xb3a58e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0xc86fb88)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0xc86fb88)', 'SCALAR(0xb504490)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xc2d7d00)', 'Error::Simple=HASH(0xc86fb88)', undef, 'ARRAY(0x767a290)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0x5ab13f0)', 'HASH(0xc2d7d00)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0x18943010)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0x18943010)', 'SCALAR(0xa6e7cd0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 """ Every 100,000 lines or so, that sort of pattern is interrupted by this sort of pattern: """ Process::MpiTiers::__ANON__('Error::Simple=HASH(0xd9816c8)', 'SCALAR(0x2558c48)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0x1bd77ab8)', 'Error::Simple=HASH(0xd9816c8)', undef, 'ARRAY(0x2561948)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xf48b0d8)', 'HASH(0x1bd77ab8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 285 Process::MpiTiers::run_all('Process::MpiTiers=HASH(0xc10a7e0)', 0) calledERROR: Failed while builing masking tiers ERROR: Can not get next level ERROR: Can't open seq file: /export/home9/jrs/bmy/maker02/bmy.min1e3.maker.output/bmy.min1e3_datastore/E0/1F/C16738816//theVoid.C16738816/query.masked.gff.seq No such file or directory at /home/jrs/maker-2.26-beta/bin/../lib/Dumper/GFF/GFFV3.pm line 173. Dumper::GFF::GFFV3::finalize('Dumper::GFF::GFFV3=HASH(0x4c328c8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 700 Process::MpiChunk::__ANON__() called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 415 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0xfc1d208)', 'HASH(0xfc1fbd8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 3768 Process::MpiChunk::_go('Process::MpiChunk=HASH(0x47162a0)', 'flow', 'HASH(0x93ba680)', 2, 0) called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 369 """ The makers kept outputting these things until I killed them all, and also mpiexec, with kill -s SIGKILL. These strack traces go on for at least the last million lines of maker.oe, which is ~100 GB. Because the log is so big, without knowing what to grep for it's hard for me to suggest what originally went wrong. The contig's datastore directory does not contain anything obvious. Its run.log indicates only that it was the second try for this contig. None of the files in this directory (including theVoid) had been updated since 3 days ago. So this contig itself may even be a red herring. maker's protein output for all the other contigs seems sane, and this one contig is probably too small to contain any proteins, so it doesn't matter to me if maker finishes it successfully. I just want to figure out how I can keep maker from hanging. Suggestions? Thanks, Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 14:42:52 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 20:42:52 +0000 Subject: [maker-devel] maker mpi failed Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D692@EXMBX05.austin.utexas.edu> Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 14:53:08 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 16:53:08 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D692@EXMBX05.austin.utexas.edu> Message-ID: You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 15:20:48 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 21:20:48 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D7C8@EXMBX05.austin.utexas.edu> Hi Carson, I didn't know maker does not work with MVAPICH2. The cluster I was installing maker on only has mvapich2. So I guess I will have to install mpich2 under my directory. However, I installed perl threading based on mvapich2. Does that mean I need to reinstall perl with mpich2 before reinstalling maker? Oscar -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Thu Oct 4 17:06:13 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Thu, 4 Oct 2012 16:06:13 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: References: Message-ID: <1764.131.215.15.234.1349391973.squirrel@webmail.caltech.edu> Hi, Thanks, reinstalling BioPerl fixed the issue. Now we are training our genome to generate training dataset for SNAP and Augustus, as per the instructions provided at http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors However the training dataset would have multiple gff3 prediction files, one for each contig. Is it recommended to cat(command) all the gff3 files to generate a single file and finally generate hmm file with the steps mentioned in the tutorial. Would the same hmm file work as training dataset for AUGUSTUS? Many thanks, Parul Kudtarkar > You probably need to reinstall Bio-perl. > > Other things that can cause the same error are setting TMP in the > maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or > sometimes setting it to an NFS mount. > A full drive can cause this as well or a broken Berkley DB. Use df -h to > see if any of the drives used are full (either current working directory > or TMP location). You can also swap Berkley DB for a different backend by > setting AnyDBM_ISA during setup. > > Example: > cd ../maker/src/ > perl Build.PL --AnyDBM_ISA SDBM_File > ./Build install > > --Carson > > > > On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: > >>I am running maker on example data that comes along with installation and >>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >> >>Please advice on the aforementioned error. >> >>--------------------- >>Maker is now finished!!! >> >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >>eins%2Efasta.repeatrunner >>deleted:0 hits >> in cluster:shadow cluster... >> i_size:0 j_size:0 >> sorting hits in shadow cluster... >>... finished. >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >>%2Efasta.blastn >>deleted:-1 hits >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >>tein%2Efasta.blastx >>WARNING: Multiple MAKER processes have been started in the >>same directory. >> >>deleted:0 hits >>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>stop here:dpp-mRNA-4 >>ERROR: Fasta index error >> >>FATAL ERROR >>Maker is now finished!!! >> >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >>eins%2Efasta.repeatrunner >>deleted:0 hits >> in cluster:shadow cluster... >> i_size:0 j_size:0 >> sorting hits in shadow cluster... >>... finished. >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >>%2Efasta.blastn >>deleted:-1 hits >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >>tein%2Efasta.blastx >>WARNING: Multiple MAKER processes have been started in the >>same directory. >> >>deleted:0 hits >>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>stop here:dpp-mRNA-4 >>ERROR: Fasta index error >> >>FATAL ERROR >>ERROR: Failed while polishig ESTs!! >> >>ERROR: Chunk failed at level 14 >>!! >>FAILED CONTIG:contig-dpp-500-500 >> >> >> >>ERROR: Chunk failed at level 14 >>!! >>FAILED CONTIG:contig-dpp-500-500 >>------------------------------------ >> >> >>Many thanks, >>Parul Kudtarkar >> >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org >> >> >>_______________________________________________ >>maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From david.powell at monash.edu Thu Oct 4 17:44:32 2012 From: david.powell at monash.edu (David Powell) Date: Fri, 5 Oct 2012 09:44:32 +1000 Subject: [maker-devel] problem running maker In-Reply-To: References: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Message-ID: I've also had the problem with a segfault when running MAKER 2.26. I'm not sure it is the one experienced here, but in the end I was able to work around it. I tracked the problem to lib/FastA.pm in the _safe_new function. Specifically, in the local handler installed for warnings. By removing the "die" call, I was able to stop the perl process from segfaulting. I suspect this is not a great fix since it will ignore any warnings from Bio::DB::Fasta, but it worked for me (I did check the warnings generated were not important). Cheers, -- David Powell On 5 October 2012 03:17, Carson Holt wrote: > Seg faults really only happen from C and not Perl, so there may be a perl > module that is broken on your machine that use C internally. > > Here are some modules that MAKER can use that I know use C and that you > might want to reinstall from CPAN. > > DB_File > Forks > Proc::ProcessTable > Inline::C > > Also during the maker install say no to the "use MPI" question. It's best > to make the installation as simple as possible before getting into MPI > configuration. > > Also after installing those and rerunning maker's 'perl Build.PL' and > './Build install' steps do your first run with 'maker --debug' (redirect > STDERR to a file). Then if it fails, you can send me the error log. > > Thanks, > Carson > > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 11:36 AM > > To: Carson Holt , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Ok. So I installed 2.26 instead. When I ran maker again I got segmentation > fault with no additional message. I am using dpp_contig.fasta, > dpp_est.fasta and dpp_protein.fasta. > > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM > To: Oscar Jiao , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Try 2.26 and let me know. > > Thanks, > Carson > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM > To: Carson Holt , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > 2.10 > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM > To: Oscar Jiao , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Are you trying to install 2.26 or 2.10? > > --Carson > > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM > To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker > > Hi, > > I just installed maker and tried to run it with these dpp example fasta > files that came with the package. However when I do "maker maker_opts.ctl > maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like > there are three errors: (1) defined array deprecated; (2) unknown state in > Signal.pm and (3) GDBM_File (etc.) missing > > The first line seems to be just warning. I got it even when I do just > "maker" or "maker ctl". The control files did get generated. And what is > the relationship between the three perl modules, GDBM/NDBM/SDBM_File, > AnyDBM and DB_File? > > Oscar > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 20:24:16 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 22:24:16 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D7C8@EXMBX05.austin.utexas.edu> Message-ID: I don't think you will need to reinstall perl (you may but I don't think so). MVAPICH2 doesn't work because they have overridden malloc with their own version which causes weirdness with shared libraries in Perl. I've submitted some bug reports to the MVAPICH2 developers and made modifications to the development version of MAKER, and had some progress, but I am still not getting reliable behavior in MVAPICH2 when compiling for the OFA-IB-Nemesis or OFA-IB-CH3 interfaces. So for now using MPICH2 is your best choice. You lose some of the infiniband optimizations but IO rather than messages passing is MAKER's true bottleneck (so you won't get a performance hit by using MPICH2). --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 5:20 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Hi Carson, I didn't know maker does not work with MVAPICH2. The cluster I was installing maker on only has mvapich2. So I guess I will have to install mpich2 under my directory. However, I installed perl threading based on mvapich2. Does that mean I need to reinstall perl with mpich2 before reinstalling maker? Oscar -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 20:51:19 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 22:51:19 -0400 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <1764.131.215.15.234.1349391973.squirrel@webmail.caltech.edu> Message-ID: Just concatenating won't work. Use the gff3_merge tool to safely merge separate GFF3 files. In the tutorial dataset their should only be one contig to train with, but with real data you will have multiple contigs and will want to merge the GFF3 files. Augustus has a separate training procedure that is a little more complicated than SNAP's. You will need to read their documentation on training which is a little cryptic. I know Jason Stajich developed a tool for training augustus from snap results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be willing to share it with you ;-) Thanks, Carson On 12-10-04 7:06 PM, "Parul Kudtarkar" wrote: >Hi, > >Thanks, reinstalling BioPerl fixed the issue. Now we are training our >genome to generate training dataset for SNAP and Augustus, as per the >instructions provided at >http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors > >However the training dataset would have multiple gff3 prediction files, >one for each contig. Is it recommended to cat(command) all the gff3 files >to generate a single file and finally generate hmm file with the steps >mentioned in the tutorial. Would the same hmm file work as training >dataset for AUGUSTUS? > >Many thanks, >Parul Kudtarkar > >> You probably need to reinstall Bio-perl. >> >> Other things that can cause the same error are setting TMP in the >> maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or >> sometimes setting it to an NFS mount. >> A full drive can cause this as well or a broken Berkley DB. Use df -h >>to >> see if any of the drives used are full (either current working directory >> or TMP location). You can also swap Berkley DB for a different backend >>by >> setting AnyDBM_ISA during setup. >> >> Example: >> cd ../maker/src/ >> perl Build.PL --AnyDBM_ISA SDBM_File >> ./Build install >> >> --Carson >> >> >> >> On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >> >>>I am running maker on example data that comes along with installation >>>and >>>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >>> >>>Please advice on the aforementioned error. >>> >>>--------------------- >>>Maker is now finished!!! >>> >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>ot >>>eins%2Efasta.repeatrunner >>>deleted:0 hits >>> in cluster:shadow cluster... >>> i_size:0 j_size:0 >>> sorting hits in shadow cluster... >>>... finished. >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>st >>>%2Efasta.blastn >>>deleted:-1 hits >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>ro >>>tein%2Efasta.blastx >>>WARNING: Multiple MAKER processes have been started in the >>>same directory. >>> >>>deleted:0 hits >>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>stop here:dpp-mRNA-4 >>>ERROR: Fasta index error >>> >>>FATAL ERROR >>>Maker is now finished!!! >>> >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>ot >>>eins%2Efasta.repeatrunner >>>deleted:0 hits >>> in cluster:shadow cluster... >>> i_size:0 j_size:0 >>> sorting hits in shadow cluster... >>>... finished. >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>st >>>%2Efasta.blastn >>>deleted:-1 hits >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>ro >>>tein%2Efasta.blastx >>>WARNING: Multiple MAKER processes have been started in the >>>same directory. >>> >>>deleted:0 hits >>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>stop here:dpp-mRNA-4 >>>ERROR: Fasta index error >>> >>>FATAL ERROR >>>ERROR: Failed while polishig ESTs!! >>> >>>ERROR: Chunk failed at level 14 >>>!! >>>FAILED CONTIG:contig-dpp-500-500 >>> >>> >>> >>>ERROR: Chunk failed at level 14 >>>!! >>>FAILED CONTIG:contig-dpp-500-500 >>>------------------------------------ >>> >>> >>>Many thanks, >>>Parul Kudtarkar >>> >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org >>> >>> >>>_______________________________________________ >>>maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > From jiao at tacc.utexas.edu Thu Oct 4 23:06:48 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 05:06:48 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868DBE8@EXMBX05.austin.utexas.edu> I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Thu Oct 4 23:34:37 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Thu, 4 Oct 2012 22:34:37 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: References: Message-ID: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> Carson, thanks for very quick response. Jason, could you please elaborate/share document if any on using zff2augustus_gbk.pl Thanks, Parul Kudtarkar > Just concatenating won't work. Use the gff3_merge tool to safely merge > separate GFF3 files. > > In the tutorial dataset their should only be one contig to train with, but > with real data you will have multiple contigs and will want to merge the > GFF3 files. > > Augustus has a separate training procedure that is a little more > complicated than SNAP's. You will need to read their documentation on > training which is a little cryptic. > > I know Jason Stajich developed a tool for training augustus from snap > results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be > willing to share it with you ;-) > > Thanks, > Carson > > > > On 12-10-04 7:06 PM, "Parul Kudtarkar" wrote: > >>Hi, >> >>Thanks, reinstalling BioPerl fixed the issue. Now we are training our >>genome to generate training dataset for SNAP and Augustus, as per the >>instructions provided at >>http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors >> >>However the training dataset would have multiple gff3 prediction files, >>one for each contig. Is it recommended to cat(command) all the gff3 files >>to generate a single file and finally generate hmm file with the steps >>mentioned in the tutorial. Would the same hmm file work as training >>dataset for AUGUSTUS? >> >>Many thanks, >>Parul Kudtarkar >> >>> You probably need to reinstall Bio-perl. >>> >>> Other things that can cause the same error are setting TMP in the >>> maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) >>> or >>> sometimes setting it to an NFS mount. >>> A full drive can cause this as well or a broken Berkley DB. Use df -h >>>to >>> see if any of the drives used are full (either current working >>> directory >>> or TMP location). You can also swap Berkley DB for a different backend >>>by >>> setting AnyDBM_ISA during setup. >>> >>> Example: >>> cd ../maker/src/ >>> perl Build.PL --AnyDBM_ISA SDBM_File >>> ./Build install >>> >>> --Carson >>> >>> >>> >>> On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >>> >>>>I am running maker on example data that comes along with installation >>>>and >>>>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >>>> >>>>Please advice on the aforementioned error. >>>> >>>>--------------------- >>>>Maker is now finished!!! >>>> >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>>ot >>>>eins%2Efasta.repeatrunner >>>>deleted:0 hits >>>> in cluster:shadow cluster... >>>> i_size:0 j_size:0 >>>> sorting hits in shadow cluster... >>>>... finished. >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>>st >>>>%2Efasta.blastn >>>>deleted:-1 hits >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>>ro >>>>tein%2Efasta.blastx >>>>WARNING: Multiple MAKER processes have been started in the >>>>same directory. >>>> >>>>deleted:0 hits >>>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>>stop here:dpp-mRNA-4 >>>>ERROR: Fasta index error >>>> >>>>FATAL ERROR >>>>Maker is now finished!!! >>>> >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>>ot >>>>eins%2Efasta.repeatrunner >>>>deleted:0 hits >>>> in cluster:shadow cluster... >>>> i_size:0 j_size:0 >>>> sorting hits in shadow cluster... >>>>... finished. >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>>st >>>>%2Efasta.blastn >>>>deleted:-1 hits >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>>ro >>>>tein%2Efasta.blastx >>>>WARNING: Multiple MAKER processes have been started in the >>>>same directory. >>>> >>>>deleted:0 hits >>>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>>stop here:dpp-mRNA-4 >>>>ERROR: Fasta index error >>>> >>>>FATAL ERROR >>>>ERROR: Failed while polishig ESTs!! >>>> >>>>ERROR: Chunk failed at level 14 >>>>!! >>>>FAILED CONTIG:contig-dpp-500-500 >>>> >>>> >>>> >>>>ERROR: Chunk failed at level 14 >>>>!! >>>>FAILED CONTIG:contig-dpp-500-500 >>>>------------------------------------ >>>> >>>> >>>>Many thanks, >>>>Parul Kudtarkar >>>> >>>>-- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org >>>> >>>> >>>>_______________________________________________ >>>>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> >> >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org >> > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jason.stajich at ucr.edu Fri Oct 5 01:02:50 2012 From: jason.stajich at ucr.edu (Jason E Stajich) Date: Fri, 5 Oct 2012 07:02:50 +0000 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> References: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> Message-ID: <41A4465BD48EEE47A449FA03108C852A073651E2@EXCH-MBOX-3.exch.ucr.edu> I wrote up how to use it on this list msg - http://brie4.cshl.edu/pipermail/gmod-help/2012-June/001724.html Script is in this github repo - https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl Patches and documentation updates are always welcomed, I'll get around to that eventually. Jason On Oct 4, 2012, at 10:34 PM, Parul Kudtarkar > wrote: Carson, thanks for very quick response. Jason, could you please elaborate/share document if any on using zff2augustus_gbk.pl Thanks, Parul Kudtarkar Just concatenating won't work. Use the gff3_merge tool to safely merge separate GFF3 files. In the tutorial dataset their should only be one contig to train with, but with real data you will have multiple contigs and will want to merge the GFF3 files. Augustus has a separate training procedure that is a little more complicated than SNAP's. You will need to read their documentation on training which is a little cryptic. I know Jason Stajich developed a tool for training augustus from snap results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be willing to share it with you ;-) Thanks, Carson On 12-10-04 7:06 PM, "Parul Kudtarkar" > wrote: Hi, Thanks, reinstalling BioPerl fixed the issue. Now we are training our genome to generate training dataset for SNAP and Augustus, as per the instructions provided at http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors However the training dataset would have multiple gff3 prediction files, one for each contig. Is it recommended to cat(command) all the gff3 files to generate a single file and finally generate hmm file with the steps mentioned in the tutorial. Would the same hmm file work as training dataset for AUGUSTUS? Many thanks, Parul Kudtarkar You probably need to reinstall Bio-perl. Other things that can cause the same error are setting TMP in the maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or sometimes setting it to an NFS mount. A full drive can cause this as well or a broken Berkley DB. Use df -h to see if any of the drives used are full (either current working directory or TMP location). You can also swap Berkley DB for a different backend by setting AnyDBM_ISA during setup. Example: cd ../maker/src/ perl Build.PL --AnyDBM_ISA SDBM_File ./Build install --Carson On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: I am running maker on example data that comes along with installation and is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note Please advice on the aforementioned error. --------------------- Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr ot eins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e st %2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p ro tein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr ot eins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e st %2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p ro tein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ------------------------------------ Many thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -- Jason E Stajich, PhD Assistant Professor Plant Pathology & Microbiology University of California, Riverside 951.827.2363 http://lab.stajich.org http://fungalgenomes.org http://fungidb.org http://1000.fungalgenomes.org/ twitter @stajichlab @hyphaltip @fungalgenomes @fungidb http://plantpathology.ucr.edu http://genomics.ucr.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From Carson.Holt at oicr.on.ca Thu Oct 4 13:19:57 2012 From: Carson.Holt at oicr.on.ca (Carson Holt) Date: Thu, 4 Oct 2012 19:19:57 +0000 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Yes. You really have to because the single exon transcripts produced from mRNA-seq assembly are strandless (you don't know where they belong). They also tend to be heavily weighted to pseudogenes and transposons. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 3:16 PM To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 Does the cufflinks2gff3 script filter out single-exon transcripts? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt > wrote: Use the cufflinks2gff3 script that comes with MAKER. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 3:07 PM To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only transcript and exon features. So I'm assuming I can simply convert the transcript features to match and the exon features to match_part and make sure the parent/child relationships are maintained with the Parent and ID attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt > wrote: Match/match_part features are expected. MAKER expects these to be polished (I.e. correctly aligned around splice sites). For example exonerate, BLAT, and cufflinks results will be correct around splice sites and can be used; BLAST results on the other hand will not be correct and should not be used. The Gap attribute is used by maker if available, but is not required (Gap describes how to reconstruct an alignment for gaps and mismatches). Otherwise MAKER assumes all positions are matches to the reference. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 Greetings. I would like to use Maker's est_gff option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 10:05:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 12:05:56 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868DBE8@EXMBX05.austin.utexas.edu> Message-ID: Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 10:34:12 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 12:34:12 -0400 Subject: [maker-devel] editing MAKER mailing list options Message-ID: Just a small note to subscribers to the MAKER e-mail list. Being as there are spurts of high volume list activity that may be a nuisance to some, if you want to edit your mailing list options visit this link --> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org You can then change to digest mode if you don't want to receive every single message posted to the list, or you can set your options to be subscribed to but not receive messages from the list (so you can post and get help but won't get everyone else's posts). Enter your e-mail address in the box at the very very bottom of the page to edit option. Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 12:19:15 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 14:19:15 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E519@EXMBX05.austin.utexas.edu> Message-ID: Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 12:16:41 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 18:16:41 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E519@EXMBX05.austin.utexas.edu> See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: maker_opts.ctl Type: application/octet-stream Size: 4468 bytes Desc: maker_opts.ctl URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mpich-maker-debug.log Type: application/octet-stream Size: 315735 bytes Desc: mpich-maker-debug.log URL: From jiao at tacc.utexas.edu Fri Oct 5 13:55:39 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 19:55:39 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E7A3@EXMBX05.austin.utexas.edu> I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 14:03:09 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 16:03:09 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E7A3@EXMBX05.austin.utexas.edu> Message-ID: What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Ap plication/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.p m line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/M PI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 14:06:02 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 20:06:02 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E866@EXMBX05.austin.utexas.edu> Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 14:07:18 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 16:07:18 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E866@EXMBX05.austin.utexas.edu> Message-ID: Is /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2 a new install of mpich2 or is it the install you did with 2.26 just copied over? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 4:06 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Ap plication/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.p m line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/M PI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 14:08:35 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 20:08:35 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E8D6@EXMBX05.austin.utexas.edu> It is a new install. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:07 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Is /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2 a new install of mpich2 or is it the install you did with 2.26 just copied over? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 4:06 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisi.hahni at gmail.com Tue Oct 9 05:07:14 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 09 Oct 2012 13:07:14 +0200 Subject: [maker-devel] install maker on cluster Message-ID: <50740562.9010904@gmail.com> Hello maker-team, I am trying to install maker 2.25 on a cluster, without root privileges (I managed before, but now after migration to a new cluster I cant seem to get it to work). All necessary prerequisite programs are installed and in the path (I followed the steps in the INSTALL file that comes with maker2.25). When I do: perl Build.PL, I get: Checking prerequisites... requires: ! DBD::SQLite is not installed ! IO::All is not installed ! Inline::C is not installed ! Perl::Unsafe::Signals is not installed ! Proc::ProcessTable is not installed ! Want is not installed ! forks is not installed ! forks::shared is not installed build_requires: ! LWP::Simple is not installed recommends: * DBD::Pg is not installed ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the versions of the modules indicated above before proceeding with this installation Run 'Build installdeps' to install missing prerequisites. MAKER supports distributed parallelization via MPI. Would you like to configure MAKER for MPI (This requires that you have an MPI client installed)? [N ]n WARNING: Apache cannot be located. The optional web based interface to MAKER will not be available to you. Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'MAKER' version '2.25' The file 'Build' has been created for you to finish installing MAKER. ============================================================================== STATUS MAKER 2.25 ============================================================================== PERL Dependencies: MISSING ! forks ! IO::All ! forks::shared ! Want ! DBD::SQLite ! Proc::ProcessTable ! Inline::C ! Perl::Unsafe::Signals External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: DISABLED MWAS Web Interface: DISABLED MAKER PACKAGE: MISSING PREREQUISITES Important Commands: ./Build installdeps #installs missing PERL dependencies ./Build installexes #installs all missing external programs ./Build install #installs MAKER ./Build status #Shows this status menu Other Commands: ./Build repeatmasker #installs RepeatMasker (asks for RepBase) ./Build blast #installs BLAST (NCBI BLAST+) ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) ./Build snap #installs SNAP ./Build augustus #installs Augustus ./Build apollo #installs Apollo ./Build gbrowse #installs GBrowse (must be root) ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) ./Build mpich2 #installs MPICH2 (but manual install recommended) So it seems some Perl Dependencies are missing. As indicated in INSTALL I followed the quick and dirty installation for Bioperl, so I am not sure what I did wrong/missed out. When I run ./Build installdeps, I get: You do not have write access to install missing Modules. I can try and install these locally (i.e. only for MAKER) in the .../maker/perl/lib directory, or you can run './Build installdeps' as root or using sudo and try again. Do want MAKER to try and build a local installation? [N ]y mkdir /usit/titan: Permission denied at /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm line 59 Sorry to bother you with this minor things!! Any help would me much appreciated! much obliged, Christoph From carsonhh at gmail.com Tue Oct 9 10:27:36 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 09 Oct 2012 12:27:36 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <50740562.9010904@gmail.com> Message-ID: You may need to reconfigure your cpan preferences. You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, or by starting cpan from your command line (command is cpan). Then run 'o conf init' inside the cpan ui. --Carson On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >Hello maker-team, > >I am trying to install maker 2.25 on a cluster, without root privileges >(I managed before, but now after migration to a new cluster I cant seem >to get it to work). All necessary prerequisite programs are installed >and in the path (I followed the steps in the INSTALL file that comes >with maker2.25). When I do: perl Build.PL, I get: >Checking prerequisites... > requires: > ! DBD::SQLite is not installed > ! IO::All is not installed > ! Inline::C is not installed > ! Perl::Unsafe::Signals is not installed > ! Proc::ProcessTable is not installed > ! Want is not installed > ! forks is not installed > ! forks::shared is not installed > build_requires: > ! LWP::Simple is not installed > recommends: > * DBD::Pg is not installed > >ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >versions >of the modules indicated above before proceeding with this installation > >Run 'Build installdeps' to install missing prerequisites. > >MAKER supports distributed parallelization via MPI. >Would you like to configure MAKER for MPI (This >requires that you have an MPI client installed)? [N ]n > >WARNING: Apache cannot be located. The optional web based >interface to MAKER will not be available to you. > >Created MYMETA.yml and MYMETA.json >Creating new 'Build' script for 'MAKER' version '2.25' > > >The file 'Build' has been created for you to finish installing MAKER. > > >========================================================================== >==== >STATUS MAKER 2.25 >========================================================================== >==== >PERL Dependencies: MISSING > ! forks > ! IO::All > ! forks::shared > ! Want > ! DBD::SQLite > ! Proc::ProcessTable > ! Inline::C > ! Perl::Unsafe::Signals > >External Programs: VERIFIED >External C Libraries: VERIFIED >MPI SUPPORT: DISABLED >MWAS Web Interface: DISABLED >MAKER PACKAGE: MISSING PREREQUISITES > > >Important Commands: > ./Build installdeps #installs missing PERL dependencies > ./Build installexes #installs all missing external programs > ./Build install #installs MAKER > ./Build status #Shows this status menu > >Other Commands: > ./Build repeatmasker #installs RepeatMasker (asks for RepBase) > ./Build blast #installs BLAST (NCBI BLAST+) > ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on >Mac OSX) > ./Build snap #installs SNAP > ./Build augustus #installs Augustus > ./Build apollo #installs Apollo > ./Build gbrowse #installs GBrowse (must be root) > ./Build jbrowse #installs JBrowse (MAKER copy, not web >accecible) > ./Build mpich2 #installs MPICH2 (but manual install >recommended) > >So it seems some Perl Dependencies are missing. As indicated in INSTALL >I followed the quick and dirty installation for Bioperl, so I am not >sure what I did wrong/missed out. > >When I run ./Build installdeps, I get: >You do not have write access to install missing Modules. >I can try and install these locally (i.e. only for MAKER) >in the .../maker/perl/lib directory, or you can run >'./Build installdeps' as root or using sudo and try again. >Do want MAKER to try and build a local installation? [N ]y >mkdir /usit/titan: Permission denied at >/cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm line >59 > >Sorry to bother you with this minor things!! Any help would me much >appreciated! > >much obliged, >Christoph > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From kippjohnson at uchicago.edu Thu Oct 11 13:43:32 2012 From: kippjohnson at uchicago.edu (Kipp Johnson) Date: Thu, 11 Oct 2012 14:43:32 -0500 Subject: [maker-devel] Interpreting Maker Results Message-ID: Hi Carson, I'm trying to get my genetic annotation out of maker. My maker run on a non-model eukaryote finished, and I used your gff3_merge script to merge the resulting files. This file is enormous, because I used snap, augustus, genemark, repeatmasker, exonerate, and blast, and has a lot of entries from all of these different programs. I want to extract only the genetic regions predicted by maker, so I used the gff3_merge script with the "-g" option. However, when I do this, I get a maker file that only has about 12,000 genes, while I was expecting around 20,000 genes for our genome. However, when I use the fasta merge tool, however, I get output files (for example, "merged.fasta.all.maker.non_overlapping_ab_initio.proteins.fasta") with about 21,000 proteins, which is closer to the gene number that I was expecting. Does the "-g" option ignore evidence from blast/exonerate or similar? How should I extract the complete set of genetic regions to blast against, so that I can go about further working on the annotation? Also, what is maker using to find these 9,000 extra proteins? Are these these all alternately sliced or something along those lines? I can't find any documentation online for how to actually get the final annotations out of maker correctly. Thanks for your time! Best, Kipp Johnson kippjohnson at uchicago.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 11 16:21:49 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 11 Oct 2012 18:21:49 -0400 Subject: [maker-devel] Interpreting Maker Results In-Reply-To: Message-ID: The non_overlapping_ab_initio.proteins.fasta file contain models that were rejected for lack of homology evidence that do not overlap the maker models. So if you accept everything it is not 21,000 proteins rather it's 33,000 (12,000 + 21,000). This is because the contents of that file do not occupy the same regions as the gene models (I.e. non-overlapping). Ab initio predictors have a tendency to overpredict which is why there are so many models. If you think you are missing genes you can try and add additional evidence to the maker run (I.e all proteins from a few related species). Also try running Cegma (http://korflab.ucdavis.edu/datasets/cegma/) to estimate the completeness of you assembly. Sometime a lower number than expected can be attributed to an incomplete assembly. Also you can run something like InterProScan to identify models in the non_overlapping_ab_initio.proteins.fasta file that contain Uniprot protein domains (likely to be real genes), then add them to your results as a second step using maker's model_gff option. I've attached a couple of scripts that can help with that. gff3_preds2models will turn a set of match/match_part features to gene/mRNA/exon/CDS features. It doesn't check translation of CDS though, so only use gene predictions which should be all CDS. gff3_select is used to select some subset of features from a GFF3 file. Useful for slicing sections of data from a GFF3 file. Thanks, Carson From: Kipp Johnson Date: Thursday, 11 October, 2012 3:43 PM To: , Carson Holt Subject: Interpreting Maker Results Hi Carson, I'm trying to get my genetic annotation out of maker. My maker run on a non-model eukaryote finished, and I used your gff3_merge script to merge the resulting files. This file is enormous, because I used snap, augustus, genemark, repeatmasker, exonerate, and blast, and has a lot of entries from all of these different programs. I want to extract only the genetic regions predicted by maker, so I used the gff3_merge script with the "-g" option. However, when I do this, I get a maker file that only has about 12,000 genes, while I was expecting around 20,000 genes for our genome. However, when I use the fasta merge tool, however, I get output files (for example, "merged.fasta.all.maker.non_overlapping_ab_initio.proteins.fasta") with about 21,000 proteins, which is closer to the gene number that I was expecting. Does the "-g" option ignore evidence from blast/exonerate or similar? How should I extract the complete set of genetic regions to blast against, so that I can go about further working on the annotation? Also, what is maker using to find these 9,000 extra proteins? Are these these all alternately sliced or something along those lines? I can't find any documentation online for how to actually get the final annotations out of maker correctly. Thanks for your time! Best, Kipp Johnson kippjohnson at uchicago.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gff3_preds2models Type: application/octet-stream Size: 4778 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gff3_select Type: application/octet-stream Size: 3237 bytes Desc: not available URL: From parulk at caltech.edu Fri Oct 12 15:46:21 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Fri, 12 Oct 2012 14:46:21 -0700 (PDT) Subject: [maker-devel] Conensus gene model Message-ID: <4092.131.215.15.234.1350078381.squirrel@webmail.caltech.edu> Hi, We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 file as input). The output that we get is combination of various gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? Thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig2106.gff Type: application/octet-stream Size: 10563 bytes Desc: not available URL: From parulk at caltech.edu Mon Oct 15 11:41:54 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 15 Oct 2012 10:41:54 -0700 (PDT) Subject: [maker-devel] Conensus gene model Message-ID: <2992.131.215.15.234.1350322914.squirrel@webmail.caltech.edu> Hello, Please advice on the aforementioned query? Thanks, Parul Kudtarkar ---------------------------- Original Message ---------------------------- Subject: [maker-devel] Conensus gene model From: "Parul Kudtarkar" Date: Fri, October 12, 2012 2:46 pm To: maker-devel at yandell-lab.org -------------------------------------------------------------------------- Hi, We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 file as input). The output that we get is combination of various gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? Thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org_______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig2106.gff Type: application/octet-stream Size: 10563 bytes Desc: not available URL: From carsonhh at gmail.com Mon Oct 15 11:58:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 15 Oct 2012 13:58:25 -0400 Subject: [maker-devel] Conensus gene model In-Reply-To: <2992.131.215.15.234.1350322914.squirrel@webmail.caltech.edu> Message-ID: The contig in question is really too small to get much out of it (only 5 kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb length or longer contigs to glean enough information for optimizing your parameters. The general suggestions for any maker run are to use proteins from a closely related organism or a couple of closely related organisms for the protein= option in maker. Also leave single_exon set to 0, except for certain eukaryotes that have a bias for single exon transcripts (i.e. some fungi and oomycetes). And leave keep_preds set to 0 because ab initio predictors tend to over-predict by a wide margin (lots of false positives). Additional training would really depend on what your other contigs look like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. Thanks, Carson On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >Hello, > >Please advice on the aforementioned query? > >Thanks, >Parul Kudtarkar >---------------------------- Original Message ---------------------------- >Subject: [maker-devel] Conensus gene model >From: "Parul Kudtarkar" >Date: Fri, October 12, 2012 2:46 pm >To: maker-devel at yandell-lab.org >-------------------------------------------------------------------------- > >Hi, > >We are using snap(training set[hmm file] generated using est,protein and >contig file), agustus,genemarkE(we ran it outside maker and have gff3 file >as input). The output that we get is combination of various >gene-predictors and evidences. I have attached sample result file. What >would you recommend to get consensus result set? Bootstrapping the >resulting gff3 file (rerunning maker)? > >Thanks, >Parul Kudtarkar >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Mon Oct 15 14:10:00 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 15 Oct 2012 16:10:00 -0400 Subject: [maker-devel] Conensus gene model In-Reply-To: <4445.131.215.15.234.1350330328.squirrel@webmail.caltech.edu> Message-ID: One thing you seem to be missing is protein evidence. Is this a sea urchin (I looked up some of the ESTs)? If so, I would recommend adding all proteins from the Strongylocentrotus purpuratus genome, then throw in another Deuterstome of your choice. Perhaps you should also add a couple of outgroup organisms like Nematostella vectensis (cnidaria) and a protostome of your choice. Be careful if adding adding to many protostome outgroups (i.e. C. elegans and Drosophila) because a big part of their evolution is gene loss (so distant cnidaria often match deuterstomes better than most protostomes do). You could take the maker results when protein data is included and use it to retrain SNAP again. Even a 22 kb contig is still really short. Is this genome primarily constituted by short contigs like this? I would recommend running CEGMA once on this genome to get an appropriate estimate of how recoverable the genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). Cegma will give you an estimate for genome completeness as well as estimates of what percentage of genes will be found in their entirety and what percent will be partial genes. This is important to do if your genome is fragmented as it will give you a reasonable expectation of what you can expected to recover (as short contigs don't annotate very well - you tend to loose a lot). Thanks, Carson On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >Hi Carson, > >Thanks. I have attached another contig which is 22 kb, with as many as 3 >exons EST alignments. Could you please recommend additional training. We >are currently running maker on the entire contig set and eventually merge >all the gff3 contig predictions. The using suggested parameter/methods we >would like to get a consensus gene-set with minimal false >positives/negatives. > >Thanks, >Parul > >> The contig in question is really too small to get much out of it (only 5 >kb). There was only one single exon EST alignments and a couple of >predictions with no evidence support. Anything smaller than 10 kb is >mostly useless for annotation purposes. You would really need a few >100kb >> length or longer contigs to glean enough information for optimizing your >parameters. >> >> The general suggestions for any maker run are to use proteins from a >closely related organism or a couple of closely related organisms for >the >> protein= option in maker. Also leave single_exon set to 0, except for >certain eukaryotes that have a bias for single exon transcripts (i.e. >some >> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >predictors tend to over-predict by a wide margin (lots of false >> positives). >> >> Additional training would really depend on what your other contigs look >like. Do you have any large contigs? I could look at one of those and >give suggestions but the provided contig is just too short to glean >much. >> >> Thanks, >> Carson >> >> >> >> >> >> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >> >>>Hello, >>>Please advice on the aforementioned query? >>>Thanks, >>>Parul Kudtarkar >>>---------------------------- Original Message >>> ---------------------------- >>>Subject: [maker-devel] Conensus gene model >>>From: "Parul Kudtarkar" >>>Date: Fri, October 12, 2012 2:46 pm >>>To: maker-devel at yandell-lab.org >>>------------------------------------------------------------------------ >>>-- >Hi, >>>We are using snap(training set[hmm file] generated using est,protein and >contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>> file >>>as input). The output that we get is combination of various >>>gene-predictors and evidences. I have attached sample result file. What >would you recommend to get consensus result set? Bootstrapping the >resulting gff3 file (rerunning maker)? >>>Thanks, >>>Parul Kudtarkar >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org_______________________________________________ >maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org_______________________________________________ >maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > From chrisi.hahni at gmail.com Tue Oct 16 08:02:36 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 16:02:36 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D68FC.10201@gmail.com> Thanks for your help Carson! I tried maker2.26, but the error still remains. I also tried to change the TMP= option in the maker_opts file, but no change either.. STATUS: Parsing control files... ERROR: The HMM file[s] provided for gmhmm do not exist: When building the maker2.26 I saw that it is complaining about one more (recommended) dependency (I must have overlooked that before): Checking prerequisites... recommends: * DBD::Pg is not installed I am having problems to get that one.. Is that causing the error? much obliged, Christoph Am 15.10.2012 20:05, schrieb Carson Holt: > It looks like either an NFS issue or perhaps an issue with the location > provided for the TMP= in the maker_opts.ctl file. > > First I would suggest trying maker 2.26 to see if the issue still happens > there (there are a few updates to how NFS locks are handled in 2.26). > > Second if you are setting TMP make sure your disk is not full. Or if TMP > was set to a tmpfs location (in memory filesystem), it could be a full > memory issue in which case you could just set TMP to somewhere else. > > Let me know if you still see the issue in 2.26 or after changing the > location of TMP. > > Thanks, > Carson > > > On 12-10-12 1:09 PM, "Christoph Hahn" wrote: > >> Hi Carson, >> >> Thanks for your reply. I managed to install maker2.25 properly now, but >> when running it I am getting strange errors (two kinds): >> type one: >> STATUS: Parsing control files... >> ERROR: The HMM file[s] provided for gmhmm do not exist: >> >> type two: >> STATUS: Parsing control files... >> ERROR: Cannot get initialization lock. >> >> I have divided the draft assembly to several files, each containing a >> subset of contigs on which I am running a maker instance. I am getting >> the above named error type one for the majority of the instances, >> although the HMM file it is complaining about (es.mod) is actually in >> the path it is naming (not shown above). In a small number of instances >> I am getting the type two error mentioned above. >> >> Can you help me? >> >> much obliged, >> Christoph >> >> >> >> On 09.10.2012 18:27, Carson Holt wrote: >>> You may need to reconfigure your cpan preferences. >>> >>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>> or >>> by starting cpan from your command line (command is cpan). Then run 'o >>> conf init' inside the cpan ui. >>> >>> --Carson >>> >>> >>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>> >>>> Hello maker-team, >>>> >>>> I am trying to install maker 2.25 on a cluster, without root privileges >>>> (I managed before, but now after migration to a new cluster I cant seem >>>> to get it to work). All necessary prerequisite programs are installed >>>> and in the path (I followed the steps in the INSTALL file that comes >>>> with maker2.25). When I do: perl Build.PL, I get: >>>> Checking prerequisites... >>>> requires: >>>> ! DBD::SQLite is not installed >>>> ! IO::All is not installed >>>> ! Inline::C is not installed >>>> ! Perl::Unsafe::Signals is not installed >>>> ! Proc::ProcessTable is not installed >>>> ! Want is not installed >>>> ! forks is not installed >>>> ! forks::shared is not installed >>>> build_requires: >>>> ! LWP::Simple is not installed >>>> recommends: >>>> * DBD::Pg is not installed >>>> >>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>> versions >>>> of the modules indicated above before proceeding with this installation >>>> >>>> Run 'Build installdeps' to install missing prerequisites. >>>> >>>> MAKER supports distributed parallelization via MPI. >>>> Would you like to configure MAKER for MPI (This >>>> requires that you have an MPI client installed)? [N ]n >>>> >>>> WARNING: Apache cannot be located. The optional web based >>>> interface to MAKER will not be available to you. >>>> >>>> Created MYMETA.yml and MYMETA.json >>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>> >>>> >>>> The file 'Build' has been created for you to finish installing MAKER. >>>> >>>> >>>> >>>> ======================================================================== >>>> == >>>> ==== >>>> STATUS MAKER 2.25 >>>> >>>> ======================================================================== >>>> == >>>> ==== >>>> PERL Dependencies: MISSING >>>> ! forks >>>> ! IO::All >>>> ! forks::shared >>>> ! Want >>>> ! DBD::SQLite >>>> ! Proc::ProcessTable >>>> ! Inline::C >>>> ! Perl::Unsafe::Signals >>>> >>>> External Programs: VERIFIED >>>> External C Libraries: VERIFIED >>>> MPI SUPPORT: DISABLED >>>> MWAS Web Interface: DISABLED >>>> MAKER PACKAGE: MISSING PREREQUISITES >>>> >>>> >>>> Important Commands: >>>> ./Build installdeps #installs missing PERL dependencies >>>> ./Build installexes #installs all missing external >>>> programs >>>> ./Build install #installs MAKER >>>> ./Build status #Shows this status menu >>>> >>>> Other Commands: >>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>> RepBase) >>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>> ./Build exonerate #installs Exonerate (v2 on UNIX / v1 >>>> on >>>> Mac OSX) >>>> ./Build snap #installs SNAP >>>> ./Build augustus #installs Augustus >>>> ./Build apollo #installs Apollo >>>> ./Build gbrowse #installs GBrowse (must be root) >>>> ./Build jbrowse #installs JBrowse (MAKER copy, not web >>>> accecible) >>>> ./Build mpich2 #installs MPICH2 (but manual install >>>> recommended) >>>> >>>> So it seems some Perl Dependencies are missing. As indicated in INSTALL >>>> I followed the quick and dirty installation for Bioperl, so I am not >>>> sure what I did wrong/missed out. >>>> >>>> When I run ./Build installdeps, I get: >>>> You do not have write access to install missing Modules. >>>> I can try and install these locally (i.e. only for MAKER) >>>> in the .../maker/perl/lib directory, or you can run >>>> './Build installdeps' as root or using sudo and try again. >>>> Do want MAKER to try and build a local installation? [N ]y >>>> mkdir /usit/titan: Permission denied at >>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>> line >>>> 59 >>>> >>>> Sorry to bother you with this minor things!! Any help would me much >>>> appreciated! >>>> >>>> much obliged, >>>> Christoph >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> > From parulk at caltech.edu Mon Oct 15 13:45:28 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 15 Oct 2012 12:45:28 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: Message-ID: <4445.131.215.15.234.1350330328.squirrel@webmail.caltech.edu> Hi Carson, Thanks. I have attached another contig which is 22 kb, with as many as 3 exons EST alignments. Could you please recommend additional training. We are currently running maker on the entire contig set and eventually merge all the gff3 contig predictions. The using suggested parameter/methods we would like to get a consensus gene-set with minimal false positives/negatives. Thanks, Parul > The contig in question is really too small to get much out of it (only 5 kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb > length or longer contigs to glean enough information for optimizing your parameters. > > The general suggestions for any maker run are to use proteins from a closely related organism or a couple of closely related organisms for the > protein= option in maker. Also leave single_exon set to 0, except for certain eukaryotes that have a bias for single exon transcripts (i.e. some > fungi and oomycetes). And leave keep_preds set to 0 because ab initio predictors tend to over-predict by a wide margin (lots of false > positives). > > Additional training would really depend on what your other contigs look like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. > > Thanks, > Carson > > > > > > On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: > >>Hello, >>Please advice on the aforementioned query? >>Thanks, >>Parul Kudtarkar >>---------------------------- Original Message >> ---------------------------- >>Subject: [maker-devel] Conensus gene model >>From: "Parul Kudtarkar" >>Date: Fri, October 12, 2012 2:46 pm >>To: maker-devel at yandell-lab.org >>-------------------------------------------------------------------------- Hi, >>We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 >> file >>as input). The output that we get is combination of various >>gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? >>Thanks, >>Parul Kudtarkar >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org_______________________________________________ maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org_______________________________________________ maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig7.gff Type: application/octet-stream Size: 68448 bytes Desc: not available URL: From carsonhh at gmail.com Tue Oct 16 08:19:21 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 16 Oct 2012 10:19:21 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <507D68FC.10201@gmail.com> Message-ID: No. that's just a recommendation for the maker2chado script to work. Is es.mod the actual HMM or a soft link to the HMM? After the error message their should be a list of the actual files. Does it print that? Are there trailing commas in you maker_opts.ctl file after the hmm file name in the control files? Are their ':' characters in the path name to the file? --Carson On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >Thanks for your help Carson! > >I tried maker2.26, but the error still remains. I also tried to change >the TMP= option in the maker_opts file, but no change either.. > >STATUS: Parsing control files... >ERROR: The HMM file[s] provided for gmhmm do not exist: > >When building the maker2.26 I saw that it is complaining about one more >(recommended) dependency (I must have overlooked that before): > Checking prerequisites... > recommends: > * DBD::Pg is not installed > >I am having problems to get that one.. Is that causing the error? > >much obliged, >Christoph > >Am 15.10.2012 20:05, schrieb Carson Holt: >> It looks like either an NFS issue or perhaps an issue with the location >> provided for the TMP= in the maker_opts.ctl file. >> >> First I would suggest trying maker 2.26 to see if the issue still >>happens >> there (there are a few updates to how NFS locks are handled in 2.26). >> >> Second if you are setting TMP make sure your disk is not full. Or if >>TMP >> was set to a tmpfs location (in memory filesystem), it could be a full >> memory issue in which case you could just set TMP to somewhere else. >> >> Let me know if you still see the issue in 2.26 or after changing the >> location of TMP. >> >> Thanks, >> Carson >> >> >> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >> >>> Hi Carson, >>> >>> Thanks for your reply. I managed to install maker2.25 properly now, but >>> when running it I am getting strange errors (two kinds): >>> type one: >>> STATUS: Parsing control files... >>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>> >>> type two: >>> STATUS: Parsing control files... >>> ERROR: Cannot get initialization lock. >>> >>> I have divided the draft assembly to several files, each containing a >>> subset of contigs on which I am running a maker instance. I am getting >>> the above named error type one for the majority of the instances, >>> although the HMM file it is complaining about (es.mod) is actually in >>> the path it is naming (not shown above). In a small number of instances >>> I am getting the type two error mentioned above. >>> >>> Can you help me? >>> >>> much obliged, >>> Christoph >>> >>> >>> >>> On 09.10.2012 18:27, Carson Holt wrote: >>>> You may need to reconfigure your cpan preferences. >>>> >>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>>> or >>>> by starting cpan from your command line (command is cpan). Then run >>>>'o >>>> conf init' inside the cpan ui. >>>> >>>> --Carson >>>> >>>> >>>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>>> >>>>> Hello maker-team, >>>>> >>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>privileges >>>>> (I managed before, but now after migration to a new cluster I cant >>>>>seem >>>>> to get it to work). All necessary prerequisite programs are installed >>>>> and in the path (I followed the steps in the INSTALL file that comes >>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>> Checking prerequisites... >>>>> requires: >>>>> ! DBD::SQLite is not installed >>>>> ! IO::All is not installed >>>>> ! Inline::C is not installed >>>>> ! Perl::Unsafe::Signals is not installed >>>>> ! Proc::ProcessTable is not installed >>>>> ! Want is not installed >>>>> ! forks is not installed >>>>> ! forks::shared is not installed >>>>> build_requires: >>>>> ! LWP::Simple is not installed >>>>> recommends: >>>>> * DBD::Pg is not installed >>>>> >>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>>> versions >>>>> of the modules indicated above before proceeding with this >>>>>installation >>>>> >>>>> Run 'Build installdeps' to install missing prerequisites. >>>>> >>>>> MAKER supports distributed parallelization via MPI. >>>>> Would you like to configure MAKER for MPI (This >>>>> requires that you have an MPI client installed)? [N ]n >>>>> >>>>> WARNING: Apache cannot be located. The optional web based >>>>> interface to MAKER will not be available to you. >>>>> >>>>> Created MYMETA.yml and MYMETA.json >>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>> >>>>> >>>>> The file 'Build' has been created for you to finish installing MAKER. >>>>> >>>>> >>>>> >>>>> >>>>>====================================================================== >>>>>== >>>>> == >>>>> ==== >>>>> STATUS MAKER 2.25 >>>>> >>>>> >>>>>====================================================================== >>>>>== >>>>> == >>>>> ==== >>>>> PERL Dependencies: MISSING >>>>> ! forks >>>>> ! IO::All >>>>> ! forks::shared >>>>> ! Want >>>>> ! DBD::SQLite >>>>> ! Proc::ProcessTable >>>>> ! Inline::C >>>>> ! Perl::Unsafe::Signals >>>>> >>>>> External Programs: VERIFIED >>>>> External C Libraries: VERIFIED >>>>> MPI SUPPORT: DISABLED >>>>> MWAS Web Interface: DISABLED >>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>> >>>>> >>>>> Important Commands: >>>>> ./Build installdeps #installs missing PERL dependencies >>>>> ./Build installexes #installs all missing external >>>>> programs >>>>> ./Build install #installs MAKER >>>>> ./Build status #Shows this status menu >>>>> >>>>> Other Commands: >>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>> RepBase) >>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>> ./Build exonerate #installs Exonerate (v2 on UNIX / >>>>>v1 >>>>> on >>>>> Mac OSX) >>>>> ./Build snap #installs SNAP >>>>> ./Build augustus #installs Augustus >>>>> ./Build apollo #installs Apollo >>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>> ./Build jbrowse #installs JBrowse (MAKER copy, not >>>>>web >>>>> accecible) >>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>install >>>>> recommended) >>>>> >>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>INSTALL >>>>> I followed the quick and dirty installation for Bioperl, so I am not >>>>> sure what I did wrong/missed out. >>>>> >>>>> When I run ./Build installdeps, I get: >>>>> You do not have write access to install missing Modules. >>>>> I can try and install these locally (i.e. only for MAKER) >>>>> in the .../maker/perl/lib directory, or you can run >>>>> './Build installdeps' as root or using sudo and try again. >>>>> Do want MAKER to try and build a local installation? [N ]y >>>>> mkdir /usit/titan: Permission denied at >>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>> line >>>>> 59 >>>>> >>>>> Sorry to bother you with this minor things!! Any help would me much >>>>> appreciated! >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> >>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>g >>>> >> > > From chrisi.hahni at gmail.com Tue Oct 16 09:14:36 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 17:14:36 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D79DC.5090702@gmail.com> Hi Carson, It was a link. The path was not correct any more because of the migration to the new cluster. I could not change it so I just gave the path to the real file in the maker_opts file. That has resolved this error, but now I am getting the following: STATUS: Parsing control files... ERROR: Cannot get initialization lock. Probably not relevant any more, but: No trailing commas, no ':' characters in the path. Thanks for your help! Christoph Am 16.10.2012 16:19, schrieb Carson Holt: > No. that's just a recommendation for the maker2chado script to work. > > Is es.mod the actual HMM or a soft link to the HMM? > After the error message their should be a list of the actual files. Does > it print that? > Are there trailing commas in you maker_opts.ctl file after the hmm file > name in the control files? > Are their ':' characters in the path name to the file? > > --Carson > > > > On 12-10-16 10:02 AM, "Christoph Hahn" wrote: > >> Thanks for your help Carson! >> >> I tried maker2.26, but the error still remains. I also tried to change >> the TMP= option in the maker_opts file, but no change either.. >> >> STATUS: Parsing control files... >> ERROR: The HMM file[s] provided for gmhmm do not exist: >> >> When building the maker2.26 I saw that it is complaining about one more >> (recommended) dependency (I must have overlooked that before): >> Checking prerequisites... >> recommends: >> * DBD::Pg is not installed >> >> I am having problems to get that one.. Is that causing the error? >> >> much obliged, >> Christoph >> >> Am 15.10.2012 20:05, schrieb Carson Holt: >>> It looks like either an NFS issue or perhaps an issue with the location >>> provided for the TMP= in the maker_opts.ctl file. >>> >>> First I would suggest trying maker 2.26 to see if the issue still >>> happens >>> there (there are a few updates to how NFS locks are handled in 2.26). >>> >>> Second if you are setting TMP make sure your disk is not full. Or if >>> TMP >>> was set to a tmpfs location (in memory filesystem), it could be a full >>> memory issue in which case you could just set TMP to somewhere else. >>> >>> Let me know if you still see the issue in 2.26 or after changing the >>> location of TMP. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>> >>>> Hi Carson, >>>> >>>> Thanks for your reply. I managed to install maker2.25 properly now, but >>>> when running it I am getting strange errors (two kinds): >>>> type one: >>>> STATUS: Parsing control files... >>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>> >>>> type two: >>>> STATUS: Parsing control files... >>>> ERROR: Cannot get initialization lock. >>>> >>>> I have divided the draft assembly to several files, each containing a >>>> subset of contigs on which I am running a maker instance. I am getting >>>> the above named error type one for the majority of the instances, >>>> although the HMM file it is complaining about (es.mod) is actually in >>>> the path it is naming (not shown above). In a small number of instances >>>> I am getting the type two error mentioned above. >>>> >>>> Can you help me? >>>> >>>> much obliged, >>>> Christoph >>>> >>>> >>>> >>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>> You may need to reconfigure your cpan preferences. >>>>> >>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>>>> or >>>>> by starting cpan from your command line (command is cpan). Then run >>>>> 'o >>>>> conf init' inside the cpan ui. >>>>> >>>>> --Carson >>>>> >>>>> >>>>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>>>> >>>>>> Hello maker-team, >>>>>> >>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>> privileges >>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>> seem >>>>>> to get it to work). All necessary prerequisite programs are installed >>>>>> and in the path (I followed the steps in the INSTALL file that comes >>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>> Checking prerequisites... >>>>>> requires: >>>>>> ! DBD::SQLite is not installed >>>>>> ! IO::All is not installed >>>>>> ! Inline::C is not installed >>>>>> ! Perl::Unsafe::Signals is not installed >>>>>> ! Proc::ProcessTable is not installed >>>>>> ! Want is not installed >>>>>> ! forks is not installed >>>>>> ! forks::shared is not installed >>>>>> build_requires: >>>>>> ! LWP::Simple is not installed >>>>>> recommends: >>>>>> * DBD::Pg is not installed >>>>>> >>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>>>> versions >>>>>> of the modules indicated above before proceeding with this >>>>>> installation >>>>>> >>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>> >>>>>> MAKER supports distributed parallelization via MPI. >>>>>> Would you like to configure MAKER for MPI (This >>>>>> requires that you have an MPI client installed)? [N ]n >>>>>> >>>>>> WARNING: Apache cannot be located. The optional web based >>>>>> interface to MAKER will not be available to you. >>>>>> >>>>>> Created MYMETA.yml and MYMETA.json >>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>> >>>>>> >>>>>> The file 'Build' has been created for you to finish installing MAKER. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ====================================================================== >>>>>> == >>>>>> == >>>>>> ==== >>>>>> STATUS MAKER 2.25 >>>>>> >>>>>> >>>>>> ====================================================================== >>>>>> == >>>>>> == >>>>>> ==== >>>>>> PERL Dependencies: MISSING >>>>>> ! forks >>>>>> ! IO::All >>>>>> ! forks::shared >>>>>> ! Want >>>>>> ! DBD::SQLite >>>>>> ! Proc::ProcessTable >>>>>> ! Inline::C >>>>>> ! Perl::Unsafe::Signals >>>>>> >>>>>> External Programs: VERIFIED >>>>>> External C Libraries: VERIFIED >>>>>> MPI SUPPORT: DISABLED >>>>>> MWAS Web Interface: DISABLED >>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>> >>>>>> >>>>>> Important Commands: >>>>>> ./Build installdeps #installs missing PERL dependencies >>>>>> ./Build installexes #installs all missing external >>>>>> programs >>>>>> ./Build install #installs MAKER >>>>>> ./Build status #Shows this status menu >>>>>> >>>>>> Other Commands: >>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>> RepBase) >>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX / >>>>>> v1 >>>>>> on >>>>>> Mac OSX) >>>>>> ./Build snap #installs SNAP >>>>>> ./Build augustus #installs Augustus >>>>>> ./Build apollo #installs Apollo >>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, not >>>>>> web >>>>>> accecible) >>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>> install >>>>>> recommended) >>>>>> >>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>> INSTALL >>>>>> I followed the quick and dirty installation for Bioperl, so I am not >>>>>> sure what I did wrong/missed out. >>>>>> >>>>>> When I run ./Build installdeps, I get: >>>>>> You do not have write access to install missing Modules. >>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>> in the .../maker/perl/lib directory, or you can run >>>>>> './Build installdeps' as root or using sudo and try again. >>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>> mkdir /usit/titan: Permission denied at >>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>> line >>>>>> 59 >>>>>> >>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>> appreciated! >>>>>> >>>>>> much obliged, >>>>>> Christoph >>>>>> >>>>>> _______________________________________________ >>>>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>> g >> > From carsonhh at gmail.com Tue Oct 16 09:18:48 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 16 Oct 2012 11:18:48 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <507D79DC.5090702@gmail.com> Message-ID: Delete any file in the maker output directory that start with the name .init or .NFS then rerun. Thanks, Carson On 12-10-16 11:14 AM, "Christoph Hahn" wrote: >Hi Carson, > >It was a link. The path was not correct any more because of the >migration to the new cluster. I could not change it so I just gave the >path to the real file in the maker_opts file. That has resolved this >error, but now I am getting the following: >STATUS: Parsing control files... >ERROR: Cannot get initialization lock. > >Probably not relevant any more, but: No trailing commas, no ':' >characters in the path. > >Thanks for your help! >Christoph > >Am 16.10.2012 16:19, schrieb Carson Holt: >> No. that's just a recommendation for the maker2chado script to work. >> >> Is es.mod the actual HMM or a soft link to the HMM? >> After the error message their should be a list of the actual files. >>Does >> it print that? >> Are there trailing commas in you maker_opts.ctl file after the hmm file >> name in the control files? >> Are their ':' characters in the path name to the file? >> >> --Carson >> >> >> >> On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >> >>> Thanks for your help Carson! >>> >>> I tried maker2.26, but the error still remains. I also tried to change >>> the TMP= option in the maker_opts file, but no change either.. >>> >>> STATUS: Parsing control files... >>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>> >>> When building the maker2.26 I saw that it is complaining about one more >>> (recommended) dependency (I must have overlooked that before): >>> Checking prerequisites... >>> recommends: >>> * DBD::Pg is not installed >>> >>> I am having problems to get that one.. Is that causing the error? >>> >>> much obliged, >>> Christoph >>> >>> Am 15.10.2012 20:05, schrieb Carson Holt: >>>> It looks like either an NFS issue or perhaps an issue with the >>>>location >>>> provided for the TMP= in the maker_opts.ctl file. >>>> >>>> First I would suggest trying maker 2.26 to see if the issue still >>>> happens >>>> there (there are a few updates to how NFS locks are handled in 2.26). >>>> >>>> Second if you are setting TMP make sure your disk is not full. Or if >>>> TMP >>>> was set to a tmpfs location (in memory filesystem), it could be a full >>>> memory issue in which case you could just set TMP to somewhere else. >>>> >>>> Let me know if you still see the issue in 2.26 or after changing the >>>> location of TMP. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>>> >>>>> Hi Carson, >>>>> >>>>> Thanks for your reply. I managed to install maker2.25 properly now, >>>>>but >>>>> when running it I am getting strange errors (two kinds): >>>>> type one: >>>>> STATUS: Parsing control files... >>>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>>> >>>>> type two: >>>>> STATUS: Parsing control files... >>>>> ERROR: Cannot get initialization lock. >>>>> >>>>> I have divided the draft assembly to several files, each containing a >>>>> subset of contigs on which I am running a maker instance. I am >>>>>getting >>>>> the above named error type one for the majority of the instances, >>>>> although the HMM file it is complaining about (es.mod) is actually in >>>>> the path it is naming (not shown above). In a small number of >>>>>instances >>>>> I am getting the type two error mentioned above. >>>>> >>>>> Can you help me? >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> >>>>> >>>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>>> You may need to reconfigure your cpan preferences. >>>>>> >>>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm >>>>>>file, >>>>>> or >>>>>> by starting cpan from your command line (command is cpan). Then run >>>>>> 'o >>>>>> conf init' inside the cpan ui. >>>>>> >>>>>> --Carson >>>>>> >>>>>> >>>>>> On 12-10-09 7:07 AM, "Christoph Hahn" >>>>>>wrote: >>>>>> >>>>>>> Hello maker-team, >>>>>>> >>>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>>> privileges >>>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>>> seem >>>>>>> to get it to work). All necessary prerequisite programs are >>>>>>>installed >>>>>>> and in the path (I followed the steps in the INSTALL file that >>>>>>>comes >>>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>>> Checking prerequisites... >>>>>>> requires: >>>>>>> ! DBD::SQLite is not installed >>>>>>> ! IO::All is not installed >>>>>>> ! Inline::C is not installed >>>>>>> ! Perl::Unsafe::Signals is not installed >>>>>>> ! Proc::ProcessTable is not installed >>>>>>> ! Want is not installed >>>>>>> ! forks is not installed >>>>>>> ! forks::shared is not installed >>>>>>> build_requires: >>>>>>> ! LWP::Simple is not installed >>>>>>> recommends: >>>>>>> * DBD::Pg is not installed >>>>>>> >>>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install >>>>>>>the >>>>>>> versions >>>>>>> of the modules indicated above before proceeding with this >>>>>>> installation >>>>>>> >>>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>>> >>>>>>> MAKER supports distributed parallelization via MPI. >>>>>>> Would you like to configure MAKER for MPI (This >>>>>>> requires that you have an MPI client installed)? [N ]n >>>>>>> >>>>>>> WARNING: Apache cannot be located. The optional web based >>>>>>> interface to MAKER will not be available to you. >>>>>>> >>>>>>> Created MYMETA.yml and MYMETA.json >>>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>>> >>>>>>> >>>>>>> The file 'Build' has been created for you to finish installing >>>>>>>MAKER. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>== >>>>>>> == >>>>>>> == >>>>>>> ==== >>>>>>> STATUS MAKER 2.25 >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>== >>>>>>> == >>>>>>> == >>>>>>> ==== >>>>>>> PERL Dependencies: MISSING >>>>>>> ! forks >>>>>>> ! IO::All >>>>>>> ! forks::shared >>>>>>> ! Want >>>>>>> ! DBD::SQLite >>>>>>> ! Proc::ProcessTable >>>>>>> ! Inline::C >>>>>>> ! Perl::Unsafe::Signals >>>>>>> >>>>>>> External Programs: VERIFIED >>>>>>> External C Libraries: VERIFIED >>>>>>> MPI SUPPORT: DISABLED >>>>>>> MWAS Web Interface: DISABLED >>>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>>> >>>>>>> >>>>>>> Important Commands: >>>>>>> ./Build installdeps #installs missing PERL >>>>>>>dependencies >>>>>>> ./Build installexes #installs all missing external >>>>>>> programs >>>>>>> ./Build install #installs MAKER >>>>>>> ./Build status #Shows this status menu >>>>>>> >>>>>>> Other Commands: >>>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>>> RepBase) >>>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX >>>>>>>/ >>>>>>> v1 >>>>>>> on >>>>>>> Mac OSX) >>>>>>> ./Build snap #installs SNAP >>>>>>> ./Build augustus #installs Augustus >>>>>>> ./Build apollo #installs Apollo >>>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, >>>>>>>not >>>>>>> web >>>>>>> accecible) >>>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>>> install >>>>>>> recommended) >>>>>>> >>>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>>> INSTALL >>>>>>> I followed the quick and dirty installation for Bioperl, so I am >>>>>>>not >>>>>>> sure what I did wrong/missed out. >>>>>>> >>>>>>> When I run ./Build installdeps, I get: >>>>>>> You do not have write access to install missing Modules. >>>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>>> in the .../maker/perl/lib directory, or you can run >>>>>>> './Build installdeps' as root or using sudo and try again. >>>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>>> mkdir /usit/titan: Permission denied at >>>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>>> line >>>>>>> 59 >>>>>>> >>>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>>> appreciated! >>>>>>> >>>>>>> much obliged, >>>>>>> Christoph >>>>>>> >>>>>>> _______________________________________________ >>>>>>> maker-devel mailing list >>>>>>> maker-devel at box290.bluehost.com >>>>>>> >>>>>>> >>>>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab. >>>>>>>or >>>>>>> g >>> >> > > From chrisi.hahni at gmail.com Tue Oct 16 10:12:21 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 18:12:21 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D8765.8010909@gmail.com> The maker output directory contains only one thing: .NFSLock.init_lock.NFSLock, which is a link to .NFSLock.init_lock.tmp.3358.21603.1715.48578929183 in the same directory. When I delete it and rerun maker I get the same error and another .NFSLock.init_lock.NFSLock is created, this one is linked to .NFSLock.init_lock.tmp.3635.21981.7756.8294996771 Thanks, Christoph Am 16.10.2012 17:18, schrieb Carson Holt: > Delete any file in the maker output directory that start with the name > .init or .NFS then rerun. > > Thanks, > Carson > > > > On 12-10-16 11:14 AM, "Christoph Hahn" wrote: > >> Hi Carson, >> >> It was a link. The path was not correct any more because of the >> migration to the new cluster. I could not change it so I just gave the >> path to the real file in the maker_opts file. That has resolved this >> error, but now I am getting the following: >> STATUS: Parsing control files... >> ERROR: Cannot get initialization lock. >> >> Probably not relevant any more, but: No trailing commas, no ':' >> characters in the path. >> >> Thanks for your help! >> Christoph >> >> Am 16.10.2012 16:19, schrieb Carson Holt: >>> No. that's just a recommendation for the maker2chado script to work. >>> >>> Is es.mod the actual HMM or a soft link to the HMM? >>> After the error message their should be a list of the actual files. >>> Does >>> it print that? >>> Are there trailing commas in you maker_opts.ctl file after the hmm file >>> name in the control files? >>> Are their ':' characters in the path name to the file? >>> >>> --Carson >>> >>> >>> >>> On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >>> >>>> Thanks for your help Carson! >>>> >>>> I tried maker2.26, but the error still remains. I also tried to change >>>> the TMP= option in the maker_opts file, but no change either.. >>>> >>>> STATUS: Parsing control files... >>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>> >>>> When building the maker2.26 I saw that it is complaining about one more >>>> (recommended) dependency (I must have overlooked that before): >>>> Checking prerequisites... >>>> recommends: >>>> * DBD::Pg is not installed >>>> >>>> I am having problems to get that one.. Is that causing the error? >>>> >>>> much obliged, >>>> Christoph >>>> >>>> Am 15.10.2012 20:05, schrieb Carson Holt: >>>>> It looks like either an NFS issue or perhaps an issue with the >>>>> location >>>>> provided for the TMP= in the maker_opts.ctl file. >>>>> >>>>> First I would suggest trying maker 2.26 to see if the issue still >>>>> happens >>>>> there (there are a few updates to how NFS locks are handled in 2.26). >>>>> >>>>> Second if you are setting TMP make sure your disk is not full. Or if >>>>> TMP >>>>> was set to a tmpfs location (in memory filesystem), it could be a full >>>>> memory issue in which case you could just set TMP to somewhere else. >>>>> >>>>> Let me know if you still see the issue in 2.26 or after changing the >>>>> location of TMP. >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>>>> >>>>>> Hi Carson, >>>>>> >>>>>> Thanks for your reply. I managed to install maker2.25 properly now, >>>>>> but >>>>>> when running it I am getting strange errors (two kinds): >>>>>> type one: >>>>>> STATUS: Parsing control files... >>>>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>>>> >>>>>> type two: >>>>>> STATUS: Parsing control files... >>>>>> ERROR: Cannot get initialization lock. >>>>>> >>>>>> I have divided the draft assembly to several files, each containing a >>>>>> subset of contigs on which I am running a maker instance. I am >>>>>> getting >>>>>> the above named error type one for the majority of the instances, >>>>>> although the HMM file it is complaining about (es.mod) is actually in >>>>>> the path it is naming (not shown above). In a small number of >>>>>> instances >>>>>> I am getting the type two error mentioned above. >>>>>> >>>>>> Can you help me? >>>>>> >>>>>> much obliged, >>>>>> Christoph >>>>>> >>>>>> >>>>>> >>>>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>>>> You may need to reconfigure your cpan preferences. >>>>>>> >>>>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm >>>>>>> file, >>>>>>> or >>>>>>> by starting cpan from your command line (command is cpan). Then run >>>>>>> 'o >>>>>>> conf init' inside the cpan ui. >>>>>>> >>>>>>> --Carson >>>>>>> >>>>>>> >>>>>>> On 12-10-09 7:07 AM, "Christoph Hahn" >>>>>>> wrote: >>>>>>> >>>>>>>> Hello maker-team, >>>>>>>> >>>>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>>>> privileges >>>>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>>>> seem >>>>>>>> to get it to work). All necessary prerequisite programs are >>>>>>>> installed >>>>>>>> and in the path (I followed the steps in the INSTALL file that >>>>>>>> comes >>>>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>>>> Checking prerequisites... >>>>>>>> requires: >>>>>>>> ! DBD::SQLite is not installed >>>>>>>> ! IO::All is not installed >>>>>>>> ! Inline::C is not installed >>>>>>>> ! Perl::Unsafe::Signals is not installed >>>>>>>> ! Proc::ProcessTable is not installed >>>>>>>> ! Want is not installed >>>>>>>> ! forks is not installed >>>>>>>> ! forks::shared is not installed >>>>>>>> build_requires: >>>>>>>> ! LWP::Simple is not installed >>>>>>>> recommends: >>>>>>>> * DBD::Pg is not installed >>>>>>>> >>>>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install >>>>>>>> the >>>>>>>> versions >>>>>>>> of the modules indicated above before proceeding with this >>>>>>>> installation >>>>>>>> >>>>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>>>> >>>>>>>> MAKER supports distributed parallelization via MPI. >>>>>>>> Would you like to configure MAKER for MPI (This >>>>>>>> requires that you have an MPI client installed)? [N ]n >>>>>>>> >>>>>>>> WARNING: Apache cannot be located. The optional web based >>>>>>>> interface to MAKER will not be available to you. >>>>>>>> >>>>>>>> Created MYMETA.yml and MYMETA.json >>>>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>>>> >>>>>>>> >>>>>>>> The file 'Build' has been created for you to finish installing >>>>>>>> MAKER. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> == >>>>>>>> == >>>>>>>> == >>>>>>>> ==== >>>>>>>> STATUS MAKER 2.25 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> == >>>>>>>> == >>>>>>>> == >>>>>>>> ==== >>>>>>>> PERL Dependencies: MISSING >>>>>>>> ! forks >>>>>>>> ! IO::All >>>>>>>> ! forks::shared >>>>>>>> ! Want >>>>>>>> ! DBD::SQLite >>>>>>>> ! Proc::ProcessTable >>>>>>>> ! Inline::C >>>>>>>> ! Perl::Unsafe::Signals >>>>>>>> >>>>>>>> External Programs: VERIFIED >>>>>>>> External C Libraries: VERIFIED >>>>>>>> MPI SUPPORT: DISABLED >>>>>>>> MWAS Web Interface: DISABLED >>>>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>>>> >>>>>>>> >>>>>>>> Important Commands: >>>>>>>> ./Build installdeps #installs missing PERL >>>>>>>> dependencies >>>>>>>> ./Build installexes #installs all missing external >>>>>>>> programs >>>>>>>> ./Build install #installs MAKER >>>>>>>> ./Build status #Shows this status menu >>>>>>>> >>>>>>>> Other Commands: >>>>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>>>> RepBase) >>>>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX >>>>>>>> / >>>>>>>> v1 >>>>>>>> on >>>>>>>> Mac OSX) >>>>>>>> ./Build snap #installs SNAP >>>>>>>> ./Build augustus #installs Augustus >>>>>>>> ./Build apollo #installs Apollo >>>>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, >>>>>>>> not >>>>>>>> web >>>>>>>> accecible) >>>>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>>>> install >>>>>>>> recommended) >>>>>>>> >>>>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>>>> INSTALL >>>>>>>> I followed the quick and dirty installation for Bioperl, so I am >>>>>>>> not >>>>>>>> sure what I did wrong/missed out. >>>>>>>> >>>>>>>> When I run ./Build installdeps, I get: >>>>>>>> You do not have write access to install missing Modules. >>>>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>>>> in the .../maker/perl/lib directory, or you can run >>>>>>>> './Build installdeps' as root or using sudo and try again. >>>>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>>>> mkdir /usit/titan: Permission denied at >>>>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>>>> line >>>>>>>> 59 >>>>>>>> >>>>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>>>> appreciated! >>>>>>>> >>>>>>>> much obliged, >>>>>>>> Christoph >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> maker-devel mailing list >>>>>>>> maker-devel at box290.bluehost.com >>>>>>>> >>>>>>>> >>>>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab. >>>>>>>> or >>>>>>>> g >> > From mikael.durling at slu.se Tue Oct 16 10:58:55 2012 From: mikael.durling at slu.se (=?iso-8859-1?Q?Mikael_Brandstr=F6m_Durling?=) Date: Tue, 16 Oct 2012 18:58:55 +0200 Subject: [maker-devel] Format of repeat_gff gff3 file Message-ID: Hi, I would like to mask my fungal genome from two different sources (ie. repbase and an inhouse repeat library). However, I suppose the that if I supply a library as rmlib in maker_opts, it will be mutually exclusive to the model_org option, in the same way as -spec and -lib options to RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the model_org option as fungi, and would like to provide maker with additional masking as a gff file. I tried by running RepeatMasker with my inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker package to obtain a gff3 file. This file was supplied to maker as rm_gff (see below for a sample from the file). The run fail with backtraces like the one paseted below. How should this gff file be formatted for maker to understand it? I see that in maker produced gff files, there are additional information found in the id of the hits. Is this required? Maybe it's easier to modify maker to make two rounds of RepeatMasker calls if both model_org and rmlib are specified? Thanks for any input, Mikael ------------- EXCEPTION: Bio::Root::Exception ------------- MSG: Must have defined a valid name for Hit STACK: Error::throw STACK: Bio::Root::Root::throw /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 STACK: Bio::Search::Hit::GenericHit::new /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:149 STACK: Bio::Search::Hit::PhatHit::Base::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm:127 STACK: Bio::Search::Hit::PhatHit::gff3::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/gff3.pm:23 STACK: GFFDB::_load_hits /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:1026 STACK: GFFDB::phathits_on_chunk /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:651 STACK: Process::MpiChunk::_go /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:752 STACK: Process::MpiChunk::run /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:331 STACK: main::node_thread /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 STACK: threads::new /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm:799 STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 ----------------------------------------------------------- Cannot restore overloading on HASH(0x2580810) (package Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at /opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm line 2256. Compilation failed in require at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. BEGIN failed--compilation aborted at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached ERROR: Could not open '/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.output/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fungi.rb.out' ERROR: Failed while doing repeat masking ERROR: Chunk failed at level:0, tier_type:1 FAILED CONTIG:scf_55419 The gff3-file looks like this: ##gff-version 3 ##sequence-region scf_42697 1 3949 scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 ##sequence-region scf_82496 1 2757 scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_family-1046 2478 4915 ##sequence-region scf_82727 1 4159 scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 23 scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_family-64 15 50 ##sequence-region scf_82785 1 4084 scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 24 scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_family-690 419 786 ##sequence-region scf_86740 1 4293 scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_family-262 1 25 scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_family-262 2 60 scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_family-262 13 60 ##sequence-region scf_86782 1 8564 scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_family-480 1027 1129 ##sequence-region scf_86808 1 4495 scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family-690 1 969 scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 74 ##sequence-region scf_86815 1 4139 scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-262 825 918 scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_family-262 526 4459 ##sequence-region scf_86823 1 2528 scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 178 scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 35 scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 45 scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 41 scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 96 scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 40 scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_family-262 881 920 ##sequence-region scf_86857 1 2778 From mike.thon at gmail.com Wed Oct 17 04:48:59 2012 From: mike.thon at gmail.com (Michael Thon) Date: Wed, 17 Oct 2012 12:48:59 +0200 Subject: [maker-devel] Format of repeat_gff gff3 file In-Reply-To: References: Message-ID: Hi - 2 ideas: 1) There is a utility included with RepeatMasker that enables you to extract sequences from it. its in the utils directory. You can run it like this: ./queryRepeatDatabase.pl -species Fungi -clade that will extract all repeat sequences belonging to Fungi or its descendants. Maybe the simplest thing for you to do is extract the sequences that you want and cat them together with your in house sequences to provide MAKER with a single file of reference sequences. 2) some of the MAKER parameters take a comma separated list of files. I don't know if this applied to rm_lib though. Mike On Oct 16, 2012, at 6:58 PM, Mikael Brandstr?m Durling wrote: > Hi, > > I would like to mask my fungal genome from two different sources (ie. repbase and an inhouse repeat library). However, I suppose the that if I supply a library as rmlib in maker_opts, it will be mutually exclusive to the model_org option, in the same way as -spec and -lib options to RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the model_org option as fungi, and would like to provide maker with additional masking as a gff file. I tried by running RepeatMasker with my inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker package to obtain a gff3 file. This file was supplied to maker as rm_gff (see below for a sample from the file). The run fail with backtraces like the one paseted below. How should this gff file be formatted for maker to understand it? I see that in maker produced gff files, there are additional information found in the id of the hits. Is this required? > > Maybe it's easier to modify maker to make two rounds of RepeatMasker calls if both model_org and rmlib are specified? > > Thanks for any input, > Mikael > > > ------------- EXCEPTION: Bio::Root::Exception ------------- > MSG: Must have defined a valid name for Hit > STACK: Error::throw > STACK: Bio::Root::Root::throw /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 > STACK: Bio::Search::Hit::GenericHit::new /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:149 > STACK: Bio::Search::Hit::PhatHit::Base::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm:127 > STACK: Bio::Search::Hit::PhatHit::gff3::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/gff3.pm:23 > STACK: GFFDB::_load_hits /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:1026 > STACK: GFFDB::phathits_on_chunk /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:651 > STACK: Process::MpiChunk::_go /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:752 > STACK: Process::MpiChunk::run /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:331 > STACK: main::node_thread /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 > STACK: threads::new /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm:799 > STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 > ----------------------------------------------------------- > Cannot restore overloading on HASH(0x2580810) (package Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at /opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm line 2256. > Compilation failed in require at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. > BEGIN failed--compilation aborted at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. > Perl exited with active threads: > 1 running and unjoined > 0 finished and unjoined > 0 running and detached > ERROR: Could not open '/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.output/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fungi.rb.out' > ERROR: Failed while doing repeat masking > ERROR: Chunk failed at level:0, tier_type:1 > FAILED CONTIG:scf_55419 > > > The gff3-file looks like this: > ##gff-version 3 > ##sequence-region scf_42697 1 3949 > scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 > scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 > scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 > ##sequence-region scf_82496 1 2757 > scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_family-1046 2478 4915 > ##sequence-region scf_82727 1 4159 > scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 > scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 23 > scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_family-64 15 50 > ##sequence-region scf_82785 1 4084 > scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 24 > scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_family-690 419 786 > ##sequence-region scf_86740 1 4293 > scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_family-262 1 25 > scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_family-262 2 60 > scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_family-262 13 60 > ##sequence-region scf_86782 1 8564 > scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_family-480 1027 1129 > ##sequence-region scf_86808 1 4495 > scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family-690 1 969 > scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 74 > ##sequence-region scf_86815 1 4139 > scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-262 825 918 > scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_family-262 526 4459 > ##sequence-region scf_86823 1 2528 > scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 > scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 > scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 178 > scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 35 > scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 45 > scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 41 > scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 96 > scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 40 > scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_family-262 881 920 > ##sequence-region scf_86857 1 2778 > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Fri Oct 19 08:00:12 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 19 Oct 2012 10:00:12 -0400 Subject: [maker-devel] Format of repeat_gff gff3 file In-Reply-To: Message-ID: This command line should add IDs to the end of the GFF3 to let it pass through without the error message. cat file.gff | perl -ane '$id; if(!/^\#/){@F = split(/\t/, $_); chomp $F[-1];$id++; $F[-1] .= "\;ID=$id"; $_ = join("\t", @F)."\n"} print $_' MAKER will use the GFF3 first if provided, then run the species specific library, and then any model organism specified. So if there is overlap and one must be excluded, then they will be kept in that same order of precedence. Thanks, Carson On 12-10-16 12:58 PM, "Mikael Brandstr?m Durling" wrote: >Hi, > >I would like to mask my fungal genome from two different sources (ie. >repbase and an inhouse repeat library). However, I suppose the that if I >supply a library as rmlib in maker_opts, it will be mutually exclusive to >the model_org option, in the same way as -spec and -lib options to >RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the >model_org option as fungi, and would like to provide maker with >additional masking as a gff file. I tried by running RepeatMasker with my >inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker >package to obtain a gff3 file. This file was supplied to maker as rm_gff >(see below for a sample from the file). The run fail with backtraces like >the one paseted below. How should this gff file be formatted for maker to >understand it? I see that in maker produced gff files, there are >additional information found in the id of the hits. Is this required? > >Maybe it's easier to modify maker to make two rounds of RepeatMasker >calls if both model_org and rmlib are specified? > >Thanks for any input, >Mikael > > >------------- EXCEPTION: Bio::Root::Exception ------------- >MSG: Must have defined a valid name for Hit >STACK: Error::throw >STACK: Bio::Root::Root::throw >/opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 >STACK: Bio::Search::Hit::GenericHit::new >/opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:14 >9 >STACK: Bio::Search::Hit::PhatHit::Base::new >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Bio/Search/Hit/PhatHit/Base.pm:127 >STACK: Bio::Search::Hit::PhatHit::gff3::new >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Bio/Search/Hit/PhatHit/gff3.pm:23 >STACK: GFFDB::_load_hits >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/GFFDB.pm:1026 >STACK: GFFDB::phathits_on_chunk >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/GFFDB.pm:651 >STACK: Process::MpiChunk::_go >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Process/MpiChunk.pm:752 >STACK: Process::MpiChunk::run >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Process/MpiChunk.pm:331 >STACK: main::node_thread >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 >STACK: threads::new >/proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16. >1/x86_64-linux/forks.pm:799 >STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 >----------------------------------------------------------- >Cannot restore overloading on HASH(0x2580810) (package >Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at >/opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at >/proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16. >1/x86_64-linux/forks.pm line 2256. >Compilation failed in require at >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. >BEGIN failed--compilation aborted at >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. >Perl exited with active threads: > 1 running and unjoined > 0 finished and unjoined > 0 running and detached >ERROR: Could not open >'/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.o >utput/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fung >i.rb.out' >ERROR: Failed while doing repeat masking >ERROR: Chunk failed at level:0, tier_type:1 >FAILED CONTIG:scf_55419 > > >The gff3-file looks like this: >##gff-version 3 >##sequence-region scf_42697 1 3949 >scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 >scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 >scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 >##sequence-region scf_82496 1 2757 >scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_fami >ly-1046 2478 4915 >##sequence-region scf_82727 1 4159 >scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 >scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 >23 >scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_fam >ily-64 15 50 >##sequence-region scf_82785 1 4084 >scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 >24 >scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_fam >ily-690 419 786 >##sequence-region scf_86740 1 4293 >scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_famil >y-262 1 25 >scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_famil >y-262 2 60 >scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_famil >y-262 13 60 >##sequence-region scf_86782 1 8564 >scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_fam >ily-480 1027 1129 >##sequence-region scf_86808 1 4495 >scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family >-690 1 969 >scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 >74 >##sequence-region scf_86815 1 4139 >scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-2 >62 825 918 >scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_fa >mily-262 526 4459 >##sequence-region scf_86823 1 2528 >scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 >scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 >scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 >178 >scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 >35 >scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 >45 >scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 >41 >scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 >96 >scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 >40 >scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_fam >ily-262 881 920 >##sequence-region scf_86857 1 2778 > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From parulk at caltech.edu Mon Oct 22 15:46:18 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 22 Oct 2012 14:46:18 -0700 (PDT) Subject: [maker-devel] gff3_preds2models script Message-ID: <3207.131.215.15.234.1350942378.squirrel@webmail.caltech.edu> Hello, I want to add gene structure(gene/mRNA/exon) to gff3 file. I am using gff3_preds2models for this purpose. However I get following error **WARNING: No top level feature found for ID WHL22.100252 I have attached the sample input gff3 file and the list of ids Thanks and regards, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: sample.gff3 Type: application/octet-stream Size: 1874 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: list Type: application/octet-stream Size: 77 bytes Desc: not available URL: From fourie.joubert at up.ac.za Wed Oct 24 01:07:13 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 09:07:13 +0200 Subject: [maker-devel] Segfault during build_fasta_index Message-ID: <508793A1.1000704@up.ac.za> Hi I am getting a segfault in maker somewhere during the build_fasta_index step. I suspect it is happening after some updates I made to bioperl. Has anyone else experienced something similar, and is there a recommended bioperl version? Thanks and regards! Fourie -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 06:28:10 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 08:28:10 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <508793A1.1000704@up.ac.za> Message-ID: Yes. Don't use the live version of BioPerl. Use the CPAN version. The live version has a bug. Thanks, Carson On 12-10-24 3:07 AM, "Fourie Joubert" wrote: >Hi > >I am getting a segfault in maker somewhere during the build_fasta_index >step. > >I suspect it is happening after some updates I made to bioperl. > >Has anyone else experienced something similar, and is there a >recommended bioperl version? > >Thanks and regards! > >Fourie > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From fourie.joubert at up.ac.za Wed Oct 24 08:05:12 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:05:12 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087F598.20303@up.ac.za> Hi I have installed from CPAN (C/CJ/CJFIELDS/BioPerl-1.6.901.tar.gz), but the error persists... Does anyone know if the versions in CPAN may have been replaced recently? Best regards! Foorie On 10/24/2012 02:28 PM, Carson Holt wrote: > Yes. Don't use the live version of BioPerl. Use the CPAN version. The > live version has a bug. > > Thanks, > Carson > > > On 12-10-24 3:07 AM, "Fourie Joubert" wrote: > >> Hi >> >> I am getting a segfault in maker somewhere during the build_fasta_index >> step. >> >> I suspect it is happening after some updates I made to bioperl. >> >> Has anyone else experienced something similar, and is there a >> recommended bioperl version? >> >> Thanks and regards! >> >> Fourie >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:10:32 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:10:32 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087F598.20303@up.ac.za> Message-ID: Run this command --> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' What does it print? --Carson On 12-10-24 10:05 AM, "Fourie Joubert" wrote: >Hi > >I have installed from CPAN (C/CJ/CJFIELDS/BioPerl-1.6.901.tar.gz), but >the error persists... > >Does anyone know if the versions in CPAN may have been replaced recently? > >Best regards! > >Foorie > > > > > > >On 10/24/2012 02:28 PM, Carson Holt wrote: >> Yes. Don't use the live version of BioPerl. Use the CPAN version. The >> live version has a bug. >> >> Thanks, >> Carson >> >> >> On 12-10-24 3:07 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> I am getting a segfault in maker somewhere during the build_fasta_index >>> step. >>> >>> I suspect it is happening after some updates I made to bioperl. >>> >>> Has anyone else experienced something similar, and is there a >>> recommended bioperl version? >>> >>> Thanks and regards! >>> >>> Fourie >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:24:43 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:24:43 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FA2B.5020702@up.ac.za> Hi 1.006901 Regards! Fourie On 10/24/2012 04:10 PM, Carson Holt wrote: > perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:26:41 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:26:41 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FA2B.5020702@up.ac.za> Message-ID: Try completely deleting the mpi_blastdb directory under the MAKER output folder before restarting. Perhaps also try reinstalling DB_File via CPAN. --Carson On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >Hi > >1.006901 > >Regards! > >Fourie > > > >On 10/24/2012 04:10 PM, Carson Holt wrote: >> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:30:15 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:30:15 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FB77.7060802@up.ac.za> Hi Darn - did both, but unfortunately still segfaults... Best regards! Fourie On 10/24/2012 04:26 PM, Carson Holt wrote: > Try completely deleting the mpi_blastdb directory under the MAKER output > folder before restarting. Perhaps also try reinstalling DB_File via CPAN. > > --Carson > > On 12-10-24 10:24 AM, "Fourie Joubert" wrote: > >> Hi >> >> 1.006901 >> >> Regards! >> >> Fourie >> >> >> >> On 10/24/2012 04:10 PM, Carson Holt wrote: >>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:33:07 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:33:07 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FB77.7060802@up.ac.za> Message-ID: I'm going to write you a test script that just creates a fasta index to see if it seg faults. I'll send it to you later. For now could you run maker with the --debug flag set, apture the STDERR and send it to me. Thanks, Carson On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >Hi > >Darn - did both, but unfortunately still segfaults... > >Best regards! > >Fourie > > >On 10/24/2012 04:26 PM, Carson Holt wrote: >> Try completely deleting the mpi_blastdb directory under the MAKER output >> folder before restarting. Perhaps also try reinstalling DB_File via >>CPAN. >> >> --Carson >> >> On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> 1.006901 >>> >>> Regards! >>> >>> Fourie >>> >>> >>> >>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:41:15 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:41:15 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FE0B.7030206@up.ac.za> Hi Really weird - as soon as I add the debug flag - no more segfault... Best regards! Fourie On 10/24/2012 04:33 PM, Carson Holt wrote: > I'm going to write you a test script that just creates a fasta index to > see if it seg faults. I'll send it to you later. For now could you run > maker with the --debug flag set, apture the STDERR and send it to me. > > Thanks, > Carson > > > On 12-10-24 10:30 AM, "Fourie Joubert" wrote: > >> Hi >> >> Darn - did both, but unfortunately still segfaults... >> >> Best regards! >> >> Fourie >> >> >> On 10/24/2012 04:26 PM, Carson Holt wrote: >>> Try completely deleting the mpi_blastdb directory under the MAKER output >>> folder before restarting. Perhaps also try reinstalling DB_File via >>> CPAN. >>> >>> --Carson >>> >>> On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> 1.006901 >>>> >>>> Regards! >>>> >>>> Fourie >>>> >>>> >>>> >>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:56:51 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:56:51 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FE0B.7030206@up.ac.za> Message-ID: Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA variable. It's set by both BioPerl and a couple of MAKER modules. If it gets set twice, sometimes you get bugs. Try upgrading to MAKER 2.26 I have a statement in that that will make sure it won't get set wrong. Otherwise I can give you instructions on editing the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER modules. Thanks, Carson On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >Hi > >Really weird - as soon as I add the debug flag - no more segfault... > >Best regards! > >Fourie > > > > >On 10/24/2012 04:33 PM, Carson Holt wrote: >> I'm going to write you a test script that just creates a fasta index to >> see if it seg faults. I'll send it to you later. For now could you run >> maker with the --debug flag set, apture the STDERR and send it to me. >> >> Thanks, >> Carson >> >> >> On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> Darn - did both, but unfortunately still segfaults... >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>output >>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>> CPAN. >>>> >>>> --Carson >>>> >>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> 1.006901 >>>>> >>>>> Regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> >>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>> perl -MBio::Root::Version -e 'print >>>>>>"$Bio::Root::Version::VERSION\n"' >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 09:19:08 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 17:19:08 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <508806EC.6050600@up.ac.za> Hi Carson We are indeed running 2.26. Instructions on editing the statements would be great. Many thanks for all the help, and sorry for the bother! Best regards! Fourie On 10/24/2012 04:56 PM, Carson Holt wrote: > Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA > variable. It's set by both BioPerl and a couple of MAKER modules. If it > gets set twice, sometimes you get bugs. > > Try upgrading to MAKER 2.26 I have a statement in that that will make sure > it won't get set wrong. Otherwise I can give you instructions on editing > the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER > modules. > > Thanks, > Carson > > > On 12-10-24 10:41 AM, "Fourie Joubert" wrote: > >> Hi >> >> Really weird - as soon as I add the debug flag - no more segfault... >> >> Best regards! >> >> Fourie >> >> >> >> >> On 10/24/2012 04:33 PM, Carson Holt wrote: >>> I'm going to write you a test script that just creates a fasta index to >>> see if it seg faults. I'll send it to you later. For now could you run >>> maker with the --debug flag set, apture the STDERR and send it to me. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> Darn - did both, but unfortunately still segfaults... >>>> >>>> Best regards! >>>> >>>> Fourie >>>> >>>> >>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>> output >>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>> CPAN. >>>>> >>>>> --Carson >>>>> >>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> 1.006901 >>>>>> >>>>>> Regards! >>>>>> >>>>>> Fourie >>>>>> >>>>>> >>>>>> >>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>> perl -MBio::Root::Version -e 'print >>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>> -- >>>>>> -------------- >>>>>> Prof Fourie Joubert >>>>>> Bioinformatics and Computational Biology Unit >>>>>> Department of Biochemistry >>>>>> University of Pretoria >>>>>> fourie.joubert at up.ac.za >>>>>> http://www.bi.up.ac.za >>>>>> Tel. +27-12-420-5825 >>>>>> Fax. +27-12-420-5800 >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> -- >>>>>> - >>>>>> This message and attachments are subject to a disclaimer. Please >>>>>> refer >>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>> details. >>>>>> >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 09:50:03 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 11:50:03 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <508806EC.6050600@up.ac.za> Message-ID: Find these lines in these files in MAKER .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); And the Bio::DB::Fasta file in BioPerl. Use this command to identify the location of Bio::DB::Fasta --> perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); Use an editor like emacs or vi to make the changes. Thanks, Carson On 12-10-24 11:19 AM, "Fourie Joubert" wrote: >Hi Carson > >We are indeed running 2.26. > >Instructions on editing the statements would be great. > >Many thanks for all the help, and sorry for the bother! > >Best regards! > >Fourie > > > > >On 10/24/2012 04:56 PM, Carson Holt wrote: >> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >> variable. It's set by both BioPerl and a couple of MAKER modules. If it >> gets set twice, sometimes you get bugs. >> >> Try upgrading to MAKER 2.26 I have a statement in that that will make >>sure >> it won't get set wrong. Otherwise I can give you instructions on >>editing >> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >> modules. >> >> Thanks, >> Carson >> >> >> On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> Really weird - as soon as I add the debug flag - no more segfault... >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> >>> >>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>> I'm going to write you a test script that just creates a fasta index >>>>to >>>> see if it seg faults. I'll send it to you later. For now could you >>>>run >>>> maker with the --debug flag set, apture the STDERR and send it to me. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> Darn - did both, but unfortunately still segfaults... >>>>> >>>>> Best regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>> output >>>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>>> CPAN. >>>>>> >>>>>> --Carson >>>>>> >>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> 1.006901 >>>>>>> >>>>>>> Regards! >>>>>>> >>>>>>> Fourie >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>> -- >>>>>>> -------------- >>>>>>> Prof Fourie Joubert >>>>>>> Bioinformatics and Computational Biology Unit >>>>>>> Department of Biochemistry >>>>>>> University of Pretoria >>>>>>> fourie.joubert at up.ac.za >>>>>>> http://www.bi.up.ac.za >>>>>>> Tel. +27-12-420-5825 >>>>>>> Fax. +27-12-420-5800 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>> -- >>>>>>> - >>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>> refer >>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>> details. >>>>>>> >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Thu Oct 25 00:51:05 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Thu, 25 Oct 2012 08:51:05 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5088E159.3050502@up.ac.za> Hi Carson All happy now - much appreciated! Best regards! Fourie On 10/24/2012 05:50 PM, Carson Holt wrote: > Find these lines in these files in MAKER > > .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > > > And the Bio::DB::Fasta file in BioPerl. Use this command to identify the > location of Bio::DB::Fasta --> > perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' > > .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > > > Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); > > Use an editor like emacs or vi to make the changes. > > Thanks, > Carson > > > > On 12-10-24 11:19 AM, "Fourie Joubert" wrote: > >> Hi Carson >> >> We are indeed running 2.26. >> >> Instructions on editing the statements would be great. >> >> Many thanks for all the help, and sorry for the bother! >> >> Best regards! >> >> Fourie >> >> >> >> >> On 10/24/2012 04:56 PM, Carson Holt wrote: >>> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >>> variable. It's set by both BioPerl and a couple of MAKER modules. If it >>> gets set twice, sometimes you get bugs. >>> >>> Try upgrading to MAKER 2.26 I have a statement in that that will make >>> sure >>> it won't get set wrong. Otherwise I can give you instructions on >>> editing >>> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >>> modules. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> Really weird - as soon as I add the debug flag - no more segfault... >>>> >>>> Best regards! >>>> >>>> Fourie >>>> >>>> >>>> >>>> >>>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>>> I'm going to write you a test script that just creates a fasta index >>>>> to >>>>> see if it seg faults. I'll send it to you later. For now could you >>>>> run >>>>> maker with the --debug flag set, apture the STDERR and send it to me. >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> Darn - did both, but unfortunately still segfaults... >>>>>> >>>>>> Best regards! >>>>>> >>>>>> Fourie >>>>>> >>>>>> >>>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>>> output >>>>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>>>> CPAN. >>>>>>> >>>>>>> --Carson >>>>>>> >>>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>>> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> 1.006901 >>>>>>>> >>>>>>>> Regards! >>>>>>>> >>>>>>>> Fourie >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>>> -- >>>>>>>> -------------- >>>>>>>> Prof Fourie Joubert >>>>>>>> Bioinformatics and Computational Biology Unit >>>>>>>> Department of Biochemistry >>>>>>>> University of Pretoria >>>>>>>> fourie.joubert at up.ac.za >>>>>>>> http://www.bi.up.ac.za >>>>>>>> Tel. +27-12-420-5825 >>>>>>>> Fax. +27-12-420-5800 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -------------------------------------------------------------------- >>>>>>>> -- >>>>>>>> -- >>>>>>>> - >>>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>>> refer >>>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>>> details. >>>>>>>> >>>>>> -- >>>>>> -------------- >>>>>> Prof Fourie Joubert >>>>>> Bioinformatics and Computational Biology Unit >>>>>> Department of Biochemistry >>>>>> University of Pretoria >>>>>> fourie.joubert at up.ac.za >>>>>> http://www.bi.up.ac.za >>>>>> Tel. +27-12-420-5825 >>>>>> Fax. +27-12-420-5800 >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> -- >>>>>> - >>>>>> This message and attachments are subject to a disclaimer. Please >>>>>> refer >>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>> details. >>>>>> >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Thu Oct 25 06:57:31 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 25 Oct 2012 08:57:31 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5088E159.3050502@up.ac.za> Message-ID: Glad I could help. Thanks, Carson On 12-10-25 2:51 AM, "Fourie Joubert" wrote: >Hi Carson > >All happy now - much appreciated! > >Best regards! > >Fourie > >On 10/24/2012 05:50 PM, Carson Holt wrote: >> Find these lines in these files in MAKER >> >> .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> >> >> And the Bio::DB::Fasta file in BioPerl. Use this command to identify >>the >> location of Bio::DB::Fasta --> >> perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' >> >> .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> >> >> Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); >> >> Use an editor like emacs or vi to make the changes. >> >> Thanks, >> Carson >> >> >> >> On 12-10-24 11:19 AM, "Fourie Joubert" wrote: >> >>> Hi Carson >>> >>> We are indeed running 2.26. >>> >>> Instructions on editing the statements would be great. >>> >>> Many thanks for all the help, and sorry for the bother! >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> >>> >>> On 10/24/2012 04:56 PM, Carson Holt wrote: >>>> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >>>> variable. It's set by both BioPerl and a couple of MAKER modules. If >>>>it >>>> gets set twice, sometimes you get bugs. >>>> >>>> Try upgrading to MAKER 2.26 I have a statement in that that will make >>>> sure >>>> it won't get set wrong. Otherwise I can give you instructions on >>>> editing >>>> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >>>> modules. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-24 10:41 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> Really weird - as soon as I add the debug flag - no more segfault... >>>>> >>>>> Best regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> >>>>> >>>>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>>>> I'm going to write you a test script that just creates a fasta index >>>>>> to >>>>>> see if it seg faults. I'll send it to you later. For now could you >>>>>> run >>>>>> maker with the --debug flag set, apture the STDERR and send it to >>>>>>me. >>>>>> >>>>>> Thanks, >>>>>> Carson >>>>>> >>>>>> >>>>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> Darn - did both, but unfortunately still segfaults... >>>>>>> >>>>>>> Best regards! >>>>>>> >>>>>>> Fourie >>>>>>> >>>>>>> >>>>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>>>> output >>>>>>>> folder before restarting. Perhaps also try reinstalling DB_File >>>>>>>>via >>>>>>>> CPAN. >>>>>>>> >>>>>>>> --Carson >>>>>>>> >>>>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> 1.006901 >>>>>>>>> >>>>>>>>> Regards! >>>>>>>>> >>>>>>>>> Fourie >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>>>> -- >>>>>>>>> -------------- >>>>>>>>> Prof Fourie Joubert >>>>>>>>> Bioinformatics and Computational Biology Unit >>>>>>>>> Department of Biochemistry >>>>>>>>> University of Pretoria >>>>>>>>> fourie.joubert at up.ac.za >>>>>>>>> http://www.bi.up.ac.za >>>>>>>>> Tel. +27-12-420-5825 >>>>>>>>> Fax. +27-12-420-5800 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>------------------------------------------------------------------ >>>>>>>>>-- >>>>>>>>> -- >>>>>>>>> -- >>>>>>>>> - >>>>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>>>> refer >>>>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>>>> details. >>>>>>>>> >>>>>>> -- >>>>>>> -------------- >>>>>>> Prof Fourie Joubert >>>>>>> Bioinformatics and Computational Biology Unit >>>>>>> Department of Biochemistry >>>>>>> University of Pretoria >>>>>>> fourie.joubert at up.ac.za >>>>>>> http://www.bi.up.ac.za >>>>>>> Tel. +27-12-420-5825 >>>>>>> Fax. +27-12-420-5800 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>> -- >>>>>>> - >>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>> refer >>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>> details. >>>>>>> >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From daniel.standage at gmail.com Thu Oct 25 13:30:44 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 25 Oct 2012 15:30:44 -0400 Subject: [maker-devel] Strange error at blastn step Message-ID: Greetings! I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. #--------- command -------------# Widget::blastn: /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn #-------------------------------# deleted:0 hits ERROR: Could not obtain lock to format database FATAL ERROR ERROR: Failed while doing blastn of ESTs!! ERROR: Chunk failed at level 8 !! FAILED CONTIG:scaffold_866 Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 11:52:43 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 13:52:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx #-------------------------------# deleted:-10 hits formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx #-------------------------------# deleted:-6 hits WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. stop here:comp59088_c1_seq7 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:scaffold_0 --Next Contig-- #--------------------------------------------------------------------- Now starting the contig!! SeqID: scaffold_1 Length: 5805686 #--------------------------------------------------------------------- My first thought based on the message is that *blastdbcmd* could not find the sequence in the database. I verified this was the case--I could not extract sequence *comp59088_c1_seq7* from the database Maker had created under /tmp. However, after removing the index files and re-running * makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully extracted the sequence. I was initially happy with this finding, but upon closer inspection it looks like Maker does not use *blastdbcmd* to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a > pretty short contig (about 3kb). However, there seems to be a hiccup during > the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db > /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 > -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 > -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 > -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp > 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis > -out > /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff > old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > > Several blastn steps appeared to have completed successfully to this one > failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From myandell at genetics.utah.edu Fri Oct 26 12:02:29 2012 From: myandell at genetics.utah.edu (Mark Yandell) Date: Fri, 26 Oct 2012 18:02:29 +0000 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: , Message-ID: <7A60AB257EFF2B48B1F4C814817EA05331BA6EC8@mxb1.hg.genetics.utah.edu> Hi Daniel, I think its your fasta-file '> comp59088_c1_seq' note the space between the chevron and the id. This isn't allowed by the fasta format. cheers, --mark Mark Yandell Professor of Human Genetics H.A. & Edna Benning Presidential Endowed Chair Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 ph:801-587-7707 ________________________________________ From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Daniel Standage [daniel.standage at gmail.com] Sent: Friday, October 26, 2012 11:52 AM To: Maker Mailing List Subject: Re: [maker-devel] Strange error at blastn step I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx #-------------------------------# deleted:-10 hits formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx #-------------------------------# deleted:-6 hits WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. stop here:comp59088_c1_seq7 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:scaffold_0 --Next Contig-- #--------------------------------------------------------------------- Now starting the contig!! SeqID: scaffold_1 Length: 5805686 #--------------------------------------------------------------------- My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage > wrote: Greetings! I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. #--------- command -------------# Widget::blastn: /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn #-------------------------------# deleted:0 hits ERROR: Could not obtain lock to format database FATAL ERROR ERROR: Failed while doing blastn of ESTs!! ERROR: Chunk failed at level 8 !! FAILED CONTIG:scaffold_866 Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University From carsonhh at gmail.com Fri Oct 26 12:09:39 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:09:39 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Check to see where /tmp is located? Some clusters have it set up as a tmpfs directory and I have had problems with fasta indexes running from tmpfs mounts in the past. --Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:05 PM To: Carson Holt Subject: Re: [maker-devel] Strange error at blastn step The maker working directory is in a cluster environment with shared scratch space (I'm guessing NFS-mounted). I didn't change the temp directory setting, so it should be the local default (/tmp). I'll give the dev version a shot. Thanks. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > Could you try this development version and tell me if the error still happens? > > Use this command to download --> > <> > > Username: <> > Password: <> > > Are you running in an NFS mounted directory or are you resetting TMP to a > different location? > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 1:52 PM > To: Maker Mailing List > Subject: Re: [maker-devel] Strange error at blastn step > > I have since installed Maker on a different machine and tried it out. The test > run completed successfully, but as I commenced with the full genome > annotation, I have noticed the following error popping up frequently. > >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions >> 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes >> -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker >> .output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0 >> .0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi. >> 10.8.blastx >> #-------------------------------# >> deleted:-10 hits >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions >> 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes >> -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker >> .output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0 >> .0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi. >> 10.9.blastx >> #-------------------------------# >> deleted:-6 hits >> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >> stop here:comp59088_c1_seq7 >> ERROR: Fasta index error >> >> FATAL ERROR >> ERROR: Failed while polishig ESTs!! >> >> ERROR: Chunk failed at level 14 >> !! >> FAILED CONTIG:scaffold_0 >> >> >> >> >> --Next Contig-- >> >> #--------------------------------------------------------------------- >> Now starting the contig!! >> SeqID: scaffold_1 >> Length: 5805686 >> #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the > sequence in the database. I verified this was the case--I could not extract > sequence comp59088_c1_seq7 from the database Maker had created under /tmp. > However, after removing the index files and re-running makeblastdb with the > -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks > like Maker does not use blastdbcmd to extract sequences, but rather its own > internal code. Therefore I'm still not sure where the problem is and how I > might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage > wrote: >> Greetings! >> >> I am doing a test run of my Maker setup on a new machine, annotating a pretty >> short contig (about 3kb). However, there seems to be a hiccup during the >> blastn stage. This is the terminal message. >> >>> #--------- command -------------# >>> Widget::blastn: >>> /share/home/01854/standage/local/bin/blastn -db >>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta >>> .mpi.10.7 -query >>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments >>> 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 >>> -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 >>> -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out >>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.out >>> put/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.P >>> dom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmom >>> atic%2Efasta.mpi.10.7.blastn >>> #-------------------------------# >>> deleted:0 hits >>> ERROR: Could not obtain lock to format database >>> >>> >>> FATAL ERROR >>> ERROR: Failed while doing blastn of ESTs!! >>> >>> ERROR: Chunk failed at level 8 >>> !! >>> FAILED CONTIG:scaffold_866 >> >> Several blastn steps appeared to have completed successfully to this one >> failing. Any ideas what could be causing this? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/mak > er-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:12:38 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:12:38 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: It looks like /tmp is indeed being used: the files I played with were under */tmp/maker_1YQF9o*. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a > tmpfs directory and I have had problems with fasta indexes running from > tmpfs mounts in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared > scratch space (I'm guessing NFS-mounted). I didn't change the temp > directory setting, so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > >> Could you try this development version and tell me if the error still >> happens? >> >> Use this command to download --> >> <> >> >> Username: <> >> Password: <> >> >> Are you running in an NFS mounted directory or are you resetting TMP to a >> different location? >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 1:52 PM >> To: Maker Mailing List >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I have since installed Maker on a different machine and tried it out. The >> test run completed successfully, but as I commenced with the full genome >> annotation, I have noticed the following error popping up frequently. >> >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >> #-------------------------------# >> deleted:-10 hits >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >> #-------------------------------# >> deleted:-6 hits >> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >> stop here:comp59088_c1_seq7 >> ERROR: Fasta index error >> >> FATAL ERROR >> ERROR: Failed while polishig ESTs!! >> >> ERROR: Chunk failed at level 14 >> !! >> FAILED CONTIG:scaffold_0 >> >> >> >> >> --Next Contig-- >> >> #--------------------------------------------------------------------- >> Now starting the contig!! >> SeqID: scaffold_1 >> Length: 5805686 >> #--------------------------------------------------------------------- >> >> >> My first thought based on the message is that *blastdbcmd* could not >> find the sequence in the database. I verified this was the case--I could >> not extract sequence *comp59088_c1_seq7* from the database Maker had >> created under /tmp. However, after removing the index files and re-running >> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >> extracted the sequence. >> >> I was initially happy with this finding, but upon closer inspection it >> looks like Maker does not use *blastdbcmd* to extract sequences, but >> rather its own internal code. Therefore I'm still not sure where the >> problem is and how I might fix it. Any insights? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >> daniel.standage at gmail.com> wrote: >> >>> Greetings! >>> >>> I am doing a test run of my Maker setup on a new machine, annotating a >>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>> the blastn stage. This is the terminal message. >>> >>> #--------- command -------------# >>> Widget::blastn: >>> /share/home/01854/standage/local/bin/blastn -db >>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>> -out >>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>> #-------------------------------# >>> deleted:0 hits >>> ERROR: Could not obtain lock to format database >>> >>> >>> FATAL ERROR >>> ERROR: Failed while doing blastn of ESTs!! >>> >>> ERROR: Chunk failed at level 8 >>> !! >>> FAILED CONTIG:scaffold_866 >>> >>> >>> Several blastn steps appeared to have completed successfully to this one >>> failing. Any ideas what could be causing this? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:14:29 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:14:29 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: The command 'df /tmp' will tell you whether /tmp is a tmpfs mount Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:12 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step It looks like /tmp is indeed being used: the files I played with were under /tmp/maker_1YQF9o. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a tmpfs > directory and I have had problems with fasta indexes running from tmpfs mounts > in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared scratch > space (I'm guessing NFS-mounted). I didn't change the temp directory setting, > so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >> Could you try this development version and tell me if the error still >> happens? >> >> Use this command to download --> >> <> >> >> Username: <> >> Password: <> >> >> Are you running in an NFS mounted directory or are you resetting TMP to a >> different location? >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 1:52 PM >> To: Maker Mailing List >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I have since installed Maker on a different machine and tried it out. The >> test run completed successfully, but as I commenced with the full genome >> annotation, I have noticed the following error popping up frequently. >> >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.make >>> r.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold >>> _0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.m >>> pi.10.8.blastx >>> #-------------------------------# >>> deleted:-10 hits >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.make >>> r.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold >>> _0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.m >>> pi.10.9.blastx >>> #-------------------------------# >>> deleted:-6 hits >>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>> stop here:comp59088_c1_seq7 >>> ERROR: Fasta index error >>> >>> FATAL ERROR >>> ERROR: Failed while polishig ESTs!! >>> >>> ERROR: Chunk failed at level 14 >>> !! >>> FAILED CONTIG:scaffold_0 >>> >>> >>> >>> >>> --Next Contig-- >>> >>> #--------------------------------------------------------------------- >>> Now starting the contig!! >>> SeqID: scaffold_1 >>> Length: 5805686 >>> #--------------------------------------------------------------------- >> >> My first thought based on the message is that blastdbcmd could not find the >> sequence in the database. I verified this was the case--I could not extract >> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >> However, after removing the index files and re-running makeblastdb with the >> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >> >> I was initially happy with this finding, but upon closer inspection it looks >> like Maker does not use blastdbcmd to extract sequences, but rather its own >> internal code. Therefore I'm still not sure where the problem is and how I >> might fix it. Any insights? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >> wrote: >>> Greetings! >>> >>> I am doing a test run of my Maker setup on a new machine, annotating a >>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>> the blastn stage. This is the terminal message. >>> >>>> #--------- command -------------# >>>> Widget::blastn: >>>> /share/home/01854/standage/local/bin/blastn -db >>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efast >>>> a.mpi.10.7 -query >>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>> -show_gis -out >>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.ou >>>> tput/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0 >>>> .Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrim >>>> momatic%2Efasta.mpi.10.7.blastn >>>> #-------------------------------# >>>> deleted:0 hits >>>> ERROR: Could not obtain lock to format database >>>> >>>> >>>> FATAL ERROR >>>> ERROR: Failed while doing blastn of ESTs!! >>>> >>>> ERROR: Chunk failed at level 8 >>>> !! >>>> FAILED CONTIG:scaffold_866 >>> >>> Several blastn steps appeared to have completed successfully to this one >>> failing. Any ideas what could be causing this? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ma >> ker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry.moore at genetics.utah.edu Fri Oct 26 12:19:06 2012 From: barry.moore at genetics.utah.edu (Barry Moore) Date: Fri, 26 Oct 2012 12:19:06 -0600 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> Hi Daniel, What version or revision of MAKER are you running. This sounds like something we were seeing here last night. We traced it as far as what appeared to be soft links in /tmp being set incorrectly. The FastaDB objects had pointers to fasta files in the void for the correct fasta file, but their dir attribute pointed to a /tmp directory in where there were soft links to another (incorrect) fasta file with it's index. It would look appeared that it was looking in the /tmp index (which pointed to an incorrect fasta file) and when it failed to find what it was looking for it would re-index the fasta file in the void (the correct one) and then look again in the index in tmp. Don't know if that helps, but the errors look similar and that's as far as I got with our error here? This was on one of Mike's annotation projects, so I don't know for sure what revision he was running, but I think it was the latest. B On Oct 26, 2012, at 11:52 AM, Daniel Standage wrote: > I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Barry Moore Research Scientist Dept. of Human Genetics University of Utah Salt Lake City, UT 84112 -------------------------------------------- (801) 585-3543 -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:19:07 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:19:07 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Unfortunately, the job is no longer running and as a result I cannot connect to the compute nodes as I could while it was running. On the interactive node, it looks like it's real disk, although it looks like there are some tmpfs mounts. [dstandag at mason src] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 462824304 180235660 259078476 42% /tmp [dstandag at mason src] df Filesystem 1K-blocks Used Available Use% Mounted on login_x86_64 16497564 3077352 13420212 19% / tmpfs 16497564 0 16497564 0% /dev/shm tmpfs 10240 0 10240 0% /var/tmp /dev/sdb2 462824304 180235660 259078476 42% /tmp AFS 9000000 0 9000000 0% /afs bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs ... ... I'll see if I can launch another short job and verify this on the compute nodes. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > The command 'df /tmp' will tell you whether /tmp is a tmpfs mount > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:12 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > It looks like /tmp is indeed being used: the files I played with were > under */tmp/maker_1YQF9o*. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > >> Check to see where /tmp is located? Some clusters have it set up as a >> tmpfs directory and I have had problems with fasta indexes running from >> tmpfs mounts in the past. >> >> --Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:05 PM >> To: Carson Holt >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> The maker working directory is in a cluster environment with shared >> scratch space (I'm guessing NFS-mounted). I didn't change the temp >> directory setting, so it should be the local default (/tmp). >> >> I'll give the dev version a shot. Thanks. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >> >>> Could you try this development version and tell me if the error still >>> happens? >>> >>> Use this command to download --> >>> <> >>> >>> Username: <> >>> Password: <> >>> >>> Are you running in an NFS mounted directory or are you resetting TMP to >>> a different location? >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 1:52 PM >>> To: Maker Mailing List >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> I have since installed Maker on a different machine and tried it out. >>> The test run completed successfully, but as I commenced with the full >>> genome annotation, I have noticed the following error popping up frequently. >>> >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>> #-------------------------------# >>> deleted:-10 hits >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>> #-------------------------------# >>> deleted:-6 hits >>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>> stop here:comp59088_c1_seq7 >>> ERROR: Fasta index error >>> >>> FATAL ERROR >>> ERROR: Failed while polishig ESTs!! >>> >>> ERROR: Chunk failed at level 14 >>> !! >>> FAILED CONTIG:scaffold_0 >>> >>> >>> >>> >>> --Next Contig-- >>> >>> #--------------------------------------------------------------------- >>> Now starting the contig!! >>> SeqID: scaffold_1 >>> Length: 5805686 >>> #--------------------------------------------------------------------- >>> >>> >>> My first thought based on the message is that *blastdbcmd* could not >>> find the sequence in the database. I verified this was the case--I could >>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>> created under /tmp. However, after removing the index files and re-running >>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>> extracted the sequence. >>> >>> I was initially happy with this finding, but upon closer inspection it >>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>> rather its own internal code. Therefore I'm still not sure where the >>> problem is and how I might fix it. Any insights? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>> daniel.standage at gmail.com> wrote: >>> >>>> Greetings! >>>> >>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>> the blastn stage. This is the terminal message. >>>> >>>> #--------- command -------------# >>>> Widget::blastn: >>>> /share/home/01854/standage/local/bin/blastn -db >>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>>> -out >>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>> #-------------------------------# >>>> deleted:0 hits >>>> ERROR: Could not obtain lock to format database >>>> >>>> >>>> FATAL ERROR >>>> ERROR: Failed while doing blastn of ESTs!! >>>> >>>> ERROR: Chunk failed at level 8 >>>> !! >>>> FAILED CONTIG:scaffold_866 >>>> >>>> >>>> Several blastn steps appeared to have completed successfully to this >>>> one failing. Any ideas what could be causing this? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:20:55 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:20:55 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> References: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> Message-ID: Running 2.10. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:19 PM, Barry Moore wrote: > Hi Daniel, > > What version or revision of MAKER are you running. This sounds like > something we were seeing here last night. We traced it as far as what > appeared to be soft links in /tmp being set incorrectly. The FastaDB > objects had pointers to fasta files in the void for the correct fasta file, > but their dir attribute pointed to a /tmp directory in where there were > soft links to another (incorrect) fasta file with it's index. It would > look appeared that it was looking in the /tmp index (which pointed to an > incorrect fasta file) and when it failed to find what it was looking for it > would re-index the fasta file in the void (the correct one) and then look > again in the index in tmp. Don't know if that helps, but the errors look > similar and that's as far as I got with our error here? This was on one of > Mike's annotation projects, so I don't know for sure what revision he was > running, but I think it was the latest. > > B > > On Oct 26, 2012, at 11:52 AM, Daniel Standage wrote: > > I have since installed Maker on a different machine and tried it out. The > test run completed successfully, but as I commenced with the full genome > annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query > /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 > -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 > -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query > /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 > -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 > -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > > My first thought based on the message is that *blastdbcmd* could not find > the sequence in the database. I verified this was the case--I could not > extract sequence *comp59088_c1_seq7* from the database Maker had created > under /tmp. However, after removing the index files and re-running * > makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully > extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it > looks like Maker does not use *blastdbcmd* to extract sequences, but > rather its own internal code. Therefore I'm still not sure where the > problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < > daniel.standage at gmail.com> wrote: > >> Greetings! >> >> I am doing a test run of my Maker setup on a new machine, annotating a >> pretty short contig (about 3kb). However, there seems to be a hiccup during >> the blastn stage. This is the terminal message. >> >> #--------- command -------------# >> Widget::blastn: >> /share/home/01854/standage/local/bin/blastn -db >> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >> -out >> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >> #-------------------------------# >> deleted:0 hits >> ERROR: Could not obtain lock to format database >> >> >> FATAL ERROR >> ERROR: Failed while doing blastn of ESTs!! >> >> ERROR: Chunk failed at level 8 >> !! >> FAILED CONTIG:scaffold_866 >> >> >> Several blastn steps appeared to have completed successfully to this one >> failing. Any ideas what could be causing this? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > Barry Moore > Research Scientist > Dept. of Human Genetics > University of Utah > Salt Lake City, UT 84112 > -------------------------------------------- > (801) 585-3543 > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:24:43 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:24:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Ok. Try the developer release and see if it still happens. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:19 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Unfortunately, the job is no longer running and as a result I cannot connect to the compute nodes as I could while it was running. On the interactive node, it looks like it's real disk, although it looks like there are some tmpfs mounts. [dstandag at mason src] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 462824304 180235660 259078476 42% /tmp [dstandag at mason src] df Filesystem 1K-blocks Used Available Use% Mounted on login_x86_64 16497564 3077352 13420212 19% / tmpfs 16497564 0 16497564 0% /dev/shm tmpfs 10240 0 10240 0% /var/tmp /dev/sdb2 462824304 180235660 259078476 42% /tmp AFS 9000000 0 9000000 0% /afs bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs ... ... I'll see if I can launch another short job and verify this on the compute nodes. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > The command 'df /tmp' will tell you whether /tmp is a tmpfs mount > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:12 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > It looks like /tmp is indeed being used: the files I played with were under > /tmp/maker_1YQF9o. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >> directory and I have had problems with fasta indexes running from tmpfs >> mounts in the past. >> >> --Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:05 PM >> To: Carson Holt >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> The maker working directory is in a cluster environment with shared scratch >> space (I'm guessing NFS-mounted). I didn't change the temp directory setting, >> so it should be the local default (/tmp). >> >> I'll give the dev version a shot. Thanks. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>> Could you try this development version and tell me if the error still >>> happens? >>> >>> Use this command to download --> >>> <> >>> >>> Username: <> >>> Password: <> >>> >>> Are you running in an NFS mounted directory or are you resetting TMP to a >>> different location? >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 1:52 PM >>> To: Maker Mailing List >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> I have since installed Maker on a different machine and tried it out. The >>> test run completed successfully, but as I commenced with the full genome >>> annotation, I have noticed the following error popping up frequently. >>> >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.mak >>>> er.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffo >>>> ld_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efa >>>> a.mpi.10.8.blastx >>>> #-------------------------------# >>>> deleted:-10 hits >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.mak >>>> er.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffo >>>> ld_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efa >>>> a.mpi.10.9.blastx >>>> #-------------------------------# >>>> deleted:-6 hits >>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>> stop here:comp59088_c1_seq7 >>>> ERROR: Fasta index error >>>> >>>> FATAL ERROR >>>> ERROR: Failed while polishig ESTs!! >>>> >>>> ERROR: Chunk failed at level 14 >>>> !! >>>> FAILED CONTIG:scaffold_0 >>>> >>>> >>>> >>>> >>>> --Next Contig-- >>>> >>>> #--------------------------------------------------------------------- >>>> Now starting the contig!! >>>> SeqID: scaffold_1 >>>> Length: 5805686 >>>> #--------------------------------------------------------------------- >>> >>> My first thought based on the message is that blastdbcmd could not find the >>> sequence in the database. I verified this was the case--I could not extract >>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>> However, after removing the index files and re-running makeblastdb with the >>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>> >>> I was initially happy with this finding, but upon closer inspection it looks >>> like Maker does not use blastdbcmd to extract sequences, but rather its own >>> internal code. Therefore I'm still not sure where the problem is and how I >>> might fix it. Any insights? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>> wrote: >>>> Greetings! >>>> >>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>> the blastn stage. This is the terminal message. >>>> >>>>> #--------- command -------------# >>>>> Widget::blastn: >>>>> /share/home/01854/standage/local/bin/blastn -db >>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efas >>>>> ta.mpi.10.7 -query >>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>> -show_gis -out >>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.o >>>>> utput/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866 >>>>> .0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ET >>>>> rimmomatic%2Efasta.mpi.10.7.blastn >>>>> #-------------------------------# >>>>> deleted:0 hits >>>>> ERROR: Could not obtain lock to format database >>>>> >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while doing blastn of ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 8 >>>>> !! >>>>> FAILED CONTIG:scaffold_866 >>>> >>>> Several blastn steps appeared to have completed successfully to this one >>>> failing. Any ideas what could be causing this? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m >>> aker-devel_yandell-lab.org >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:29:18 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:29:18 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot > connect to the compute nodes as I could while it was running. On the > interactive node, it looks like it's real disk, although it looks like > there are some tmpfs mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were >> under */tmp/maker_1YQF9o*. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >> >>> Check to see where /tmp is located? Some clusters have it set up as a >>> tmpfs directory and I have had problems with fasta indexes running from >>> tmpfs mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared >>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>> directory setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>> >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to >>>> a different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. >>>> The test run completed successfully, but as I commenced with the full >>>> genome annotation, I have noticed the following error popping up frequently. >>>> >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>> #-------------------------------# >>>> deleted:-10 hits >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>> #-------------------------------# >>>> deleted:-6 hits >>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>> stop here:comp59088_c1_seq7 >>>> ERROR: Fasta index error >>>> >>>> FATAL ERROR >>>> ERROR: Failed while polishig ESTs!! >>>> >>>> ERROR: Chunk failed at level 14 >>>> !! >>>> FAILED CONTIG:scaffold_0 >>>> >>>> >>>> >>>> >>>> --Next Contig-- >>>> >>>> #--------------------------------------------------------------------- >>>> Now starting the contig!! >>>> SeqID: scaffold_1 >>>> Length: 5805686 >>>> #--------------------------------------------------------------------- >>>> >>>> >>>> My first thought based on the message is that *blastdbcmd* could not >>>> find the sequence in the database. I verified this was the case--I could >>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>> created under /tmp. However, after removing the index files and re-running >>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>> extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>> rather its own internal code. Therefore I'm still not sure where the >>>> problem is and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>> daniel.standage at gmail.com> wrote: >>>> >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>>> the blastn stage. This is the terminal message. >>>>> >>>>> #--------- command -------------# >>>>> Widget::blastn: >>>>> /share/home/01854/standage/local/bin/blastn -db >>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>>>> -out >>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>> #-------------------------------# >>>>> deleted:0 hits >>>>> ERROR: Could not obtain lock to format database >>>>> >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while doing blastn of ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 8 >>>>> !! >>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> >>>>> Several blastn steps appeared to have completed successfully to this >>>>> one failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>> _______________________________________________ maker-devel mailing >>>> list maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:32:35 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:32:35 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: If running './Build install_deps' to get the prereqs automatically, you can say yes to the local version question if you want those prereqs to be installed just for MAKER rather than globally to whatever perl you are using. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:29 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot connect > to the compute nodes as I could while it was running. On the interactive node, > it looks like it's real disk, although it looks like there are some tmpfs > mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were under >> /tmp/maker_1YQF9o. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >>> directory and I have had problems with fasta indexes running from tmpfs >>> mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared scratch >>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>> setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>> different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. The >>>> test run completed successfully, but as I commenced with the full genome >>>> annotation, I have noticed the following error popping up frequently. >>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>> >>>> My first thought based on the message is that blastdbcmd could not find the >>>> sequence in the database. I verified this was the case--I could not extract >>>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>>> However, after removing the index files and re-running makeblastdb with the >>>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>> its own internal code. Therefore I'm still not sure where the problem is >>>> and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>> wrote: >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>> during the blastn stage. This is the terminal message. >>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efa >>>>>> sta.mpi.10.7 -query >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>> -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker. >>>>>> output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_8 >>>>>> 66.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity% >>>>>> 2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> Several blastn steps appeared to have completed successfully to this one >>>>> failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>> >>>> _______________________________________________ maker-devel mailing list >>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ >>>> maker-devel_yandell-lab.org >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:47:43 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:47:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I've got a test run (with the dev version) waiting in the queue. But just to be clear, if the temp directory is indeed the problem, I'm assuming that would that be fixed by setting a different temp directory on the same disk as the scratch disk? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:32 PM, Carson Holt wrote: > If running './Build install_deps' to get the prereqs automatically, you > can say yes to the local version question if you want those prereqs to be > installed just for MAKER rather than globally to whatever perl you are > using. > > Thanks, > Carson > > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot >> connect to the compute nodes as I could while it was running. On the >> interactive node, it looks like it's real disk, although it looks like >> there are some tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were >>> under */tmp/maker_1YQF9o*. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared >>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>> directory setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>> to a different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. >>>>> The test run completed successfully, but as I commenced with the full >>>>> genome annotation, I have noticed the following error popping up frequently. >>>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>>> >>>>> >>>>> My first thought based on the message is that *blastdbcmd* could not >>>>> find the sequence in the database. I verified this was the case--I could >>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>> created under /tmp. However, after removing the index files and re-running >>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>>> rather its own internal code. Therefore I'm still not sure where the >>>>> problem is and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>> daniel.standage at gmail.com> wrote: >>>>> >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>> true -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this >>>>>> one failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>> _______________________________________________ maker-devel mailing >>>>> list maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:56:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:56:30 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Yes. That is correct. Also I've made one more update to the development release with respect to indexes for your test run. Could you run 'svn update' inside the devel maker directory. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:47 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step I've got a test run (with the dev version) waiting in the queue. But just to be clear, if the temp directory is indeed the problem, I'm assuming that would that be fixed by setting a different temp directory on the same disk as the scratch disk? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:32 PM, Carson Holt wrote: > If running './Build install_deps' to get the prereqs automatically, you can > say yes to the local version question if you want those prereqs to be > installed just for MAKER rather than globally to whatever perl you are using. > > Thanks, > Carson > > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot connect >> to the compute nodes as I could while it was running. On the interactive >> node, it looks like it's real disk, although it looks like there are some >> tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were under >>> /tmp/maker_1YQF9o. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared scratch >>>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>>> setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>>> different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. The >>>>> test run completed successfully, but as I commenced with the full genome >>>>> annotation, I have noticed the following error popping up frequently. >>>>> >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.m >>>>>> aker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/sc >>>>>> affold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E4 >>>>>> 7%2Efaa.mpi.10.8.blastx >>>>>> #-------------------------------# >>>>>> deleted:-10 hits >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.m >>>>>> aker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/sc >>>>>> affold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E4 >>>>>> 7%2Efaa.mpi.10.9.blastx >>>>>> #-------------------------------# >>>>>> deleted:-6 hits >>>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>>> stop here:comp59088_c1_seq7 >>>>>> ERROR: Fasta index error >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while polishig ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 14 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --Next Contig-- >>>>>> >>>>>> #--------------------------------------------------------------------- >>>>>> Now starting the contig!! >>>>>> SeqID: scaffold_1 >>>>>> Length: 5805686 >>>>>> #--------------------------------------------------------------------- >>>>> >>>>> My first thought based on the message is that blastdbcmd could not find >>>>> the sequence in the database. I verified this was the case--I could not >>>>> extract sequence comp59088_c1_seq7 from the database Maker had created >>>>> under /tmp. However, after removing the index files and re-running >>>>> makeblastdb with the -parse_seqids option set, blastdbcmd successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>>> its own internal code. Therefore I'm still not sure where the problem is >>>>> and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>>> wrote: >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>>> #--------- command -------------# >>>>>>> Widget::blastn: >>>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Ef >>>>>>> asta.mpi.10.7 -query >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>>> -show_gis -out >>>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker >>>>>>> .output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold >>>>>>> _866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrini >>>>>>> ty%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>>> #-------------------------------# >>>>>>> deleted:0 hits >>>>>>> ERROR: Could not obtain lock to format database >>>>>>> >>>>>>> >>>>>>> FATAL ERROR >>>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>>> >>>>>>> ERROR: Chunk failed at level 8 >>>>>>> !! >>>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this one >>>>>> failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>> >>>>> _______________________________________________ maker-devel mailing list >>>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo >>>>> /maker-devel_yandell-lab.org >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason.stajich at gmail.com Fri Oct 26 14:49:57 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Fri, 26 Oct 2012 13:49:57 -0700 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Right but is tmp on a disk or a tmpfs df -h /tmp would give you the first hint at whether it is a local drive or something else. On Oct 26, 2012, at 11:12 AM, Daniel Standage wrote: > It looks like /tmp is indeed being used: the files I played with were under /tmp/maker_1YQF9o. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a tmpfs directory and I have had problems with fasta indexes running from tmpfs mounts in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared scratch space (I'm guessing NFS-mounted). I didn't change the temp directory setting, so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > Could you try this development version and tell me if the error still happens? > > Use this command to download --> > <> > > Username: <> > Password: <> > > Are you running in an NFS mounted directory or are you resetting TMP to a different location? > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 1:52 PM > To: Maker Mailing List > Subject: Re: [maker-devel] Strange error at blastn step > > I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 16:59:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 18:59:30 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: From: Carson Holt Date: Friday, 26 October, 2012 6:10 PM To: Daniel Standage Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step I've been going over the indexing code using different scenarios and I may have isolated a candidate for what is causing this. Could you do one more 'svn update' inside the maker devel directory before running a test job? Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:29 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot connect > to the compute nodes as I could while it was running. On the interactive node, > it looks like it's real disk, although it looks like there are some tmpfs > mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were under >> /tmp/maker_1YQF9o. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >>> directory and I have had problems with fasta indexes running from tmpfs >>> mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared scratch >>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>> setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>> different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. The >>>> test run completed successfully, but as I commenced with the full genome >>>> annotation, I have noticed the following error popping up frequently. >>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>> >>>> My first thought based on the message is that blastdbcmd could not find the >>>> sequence in the database. I verified this was the case--I could not extract >>>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>>> However, after removing the index files and re-running makeblastdb with the >>>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>> its own internal code. Therefore I'm still not sure where the problem is >>>> and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>> wrote: >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>> during the blastn stage. This is the terminal message. >>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efa >>>>>> sta.mpi.10.7 -query >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>> -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker. >>>>>> output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_8 >>>>>> 66.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity% >>>>>> 2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> Several blastn steps appeared to have completed successfully to this one >>>>> failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>> >>>> _______________________________________________ maker-devel mailing list >>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ >>>> maker-devel_yandell-lab.org >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kokwei86 at gmail.com Sun Oct 28 12:50:19 2012 From: kokwei86 at gmail.com (kokwei) Date: Mon, 29 Oct 2012 02:50:19 +0800 Subject: [maker-devel] Query on genemark results Message-ID: <508D7E6B.3060001@gmail.com> Hi, I have the same problems as posted by Andr? Gomes on 10/4/10. I still have the same problems even though using the current available version of maker (maker 2.1 and maker 2.26 beta version). I have tried to do the gene prediction on eukaryotic genome using 3 ab-initio gene predictors (SNAP, Augustus and GeneMark-ES) using Maker. >From the fasta_merge output, I have 3 separate files of gene models ($prefix.all.maker.augustus_masked.proteins.fasta, $prefix.all.maker.snap_masked.proteins.fasta and $prefix.all.maker.genemark.proteins.fasta) and one $prefix.all.maker.proteins.fasta (I presume this should be the consensus gene models from 3 predictors' results, right?). From the consensus gene models file, I don't see even one result from genemark but all from snap/augustus only. Is that normal? Also from the file naming and gene model label in final gff file, it's showing that genemark is not masked like those of augustus and snap? Is that true? Why only augustus and snap masked but not genemark? Please assist and thanks for your helps. Kok Wei From daniel.standage at gmail.com Mon Oct 29 07:39:05 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Mon, 29 Oct 2012 09:39:05 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Carson, I was able to run the first svn update before the test job ran, but I didn't get the message about this second update until after it has already executed and failed. I got about 372 lines of such. STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_datastore To access files for individual sequences use the datastore index: /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_master_datastore_index.log STATUS: Now running MAKER... WARNING: Cannot find >scaffold_0, trying to re-index the fasta. stop here: scaffold_0 ERROR: Fasta index error at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 239. Process::MpiChunk::_prepare('Process::MpiChunk=HASH(0x3c8b300)', 'HASH(0x3c80820)', 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 73 Process::MpiTiers::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c808b0)', 'HASH(0x3c8ec40)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 79 Process::MpiTiers::_prepare('Process::MpiTiers=HASH(0x3c71f30)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 56 Process::MpiTiers::new('Process::MpiTiers', 'HASH(0x3c80640)', 0, 'Process::MpiChunk') called at /N/u/dstandag/Mason/local/src/maker-dev/bin/maker line 627 --> rank=NA, hostname=c4 ERROR: Failed in tier preparation WARNING: You must always set a rank before running MpiTiers FATAL: argument `seq_id` does not exist in MpiTier object at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 86. Process::MpiChunk::_initialize_vars('Process::MpiChunk=HASH(0x3cc93d0)', 'HASH(0x3cc9400)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 47 Process::MpiChunk::new('Process::MpiChunk', 'HASH(0x3c80820)', 0, 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 407 Process::MpiChunk::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c8f498)', 'HASH(0x3cc8f20)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 3811 Process::MpiChunk::_go('Process::MpiChunk=HASH(0x3c8b300)', 'load', 'HASH(0x3c80820)', 0, 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 310 Process::MpiChunk::_loader('Process::MpiChunk=HASH(0x3c8b300)', 'HASH(0x3c80820)', 0, 0, 'Process::MpiTiers=HASH(0x3c71f30)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 364 Process::MpiTiers::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c8f9c0)', 'HASH(0x3c8fb28)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 375 I'll try the next update and run the test again. I see a lot of references in there to MPI, and just thought I would make it clear that although I am running in a cluster environment, I am using the default serial version of Maker, not the parallel MPI version. Also, after this failed, I tried changing the TMP directory so that it was located on the same NFS mount as the scratch disk to which the output was written. This did not seem to have any affect, and I saw the same issues with the EST sequences unable to be found. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 6:59 PM, Carson Holt wrote: > > > From: Carson Holt > Date: Friday, 26 October, 2012 6:10 PM > To: Daniel Standage > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > I've been going over the indexing code using different scenarios and I may > have isolated a candidate for what is causing this. Could you do one more > 'svn update' inside the maker devel directory before running a test job? > > Thanks, > Carson > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot >> connect to the compute nodes as I could while it was running. On the >> interactive node, it looks like it's real disk, although it looks like >> there are some tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were >>> under */tmp/maker_1YQF9o*. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared >>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>> directory setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>> to a different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. >>>>> The test run completed successfully, but as I commenced with the full >>>>> genome annotation, I have noticed the following error popping up frequently. >>>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>>> >>>>> >>>>> My first thought based on the message is that *blastdbcmd* could not >>>>> find the sequence in the database. I verified this was the case--I could >>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>> created under /tmp. However, after removing the index files and re-running >>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>>> rather its own internal code. Therefore I'm still not sure where the >>>>> problem is and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>> daniel.standage at gmail.com> wrote: >>>>> >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>> true -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this >>>>>> one failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>> _______________________________________________ maker-devel mailing >>>>> list maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Mon Oct 29 07:43:46 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Mon, 29 Oct 2012 09:43:46 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I just ran svn update and tried again with the development version of Maker. Same long list of errors as in the last message. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Mon, Oct 29, 2012 at 9:39 AM, Daniel Standage wrote: > Carson, > > I was able to run the first svn update before the test job ran, but I > didn't get the message about this second update until after it has already > executed and failed. I got about 372 lines of such. > > STATUS: Parsing control files... > STATUS: Processing and indexing input FASTA files... > STATUS: Setting up database for any GFF3 input... > A data structure will be created for you at: > > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_datastore > > To access files for individual sequences use the datastore index: > > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_master_datastore_index.log > > STATUS: Now running MAKER... > WARNING: Cannot find >scaffold_0, trying to re-index the fasta. > stop here: scaffold_0 > ERROR: Fasta index error > at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 239. > Process::MpiChunk::_prepare('Process::MpiChunk=HASH(0x3c8b300)', > 'HASH(0x3c80820)', 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 73 > Process::MpiTiers::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c808b0)', 'HASH(0x3c8ec40)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 79 > Process::MpiTiers::_prepare('Process::MpiTiers=HASH(0x3c71f30)') > called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 56 > Process::MpiTiers::new('Process::MpiTiers', 'HASH(0x3c80640)', 0, > 'Process::MpiChunk') called at > /N/u/dstandag/Mason/local/src/maker-dev/bin/maker line 627 > --> rank=NA, hostname=c4 > ERROR: Failed in tier preparation > WARNING: You must always set a rank before running MpiTiers > FATAL: argument `seq_id` does not exist in MpiTier object > at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 86. > > Process::MpiChunk::_initialize_vars('Process::MpiChunk=HASH(0x3cc93d0)', > 'HASH(0x3cc9400)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 47 > Process::MpiChunk::new('Process::MpiChunk', 'HASH(0x3c80820)', 0, > 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 407 > Process::MpiChunk::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c8f498)', 'HASH(0x3cc8f20)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 3811 > Process::MpiChunk::_go('Process::MpiChunk=HASH(0x3c8b300)', > 'load', 'HASH(0x3c80820)', 0, 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 310 > Process::MpiChunk::_loader('Process::MpiChunk=HASH(0x3c8b300)', > 'HASH(0x3c80820)', 0, 0, 'Process::MpiTiers=HASH(0x3c71f30)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 364 > Process::MpiTiers::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c8f9c0)', 'HASH(0x3c8fb28)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 375 > > > I'll try the next update and run the test again. I see a lot of references > in there to MPI, and just thought I would make it clear that although I am > running in a cluster environment, I am using the default serial version of > Maker, not the parallel MPI version. > > Also, after this failed, I tried changing the TMP directory so that it was > located on the same NFS mount as the scratch disk to which the output was > written. This did not seem to have any affect, and I saw the same issues > with the EST sequences unable to be found. > > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 6:59 PM, Carson Holt wrote: > >> >> >> From: Carson Holt >> Date: Friday, 26 October, 2012 6:10 PM >> To: Daniel Standage >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I've been going over the indexing code using different scenarios and I >> may have isolated a candidate for what is causing this. Could you do one >> more 'svn update' inside the maker devel directory before running a test >> job? >> >> Thanks, >> Carson >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:29 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Got this from the compute node. Looks like native disk space to me. >> >> [dstandag at mason ~] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sda1 478573472 12319684 441943620 3% /tmp >> >> Installing a bundle of Perl prereqs for development version, will try >> that soon. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: >> >>> Ok. Try the developer release and see if it still happens. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:19 PM >>> >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> Unfortunately, the job is no longer running and as a result I cannot >>> connect to the compute nodes as I could while it was running. On the >>> interactive node, it looks like it's real disk, although it looks like >>> there are some tmpfs mounts. >>> >>> [dstandag at mason src] df /tmp >>> Filesystem 1K-blocks Used Available Use% Mounted on >>> /dev/sdb2 462824304 180235660 259078476 42% /tmp >>> [dstandag at mason src] df >>> Filesystem 1K-blocks Used Available Use% Mounted on >>> login_x86_64 16497564 3077352 13420212 19% / >>> tmpfs 16497564 0 16497564 0% /dev/shm >>> tmpfs 10240 0 10240 0% /var/tmp >>> /dev/sdb2 462824304 180235660 259078476 42% /tmp >>> AFS 9000000 0 9000000 0% /afs >>> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >>> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >>> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >>> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >>> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >>> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >>> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >>> ... >>> ... >>> >>> I'll see if I can launch another short job and verify this on the >>> compute nodes. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >>> >>>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:12 PM >>>> To: Carson Holt >>>> Cc: "maker-devel at yandell-lab.org" >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> It looks like /tmp is indeed being used: the files I played with were >>>> under */tmp/maker_1YQF9o*. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>>> >>>>> Check to see where /tmp is located? Some clusters have it set up as a >>>>> tmpfs directory and I have had problems with fasta indexes running from >>>>> tmpfs mounts in the past. >>>>> >>>>> --Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 2:05 PM >>>>> To: Carson Holt >>>>> >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> The maker working directory is in a cluster environment with shared >>>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>>> directory setting, so it should be the local default (/tmp). >>>>> >>>>> I'll give the dev version a shot. Thanks. >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>>> >>>>>> Could you try this development version and tell me if the error still >>>>>> happens? >>>>>> >>>>>> Use this command to download --> >>>>>> <> >>>>>> >>>>>> Username: <> >>>>>> Password: <> >>>>>> >>>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>>> to a different location? >>>>>> >>>>>> Thanks, >>>>>> Carson >>>>>> >>>>>> >>>>>> From: Daniel Standage >>>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>>> To: Maker Mailing List >>>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>>> >>>>>> I have since installed Maker on a different machine and tried it out. >>>>>> The test run completed successfully, but as I commenced with the full >>>>>> genome annotation, I have noticed the following error popping up frequently. >>>>>> >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>>> #-------------------------------# >>>>>> deleted:-10 hits >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>>> #-------------------------------# >>>>>> deleted:-6 hits >>>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>>> stop here:comp59088_c1_seq7 >>>>>> ERROR: Fasta index error >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while polishig ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 14 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --Next Contig-- >>>>>> >>>>>> #--------------------------------------------------------------------- >>>>>> Now starting the contig!! >>>>>> SeqID: scaffold_1 >>>>>> Length: 5805686 >>>>>> #--------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> My first thought based on the message is that *blastdbcmd* could not >>>>>> find the sequence in the database. I verified this was the case--I could >>>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>>> created under /tmp. However, after removing the index files and re-running >>>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>>> extracted the sequence. >>>>>> >>>>>> I was initially happy with this finding, but upon closer inspection >>>>>> it looks like Maker does not use *blastdbcmd* to extract sequences, >>>>>> but rather its own internal code. Therefore I'm still not sure where the >>>>>> problem is and how I might fix it. Any insights? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>>> daniel.standage at gmail.com> wrote: >>>>>> >>>>>>> Greetings! >>>>>>> >>>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>>> during the blastn stage. This is the terminal message. >>>>>>> >>>>>>> #--------- command -------------# >>>>>>> Widget::blastn: >>>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>>> true -show_gis -out >>>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>>> #-------------------------------# >>>>>>> deleted:0 hits >>>>>>> ERROR: Could not obtain lock to format database >>>>>>> >>>>>>> >>>>>>> FATAL ERROR >>>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>>> >>>>>>> ERROR: Chunk failed at level 8 >>>>>>> !! >>>>>>> FAILED CONTIG:scaffold_866 >>>>>>> >>>>>>> >>>>>>> Several blastn steps appeared to have completed successfully to this >>>>>>> one failing. Any ideas what could be causing this? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> -- >>>>>>> Daniel S. Standage >>>>>>> Ph.D. Candidate >>>>>>> Bioinformatics and Computational Biology Program >>>>>>> Department of Genetics, Development, and Cell Biology >>>>>>> Iowa State University >>>>>>> >>>>>>> >>>>>> _______________________________________________ maker-devel mailing >>>>>> list maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Tue Oct 30 15:18:06 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Tue, 30 Oct 2012 14:18:06 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: Message-ID: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Hello Carson and maker community, Thank you very much for your guidelines on using the maker-pipeline. Yes, green sea urchin genome that we are trying to annotate. We are running the on scaffolds and most of these scaffolds are small in size(very first genome assembly). We would typically expect 20,000 genes in this genome. So we are running maker using EST and proteins from the genome and out-groups to generate training dataset for SNAP and Augustus. Depending on the resulting predictions we may bootstrap the predicted genes once again using EST and proteins. Do you have any further suggestions? Also could you point how to convert training set generated for SNAP to be used as training set for Augustus as well? Would maker give equal weightage to SNAP and Augustus predictions for generating gene model? Thanks and regards, Parul Kudtarkar > One thing you seem to be missing is protein evidence. > > Is this a sea urchin (I looked up some of the ESTs)? If so, I would recommend adding all proteins from the Strongylocentrotus purpuratus genome, then throw in another Deuterstome of your choice. Perhaps you should also add a couple of outgroup organisms like Nematostella vectensis > (cnidaria) and a protostome of your choice. Be careful if adding adding to many protostome outgroups (i.e. C. elegans and Drosophila) because a big part of their evolution is gene loss (so distant cnidaria often match > deuterstomes better than most protostomes do). > > You could take the maker results when protein data is included and use it > to retrain SNAP again. > > Even a 22 kb contig is still really short. Is this genome primarily constituted by short contigs like this? I would recommend running CEGMA once on this genome to get an appropriate estimate of how recoverable the > genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). Cegma > will give you an estimate for genome completeness as well as estimates of > what percentage of genes will be found in their entirety and what percent > will be partial genes. This is important to do if your genome is fragmented as it will give you a reasonable expectation of what you can expected to recover (as short contigs don't annotate very well - you tend > to loose a lot). > > Thanks, > Carson > > > On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: > >>Hi Carson, >>Thanks. I have attached another contig which is 22 kb, with as many as 3 exons EST alignments. Could you please recommend additional training. We are currently running maker on the entire contig set and eventually merge >>all the gff3 contig predictions. The using suggested parameter/methods we >>would like to get a consensus gene-set with minimal false >>positives/negatives. >>Thanks, >>Parul >>> The contig in question is really too small to get much out of it (only 5 >>kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb >>> length or longer contigs to glean enough information for optimizing your >>parameters. >>> The general suggestions for any maker run are to use proteins from a >>closely related organism or a couple of closely related organisms for the >>> protein= option in maker. Also leave single_exon set to 0, except for >>certain eukaryotes that have a bias for single exon transcripts (i.e. some >>> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >>predictors tend to over-predict by a wide margin (lots of false >>> positives). >>> Additional training would really depend on what your other contigs look >>like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. >>> Thanks, >>> Carson >>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>Hello, >>>>Please advice on the aforementioned query? >>>>Thanks, >>>>Parul Kudtarkar >>>>---------------------------- Original Message >>>> ---------------------------- >>>>Subject: [maker-devel] Conensus gene model >>>>From: "Parul Kudtarkar" >>>>Date: Fri, October 12, 2012 2:46 pm >>>>To: maker-devel at yandell-lab.org >>>>------------------------------------------------------------------------ -- >>Hi, >>>>We are using snap(training set[hmm file] generated using est,protein >>>> and >>contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>> file >>>>as input). The output that we get is combination of various >>>>gene-predictors and evidences. I have attached sample result file. What >>would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? >>>>Thanks, >>>>Parul Kudtarkar >>>>-- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org_______________________________________________ >>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org_______________________________________________ >>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jason.stajich at gmail.com Tue Oct 30 18:52:43 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Tue, 30 Oct 2012 17:52:43 -0700 Subject: [maker-devel] Conensus gene model In-Reply-To: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> References: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Message-ID: Paul - I think I've posted on this before here if you are asking how to go from SNAP training to Augustus training. http://sourceforge.net/mailarchive/message.php?msg_id=29361270 I do this type of training a lot - here some pointers. I often train by generating models using cegma on the genome and get these 400 or so good models as my training set. when I have EST or RNA-Seq I use PASA to generate the best set of annotations. For CEGMA - then I run this script that comes with MAKER: cegma2zff output.cegma.gff genome.fa Then I follow the SNAP directions fathom genome.ann genome.dna -categorize 1000 fathom uni.ann uni.dna -export 1000 -plus mkdir MYGENOME cd MYGENOME forge ../export.ann ../export.dna --OPTIONS cd ../MYGENOME hmm-assembler.pl MYGENOME MYGENOME > MYGENOME.snap.hmm I then also make the augustus training data like this running in the directory that has the export.ann and export.dna files: perl gene_prediction/zff2augustus_gbk.pl > train.gb using this script: https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl I also make ZFF from GFF with this script if I got the RNA-Seq aligned and best models from PASA and incorporate all these data in to my SNAP training set, and then export again back to gbk for the augustus training. https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/pasatraining2zff.pl Then you just need to run the Augustus training (autoAugTrain.pl) on the train.gb file. Jason On Oct 30, 2012, at 2:18 PM, Parul Kudtarkar wrote: > Hello Carson and maker community, > > Thank you very much for your guidelines on using the maker-pipeline. Yes, > green sea urchin genome that we are trying to annotate. > We are running the on scaffolds and most of these scaffolds are small in > size(very first genome assembly). We would typically expect 20,000 genes > in this genome. So we are running maker using EST and proteins from the > genome and out-groups to generate training dataset for SNAP and Augustus. > Depending on the resulting predictions we may bootstrap the predicted > genes once again using EST and proteins. > > Do you have any further suggestions? Also could you point how to convert > training set generated for SNAP to be used as training set for Augustus as > well? Would maker give equal weightage to SNAP and Augustus predictions > for generating gene model? > > Thanks and regards, > Parul Kudtarkar > >> One thing you seem to be missing is protein evidence. >> >> Is this a sea urchin (I looked up some of the ESTs)? If so, I would > recommend adding all proteins from the Strongylocentrotus purpuratus > genome, then throw in another Deuterstome of your choice. Perhaps you > should also add a couple of outgroup organisms like Nematostella > vectensis >> (cnidaria) and a protostome of your choice. Be careful if adding adding > to many protostome outgroups (i.e. C. elegans and Drosophila) because a > big part of their evolution is gene loss (so distant cnidaria often > match >> deuterstomes better than most protostomes do). >> >> You could take the maker results when protein data is included and use > it >> to retrain SNAP again. >> >> Even a 22 kb contig is still really short. Is this genome primarily > constituted by short contigs like this? I would recommend running CEGMA > once on this genome to get an appropriate estimate of how recoverable > the >> genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). > Cegma >> will give you an estimate for genome completeness as well as estimates > of >> what percentage of genes will be found in their entirety and what > percent >> will be partial genes. This is important to do if your genome is > fragmented as it will give you a reasonable expectation of what you can > expected to recover (as short contigs don't annotate very well - you > tend >> to loose a lot). >> >> Thanks, >> Carson >> >> >> On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >> >>> Hi Carson, >>> Thanks. I have attached another contig which is 22 kb, with as many as 3 > exons EST alignments. Could you please recommend additional training. We > are currently running maker on the entire contig set and eventually > merge >>> all the gff3 contig predictions. The using suggested parameter/methods > we >>> would like to get a consensus gene-set with minimal false >>> positives/negatives. >>> Thanks, >>> Parul >>>> The contig in question is really too small to get much out of it (only 5 >>> kb). There was only one single exon EST alignments and a couple of > predictions with no evidence support. Anything smaller than 10 kb is > mostly useless for annotation purposes. You would really need a few > 100kb >>>> length or longer contigs to glean enough information for optimizing your >>> parameters. >>>> The general suggestions for any maker run are to use proteins from a >>> closely related organism or a couple of closely related organisms for the >>>> protein= option in maker. Also leave single_exon set to 0, except for >>> certain eukaryotes that have a bias for single exon transcripts (i.e. some >>>> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >>> predictors tend to over-predict by a wide margin (lots of false >>>> positives). >>>> Additional training would really depend on what your other contigs > look >>> like. Do you have any large contigs? I could look at one of those and > give suggestions but the provided contig is just too short to glean > much. >>>> Thanks, >>>> Carson >>>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>> Hello, >>>>> Please advice on the aforementioned query? >>>>> Thanks, >>>>> Parul Kudtarkar >>>>> ---------------------------- Original Message >>>>> ---------------------------- >>>>> Subject: [maker-devel] Conensus gene model >>>>> From: "Parul Kudtarkar" >>>>> Date: Fri, October 12, 2012 2:46 pm >>>>> To: maker-devel at yandell-lab.org >>>>> ------------------------------------------------------------------------ > -- >>> Hi, >>>>> We are using snap(training set[hmm file] generated using est,protein >>>>> and >>> contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>>> file >>>>> as input). The output that we get is combination of various >>>>> gene-predictors and evidences. I have attached sample result file. > What >>> would you recommend to get consensus result set? Bootstrapping the > resulting gff3 file (rerunning maker)? >>>>> Thanks, >>>>> Parul Kudtarkar >>>>> -- >>>>> Scientific Programmer >>>>> Center for Computational Regulatory Genomics >>>>> Beckman Institute, >>>>> California Institute of Technology >>>>> http://www.spbase.org_______________________________________________ >>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- >>>>> Scientific Programmer >>>>> Center for Computational Regulatory Genomics >>>>> Beckman Institute, >>>>> California Institute of Technology >>>>> http://www.spbase.org_______________________________________________ >>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> -- >>> Scientific Programmer >>> Center for Computational Regulatory Genomics >>> Beckman Institute, >>> California Institute of Technology >>> http://www.spbase.org >> >> >> > > > -- > Scientific Programmer > Center for Computational Regulatory Genomics > Beckman Institute, > California Institute of Technology > http://www.spbase.org > > > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org From parulk at caltech.edu Wed Oct 31 18:04:33 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Wed, 31 Oct 2012 17:04:33 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Message-ID: <1640.131.215.15.234.1351728273.squirrel@webmail.caltech.edu> Hi Jason, thanks for directions on generating training-set for augustus. Also as alignment evidence if we are providing protein sequences from the primary organism as well as other closely related species is there an option to give the primary protein file precedence over others? At the moment I have all the proteins(from primary organism as well as related species) into a single file as protein option in maker_opts.ctl Thanks and regards, Parul Kudtarkar > Paul - > > I think I've posted on this before here if you are asking how to go from > SNAP training to Augustus training. > http://sourceforge.net/mailarchive/message.php?msg_id=29361270 > > I do this type of training a lot - here some pointers. > > I often train by generating models using cegma on the genome and get these > 400 or so good models as my training set. when I have EST or RNA-Seq I > use PASA to generate the best set of annotations. > > For CEGMA - then I run this script that comes with MAKER: > cegma2zff output.cegma.gff genome.fa > > Then I follow the SNAP directions > > fathom genome.ann genome.dna -categorize 1000 > fathom uni.ann uni.dna -export 1000 -plus > mkdir MYGENOME > cd MYGENOME > forge ../export.ann ../export.dna --OPTIONS > cd ../MYGENOME > hmm-assembler.pl MYGENOME MYGENOME > MYGENOME.snap.hmm > > I then also make the augustus training data like this running in the > directory that has the export.ann and export.dna files: > perl gene_prediction/zff2augustus_gbk.pl > train.gb > > using this script: > https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl > > I also make ZFF from GFF with this script if I got the RNA-Seq aligned and > best models from PASA and incorporate all these data in to my SNAP > training set, and then export again back to gbk for the augustus training. > https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/pasatraining2zff.pl > > Then you just need to run the Augustus training (autoAugTrain.pl) on the > train.gb file. > > Jason > > On Oct 30, 2012, at 2:18 PM, Parul Kudtarkar wrote: > >> Hello Carson and maker community, >> >> Thank you very much for your guidelines on using the maker-pipeline. >> Yes, >> green sea urchin genome that we are trying to annotate. >> We are running the on scaffolds and most of these scaffolds are small in >> size(very first genome assembly). We would typically expect 20,000 genes >> in this genome. So we are running maker using EST and proteins from the >> genome and out-groups to generate training dataset for SNAP and >> Augustus. >> Depending on the resulting predictions we may bootstrap the predicted >> genes once again using EST and proteins. >> >> Do you have any further suggestions? Also could you point how to convert >> training set generated for SNAP to be used as training set for Augustus >> as >> well? Would maker give equal weightage to SNAP and Augustus predictions >> for generating gene model? >> >> Thanks and regards, >> Parul Kudtarkar >> >>> One thing you seem to be missing is protein evidence. >>> >>> Is this a sea urchin (I looked up some of the ESTs)? If so, I would >> recommend adding all proteins from the Strongylocentrotus purpuratus >> genome, then throw in another Deuterstome of your choice. Perhaps you >> should also add a couple of outgroup organisms like Nematostella >> vectensis >>> (cnidaria) and a protostome of your choice. Be careful if adding >>> adding >> to many protostome outgroups (i.e. C. elegans and Drosophila) because a >> big part of their evolution is gene loss (so distant cnidaria often >> match >>> deuterstomes better than most protostomes do). >>> >>> You could take the maker results when protein data is included and use >> it >>> to retrain SNAP again. >>> >>> Even a 22 kb contig is still really short. Is this genome primarily >> constituted by short contigs like this? I would recommend running CEGMA >> once on this genome to get an appropriate estimate of how recoverable >> the >>> genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). >> Cegma >>> will give you an estimate for genome completeness as well as estimates >> of >>> what percentage of genes will be found in their entirety and what >> percent >>> will be partial genes. This is important to do if your genome is >> fragmented as it will give you a reasonable expectation of what you can >> expected to recover (as short contigs don't annotate very well - you >> tend >>> to loose a lot). >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >>> >>>> Hi Carson, >>>> Thanks. I have attached another contig which is 22 kb, with as many as >>>> 3 >> exons EST alignments. Could you please recommend additional training. We >> are currently running maker on the entire contig set and eventually >> merge >>>> all the gff3 contig predictions. The using suggested parameter/methods >> we >>>> would like to get a consensus gene-set with minimal false >>>> positives/negatives. >>>> Thanks, >>>> Parul >>>>> The contig in question is really too small to get much out of it >>>>> (only 5 >>>> kb). There was only one single exon EST alignments and a couple of >> predictions with no evidence support. Anything smaller than 10 kb is >> mostly useless for annotation purposes. You would really need a few >> 100kb >>>>> length or longer contigs to glean enough information for optimizing >>>>> your >>>> parameters. >>>>> The general suggestions for any maker run are to use proteins from a >>>> closely related organism or a couple of closely related organisms for >>>> the >>>>> protein= option in maker. Also leave single_exon set to 0, except >>>>> for >>>> certain eukaryotes that have a bias for single exon transcripts (i.e. >>>> some >>>>> fungi and oomycetes). And leave keep_preds set to 0 because ab >>>>> initio >>>> predictors tend to over-predict by a wide margin (lots of false >>>>> positives). >>>>> Additional training would really depend on what your other contigs >> look >>>> like. Do you have any large contigs? I could look at one of those >>>> and >> give suggestions but the provided contig is just too short to glean >> much. >>>>> Thanks, >>>>> Carson >>>>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>>> Hello, >>>>>> Please advice on the aforementioned query? >>>>>> Thanks, >>>>>> Parul Kudtarkar >>>>>> ---------------------------- Original Message >>>>>> ---------------------------- >>>>>> Subject: [maker-devel] Conensus gene model >>>>>> From: "Parul Kudtarkar" >>>>>> Date: Fri, October 12, 2012 2:46 pm >>>>>> To: maker-devel at yandell-lab.org >>>>>> ------------------------------------------------------------------------ >> -- >>>> Hi, >>>>>> We are using snap(training set[hmm file] generated using est,protein >>>>>> and >>>> contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>>>> file >>>>>> as input). The output that we get is combination of various >>>>>> gene-predictors and evidences. I have attached sample result file. >> What >>>> would you recommend to get consensus result set? Bootstrapping the >> resulting gff3 file (rerunning maker)? >>>>>> Thanks, >>>>>> Parul Kudtarkar >>>>>> -- >>>>>> Scientific Programmer >>>>>> Center for Computational Regulatory Genomics >>>>>> Beckman Institute, >>>>>> California Institute of Technology >>>>>> http://www.spbase.org_______________________________________________ >>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> -- >>>>>> Scientific Programmer >>>>>> Center for Computational Regulatory Genomics >>>>>> Beckman Institute, >>>>>> California Institute of Technology >>>>>> http://www.spbase.org_______________________________________________ >>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> -- >>>> Scientific Programmer >>>> Center for Computational Regulatory Genomics >>>> Beckman Institute, >>>> California Institute of Technology >>>> http://www.spbase.org >>> >>> >>> >> >> >> -- >> Scientific Programmer >> Center for Computational Regulatory Genomics >> Beckman Institute, >> California Institute of Technology >> http://www.spbase.org >> >> >> >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > Jason Stajich > jason.stajich at gmail.com > jason at bioperl.org > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From carsonhh at gmail.com Mon Oct 1 08:01:01 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 01 Oct 2012 10:01:01 -0400 Subject: [maker-devel] model_gff question In-Reply-To: Message-ID: They can be replaced under two circumstances. 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. Thanks, Carson From: Michael Thon Date: Monday, 1 October, 2012 1:53 AM To: Subject: [maker-devel] model_gff question Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. -Mike _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.thon at gmail.com Mon Oct 1 23:01:09 2012 From: mike.thon at gmail.com (Michael Thon) Date: Tue, 2 Oct 2012 07:01:09 +0200 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: I looked at two cases in which the model_gff disappeared and they occurred in regions where there are multiple overlapping cufflinks features. One model that I'm looking at right now has overlapping protein2genome and a SNAP feature overlapping it but it was still not included in the output. it could be a problem in MAKER or it could be a problem with my RNA Seq data. I aligned the RNA Seq data using tophat/cufflinks and converted the transcripts.gtf file to gff using cufflinks2gff3 script. Is it better to use RNA Seq feature from tophat or cufflinks? On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > They can be replaced under two circumstances. > 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. > 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). > > MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. > > Thanks, > Carson > > > From: Michael Thon > Date: Monday, 1 October, 2012 1:53 AM > To: > Subject: [maker-devel] model_gff question > > Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: > https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion > > That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. > -Mike > > _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mike.thon at gmail.com Tue Oct 2 01:50:04 2012 From: mike.thon at gmail.com (Michael Thon) Date: Tue, 2 Oct 2012 09:50:04 +0200 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: <8E7B4629-B071-44A6-8E7F-2B1133A946D0@gmail.com> It seems to have disappeared completely. I'm running MAKER again now using the tophat alignments that I fed to cufflinks, instead of the cufflinks data. So far the two models visually checked as missing with the cufflinks data are present as they should be. I have to wait for the run to finish to get a whole genome count though. Maybe I need to look more closely at the cufflinks run that I did. The RNA-Seq data are from the NCBI SRA and I didn't do anything to clean them up before I ran tophat. On Oct 2, 2012, at 9:10 AM, Daniel Hughes wrote: > Did the whole model vanish or just the protein product - contaminated rnaseq that hasn't been cleaned up enough will regularly cause the later to become part of a bad utr. > > Dan > > On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: > I looked at two cases in which the model_gff disappeared and they occurred in regions where there are multiple overlapping cufflinks features. One model that I'm looking at right now has overlapping protein2genome and a SNAP feature overlapping it but it was still not included in the output. it could be a problem in MAKER or it could be a problem with my RNA Seq data. I aligned the RNA Seq data using tophat/cufflinks and converted the transcripts.gtf file to gff using cufflinks2gff3 script. > > Is it better to use RNA Seq feature from tophat or cufflinks? > > > On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > >> They can be replaced under two circumstances. >> 1. If you provide two model_gff files (comma separated list), in which case MAKER thinks it is merging legacy annotations and will only keep one or the other if models overlap. >> 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this as a cue that if these other programs produce a better model, it can replace the current model. If you set map_forward=1, MAKER will conserve the name of the previous model (so models change structure but names are conserved); otherwise, it gets a new name. Sometimes groups like to rename models every time their is a structural change. I think you are supposed to get the Alias attribute set when you don't get names mapped forward though (I can't remember if I added this or just planned on adding the Alias mapping though). >> >> MAKER should never drop a model_gff model. It can only replace it if something better comes along, but it should not disappear. >> >> Thanks, >> Carson >> >> >> From: Michael Thon >> Date: Monday, 1 October, 2012 1:53 AM >> To: >> Subject: [maker-devel] model_gff question >> >> Under what circumstances will maker not include a gene model from the model_gff file in its final output? It was my understanding from this post: >> https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion >> >> That maker will keep or replace models in model_gff and never remove them. I'm reannotating a fungal genome and in model_gff I'm providing the gene models originally made by the sequencing center. I have 12006 models in the file I specify in model_gff but maker's final annotation has only 10727 models in it. >> -Mike >> >> _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Oct 2 12:05:04 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 02 Oct 2012 14:05:04 -0400 Subject: [maker-devel] model_gff question In-Reply-To: <8E7B4629-B071-44A6-8E7F-2B1133A946D0@gmail.com> Message-ID: If it overlaps the UTR of some other chosen model it might have been excluded for that reason as well. Sometimes this can happen when you have RNA-seq and high gene density (so models wander into each other). Try setting the the correct_est_fusion option to 1. This will take steps to trim UTR that might cause neighboring models to be left out because of UTR overlap (it also helps with false fusions caused by cufflinks results). I would not be surprised if the model being left out was because of UTR overlap. I recommend using the cufflinks data and leaving tophat out. Tophat results tend to be very noisy and can span large rather weird regions. Thanks, Carson From: Michael Thon Date: Tuesday, 2 October, 2012 3:50 AM To: Daniel Hughes Cc: Michael Thon , , Carson Holt Subject: Re: [maker-devel] model_gff question It seems to have disappeared completely. I'm running MAKER again now using the tophat alignments that I fed to cufflinks, instead of the cufflinks data. So far the two models visually checked as missing with the cufflinks data are present as they should be. I have to wait for the run to finish to get a whole genome count though. Maybe I need to look more closely at the cufflinks run that I did. The RNA-Seq data are from the NCBI SRA and I didn't do anything to clean them up before I ran tophat. On Oct 2, 2012, at 9:10 AM, Daniel Hughes wrote: > > Did the whole model vanish or just the protein product - contaminated rnaseq > that hasn't been cleaned up enough will regularly cause the later to become > part of a bad utr. > > Dan > > On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: >> I looked at two cases in which the model_gff disappeared and they occurred in >> regions where there are multiple overlapping cufflinks features. One model >> that I'm looking at right now has overlapping protein2genome and a SNAP >> feature overlapping it but it was still not included in the output. it could >> be a problem in MAKER or it could be a problem with my RNA Seq data. I >> aligned the RNA Seq data using tophat/cufflinks and converted the >> transcripts.gtf file to gff using cufflinks2gff3 script. >> >> Is it better to use RNA Seq feature from tophat or cufflinks? >> >> >> On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: >> >>> They can be replaced under two circumstances. >>> 1. If you provide two model_gff files (comma separated list), in which case >>> MAKER thinks it is merging legacy annotations and will only keep one or the >>> other if models overlap. >>> 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees this >>> as a cue that if these other programs produce a better model, it can replace >>> the current model. If you set map_forward=1, MAKER will conserve the name >>> of the previous model (so models change structure but names are conserved); >>> otherwise, it gets a new name. Sometimes groups like to rename models every >>> time their is a structural change. I think you are supposed to get the >>> Alias attribute set when you don't get names mapped forward though (I can't >>> remember if I added this or just planned on adding the Alias mapping >>> though). >>> >>> MAKER should never drop a model_gff model. It can only replace it if >>> something better comes along, but it should not disappear. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Michael Thon >>> Date: Monday, 1 October, 2012 1:53 AM >>> To: >>> Subject: [maker-devel] model_gff question >>> >>> Under what circumstances will maker not include a gene model from the >>> model_gff file in its final output? It was my understanding from this post: >>> https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion >>> >>> That maker will keep or replace models in model_gff and never remove them. >>> I'm reannotating a fungal genome and in model_gff I'm providing the gene >>> models originally made by the sequencing center. I have 12006 models in the >>> file I specify in model_gff but maker's final annotation has only 10727 >>> models in it. >>> -Mike >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m >>> aker-devel_yandell-lab.org >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dsthughes at gmail.com Tue Oct 2 01:10:19 2012 From: dsthughes at gmail.com (Daniel Hughes) Date: Tue, 2 Oct 2012 08:10:19 +0100 Subject: [maker-devel] model_gff question In-Reply-To: References: Message-ID: Did the whole model vanish or just the protein product - contaminated rnaseq that hasn't been cleaned up enough will regularly cause the later to become part of a bad utr. Dan On Oct 2, 2012 6:01 AM, "Michael Thon" wrote: > I looked at two cases in which the model_gff disappeared and they occurred > in regions where there are multiple overlapping cufflinks features. One > model that I'm looking at right now has overlapping protein2genome and a > SNAP feature overlapping it but it was still not included in the output. > it could be a problem in MAKER or it could be a problem with my RNA Seq > data. I aligned the RNA Seq data using tophat/cufflinks and converted the > transcripts.gtf file to gff using cufflinks2gff3 script. > > Is it better to use RNA Seq feature from tophat or cufflinks? > > > On Oct 1, 2012, at 4:01 PM, Carson Holt wrote: > > They can be replaced under two circumstances. > 1. If you provide two model_gff files (comma separated list), in which > case MAKER thinks it is merging legacy annotations and will only keep one > or the other if models overlap. > 2. If you turn snap, augusutus, genemark, or est2genome on. MAKER sees > this as a cue that if these other programs produce a better model, it can > replace the current model. If you set map_forward=1, MAKER will conserve > the name of the previous model (so models change structure but names are > conserved); otherwise, it gets a new name. Sometimes groups like to rename > models every time their is a structural change. I think you are supposed > to get the Alias attribute set when you don't get names mapped forward > though (I can't remember if I added this or just planned on adding the > Alias mapping though). > > MAKER should never drop a model_gff model. It can only replace it if > something better comes along, but it should not disappear. > > Thanks, > Carson > > > From: Michael Thon > Date: Monday, 1 October, 2012 1:53 AM > To: > Subject: [maker-devel] model_gff question > > Under what circumstances will maker not include a gene model from the > model_gff file in its final output? It was my understanding from this post: > https://groups.google.com/d/topic/maker-devel/Y5jSdZ1Olcc/discussion > > That maker will keep or replace models in model_gff and never remove them. > I'm reannotating a fungal genome and in model_gff I'm providing the gene > models originally made by the sequencing center. I have 12006 models in > the file I specify in model_gff but maker's final annotation has only 10727 > models in it. > -Mike > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From amelia.ireland at gmod.org Tue Oct 2 15:00:57 2012 From: amelia.ireland at gmod.org (Amelia Ireland) Date: Tue, 2 Oct 2012 14:00:57 -0700 Subject: [maker-devel] problem installing maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868AF3A@EXMBX05.austin.utexas.edu> References: <1B4E4BB25B711A4FBC93E4B3AFEB2A3218689DC6@EXMBX05.austin.utexas.edu> <1B4E4BB25B711A4FBC93E4B3AFEB2A321868AF3A@EXMBX05.austin.utexas.edu> Message-ID: Hello Oscar, I'm forwarding your email on to the MAKER mailing list and one of the developers; they should be able to answer your query. Thanks, Amelia. On Tue, Oct 2, 2012 at 1:55 PM, Dian "Oscar" Jiao wrote: > Hi, > > I was trying to install Maker. All the prerequisite seemed installed okay. > However, when I ran maker or mpimaker, I got the following error: > /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_File/DB_File.so: > undefined symbol: db_version". I have edited config.in before compiling to > make sure the paths are correct for db.h and libdb. Do you have any idea > what is causing this? > > It seems to have to do with the berkeleydb and DB_File. I have BerkeleyDB > 5.3.21 installed and set the paths correctly for db.h and libdb in config.in > when I was compiling DB_File, but it still failed. Do you know what is > causing this? > > Thanks > Oscar > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > -- > You received this message because you are subscribed to the Google Groups > "GMOD Help Desk" group. > To post to this group, send email to gmod-help-desk at googlegroups.com. > To unsubscribe from this group, send email to > gmod-help-desk+unsubscribe at googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out. > > -- Amelia Ireland GMOD Community Support Specialist || http://gmod.org From carsonhh at gmail.com Wed Oct 3 08:13:20 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 03 Oct 2012 10:13:20 -0400 Subject: [maker-devel] problem installing maker In-Reply-To: Message-ID: You may need to completely delete this directory before reinstalling DB_File. --> /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_Fil e Also if that doesn't work you can skip DB_file completely, and use a different database backend During the MAKER install do the following: perl ./Build.PL --AnyDBM_ISA SDBM_File ./Build install Then test maker once on a new job. The --AnyDBM_ISA sets a different database than Berkley DB (All of which are slower than Berkley DB). --Carson On 12-10-02 5:00 PM, "Amelia Ireland" wrote: >Hello Oscar, > >I'm forwarding your email on to the MAKER mailing list and one of the >developers; they should be able to answer your query. > >Thanks, >Amelia. > >On Tue, Oct 2, 2012 at 1:55 PM, Dian "Oscar" Jiao >wrote: >> Hi, >> >> I was trying to install Maker. All the prerequisite seemed installed >>okay. >> However, when I ran maker or mpimaker, I got the following error: >> >>/work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/auto/DB_F >>ile/DB_File.so: >> undefined symbol: db_version". I have edited config.in before compiling >>to >> make sure the paths are correct for db.h and libdb. Do you have any idea >> what is causing this? >> >> It seems to have to do with the berkeleydb and DB_File. I have >>BerkeleyDB >> 5.3.21 installed and set the paths correctly for db.h and libdb in >>config.in >> when I was compiling DB_File, but it still failed. Do you know what is >> causing this? >> >> Thanks >> Oscar >> -- >> Dian (Oscar) Jiao, Ph.D., >> Research Associate, Life Sciences Computing Group >> Texas Advanced Computing Center >> University of Texas at Austin >> jiao at tacc.utexas.edu | (832) 303-1166 >> >> -- >> You received this message because you are subscribed to the Google >>Groups >> "GMOD Help Desk" group. >> To post to this group, send email to gmod-help-desk at googlegroups.com. >> To unsubscribe from this group, send email to >> gmod-help-desk+unsubscribe at googlegroups.com. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> > > > >-- >Amelia Ireland >GMOD Community Support Specialist || http://gmod.org > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From parulk at caltech.edu Wed Oct 3 20:14:36 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Wed, 3 Oct 2012 19:14:36 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs Message-ID: <2553.131.215.15.234.1349316876.squirrel@webmail.caltech.edu> I am running maker on example data that comes along with installation and is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note Please advice on the aforementioned error. --------------------- Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_proteins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est%2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_protein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_proteins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est%2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/contig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_protein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ------------------------------------ Many thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jiao at tacc.utexas.edu Wed Oct 3 22:04:32 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 04:04:32 +0000 Subject: [maker-devel] problem running maker Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868CD39@EXMBX05.austin.utexas.edu> Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 06:27:05 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 08:27:05 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868CD39@EXMBX05.austin.utexas.edu> Message-ID: Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 06:34:38 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 08:34:38 -0400 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <2553.131.215.15.234.1349316876.squirrel@webmail.caltech.edu> Message-ID: You probably need to reinstall Bio-perl. Other things that can cause the same error are setting TMP in the maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or sometimes setting it to an NFS mount. A full drive can cause this as well or a broken Berkley DB. Use df -h to see if any of the drives used are full (either current working directory or TMP location). You can also swap Berkley DB for a different backend by setting AnyDBM_ISA during setup. Example: cd ../maker/src/ perl Build.PL --AnyDBM_ISA SDBM_File ./Build install --Carson On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >I am running maker on example data that comes along with installation and >is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note > >Please advice on the aforementioned error. > >--------------------- >Maker is now finished!!! > >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >eins%2Efasta.repeatrunner >deleted:0 hits > in cluster:shadow cluster... > i_size:0 j_size:0 > sorting hits in shadow cluster... >... finished. >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >%2Efasta.blastn >deleted:-1 hits >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >tein%2Efasta.blastx >WARNING: Multiple MAKER processes have been started in the >same directory. > >deleted:0 hits >WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >stop here:dpp-mRNA-4 >ERROR: Fasta index error > >FATAL ERROR >Maker is now finished!!! > >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >eins%2Efasta.repeatrunner >deleted:0 hits > in cluster:shadow cluster... > i_size:0 j_size:0 > sorting hits in shadow cluster... >... finished. >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >%2Efasta.blastn >deleted:-1 hits >re reading blast report. >/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >tein%2Efasta.blastx >WARNING: Multiple MAKER processes have been started in the >same directory. > >deleted:0 hits >WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >stop here:dpp-mRNA-4 >ERROR: Fasta index error > >FATAL ERROR >ERROR: Failed while polishig ESTs!! > >ERROR: Chunk failed at level 14 >!! >FAILED CONTIG:contig-dpp-500-500 > > > >ERROR: Chunk failed at level 14 >!! >FAILED CONTIG:contig-dpp-500-500 >------------------------------------ > > >Many thanks, >Parul Kudtarkar > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From jiao at tacc.utexas.edu Thu Oct 4 08:02:14 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 14:02:14 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D305@EXMBX05.austin.utexas.edu> 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 08:03:01 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 10:03:01 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D305@EXMBX05.austin.utexas.edu> Message-ID: Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 09:36:22 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 15:36:22 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 11:17:08 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 13:17:08 -0400 Subject: [maker-devel] problem running maker In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Message-ID: Seg faults really only happen from C and not Perl, so there may be a perl module that is broken on your machine that use C internally. Here are some modules that MAKER can use that I know use C and that you might want to reinstall from CPAN. DB_File Forks Proc::ProcessTable Inline::C Also during the maker install say no to the "use MPI" question. It's best to make the installation as simple as possible before getting into MPI configuration. Also after installing those and rerunning maker's 'perl Build.PL' and './Build install' steps do your first run with 'maker --debug' (redirect STDERR to a file). Then if it fails, you can send me the error log. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 11:36 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 11:23:52 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 17:23:52 +0000 Subject: [maker-devel] problem running maker In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D36D@EXMBX05.austin.utexas.edu> I tried the debug trick and it worked (at least it finished). I however when I looked at the stderr, I again noticed the "can't locate GDBM_File for @AnyDBM_File::ISA" error. Although it did not affect the output, is it what is causing the segmentation error? Because this is the only error other than a bunch of "UNKOWN Bio" messages. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 12:17 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Seg faults really only happen from C and not Perl, so there may be a perl module that is broken on your machine that use C internally. Here are some modules that MAKER can use that I know use C and that you might want to reinstall from CPAN. DB_File Forks Proc::ProcessTable Inline::C Also during the maker install say no to the "use MPI" question. It's best to make the installation as simple as possible before getting into MPI configuration. Also after installing those and rerunning maker's 'perl Build.PL' and './Build install' steps do your first run with 'maker --debug' (redirect STDERR to a file). Then if it fails, you can send me the error log. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 11:36 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Ok. So I installed 2.26 instead. When I ran maker again I got segmentation fault with no additional message. I am using dpp_contig.fasta, dpp_est.fasta and dpp_protein.fasta. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Try 2.26 and let me know. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker 2.10 -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] problem running maker Are you trying to install 2.26 or 2.10? --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker Hi, I just installed maker and tried to run it with these dpp example fasta files that came with the package. However when I do "maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like there are three errors: (1) defined array deprecated; (2) unknown state in Signal.pm and (3) GDBM_File (etc.) missing The first line seems to be just warning. I got it even when I do just "maker" or "maker ctl". The control files did get generated. And what is the relationship between the three perl modules, GDBM/NDBM/SDBM_File, AnyDBM and DB_File? Oscar _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 12:52:51 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 14:52:51 -0400 Subject: [maker-devel] Expected format for EST GFF3 Message-ID: Greetings. I would like to use Maker's *est_gff* option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 12:58:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 14:58:56 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Match/match_part features are expected. MAKER expects these to be polished (I.e. correctly aligned around splice sites). For example exonerate, BLAT, and cufflinks results will be correct around splice sites and can be used; BLAST results on the other hand will not be correct and should not be used. The Gap attribute is used by maker if available, but is not required (Gap describes how to reconstruct an alignment for gaps and mismatches). Otherwise MAKER assumes all positions are matches to the reference. Thanks, Carson From: Daniel Standage Date: Thursday, 4 October, 2012 2:52 PM To: Maker Mailing List Subject: [maker-devel] Expected format for EST GFF3 Greetings. I would like to use Maker's est_gff option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:07:18 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:07:18 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the *transcript* features to *match* and the *exon* features to *match_part *and make sure the parent/child relationships are maintained with the *Parent* and *ID* attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > Match/match_part features are expected. MAKER expects these to be > polished (I.e. correctly aligned around splice sites). For example > exonerate, BLAT, and cufflinks results will be correct around splice sites > and can be used; BLAST results on the other hand will not be correct and > should not be used. > > The Gap attribute is used by maker if available, but is not required (Gap > describes how to reconstruct an alignment for gaps and mismatches). > Otherwise MAKER assumes all positions are matches to the reference. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM > To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 > > Greetings. > > I would like to use Maker's *est_gff* option to include EST evidence from > an external GFF3 file. In addition to being a valid, well-formed GFF3 file, > what expectations does Maker assume about the contents of this file? Which > feature types are expected and/or supported? Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 13:09:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 15:09:25 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Use the cufflinks2gff3 script that comes with MAKER. Thanks, Carson From: Daniel Standage Date: Thursday, 4 October, 2012 3:07 PM To: Carson Holt Cc: Maker Mailing List Subject: Re: [maker-devel] Expected format for EST GFF3 Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only transcript and exon features. So I'm assuming I can simply convert the transcript features to match and the exon features to match_part and make sure the parent/child relationships are maintained with the Parent and ID attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > Match/match_part features are expected. MAKER expects these to be polished > (I.e. correctly aligned around splice sites). For example exonerate, BLAT, > and cufflinks results will be correct around splice sites and can be used; > BLAST results on the other hand will not be correct and should not be used. > > The Gap attribute is used by maker if available, but is not required (Gap > describes how to reconstruct an alignment for gaps and mismatches). Otherwise > MAKER assumes all positions are matches to the reference. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM > To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 > > Greetings. > > I would like to use Maker's est_gff option to include EST evidence from an > external GFF3 file. In addition to being a valid, well-formed GFF3 file, what > expectations does Maker assume about the contents of this file? Which feature > types are expected and/or supported? Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/mak > er-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:16:53 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:16:53 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Does the cufflinks2gff3 script filter out single-exon transcripts? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt wrote: > Use the cufflinks2gff3 script that comes with MAKER. > > Thanks, > Carson > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 3:07 PM > To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 > > Great. I am using PE Illumina reads mapped by Tophat and assembled by > Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the > *transcript* features to *match* and the *exon* features to *match_part *and > make sure the parent/child relationships are maintained with the *Parent* and > *ID* attributes? > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: > >> Match/match_part features are expected. MAKER expects these to be >> polished (I.e. correctly aligned around splice sites). For example >> exonerate, BLAT, and cufflinks results will be correct around splice sites >> and can be used; BLAST results on the other hand will not be correct and >> should not be used. >> >> The Gap attribute is used by maker if available, but is not required (Gap >> describes how to reconstruct an alignment for gaps and mismatches). >> Otherwise MAKER assumes all positions are matches to the reference. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Thursday, 4 October, 2012 2:52 PM >> To: Maker Mailing List >> Subject: [maker-devel] Expected format for EST GFF3 >> >> Greetings. >> >> I would like to use Maker's *est_gff* option to include EST evidence >> from an external GFF3 file. In addition to being a valid, well-formed GFF3 >> file, what expectations does Maker assume about the contents of this file? >> Which feature types are expected and/or supported? Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Thu Oct 4 13:21:52 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 4 Oct 2012 15:21:52 -0400 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: References: Message-ID: Good to know. Thanks for your help! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:19 PM, Carson Holt wrote: > Yes. You really have to because the single exon transcripts produced > from mRNA-seq assembly are strandless (you don't know where they belong). > They also tend to be heavily weighted to pseudogenes and transposons. > > Thanks, > Carson > > > > > From: Daniel Standage > Date: Thursday, 4 October, 2012 3:16 PM > > To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 > > Does the cufflinks2gff3 script filter out single-exon transcripts? > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt wrote: > >> Use the cufflinks2gff3 script that comes with MAKER. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Thursday, 4 October, 2012 3:07 PM >> To: Carson Holt >> Cc: Maker Mailing List >> Subject: Re: [maker-devel] Expected format for EST GFF3 >> >> Great. I am using PE Illumina reads mapped by Tophat and assembled by >> Cufflinks. The GTF file Cufflinks produces only *transcript* and *exon*features. So I'm assuming I can simply convert the >> *transcript* features to *match* and the *exon* features to *match_part *and >> make sure the parent/child relationships are maintained with the *Parent* and >> *ID* attributes? >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt wrote: >> >>> Match/match_part features are expected. MAKER expects these to be >>> polished (I.e. correctly aligned around splice sites). For example >>> exonerate, BLAT, and cufflinks results will be correct around splice sites >>> and can be used; BLAST results on the other hand will not be correct and >>> should not be used. >>> >>> The Gap attribute is used by maker if available, but is not required >>> (Gap describes how to reconstruct an alignment for gaps and mismatches). >>> Otherwise MAKER assumes all positions are matches to the reference. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Thursday, 4 October, 2012 2:52 PM >>> To: Maker Mailing List >>> Subject: [maker-devel] Expected format for EST GFF3 >>> >>> Greetings. >>> >>> I would like to use Maker's *est_gff* option to include EST evidence >>> from an external GFF3 file. In addition to being a valid, well-formed GFF3 >>> file, what expectations does Maker assume about the contents of this file? >>> Which feature types are expected and/or supported? Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> _______________________________________________ maker-devel mailing >>> list maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeremy.semeiks at utsw.edu Thu Oct 4 13:39:25 2012 From: jeremy.semeiks at utsw.edu (Jeremy Semeiks) Date: Thu, 4 Oct 2012 14:39:25 -0500 Subject: [maker-devel] maker hung on a contig Message-ID: Hi, I ran maker 2.26 beta on a mammalian assembly containing ~200,000 scaffolds. I ran like this: $ /usr/bin/time mpiexec -n 30 maker -q < /dev/null > maker.oe 2>&1 & disown -h After 1--2 weeks, all but one contig had finished successfully, although maker needed a retry on ~80 others. The one contig that failed is small (1,146 nt) and not obviously weird. maker hung on this one contig, outputting the following type of stack-trace-looking thing to maker.oe, over and over: """ eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xdd9d480)', 'Error::Simple=HASH(0xa9a56f8)', undef, 'ARRAY(0x9ecc710)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xf1b6a40)', 'HASH(0xdd9d480)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0xab33f30)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0xab33f30)', 'SCALAR(0x17cd3a90)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xb3a58e0)', 'Error::Simple=HASH(0xab33f30)', undef, 'ARRAY(0x77ffae0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xbe47098)', 'HASH(0xb3a58e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0xc86fb88)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0xc86fb88)', 'SCALAR(0xb504490)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0xc2d7d00)', 'Error::Simple=HASH(0xc86fb88)', undef, 'ARRAY(0x767a290)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0x5ab13f0)', 'HASH(0xc2d7d00)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 815 Process::MpiTiers::_handler('Process::MpiTiers=HASH(0xc10a7e0)', 'Error::Simple=HASH(0x18943010)', 'Can not get next level') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 336 Process::MpiTiers::__ANON__('Error::Simple=HASH(0x18943010)', 'SCALAR(0xa6e7cd0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 """ Every 100,000 lines or so, that sort of pattern is interrupted by this sort of pattern: """ Process::MpiTiers::__ANON__('Error::Simple=HASH(0xd9816c8)', 'SCALAR(0x2558c48)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 339 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 329 Error::subs::run_clauses('HASH(0x1bd77ab8)', 'Error::Simple=HASH(0xd9816c8)', undef, 'ARRAY(0x2561948)') called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 426 Error::subs::try('CODE(0xf48b0d8)', 'HASH(0x1bd77ab8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 337 Process::MpiTiers::_next_level('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 179 Process::MpiTiers::next_chunk('Process::MpiTiers=HASH(0xc10a7e0)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiTiers.pm line 285 Process::MpiTiers::run_all('Process::MpiTiers=HASH(0xc10a7e0)', 0) calledERROR: Failed while builing masking tiers ERROR: Can not get next level ERROR: Can't open seq file: /export/home9/jrs/bmy/maker02/bmy.min1e3.maker.output/bmy.min1e3_datastore/E0/1F/C16738816//theVoid.C16738816/query.masked.gff.seq No such file or directory at /home/jrs/maker-2.26-beta/bin/../lib/Dumper/GFF/GFFV3.pm line 173. Dumper::GFF::GFFV3::finalize('Dumper::GFF::GFFV3=HASH(0x4c328c8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 700 Process::MpiChunk::__ANON__() called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 415 eval {...} called at /home/jrs/maker-2.26-beta/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0xfc1d208)', 'HASH(0xfc1fbd8)') called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 3768 Process::MpiChunk::_go('Process::MpiChunk=HASH(0x47162a0)', 'flow', 'HASH(0x93ba680)', 2, 0) called at /home/jrs/maker-2.26-beta/bin/../lib/Process/MpiChunk.pm line 369 """ The makers kept outputting these things until I killed them all, and also mpiexec, with kill -s SIGKILL. These strack traces go on for at least the last million lines of maker.oe, which is ~100 GB. Because the log is so big, without knowing what to grep for it's hard for me to suggest what originally went wrong. The contig's datastore directory does not contain anything obvious. Its run.log indicates only that it was the second try for this contig. None of the files in this directory (including theVoid) had been updated since 3 days ago. So this contig itself may even be a red herring. maker's protein output for all the other contigs seems sane, and this one contig is probably too small to contain any proteins, so it doesn't matter to me if maker finishes it successfully. I just want to figure out how I can keep maker from hanging. Suggestions? Thanks, Jeremy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 14:42:52 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 20:42:52 +0000 Subject: [maker-devel] maker mpi failed Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D692@EXMBX05.austin.utexas.edu> Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 14:53:08 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 16:53:08 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D692@EXMBX05.austin.utexas.edu> Message-ID: You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Thu Oct 4 15:20:48 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Thu, 4 Oct 2012 21:20:48 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D7C8@EXMBX05.austin.utexas.edu> Hi Carson, I didn't know maker does not work with MVAPICH2. The cluster I was installing maker on only has mvapich2. So I guess I will have to install mpich2 under my directory. However, I installed perl threading based on mvapich2. Does that mean I need to reinstall perl with mpich2 before reinstalling maker? Oscar -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Thu Oct 4 17:06:13 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Thu, 4 Oct 2012 16:06:13 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: References: Message-ID: <1764.131.215.15.234.1349391973.squirrel@webmail.caltech.edu> Hi, Thanks, reinstalling BioPerl fixed the issue. Now we are training our genome to generate training dataset for SNAP and Augustus, as per the instructions provided at http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors However the training dataset would have multiple gff3 prediction files, one for each contig. Is it recommended to cat(command) all the gff3 files to generate a single file and finally generate hmm file with the steps mentioned in the tutorial. Would the same hmm file work as training dataset for AUGUSTUS? Many thanks, Parul Kudtarkar > You probably need to reinstall Bio-perl. > > Other things that can cause the same error are setting TMP in the > maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or > sometimes setting it to an NFS mount. > A full drive can cause this as well or a broken Berkley DB. Use df -h to > see if any of the drives used are full (either current working directory > or TMP location). You can also swap Berkley DB for a different backend by > setting AnyDBM_ISA during setup. > > Example: > cd ../maker/src/ > perl Build.PL --AnyDBM_ISA SDBM_File > ./Build install > > --Carson > > > > On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: > >>I am running maker on example data that comes along with installation and >>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >> >>Please advice on the aforementioned error. >> >>--------------------- >>Maker is now finished!!! >> >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >>eins%2Efasta.repeatrunner >>deleted:0 hits >> in cluster:shadow cluster... >> i_size:0 j_size:0 >> sorting hits in shadow cluster... >>... finished. >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >>%2Efasta.blastn >>deleted:-1 hits >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >>tein%2Efasta.blastx >>WARNING: Multiple MAKER processes have been started in the >>same directory. >> >>deleted:0 hits >>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>stop here:dpp-mRNA-4 >>ERROR: Fasta index error >> >>FATAL ERROR >>Maker is now finished!!! >> >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_prot >>eins%2Efasta.repeatrunner >>deleted:0 hits >> in cluster:shadow cluster... >> i_size:0 j_size:0 >> sorting hits in shadow cluster... >>... finished. >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_est >>%2Efasta.blastn >>deleted:-1 hits >>re reading blast report. >>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F/c >>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_pro >>tein%2Efasta.blastx >>WARNING: Multiple MAKER processes have been started in the >>same directory. >> >>deleted:0 hits >>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>stop here:dpp-mRNA-4 >>ERROR: Fasta index error >> >>FATAL ERROR >>ERROR: Failed while polishig ESTs!! >> >>ERROR: Chunk failed at level 14 >>!! >>FAILED CONTIG:contig-dpp-500-500 >> >> >> >>ERROR: Chunk failed at level 14 >>!! >>FAILED CONTIG:contig-dpp-500-500 >>------------------------------------ >> >> >>Many thanks, >>Parul Kudtarkar >> >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org >> >> >>_______________________________________________ >>maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From david.powell at monash.edu Thu Oct 4 17:44:32 2012 From: david.powell at monash.edu (David Powell) Date: Fri, 5 Oct 2012 09:44:32 +1000 Subject: [maker-devel] problem running maker In-Reply-To: References: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D336@EXMBX05.austin.utexas.edu> Message-ID: I've also had the problem with a segfault when running MAKER 2.26. I'm not sure it is the one experienced here, but in the end I was able to work around it. I tracked the problem to lib/FastA.pm in the _safe_new function. Specifically, in the local handler installed for warnings. By removing the "die" call, I was able to stop the perl process from segfaulting. I suspect this is not a great fix since it will ignore any warnings from Bio::DB::Fasta, but it worked for me (I did check the warnings generated were not important). Cheers, -- David Powell On 5 October 2012 03:17, Carson Holt wrote: > Seg faults really only happen from C and not Perl, so there may be a perl > module that is broken on your machine that use C internally. > > Here are some modules that MAKER can use that I know use C and that you > might want to reinstall from CPAN. > > DB_File > Forks > Proc::ProcessTable > Inline::C > > Also during the maker install say no to the "use MPI" question. It's best > to make the installation as simple as possible before getting into MPI > configuration. > > Also after installing those and rerunning maker's 'perl Build.PL' and > './Build install' steps do your first run with 'maker --debug' (redirect > STDERR to a file). Then if it fails, you can send me the error log. > > Thanks, > Carson > > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 11:36 AM > > To: Carson Holt , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Ok. So I installed 2.26 instead. When I ran maker again I got segmentation > fault with no additional message. I am using dpp_contig.fasta, > dpp_est.fasta and dpp_protein.fasta. > > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > From: Carson Holt > Date: Thursday, October 4, 2012 9:03 AM > To: Oscar Jiao , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Try 2.26 and let me know. > > Thanks, > Carson > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 10:02 AM > To: Carson Holt , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > 2.10 > -- > Dian (Oscar) Jiao, Ph.D., > Research Associate, Life Sciences Computing Group > Texas Advanced Computing Center > University of Texas at Austin > jiao at tacc.utexas.edu | (832) 303-1166 > > From: Carson Holt > Date: Thursday, October 4, 2012 7:27 AM > To: Oscar Jiao , "maker-devel at yandell-lab.org" < > maker-devel at yandell-lab.org> > Subject: Re: [maker-devel] problem running maker > > Are you trying to install 2.26 or 2.10? > > --Carson > > > From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 12:04 AM > To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] problem running maker > > Hi, > > I just installed maker and tried to run it with these dpp example fasta > files that came with the package. However when I do "maker maker_opts.ctl > maker_bopts.ctl maker_exe.ctl", I got the following errors. It looks like > there are three errors: (1) defined array deprecated; (2) unknown state in > Signal.pm and (3) GDBM_File (etc.) missing > > The first line seems to be just warning. I got it even when I do just > "maker" or "maker ctl". The control files did get generated. And what is > the relationship between the three perl modules, GDBM/NDBM/SDBM_File, > AnyDBM and DB_File? > > Oscar > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 20:24:16 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 22:24:16 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868D7C8@EXMBX05.austin.utexas.edu> Message-ID: I don't think you will need to reinstall perl (you may but I don't think so). MVAPICH2 doesn't work because they have overridden malloc with their own version which causes weirdness with shared libraries in Perl. I've submitted some bug reports to the MVAPICH2 developers and made modifications to the development version of MAKER, and had some progress, but I am still not getting reliable behavior in MVAPICH2 when compiling for the OFA-IB-Nemesis or OFA-IB-CH3 interfaces. So for now using MPICH2 is your best choice. You lose some of the infiniband optimizations but IO rather than messages passing is MAKER's true bottleneck (so you won't get a performance hit by using MPICH2). --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 5:20 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Hi Carson, I didn't know maker does not work with MVAPICH2. The cluster I was installing maker on only has mvapich2. So I guess I will have to install mpich2 under my directory. However, I installed perl threading based on mvapich2. Does that mean I need to reinstall perl with mpich2 before reinstalling maker? Oscar -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 4 20:51:19 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 04 Oct 2012 22:51:19 -0400 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <1764.131.215.15.234.1349391973.squirrel@webmail.caltech.edu> Message-ID: Just concatenating won't work. Use the gff3_merge tool to safely merge separate GFF3 files. In the tutorial dataset their should only be one contig to train with, but with real data you will have multiple contigs and will want to merge the GFF3 files. Augustus has a separate training procedure that is a little more complicated than SNAP's. You will need to read their documentation on training which is a little cryptic. I know Jason Stajich developed a tool for training augustus from snap results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be willing to share it with you ;-) Thanks, Carson On 12-10-04 7:06 PM, "Parul Kudtarkar" wrote: >Hi, > >Thanks, reinstalling BioPerl fixed the issue. Now we are training our >genome to generate training dataset for SNAP and Augustus, as per the >instructions provided at >http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors > >However the training dataset would have multiple gff3 prediction files, >one for each contig. Is it recommended to cat(command) all the gff3 files >to generate a single file and finally generate hmm file with the steps >mentioned in the tutorial. Would the same hmm file work as training >dataset for AUGUSTUS? > >Many thanks, >Parul Kudtarkar > >> You probably need to reinstall Bio-perl. >> >> Other things that can cause the same error are setting TMP in the >> maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or >> sometimes setting it to an NFS mount. >> A full drive can cause this as well or a broken Berkley DB. Use df -h >>to >> see if any of the drives used are full (either current working directory >> or TMP location). You can also swap Berkley DB for a different backend >>by >> setting AnyDBM_ISA during setup. >> >> Example: >> cd ../maker/src/ >> perl Build.PL --AnyDBM_ISA SDBM_File >> ./Build install >> >> --Carson >> >> >> >> On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >> >>>I am running maker on example data that comes along with installation >>>and >>>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >>> >>>Please advice on the aforementioned error. >>> >>>--------------------- >>>Maker is now finished!!! >>> >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>ot >>>eins%2Efasta.repeatrunner >>>deleted:0 hits >>> in cluster:shadow cluster... >>> i_size:0 j_size:0 >>> sorting hits in shadow cluster... >>>... finished. >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>st >>>%2Efasta.blastn >>>deleted:-1 hits >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>ro >>>tein%2Efasta.blastx >>>WARNING: Multiple MAKER processes have been started in the >>>same directory. >>> >>>deleted:0 hits >>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>stop here:dpp-mRNA-4 >>>ERROR: Fasta index error >>> >>>FATAL ERROR >>>Maker is now finished!!! >>> >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>ot >>>eins%2Efasta.repeatrunner >>>deleted:0 hits >>> in cluster:shadow cluster... >>> i_size:0 j_size:0 >>> sorting hits in shadow cluster... >>>... finished. >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>st >>>%2Efasta.blastn >>>deleted:-1 hits >>>re reading blast report. >>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>/c >>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>ro >>>tein%2Efasta.blastx >>>WARNING: Multiple MAKER processes have been started in the >>>same directory. >>> >>>deleted:0 hits >>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>stop here:dpp-mRNA-4 >>>ERROR: Fasta index error >>> >>>FATAL ERROR >>>ERROR: Failed while polishig ESTs!! >>> >>>ERROR: Chunk failed at level 14 >>>!! >>>FAILED CONTIG:contig-dpp-500-500 >>> >>> >>> >>>ERROR: Chunk failed at level 14 >>>!! >>>FAILED CONTIG:contig-dpp-500-500 >>>------------------------------------ >>> >>> >>>Many thanks, >>>Parul Kudtarkar >>> >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org >>> >>> >>>_______________________________________________ >>>maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > From jiao at tacc.utexas.edu Thu Oct 4 23:06:48 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 05:06:48 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868DBE8@EXMBX05.austin.utexas.edu> I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Thu Oct 4 23:34:37 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Thu, 4 Oct 2012 22:34:37 -0700 (PDT) Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: References: Message-ID: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> Carson, thanks for very quick response. Jason, could you please elaborate/share document if any on using zff2augustus_gbk.pl Thanks, Parul Kudtarkar > Just concatenating won't work. Use the gff3_merge tool to safely merge > separate GFF3 files. > > In the tutorial dataset their should only be one contig to train with, but > with real data you will have multiple contigs and will want to merge the > GFF3 files. > > Augustus has a separate training procedure that is a little more > complicated than SNAP's. You will need to read their documentation on > training which is a little cryptic. > > I know Jason Stajich developed a tool for training augustus from snap > results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be > willing to share it with you ;-) > > Thanks, > Carson > > > > On 12-10-04 7:06 PM, "Parul Kudtarkar" wrote: > >>Hi, >> >>Thanks, reinstalling BioPerl fixed the issue. Now we are training our >>genome to generate training dataset for SNAP and Augustus, as per the >>instructions provided at >>http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors >> >>However the training dataset would have multiple gff3 prediction files, >>one for each contig. Is it recommended to cat(command) all the gff3 files >>to generate a single file and finally generate hmm file with the steps >>mentioned in the tutorial. Would the same hmm file work as training >>dataset for AUGUSTUS? >> >>Many thanks, >>Parul Kudtarkar >> >>> You probably need to reinstall Bio-perl. >>> >>> Other things that can cause the same error are setting TMP in the >>> maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) >>> or >>> sometimes setting it to an NFS mount. >>> A full drive can cause this as well or a broken Berkley DB. Use df -h >>>to >>> see if any of the drives used are full (either current working >>> directory >>> or TMP location). You can also swap Berkley DB for a different backend >>>by >>> setting AnyDBM_ISA during setup. >>> >>> Example: >>> cd ../maker/src/ >>> perl Build.PL --AnyDBM_ISA SDBM_File >>> ./Build install >>> >>> --Carson >>> >>> >>> >>> On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: >>> >>>>I am running maker on example data that comes along with installation >>>>and >>>>is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note >>>> >>>>Please advice on the aforementioned error. >>>> >>>>--------------------- >>>>Maker is now finished!!! >>>> >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>>ot >>>>eins%2Efasta.repeatrunner >>>>deleted:0 hits >>>> in cluster:shadow cluster... >>>> i_size:0 j_size:0 >>>> sorting hits in shadow cluster... >>>>... finished. >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>>st >>>>%2Efasta.blastn >>>>deleted:-1 hits >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>>ro >>>>tein%2Efasta.blastx >>>>WARNING: Multiple MAKER processes have been started in the >>>>same directory. >>>> >>>>deleted:0 hits >>>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>>stop here:dpp-mRNA-4 >>>>ERROR: Fasta index error >>>> >>>>FATAL ERROR >>>>Maker is now finished!!! >>>> >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr >>>>ot >>>>eins%2Efasta.repeatrunner >>>>deleted:0 hits >>>> in cluster:shadow cluster... >>>> i_size:0 j_size:0 >>>> sorting hits in shadow cluster... >>>>... finished. >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e >>>>st >>>>%2Efasta.blastn >>>>deleted:-1 hits >>>>re reading blast report. >>>>/usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F >>>>/c >>>>ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p >>>>ro >>>>tein%2Efasta.blastx >>>>WARNING: Multiple MAKER processes have been started in the >>>>same directory. >>>> >>>>deleted:0 hits >>>>WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. >>>>stop here:dpp-mRNA-4 >>>>ERROR: Fasta index error >>>> >>>>FATAL ERROR >>>>ERROR: Failed while polishig ESTs!! >>>> >>>>ERROR: Chunk failed at level 14 >>>>!! >>>>FAILED CONTIG:contig-dpp-500-500 >>>> >>>> >>>> >>>>ERROR: Chunk failed at level 14 >>>>!! >>>>FAILED CONTIG:contig-dpp-500-500 >>>>------------------------------------ >>>> >>>> >>>>Many thanks, >>>>Parul Kudtarkar >>>> >>>>-- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org >>>> >>>> >>>>_______________________________________________ >>>>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> >> >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org >> > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jason.stajich at ucr.edu Fri Oct 5 01:02:50 2012 From: jason.stajich at ucr.edu (Jason E Stajich) Date: Fri, 5 Oct 2012 07:02:50 +0000 Subject: [maker-devel] Failed while polishig ESTs In-Reply-To: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> References: <54438.108.85.195.190.1349415277.squirrel@webmail.caltech.edu> Message-ID: <41A4465BD48EEE47A449FA03108C852A073651E2@EXCH-MBOX-3.exch.ucr.edu> I wrote up how to use it on this list msg - http://brie4.cshl.edu/pipermail/gmod-help/2012-June/001724.html Script is in this github repo - https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl Patches and documentation updates are always welcomed, I'll get around to that eventually. Jason On Oct 4, 2012, at 10:34 PM, Parul Kudtarkar > wrote: Carson, thanks for very quick response. Jason, could you please elaborate/share document if any on using zff2augustus_gbk.pl Thanks, Parul Kudtarkar Just concatenating won't work. Use the gff3_merge tool to safely merge separate GFF3 files. In the tutorial dataset their should only be one contig to train with, but with real data you will have multiple contigs and will want to merge the GFF3 files. Augustus has a separate training procedure that is a little more complicated than SNAP's. You will need to read their documentation on training which is a little cryptic. I know Jason Stajich developed a tool for training augustus from snap results called zff2augustus_gbk.pl (I've CC'd him). Perhaps he would be willing to share it with you ;-) Thanks, Carson On 12-10-04 7:06 PM, "Parul Kudtarkar" > wrote: Hi, Thanks, reinstalling BioPerl fixed the issue. Now we are training our genome to generate training dataset for SNAP and Augustus, as per the instructions provided at http://gmod.org/wiki/MAKER_Tutorial#Training_ab_initio_Gene_Predictors However the training dataset would have multiple gff3 prediction files, one for each contig. Is it recommended to cat(command) all the gff3 files to generate a single file and finally generate hmm file with the steps mentioned in the tutorial. Would the same hmm file work as training dataset for AUGUSTUS? Many thanks, Parul Kudtarkar You probably need to reinstall Bio-perl. Other things that can cause the same error are setting TMP in the maker_opts.ctl file to a tmpfs type filesystem (i.e. in memory drive) or sometimes setting it to an NFS mount. A full drive can cause this as well or a broken Berkley DB. Use df -h to see if any of the drives used are full (either current working directory or TMP location). You can also swap Berkley DB for a different backend by setting AnyDBM_ISA during setup. Example: cd ../maker/src/ perl Build.PL --AnyDBM_ISA SDBM_File ./Build install --Carson On 12-10-03 10:14 PM, "Parul Kudtarkar" wrote: I am running maker on example data that comes along with installation and is cited at http://gmod.org/wiki/MAKER_Tutorial_2012#Note Please advice on the aforementioned error. --------------------- Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr ot eins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e st %2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p ro tein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR Maker is now finished!!! re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.te_pr ot eins%2Efasta.repeatrunner deleted:0 hits in cluster:shadow cluster... i_size:0 j_size:0 sorting hits in shadow cluster... ... finished. re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_e st %2Efasta.blastn deleted:-1 hits re reading blast report. /usr/local/maker/test/dpp_contig.maker.output/dpp_contig_datastore/05/1F /c ontig-dpp-500-500//theVoid.contig-dpp-500-500/contig-dpp-500-500.0.dpp_p ro tein%2Efasta.blastx WARNING: Multiple MAKER processes have been started in the same directory. deleted:0 hits WARNING: Cannot find> dpp-mRNA-4, trying to re-index the fasta. stop here:dpp-mRNA-4 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level 14 !! FAILED CONTIG:contig-dpp-500-500 ------------------------------------ Many thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -- Jason E Stajich, PhD Assistant Professor Plant Pathology & Microbiology University of California, Riverside 951.827.2363 http://lab.stajich.org http://fungalgenomes.org http://fungidb.org http://1000.fungalgenomes.org/ twitter @stajichlab @hyphaltip @fungalgenomes @fungidb http://plantpathology.ucr.edu http://genomics.ucr.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From Carson.Holt at oicr.on.ca Thu Oct 4 13:19:57 2012 From: Carson.Holt at oicr.on.ca (Carson Holt) Date: Thu, 4 Oct 2012 19:19:57 +0000 Subject: [maker-devel] Expected format for EST GFF3 In-Reply-To: Message-ID: Yes. You really have to because the single exon transcripts produced from mRNA-seq assembly are strandless (you don't know where they belong). They also tend to be heavily weighted to pseudogenes and transposons. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 3:16 PM To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 Does the cufflinks2gff3 script filter out single-exon transcripts? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 3:09 PM, Carson Holt > wrote: Use the cufflinks2gff3 script that comes with MAKER. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 3:07 PM To: Carson Holt > Cc: Maker Mailing List > Subject: Re: [maker-devel] Expected format for EST GFF3 Great. I am using PE Illumina reads mapped by Tophat and assembled by Cufflinks. The GTF file Cufflinks produces only transcript and exon features. So I'm assuming I can simply convert the transcript features to match and the exon features to match_part and make sure the parent/child relationships are maintained with the Parent and ID attributes? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 4, 2012 at 2:58 PM, Carson Holt > wrote: Match/match_part features are expected. MAKER expects these to be polished (I.e. correctly aligned around splice sites). For example exonerate, BLAT, and cufflinks results will be correct around splice sites and can be used; BLAST results on the other hand will not be correct and should not be used. The Gap attribute is used by maker if available, but is not required (Gap describes how to reconstruct an alignment for gaps and mismatches). Otherwise MAKER assumes all positions are matches to the reference. Thanks, Carson From: Daniel Standage > Date: Thursday, 4 October, 2012 2:52 PM To: Maker Mailing List > Subject: [maker-devel] Expected format for EST GFF3 Greetings. I would like to use Maker's est_gff option to include EST evidence from an external GFF3 file. In addition to being a valid, well-formed GFF3 file, what expectations does Maker assume about the contents of this file? Which feature types are expected and/or supported? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 10:05:56 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 12:05:56 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868DBE8@EXMBX05.austin.utexas.edu> Message-ID: Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 10:34:12 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 12:34:12 -0400 Subject: [maker-devel] editing MAKER mailing list options Message-ID: Just a small note to subscribers to the MAKER e-mail list. Being as there are spurts of high volume list activity that may be a nuisance to some, if you want to edit your mailing list options visit this link --> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org You can then change to digest mode if you don't want to receive every single message posted to the list, or you can set your options to be subscribed to but not receive messages from the list (so you can post and get help but won't get everyone else's posts). Enter your e-mail address in the box at the very very bottom of the page to edit option. Thanks, Carson -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 12:19:15 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 14:19:15 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E519@EXMBX05.austin.utexas.edu> Message-ID: Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 12:16:41 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 18:16:41 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E519@EXMBX05.austin.utexas.edu> See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: maker_opts.ctl Type: application/octet-stream Size: 4468 bytes Desc: maker_opts.ctl URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: mpich-maker-debug.log Type: application/octet-stream Size: 315735 bytes Desc: mpich-maker-debug.log URL: From jiao at tacc.utexas.edu Fri Oct 5 13:55:39 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 19:55:39 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E7A3@EXMBX05.austin.utexas.edu> I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 14:03:09 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 16:03:09 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E7A3@EXMBX05.austin.utexas.edu> Message-ID: What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Ap plication/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.p m line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/M PI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 14:06:02 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 20:06:02 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E866@EXMBX05.austin.utexas.edu> Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 5 14:07:18 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 05 Oct 2012 16:07:18 -0400 Subject: [maker-devel] maker mpi failed In-Reply-To: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E866@EXMBX05.austin.utexas.edu> Message-ID: Is /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2 a new install of mpich2 or is it the install you did with 2.26 just copied over? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 4:06 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Ap plication/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.p m line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/M PI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao , "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/d pp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From jiao at tacc.utexas.edu Fri Oct 5 14:08:35 2012 From: jiao at tacc.utexas.edu (Dian "Oscar" Jiao) Date: Fri, 5 Oct 2012 20:08:35 +0000 Subject: [maker-devel] maker mpi failed In-Reply-To: Message-ID: <1B4E4BB25B711A4FBC93E4B3AFEB2A321868E8D6@EXMBX05.austin.utexas.edu> It is a new install. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:07 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Is /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2 a new install of mpich2 or is it the install you did with 2.26 just copied over? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 4:06 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Just default, /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/bin/mpicc and /work/02045/jiao/modules/Maker-GMOD/maker-dev/exe/mpich2/include -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 3:03 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed What were you answers to the mpicc and mpi.h questions during setup? --Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 3:55 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I got problem installing the dev version Maker. Here is what I did: ./Build mpich2 perl Build.PL (yes to MPI) ./Build clean ./Build install The install fails while configuring Maker with MPI support: Had problems bootstrapping Inline module 'Parallel::Application::MPI' Can't load '/work/02045/jiao/modules/Maker-GMOD/maker-dev/src/blib/lib/auto/Parallel/Application/MPI/MPI.so' for module Parallel::Application::MPI: libifport.so.5: cannot open shared object file: No such file or directory at /work/02045/jiao/localperl/lib/5.16.1/x86_64-linux-thread-multi/DynaLoader.pm line 190. at /work/02045/jiao/localperl/add-on/lib/site_perl/5.16.1/Inline.pm line 536. at /work/02045/jiao/modules/Maker-GMOD/maker-dev/src/lib/Parallel/Application/MPI.pm line 223. ** If you are running using OpenMPI, you may have to preload object files ** for shared libraries to work. For bash, try executing a command ** similar to the following with the appropriate file location. ** Example --> export LD_PRELOAD=.../openmpi/lib/libmpi.so ** Please do this before trying to run MAKER again!! Something wrong with my DynaLoader/Inline modules? Version 2.26 installed ok. What is the difference between 2.26 and the development version? -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Friday, October 5, 2012 1:19 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Could you try the development version anyways with MPI using the same procedure and send me the results as you did with 2.26? Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 2:16 PM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed See the attached files. The command I used "/work/02045/jiao/modules/Maker-GMOD/maker2.26/exe/mpich2/bin/mpiexec -n 12 /work/02045/jiao/modules/Maker-GMOD/maker2.26/bin/maker -debug maker_exe.ctl maker_opts.ctl maker_bopts.ctl 2> mpich-maker-debug.log" I am still using Maker 2.26. These problems I had earlier with AnyDBM_File were resolved (manually), so I did not install the development version. ~Oscar From: Carson Holt > Date: Friday, October 5, 2012 11:05 AM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed Are you using the subversion version of MAKER I sent? If so send me the following. Run under MPI with the --debug flag set on maker. When running, supply the full path to both the maker and mpiexec executables. Capture the STDERR to a file. Send me that STDERR file together with the maker_opts.ctl file you are using. Thanks, Carson From: "Dian \"Oscar\" Jiao" > Date: Friday, 5 October, 2012 1:06 AM To: Carson Holt >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed I followed your instructions to compile Maker with MPICH2 (the one comes with Maker). But when I run maker with mpi, it terminates with the message: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... =========================================================================== ========== = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES = EXIT CODE: 11 = CLEANING UP REMAINING PROCESSES = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES =========================================================================== ========== APPLICATION TERMINATED WITH THE EXIT STRING: Segmentation fault (signal 11) The serial version of Maker still works. I tried to install MPICH2 manually outside maker and got the same error. -- Dian (Oscar) Jiao, Ph.D., Research Associate, Life Sciences Computing Group Texas Advanced Computing Center University of Texas at Austin jiao at tacc.utexas.edu | (832) 303-1166 From: Carson Holt > Date: Thursday, October 4, 2012 3:53 PM To: Oscar Jiao >, "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] maker mpi failed You either tried to run with the -dsindex option before the log file existed or you tried to launch via MPI when not compiled for MPI or not compiled correctly. It's as if you called MAKER serially multiple times in a row which you can do, but if you start too many before one of them gets a chance to get the log file started a race condition exists. Which MPI flavor are you using? MVAPICH2 won't work with MAKER, MPICH2 will, and OpenMPI may or may not (non-robust shared library behavior). You may have to reinstall MPICH2. MAKER can try and do this for you using necessary flags to make it easier To do this --> cd ./maker/src ./Build mpich2 Then reinstall maker cd ./maker/src perl Build.PL ./Build clean ./Build install Then use this executable to launch --> ./maker/exe/mpich2/bin/mpiexec Note you cannot use other launchers. You have to use the one you compiled with. --Carson From: "Dian \"Oscar\" Jiao" > Date: Thursday, 4 October, 2012 4:42 PM To: "maker-devel at yandell-lab.org" > Subject: [maker-devel] maker mpi failed Hi, I was trying to run maker 2.26. It was compiled with MPI. The non-mpi executable works just fine. But I got the error below while running mpirun ?n maker ?? What is this Iterator::Fasta::skip_file? Any idea what is going on here? Oscar STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_datastore To access files for individual sequences use the datastore index: /work/02045/jiao/modules/Maker-GMOD/maker2.26/data/dpp_contig.maker.output/dpp_contig_master_datastore_index.log ERROR: Log file does not exist in Iterator::Fasta::skip_file MPI process (rank: 8) terminated unexpectedly on c341-213.ls4.tacc.utexas.edu Exit code -5 signaled from c341-213 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From chrisi.hahni at gmail.com Tue Oct 9 05:07:14 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 09 Oct 2012 13:07:14 +0200 Subject: [maker-devel] install maker on cluster Message-ID: <50740562.9010904@gmail.com> Hello maker-team, I am trying to install maker 2.25 on a cluster, without root privileges (I managed before, but now after migration to a new cluster I cant seem to get it to work). All necessary prerequisite programs are installed and in the path (I followed the steps in the INSTALL file that comes with maker2.25). When I do: perl Build.PL, I get: Checking prerequisites... requires: ! DBD::SQLite is not installed ! IO::All is not installed ! Inline::C is not installed ! Perl::Unsafe::Signals is not installed ! Proc::ProcessTable is not installed ! Want is not installed ! forks is not installed ! forks::shared is not installed build_requires: ! LWP::Simple is not installed recommends: * DBD::Pg is not installed ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the versions of the modules indicated above before proceeding with this installation Run 'Build installdeps' to install missing prerequisites. MAKER supports distributed parallelization via MPI. Would you like to configure MAKER for MPI (This requires that you have an MPI client installed)? [N ]n WARNING: Apache cannot be located. The optional web based interface to MAKER will not be available to you. Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'MAKER' version '2.25' The file 'Build' has been created for you to finish installing MAKER. ============================================================================== STATUS MAKER 2.25 ============================================================================== PERL Dependencies: MISSING ! forks ! IO::All ! forks::shared ! Want ! DBD::SQLite ! Proc::ProcessTable ! Inline::C ! Perl::Unsafe::Signals External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: DISABLED MWAS Web Interface: DISABLED MAKER PACKAGE: MISSING PREREQUISITES Important Commands: ./Build installdeps #installs missing PERL dependencies ./Build installexes #installs all missing external programs ./Build install #installs MAKER ./Build status #Shows this status menu Other Commands: ./Build repeatmasker #installs RepeatMasker (asks for RepBase) ./Build blast #installs BLAST (NCBI BLAST+) ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) ./Build snap #installs SNAP ./Build augustus #installs Augustus ./Build apollo #installs Apollo ./Build gbrowse #installs GBrowse (must be root) ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) ./Build mpich2 #installs MPICH2 (but manual install recommended) So it seems some Perl Dependencies are missing. As indicated in INSTALL I followed the quick and dirty installation for Bioperl, so I am not sure what I did wrong/missed out. When I run ./Build installdeps, I get: You do not have write access to install missing Modules. I can try and install these locally (i.e. only for MAKER) in the .../maker/perl/lib directory, or you can run './Build installdeps' as root or using sudo and try again. Do want MAKER to try and build a local installation? [N ]y mkdir /usit/titan: Permission denied at /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm line 59 Sorry to bother you with this minor things!! Any help would me much appreciated! much obliged, Christoph From carsonhh at gmail.com Tue Oct 9 10:27:36 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 09 Oct 2012 12:27:36 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <50740562.9010904@gmail.com> Message-ID: You may need to reconfigure your cpan preferences. You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, or by starting cpan from your command line (command is cpan). Then run 'o conf init' inside the cpan ui. --Carson On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >Hello maker-team, > >I am trying to install maker 2.25 on a cluster, without root privileges >(I managed before, but now after migration to a new cluster I cant seem >to get it to work). All necessary prerequisite programs are installed >and in the path (I followed the steps in the INSTALL file that comes >with maker2.25). When I do: perl Build.PL, I get: >Checking prerequisites... > requires: > ! DBD::SQLite is not installed > ! IO::All is not installed > ! Inline::C is not installed > ! Perl::Unsafe::Signals is not installed > ! Proc::ProcessTable is not installed > ! Want is not installed > ! forks is not installed > ! forks::shared is not installed > build_requires: > ! LWP::Simple is not installed > recommends: > * DBD::Pg is not installed > >ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >versions >of the modules indicated above before proceeding with this installation > >Run 'Build installdeps' to install missing prerequisites. > >MAKER supports distributed parallelization via MPI. >Would you like to configure MAKER for MPI (This >requires that you have an MPI client installed)? [N ]n > >WARNING: Apache cannot be located. The optional web based >interface to MAKER will not be available to you. > >Created MYMETA.yml and MYMETA.json >Creating new 'Build' script for 'MAKER' version '2.25' > > >The file 'Build' has been created for you to finish installing MAKER. > > >========================================================================== >==== >STATUS MAKER 2.25 >========================================================================== >==== >PERL Dependencies: MISSING > ! forks > ! IO::All > ! forks::shared > ! Want > ! DBD::SQLite > ! Proc::ProcessTable > ! Inline::C > ! Perl::Unsafe::Signals > >External Programs: VERIFIED >External C Libraries: VERIFIED >MPI SUPPORT: DISABLED >MWAS Web Interface: DISABLED >MAKER PACKAGE: MISSING PREREQUISITES > > >Important Commands: > ./Build installdeps #installs missing PERL dependencies > ./Build installexes #installs all missing external programs > ./Build install #installs MAKER > ./Build status #Shows this status menu > >Other Commands: > ./Build repeatmasker #installs RepeatMasker (asks for RepBase) > ./Build blast #installs BLAST (NCBI BLAST+) > ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on >Mac OSX) > ./Build snap #installs SNAP > ./Build augustus #installs Augustus > ./Build apollo #installs Apollo > ./Build gbrowse #installs GBrowse (must be root) > ./Build jbrowse #installs JBrowse (MAKER copy, not web >accecible) > ./Build mpich2 #installs MPICH2 (but manual install >recommended) > >So it seems some Perl Dependencies are missing. As indicated in INSTALL >I followed the quick and dirty installation for Bioperl, so I am not >sure what I did wrong/missed out. > >When I run ./Build installdeps, I get: >You do not have write access to install missing Modules. >I can try and install these locally (i.e. only for MAKER) >in the .../maker/perl/lib directory, or you can run >'./Build installdeps' as root or using sudo and try again. >Do want MAKER to try and build a local installation? [N ]y >mkdir /usit/titan: Permission denied at >/cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm line >59 > >Sorry to bother you with this minor things!! Any help would me much >appreciated! > >much obliged, >Christoph > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From kippjohnson at uchicago.edu Thu Oct 11 13:43:32 2012 From: kippjohnson at uchicago.edu (Kipp Johnson) Date: Thu, 11 Oct 2012 14:43:32 -0500 Subject: [maker-devel] Interpreting Maker Results Message-ID: Hi Carson, I'm trying to get my genetic annotation out of maker. My maker run on a non-model eukaryote finished, and I used your gff3_merge script to merge the resulting files. This file is enormous, because I used snap, augustus, genemark, repeatmasker, exonerate, and blast, and has a lot of entries from all of these different programs. I want to extract only the genetic regions predicted by maker, so I used the gff3_merge script with the "-g" option. However, when I do this, I get a maker file that only has about 12,000 genes, while I was expecting around 20,000 genes for our genome. However, when I use the fasta merge tool, however, I get output files (for example, "merged.fasta.all.maker.non_overlapping_ab_initio.proteins.fasta") with about 21,000 proteins, which is closer to the gene number that I was expecting. Does the "-g" option ignore evidence from blast/exonerate or similar? How should I extract the complete set of genetic regions to blast against, so that I can go about further working on the annotation? Also, what is maker using to find these 9,000 extra proteins? Are these these all alternately sliced or something along those lines? I can't find any documentation online for how to actually get the final annotations out of maker correctly. Thanks for your time! Best, Kipp Johnson kippjohnson at uchicago.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Oct 11 16:21:49 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 11 Oct 2012 18:21:49 -0400 Subject: [maker-devel] Interpreting Maker Results In-Reply-To: Message-ID: The non_overlapping_ab_initio.proteins.fasta file contain models that were rejected for lack of homology evidence that do not overlap the maker models. So if you accept everything it is not 21,000 proteins rather it's 33,000 (12,000 + 21,000). This is because the contents of that file do not occupy the same regions as the gene models (I.e. non-overlapping). Ab initio predictors have a tendency to overpredict which is why there are so many models. If you think you are missing genes you can try and add additional evidence to the maker run (I.e all proteins from a few related species). Also try running Cegma (http://korflab.ucdavis.edu/datasets/cegma/) to estimate the completeness of you assembly. Sometime a lower number than expected can be attributed to an incomplete assembly. Also you can run something like InterProScan to identify models in the non_overlapping_ab_initio.proteins.fasta file that contain Uniprot protein domains (likely to be real genes), then add them to your results as a second step using maker's model_gff option. I've attached a couple of scripts that can help with that. gff3_preds2models will turn a set of match/match_part features to gene/mRNA/exon/CDS features. It doesn't check translation of CDS though, so only use gene predictions which should be all CDS. gff3_select is used to select some subset of features from a GFF3 file. Useful for slicing sections of data from a GFF3 file. Thanks, Carson From: Kipp Johnson Date: Thursday, 11 October, 2012 3:43 PM To: , Carson Holt Subject: Interpreting Maker Results Hi Carson, I'm trying to get my genetic annotation out of maker. My maker run on a non-model eukaryote finished, and I used your gff3_merge script to merge the resulting files. This file is enormous, because I used snap, augustus, genemark, repeatmasker, exonerate, and blast, and has a lot of entries from all of these different programs. I want to extract only the genetic regions predicted by maker, so I used the gff3_merge script with the "-g" option. However, when I do this, I get a maker file that only has about 12,000 genes, while I was expecting around 20,000 genes for our genome. However, when I use the fasta merge tool, however, I get output files (for example, "merged.fasta.all.maker.non_overlapping_ab_initio.proteins.fasta") with about 21,000 proteins, which is closer to the gene number that I was expecting. Does the "-g" option ignore evidence from blast/exonerate or similar? How should I extract the complete set of genetic regions to blast against, so that I can go about further working on the annotation? Also, what is maker using to find these 9,000 extra proteins? Are these these all alternately sliced or something along those lines? I can't find any documentation online for how to actually get the final annotations out of maker correctly. Thanks for your time! Best, Kipp Johnson kippjohnson at uchicago.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gff3_preds2models Type: application/octet-stream Size: 4778 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gff3_select Type: application/octet-stream Size: 3237 bytes Desc: not available URL: From parulk at caltech.edu Fri Oct 12 15:46:21 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Fri, 12 Oct 2012 14:46:21 -0700 (PDT) Subject: [maker-devel] Conensus gene model Message-ID: <4092.131.215.15.234.1350078381.squirrel@webmail.caltech.edu> Hi, We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 file as input). The output that we get is combination of various gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? Thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig2106.gff Type: application/octet-stream Size: 10563 bytes Desc: not available URL: From parulk at caltech.edu Mon Oct 15 11:41:54 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 15 Oct 2012 10:41:54 -0700 (PDT) Subject: [maker-devel] Conensus gene model Message-ID: <2992.131.215.15.234.1350322914.squirrel@webmail.caltech.edu> Hello, Please advice on the aforementioned query? Thanks, Parul Kudtarkar ---------------------------- Original Message ---------------------------- Subject: [maker-devel] Conensus gene model From: "Parul Kudtarkar" Date: Fri, October 12, 2012 2:46 pm To: maker-devel at yandell-lab.org -------------------------------------------------------------------------- Hi, We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 file as input). The output that we get is combination of various gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? Thanks, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org_______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig2106.gff Type: application/octet-stream Size: 10563 bytes Desc: not available URL: From carsonhh at gmail.com Mon Oct 15 11:58:25 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 15 Oct 2012 13:58:25 -0400 Subject: [maker-devel] Conensus gene model In-Reply-To: <2992.131.215.15.234.1350322914.squirrel@webmail.caltech.edu> Message-ID: The contig in question is really too small to get much out of it (only 5 kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb length or longer contigs to glean enough information for optimizing your parameters. The general suggestions for any maker run are to use proteins from a closely related organism or a couple of closely related organisms for the protein= option in maker. Also leave single_exon set to 0, except for certain eukaryotes that have a bias for single exon transcripts (i.e. some fungi and oomycetes). And leave keep_preds set to 0 because ab initio predictors tend to over-predict by a wide margin (lots of false positives). Additional training would really depend on what your other contigs look like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. Thanks, Carson On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >Hello, > >Please advice on the aforementioned query? > >Thanks, >Parul Kudtarkar >---------------------------- Original Message ---------------------------- >Subject: [maker-devel] Conensus gene model >From: "Parul Kudtarkar" >Date: Fri, October 12, 2012 2:46 pm >To: maker-devel at yandell-lab.org >-------------------------------------------------------------------------- > >Hi, > >We are using snap(training set[hmm file] generated using est,protein and >contig file), agustus,genemarkE(we ran it outside maker and have gff3 file >as input). The output that we get is combination of various >gene-predictors and evidences. I have attached sample result file. What >would you recommend to get consensus result set? Bootstrapping the >resulting gff3 file (rerunning maker)? > >Thanks, >Parul Kudtarkar >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Mon Oct 15 14:10:00 2012 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 15 Oct 2012 16:10:00 -0400 Subject: [maker-devel] Conensus gene model In-Reply-To: <4445.131.215.15.234.1350330328.squirrel@webmail.caltech.edu> Message-ID: One thing you seem to be missing is protein evidence. Is this a sea urchin (I looked up some of the ESTs)? If so, I would recommend adding all proteins from the Strongylocentrotus purpuratus genome, then throw in another Deuterstome of your choice. Perhaps you should also add a couple of outgroup organisms like Nematostella vectensis (cnidaria) and a protostome of your choice. Be careful if adding adding to many protostome outgroups (i.e. C. elegans and Drosophila) because a big part of their evolution is gene loss (so distant cnidaria often match deuterstomes better than most protostomes do). You could take the maker results when protein data is included and use it to retrain SNAP again. Even a 22 kb contig is still really short. Is this genome primarily constituted by short contigs like this? I would recommend running CEGMA once on this genome to get an appropriate estimate of how recoverable the genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). Cegma will give you an estimate for genome completeness as well as estimates of what percentage of genes will be found in their entirety and what percent will be partial genes. This is important to do if your genome is fragmented as it will give you a reasonable expectation of what you can expected to recover (as short contigs don't annotate very well - you tend to loose a lot). Thanks, Carson On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >Hi Carson, > >Thanks. I have attached another contig which is 22 kb, with as many as 3 >exons EST alignments. Could you please recommend additional training. We >are currently running maker on the entire contig set and eventually merge >all the gff3 contig predictions. The using suggested parameter/methods we >would like to get a consensus gene-set with minimal false >positives/negatives. > >Thanks, >Parul > >> The contig in question is really too small to get much out of it (only 5 >kb). There was only one single exon EST alignments and a couple of >predictions with no evidence support. Anything smaller than 10 kb is >mostly useless for annotation purposes. You would really need a few >100kb >> length or longer contigs to glean enough information for optimizing your >parameters. >> >> The general suggestions for any maker run are to use proteins from a >closely related organism or a couple of closely related organisms for >the >> protein= option in maker. Also leave single_exon set to 0, except for >certain eukaryotes that have a bias for single exon transcripts (i.e. >some >> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >predictors tend to over-predict by a wide margin (lots of false >> positives). >> >> Additional training would really depend on what your other contigs look >like. Do you have any large contigs? I could look at one of those and >give suggestions but the provided contig is just too short to glean >much. >> >> Thanks, >> Carson >> >> >> >> >> >> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >> >>>Hello, >>>Please advice on the aforementioned query? >>>Thanks, >>>Parul Kudtarkar >>>---------------------------- Original Message >>> ---------------------------- >>>Subject: [maker-devel] Conensus gene model >>>From: "Parul Kudtarkar" >>>Date: Fri, October 12, 2012 2:46 pm >>>To: maker-devel at yandell-lab.org >>>------------------------------------------------------------------------ >>>-- >Hi, >>>We are using snap(training set[hmm file] generated using est,protein and >contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>> file >>>as input). The output that we get is combination of various >>>gene-predictors and evidences. I have attached sample result file. What >would you recommend to get consensus result set? Bootstrapping the >resulting gff3 file (rerunning maker)? >>>Thanks, >>>Parul Kudtarkar >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org_______________________________________________ >maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>-- >>>Scientific Programmer >>>Center for Computational Regulatory Genomics >>>Beckman Institute, >>>California Institute of Technology >>>http://www.spbase.org_______________________________________________ >maker-devel mailing list >>>maker-devel at box290.bluehost.com >>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > >-- >Scientific Programmer >Center for Computational Regulatory Genomics >Beckman Institute, >California Institute of Technology >http://www.spbase.org > From chrisi.hahni at gmail.com Tue Oct 16 08:02:36 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 16:02:36 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D68FC.10201@gmail.com> Thanks for your help Carson! I tried maker2.26, but the error still remains. I also tried to change the TMP= option in the maker_opts file, but no change either.. STATUS: Parsing control files... ERROR: The HMM file[s] provided for gmhmm do not exist: When building the maker2.26 I saw that it is complaining about one more (recommended) dependency (I must have overlooked that before): Checking prerequisites... recommends: * DBD::Pg is not installed I am having problems to get that one.. Is that causing the error? much obliged, Christoph Am 15.10.2012 20:05, schrieb Carson Holt: > It looks like either an NFS issue or perhaps an issue with the location > provided for the TMP= in the maker_opts.ctl file. > > First I would suggest trying maker 2.26 to see if the issue still happens > there (there are a few updates to how NFS locks are handled in 2.26). > > Second if you are setting TMP make sure your disk is not full. Or if TMP > was set to a tmpfs location (in memory filesystem), it could be a full > memory issue in which case you could just set TMP to somewhere else. > > Let me know if you still see the issue in 2.26 or after changing the > location of TMP. > > Thanks, > Carson > > > On 12-10-12 1:09 PM, "Christoph Hahn" wrote: > >> Hi Carson, >> >> Thanks for your reply. I managed to install maker2.25 properly now, but >> when running it I am getting strange errors (two kinds): >> type one: >> STATUS: Parsing control files... >> ERROR: The HMM file[s] provided for gmhmm do not exist: >> >> type two: >> STATUS: Parsing control files... >> ERROR: Cannot get initialization lock. >> >> I have divided the draft assembly to several files, each containing a >> subset of contigs on which I am running a maker instance. I am getting >> the above named error type one for the majority of the instances, >> although the HMM file it is complaining about (es.mod) is actually in >> the path it is naming (not shown above). In a small number of instances >> I am getting the type two error mentioned above. >> >> Can you help me? >> >> much obliged, >> Christoph >> >> >> >> On 09.10.2012 18:27, Carson Holt wrote: >>> You may need to reconfigure your cpan preferences. >>> >>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>> or >>> by starting cpan from your command line (command is cpan). Then run 'o >>> conf init' inside the cpan ui. >>> >>> --Carson >>> >>> >>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>> >>>> Hello maker-team, >>>> >>>> I am trying to install maker 2.25 on a cluster, without root privileges >>>> (I managed before, but now after migration to a new cluster I cant seem >>>> to get it to work). All necessary prerequisite programs are installed >>>> and in the path (I followed the steps in the INSTALL file that comes >>>> with maker2.25). When I do: perl Build.PL, I get: >>>> Checking prerequisites... >>>> requires: >>>> ! DBD::SQLite is not installed >>>> ! IO::All is not installed >>>> ! Inline::C is not installed >>>> ! Perl::Unsafe::Signals is not installed >>>> ! Proc::ProcessTable is not installed >>>> ! Want is not installed >>>> ! forks is not installed >>>> ! forks::shared is not installed >>>> build_requires: >>>> ! LWP::Simple is not installed >>>> recommends: >>>> * DBD::Pg is not installed >>>> >>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>> versions >>>> of the modules indicated above before proceeding with this installation >>>> >>>> Run 'Build installdeps' to install missing prerequisites. >>>> >>>> MAKER supports distributed parallelization via MPI. >>>> Would you like to configure MAKER for MPI (This >>>> requires that you have an MPI client installed)? [N ]n >>>> >>>> WARNING: Apache cannot be located. The optional web based >>>> interface to MAKER will not be available to you. >>>> >>>> Created MYMETA.yml and MYMETA.json >>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>> >>>> >>>> The file 'Build' has been created for you to finish installing MAKER. >>>> >>>> >>>> >>>> ======================================================================== >>>> == >>>> ==== >>>> STATUS MAKER 2.25 >>>> >>>> ======================================================================== >>>> == >>>> ==== >>>> PERL Dependencies: MISSING >>>> ! forks >>>> ! IO::All >>>> ! forks::shared >>>> ! Want >>>> ! DBD::SQLite >>>> ! Proc::ProcessTable >>>> ! Inline::C >>>> ! Perl::Unsafe::Signals >>>> >>>> External Programs: VERIFIED >>>> External C Libraries: VERIFIED >>>> MPI SUPPORT: DISABLED >>>> MWAS Web Interface: DISABLED >>>> MAKER PACKAGE: MISSING PREREQUISITES >>>> >>>> >>>> Important Commands: >>>> ./Build installdeps #installs missing PERL dependencies >>>> ./Build installexes #installs all missing external >>>> programs >>>> ./Build install #installs MAKER >>>> ./Build status #Shows this status menu >>>> >>>> Other Commands: >>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>> RepBase) >>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>> ./Build exonerate #installs Exonerate (v2 on UNIX / v1 >>>> on >>>> Mac OSX) >>>> ./Build snap #installs SNAP >>>> ./Build augustus #installs Augustus >>>> ./Build apollo #installs Apollo >>>> ./Build gbrowse #installs GBrowse (must be root) >>>> ./Build jbrowse #installs JBrowse (MAKER copy, not web >>>> accecible) >>>> ./Build mpich2 #installs MPICH2 (but manual install >>>> recommended) >>>> >>>> So it seems some Perl Dependencies are missing. As indicated in INSTALL >>>> I followed the quick and dirty installation for Bioperl, so I am not >>>> sure what I did wrong/missed out. >>>> >>>> When I run ./Build installdeps, I get: >>>> You do not have write access to install missing Modules. >>>> I can try and install these locally (i.e. only for MAKER) >>>> in the .../maker/perl/lib directory, or you can run >>>> './Build installdeps' as root or using sudo and try again. >>>> Do want MAKER to try and build a local installation? [N ]y >>>> mkdir /usit/titan: Permission denied at >>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>> line >>>> 59 >>>> >>>> Sorry to bother you with this minor things!! Any help would me much >>>> appreciated! >>>> >>>> much obliged, >>>> Christoph >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> > From parulk at caltech.edu Mon Oct 15 13:45:28 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 15 Oct 2012 12:45:28 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: Message-ID: <4445.131.215.15.234.1350330328.squirrel@webmail.caltech.edu> Hi Carson, Thanks. I have attached another contig which is 22 kb, with as many as 3 exons EST alignments. Could you please recommend additional training. We are currently running maker on the entire contig set and eventually merge all the gff3 contig predictions. The using suggested parameter/methods we would like to get a consensus gene-set with minimal false positives/negatives. Thanks, Parul > The contig in question is really too small to get much out of it (only 5 kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb > length or longer contigs to glean enough information for optimizing your parameters. > > The general suggestions for any maker run are to use proteins from a closely related organism or a couple of closely related organisms for the > protein= option in maker. Also leave single_exon set to 0, except for certain eukaryotes that have a bias for single exon transcripts (i.e. some > fungi and oomycetes). And leave keep_preds set to 0 because ab initio predictors tend to over-predict by a wide margin (lots of false > positives). > > Additional training would really depend on what your other contigs look like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. > > Thanks, > Carson > > > > > > On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: > >>Hello, >>Please advice on the aforementioned query? >>Thanks, >>Parul Kudtarkar >>---------------------------- Original Message >> ---------------------------- >>Subject: [maker-devel] Conensus gene model >>From: "Parul Kudtarkar" >>Date: Fri, October 12, 2012 2:46 pm >>To: maker-devel at yandell-lab.org >>-------------------------------------------------------------------------- Hi, >>We are using snap(training set[hmm file] generated using est,protein and contig file), agustus,genemarkE(we ran it outside maker and have gff3 >> file >>as input). The output that we get is combination of various >>gene-predictors and evidences. I have attached sample result file. What would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? >>Thanks, >>Parul Kudtarkar >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org_______________________________________________ maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org_______________________________________________ maker-devel mailing list >>maker-devel at box290.bluehost.com >>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: Contig7.gff Type: application/octet-stream Size: 68448 bytes Desc: not available URL: From carsonhh at gmail.com Tue Oct 16 08:19:21 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 16 Oct 2012 10:19:21 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <507D68FC.10201@gmail.com> Message-ID: No. that's just a recommendation for the maker2chado script to work. Is es.mod the actual HMM or a soft link to the HMM? After the error message their should be a list of the actual files. Does it print that? Are there trailing commas in you maker_opts.ctl file after the hmm file name in the control files? Are their ':' characters in the path name to the file? --Carson On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >Thanks for your help Carson! > >I tried maker2.26, but the error still remains. I also tried to change >the TMP= option in the maker_opts file, but no change either.. > >STATUS: Parsing control files... >ERROR: The HMM file[s] provided for gmhmm do not exist: > >When building the maker2.26 I saw that it is complaining about one more >(recommended) dependency (I must have overlooked that before): > Checking prerequisites... > recommends: > * DBD::Pg is not installed > >I am having problems to get that one.. Is that causing the error? > >much obliged, >Christoph > >Am 15.10.2012 20:05, schrieb Carson Holt: >> It looks like either an NFS issue or perhaps an issue with the location >> provided for the TMP= in the maker_opts.ctl file. >> >> First I would suggest trying maker 2.26 to see if the issue still >>happens >> there (there are a few updates to how NFS locks are handled in 2.26). >> >> Second if you are setting TMP make sure your disk is not full. Or if >>TMP >> was set to a tmpfs location (in memory filesystem), it could be a full >> memory issue in which case you could just set TMP to somewhere else. >> >> Let me know if you still see the issue in 2.26 or after changing the >> location of TMP. >> >> Thanks, >> Carson >> >> >> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >> >>> Hi Carson, >>> >>> Thanks for your reply. I managed to install maker2.25 properly now, but >>> when running it I am getting strange errors (two kinds): >>> type one: >>> STATUS: Parsing control files... >>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>> >>> type two: >>> STATUS: Parsing control files... >>> ERROR: Cannot get initialization lock. >>> >>> I have divided the draft assembly to several files, each containing a >>> subset of contigs on which I am running a maker instance. I am getting >>> the above named error type one for the majority of the instances, >>> although the HMM file it is complaining about (es.mod) is actually in >>> the path it is naming (not shown above). In a small number of instances >>> I am getting the type two error mentioned above. >>> >>> Can you help me? >>> >>> much obliged, >>> Christoph >>> >>> >>> >>> On 09.10.2012 18:27, Carson Holt wrote: >>>> You may need to reconfigure your cpan preferences. >>>> >>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>>> or >>>> by starting cpan from your command line (command is cpan). Then run >>>>'o >>>> conf init' inside the cpan ui. >>>> >>>> --Carson >>>> >>>> >>>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>>> >>>>> Hello maker-team, >>>>> >>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>privileges >>>>> (I managed before, but now after migration to a new cluster I cant >>>>>seem >>>>> to get it to work). All necessary prerequisite programs are installed >>>>> and in the path (I followed the steps in the INSTALL file that comes >>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>> Checking prerequisites... >>>>> requires: >>>>> ! DBD::SQLite is not installed >>>>> ! IO::All is not installed >>>>> ! Inline::C is not installed >>>>> ! Perl::Unsafe::Signals is not installed >>>>> ! Proc::ProcessTable is not installed >>>>> ! Want is not installed >>>>> ! forks is not installed >>>>> ! forks::shared is not installed >>>>> build_requires: >>>>> ! LWP::Simple is not installed >>>>> recommends: >>>>> * DBD::Pg is not installed >>>>> >>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>>> versions >>>>> of the modules indicated above before proceeding with this >>>>>installation >>>>> >>>>> Run 'Build installdeps' to install missing prerequisites. >>>>> >>>>> MAKER supports distributed parallelization via MPI. >>>>> Would you like to configure MAKER for MPI (This >>>>> requires that you have an MPI client installed)? [N ]n >>>>> >>>>> WARNING: Apache cannot be located. The optional web based >>>>> interface to MAKER will not be available to you. >>>>> >>>>> Created MYMETA.yml and MYMETA.json >>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>> >>>>> >>>>> The file 'Build' has been created for you to finish installing MAKER. >>>>> >>>>> >>>>> >>>>> >>>>>====================================================================== >>>>>== >>>>> == >>>>> ==== >>>>> STATUS MAKER 2.25 >>>>> >>>>> >>>>>====================================================================== >>>>>== >>>>> == >>>>> ==== >>>>> PERL Dependencies: MISSING >>>>> ! forks >>>>> ! IO::All >>>>> ! forks::shared >>>>> ! Want >>>>> ! DBD::SQLite >>>>> ! Proc::ProcessTable >>>>> ! Inline::C >>>>> ! Perl::Unsafe::Signals >>>>> >>>>> External Programs: VERIFIED >>>>> External C Libraries: VERIFIED >>>>> MPI SUPPORT: DISABLED >>>>> MWAS Web Interface: DISABLED >>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>> >>>>> >>>>> Important Commands: >>>>> ./Build installdeps #installs missing PERL dependencies >>>>> ./Build installexes #installs all missing external >>>>> programs >>>>> ./Build install #installs MAKER >>>>> ./Build status #Shows this status menu >>>>> >>>>> Other Commands: >>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>> RepBase) >>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>> ./Build exonerate #installs Exonerate (v2 on UNIX / >>>>>v1 >>>>> on >>>>> Mac OSX) >>>>> ./Build snap #installs SNAP >>>>> ./Build augustus #installs Augustus >>>>> ./Build apollo #installs Apollo >>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>> ./Build jbrowse #installs JBrowse (MAKER copy, not >>>>>web >>>>> accecible) >>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>install >>>>> recommended) >>>>> >>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>INSTALL >>>>> I followed the quick and dirty installation for Bioperl, so I am not >>>>> sure what I did wrong/missed out. >>>>> >>>>> When I run ./Build installdeps, I get: >>>>> You do not have write access to install missing Modules. >>>>> I can try and install these locally (i.e. only for MAKER) >>>>> in the .../maker/perl/lib directory, or you can run >>>>> './Build installdeps' as root or using sudo and try again. >>>>> Do want MAKER to try and build a local installation? [N ]y >>>>> mkdir /usit/titan: Permission denied at >>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>> line >>>>> 59 >>>>> >>>>> Sorry to bother you with this minor things!! Any help would me much >>>>> appreciated! >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> >>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>g >>>> >> > > From chrisi.hahni at gmail.com Tue Oct 16 09:14:36 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 17:14:36 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D79DC.5090702@gmail.com> Hi Carson, It was a link. The path was not correct any more because of the migration to the new cluster. I could not change it so I just gave the path to the real file in the maker_opts file. That has resolved this error, but now I am getting the following: STATUS: Parsing control files... ERROR: Cannot get initialization lock. Probably not relevant any more, but: No trailing commas, no ':' characters in the path. Thanks for your help! Christoph Am 16.10.2012 16:19, schrieb Carson Holt: > No. that's just a recommendation for the maker2chado script to work. > > Is es.mod the actual HMM or a soft link to the HMM? > After the error message their should be a list of the actual files. Does > it print that? > Are there trailing commas in you maker_opts.ctl file after the hmm file > name in the control files? > Are their ':' characters in the path name to the file? > > --Carson > > > > On 12-10-16 10:02 AM, "Christoph Hahn" wrote: > >> Thanks for your help Carson! >> >> I tried maker2.26, but the error still remains. I also tried to change >> the TMP= option in the maker_opts file, but no change either.. >> >> STATUS: Parsing control files... >> ERROR: The HMM file[s] provided for gmhmm do not exist: >> >> When building the maker2.26 I saw that it is complaining about one more >> (recommended) dependency (I must have overlooked that before): >> Checking prerequisites... >> recommends: >> * DBD::Pg is not installed >> >> I am having problems to get that one.. Is that causing the error? >> >> much obliged, >> Christoph >> >> Am 15.10.2012 20:05, schrieb Carson Holt: >>> It looks like either an NFS issue or perhaps an issue with the location >>> provided for the TMP= in the maker_opts.ctl file. >>> >>> First I would suggest trying maker 2.26 to see if the issue still >>> happens >>> there (there are a few updates to how NFS locks are handled in 2.26). >>> >>> Second if you are setting TMP make sure your disk is not full. Or if >>> TMP >>> was set to a tmpfs location (in memory filesystem), it could be a full >>> memory issue in which case you could just set TMP to somewhere else. >>> >>> Let me know if you still see the issue in 2.26 or after changing the >>> location of TMP. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>> >>>> Hi Carson, >>>> >>>> Thanks for your reply. I managed to install maker2.25 properly now, but >>>> when running it I am getting strange errors (two kinds): >>>> type one: >>>> STATUS: Parsing control files... >>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>> >>>> type two: >>>> STATUS: Parsing control files... >>>> ERROR: Cannot get initialization lock. >>>> >>>> I have divided the draft assembly to several files, each containing a >>>> subset of contigs on which I am running a maker instance. I am getting >>>> the above named error type one for the majority of the instances, >>>> although the HMM file it is complaining about (es.mod) is actually in >>>> the path it is naming (not shown above). In a small number of instances >>>> I am getting the type two error mentioned above. >>>> >>>> Can you help me? >>>> >>>> much obliged, >>>> Christoph >>>> >>>> >>>> >>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>> You may need to reconfigure your cpan preferences. >>>>> >>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm file, >>>>> or >>>>> by starting cpan from your command line (command is cpan). Then run >>>>> 'o >>>>> conf init' inside the cpan ui. >>>>> >>>>> --Carson >>>>> >>>>> >>>>> On 12-10-09 7:07 AM, "Christoph Hahn" wrote: >>>>> >>>>>> Hello maker-team, >>>>>> >>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>> privileges >>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>> seem >>>>>> to get it to work). All necessary prerequisite programs are installed >>>>>> and in the path (I followed the steps in the INSTALL file that comes >>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>> Checking prerequisites... >>>>>> requires: >>>>>> ! DBD::SQLite is not installed >>>>>> ! IO::All is not installed >>>>>> ! Inline::C is not installed >>>>>> ! Perl::Unsafe::Signals is not installed >>>>>> ! Proc::ProcessTable is not installed >>>>>> ! Want is not installed >>>>>> ! forks is not installed >>>>>> ! forks::shared is not installed >>>>>> build_requires: >>>>>> ! LWP::Simple is not installed >>>>>> recommends: >>>>>> * DBD::Pg is not installed >>>>>> >>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install the >>>>>> versions >>>>>> of the modules indicated above before proceeding with this >>>>>> installation >>>>>> >>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>> >>>>>> MAKER supports distributed parallelization via MPI. >>>>>> Would you like to configure MAKER for MPI (This >>>>>> requires that you have an MPI client installed)? [N ]n >>>>>> >>>>>> WARNING: Apache cannot be located. The optional web based >>>>>> interface to MAKER will not be available to you. >>>>>> >>>>>> Created MYMETA.yml and MYMETA.json >>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>> >>>>>> >>>>>> The file 'Build' has been created for you to finish installing MAKER. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ====================================================================== >>>>>> == >>>>>> == >>>>>> ==== >>>>>> STATUS MAKER 2.25 >>>>>> >>>>>> >>>>>> ====================================================================== >>>>>> == >>>>>> == >>>>>> ==== >>>>>> PERL Dependencies: MISSING >>>>>> ! forks >>>>>> ! IO::All >>>>>> ! forks::shared >>>>>> ! Want >>>>>> ! DBD::SQLite >>>>>> ! Proc::ProcessTable >>>>>> ! Inline::C >>>>>> ! Perl::Unsafe::Signals >>>>>> >>>>>> External Programs: VERIFIED >>>>>> External C Libraries: VERIFIED >>>>>> MPI SUPPORT: DISABLED >>>>>> MWAS Web Interface: DISABLED >>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>> >>>>>> >>>>>> Important Commands: >>>>>> ./Build installdeps #installs missing PERL dependencies >>>>>> ./Build installexes #installs all missing external >>>>>> programs >>>>>> ./Build install #installs MAKER >>>>>> ./Build status #Shows this status menu >>>>>> >>>>>> Other Commands: >>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>> RepBase) >>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX / >>>>>> v1 >>>>>> on >>>>>> Mac OSX) >>>>>> ./Build snap #installs SNAP >>>>>> ./Build augustus #installs Augustus >>>>>> ./Build apollo #installs Apollo >>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, not >>>>>> web >>>>>> accecible) >>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>> install >>>>>> recommended) >>>>>> >>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>> INSTALL >>>>>> I followed the quick and dirty installation for Bioperl, so I am not >>>>>> sure what I did wrong/missed out. >>>>>> >>>>>> When I run ./Build installdeps, I get: >>>>>> You do not have write access to install missing Modules. >>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>> in the .../maker/perl/lib directory, or you can run >>>>>> './Build installdeps' as root or using sudo and try again. >>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>> mkdir /usit/titan: Permission denied at >>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>> line >>>>>> 59 >>>>>> >>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>> appreciated! >>>>>> >>>>>> much obliged, >>>>>> Christoph >>>>>> >>>>>> _______________________________________________ >>>>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.or >>>>>> g >> > From carsonhh at gmail.com Tue Oct 16 09:18:48 2012 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 16 Oct 2012 11:18:48 -0400 Subject: [maker-devel] install maker on cluster In-Reply-To: <507D79DC.5090702@gmail.com> Message-ID: Delete any file in the maker output directory that start with the name .init or .NFS then rerun. Thanks, Carson On 12-10-16 11:14 AM, "Christoph Hahn" wrote: >Hi Carson, > >It was a link. The path was not correct any more because of the >migration to the new cluster. I could not change it so I just gave the >path to the real file in the maker_opts file. That has resolved this >error, but now I am getting the following: >STATUS: Parsing control files... >ERROR: Cannot get initialization lock. > >Probably not relevant any more, but: No trailing commas, no ':' >characters in the path. > >Thanks for your help! >Christoph > >Am 16.10.2012 16:19, schrieb Carson Holt: >> No. that's just a recommendation for the maker2chado script to work. >> >> Is es.mod the actual HMM or a soft link to the HMM? >> After the error message their should be a list of the actual files. >>Does >> it print that? >> Are there trailing commas in you maker_opts.ctl file after the hmm file >> name in the control files? >> Are their ':' characters in the path name to the file? >> >> --Carson >> >> >> >> On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >> >>> Thanks for your help Carson! >>> >>> I tried maker2.26, but the error still remains. I also tried to change >>> the TMP= option in the maker_opts file, but no change either.. >>> >>> STATUS: Parsing control files... >>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>> >>> When building the maker2.26 I saw that it is complaining about one more >>> (recommended) dependency (I must have overlooked that before): >>> Checking prerequisites... >>> recommends: >>> * DBD::Pg is not installed >>> >>> I am having problems to get that one.. Is that causing the error? >>> >>> much obliged, >>> Christoph >>> >>> Am 15.10.2012 20:05, schrieb Carson Holt: >>>> It looks like either an NFS issue or perhaps an issue with the >>>>location >>>> provided for the TMP= in the maker_opts.ctl file. >>>> >>>> First I would suggest trying maker 2.26 to see if the issue still >>>> happens >>>> there (there are a few updates to how NFS locks are handled in 2.26). >>>> >>>> Second if you are setting TMP make sure your disk is not full. Or if >>>> TMP >>>> was set to a tmpfs location (in memory filesystem), it could be a full >>>> memory issue in which case you could just set TMP to somewhere else. >>>> >>>> Let me know if you still see the issue in 2.26 or after changing the >>>> location of TMP. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>>> >>>>> Hi Carson, >>>>> >>>>> Thanks for your reply. I managed to install maker2.25 properly now, >>>>>but >>>>> when running it I am getting strange errors (two kinds): >>>>> type one: >>>>> STATUS: Parsing control files... >>>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>>> >>>>> type two: >>>>> STATUS: Parsing control files... >>>>> ERROR: Cannot get initialization lock. >>>>> >>>>> I have divided the draft assembly to several files, each containing a >>>>> subset of contigs on which I am running a maker instance. I am >>>>>getting >>>>> the above named error type one for the majority of the instances, >>>>> although the HMM file it is complaining about (es.mod) is actually in >>>>> the path it is naming (not shown above). In a small number of >>>>>instances >>>>> I am getting the type two error mentioned above. >>>>> >>>>> Can you help me? >>>>> >>>>> much obliged, >>>>> Christoph >>>>> >>>>> >>>>> >>>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>>> You may need to reconfigure your cpan preferences. >>>>>> >>>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm >>>>>>file, >>>>>> or >>>>>> by starting cpan from your command line (command is cpan). Then run >>>>>> 'o >>>>>> conf init' inside the cpan ui. >>>>>> >>>>>> --Carson >>>>>> >>>>>> >>>>>> On 12-10-09 7:07 AM, "Christoph Hahn" >>>>>>wrote: >>>>>> >>>>>>> Hello maker-team, >>>>>>> >>>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>>> privileges >>>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>>> seem >>>>>>> to get it to work). All necessary prerequisite programs are >>>>>>>installed >>>>>>> and in the path (I followed the steps in the INSTALL file that >>>>>>>comes >>>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>>> Checking prerequisites... >>>>>>> requires: >>>>>>> ! DBD::SQLite is not installed >>>>>>> ! IO::All is not installed >>>>>>> ! Inline::C is not installed >>>>>>> ! Perl::Unsafe::Signals is not installed >>>>>>> ! Proc::ProcessTable is not installed >>>>>>> ! Want is not installed >>>>>>> ! forks is not installed >>>>>>> ! forks::shared is not installed >>>>>>> build_requires: >>>>>>> ! LWP::Simple is not installed >>>>>>> recommends: >>>>>>> * DBD::Pg is not installed >>>>>>> >>>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install >>>>>>>the >>>>>>> versions >>>>>>> of the modules indicated above before proceeding with this >>>>>>> installation >>>>>>> >>>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>>> >>>>>>> MAKER supports distributed parallelization via MPI. >>>>>>> Would you like to configure MAKER for MPI (This >>>>>>> requires that you have an MPI client installed)? [N ]n >>>>>>> >>>>>>> WARNING: Apache cannot be located. The optional web based >>>>>>> interface to MAKER will not be available to you. >>>>>>> >>>>>>> Created MYMETA.yml and MYMETA.json >>>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>>> >>>>>>> >>>>>>> The file 'Build' has been created for you to finish installing >>>>>>>MAKER. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>== >>>>>>> == >>>>>>> == >>>>>>> ==== >>>>>>> STATUS MAKER 2.25 >>>>>>> >>>>>>> >>>>>>> >>>>>>>==================================================================== >>>>>>>== >>>>>>> == >>>>>>> == >>>>>>> ==== >>>>>>> PERL Dependencies: MISSING >>>>>>> ! forks >>>>>>> ! IO::All >>>>>>> ! forks::shared >>>>>>> ! Want >>>>>>> ! DBD::SQLite >>>>>>> ! Proc::ProcessTable >>>>>>> ! Inline::C >>>>>>> ! Perl::Unsafe::Signals >>>>>>> >>>>>>> External Programs: VERIFIED >>>>>>> External C Libraries: VERIFIED >>>>>>> MPI SUPPORT: DISABLED >>>>>>> MWAS Web Interface: DISABLED >>>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>>> >>>>>>> >>>>>>> Important Commands: >>>>>>> ./Build installdeps #installs missing PERL >>>>>>>dependencies >>>>>>> ./Build installexes #installs all missing external >>>>>>> programs >>>>>>> ./Build install #installs MAKER >>>>>>> ./Build status #Shows this status menu >>>>>>> >>>>>>> Other Commands: >>>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>>> RepBase) >>>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX >>>>>>>/ >>>>>>> v1 >>>>>>> on >>>>>>> Mac OSX) >>>>>>> ./Build snap #installs SNAP >>>>>>> ./Build augustus #installs Augustus >>>>>>> ./Build apollo #installs Apollo >>>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, >>>>>>>not >>>>>>> web >>>>>>> accecible) >>>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>>> install >>>>>>> recommended) >>>>>>> >>>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>>> INSTALL >>>>>>> I followed the quick and dirty installation for Bioperl, so I am >>>>>>>not >>>>>>> sure what I did wrong/missed out. >>>>>>> >>>>>>> When I run ./Build installdeps, I get: >>>>>>> You do not have write access to install missing Modules. >>>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>>> in the .../maker/perl/lib directory, or you can run >>>>>>> './Build installdeps' as root or using sudo and try again. >>>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>>> mkdir /usit/titan: Permission denied at >>>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>>> line >>>>>>> 59 >>>>>>> >>>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>>> appreciated! >>>>>>> >>>>>>> much obliged, >>>>>>> Christoph >>>>>>> >>>>>>> _______________________________________________ >>>>>>> maker-devel mailing list >>>>>>> maker-devel at box290.bluehost.com >>>>>>> >>>>>>> >>>>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab. >>>>>>>or >>>>>>> g >>> >> > > From chrisi.hahni at gmail.com Tue Oct 16 10:12:21 2012 From: chrisi.hahni at gmail.com (Christoph Hahn) Date: Tue, 16 Oct 2012 18:12:21 +0200 Subject: [maker-devel] install maker on cluster In-Reply-To: References: Message-ID: <507D8765.8010909@gmail.com> The maker output directory contains only one thing: .NFSLock.init_lock.NFSLock, which is a link to .NFSLock.init_lock.tmp.3358.21603.1715.48578929183 in the same directory. When I delete it and rerun maker I get the same error and another .NFSLock.init_lock.NFSLock is created, this one is linked to .NFSLock.init_lock.tmp.3635.21981.7756.8294996771 Thanks, Christoph Am 16.10.2012 17:18, schrieb Carson Holt: > Delete any file in the maker output directory that start with the name > .init or .NFS then rerun. > > Thanks, > Carson > > > > On 12-10-16 11:14 AM, "Christoph Hahn" wrote: > >> Hi Carson, >> >> It was a link. The path was not correct any more because of the >> migration to the new cluster. I could not change it so I just gave the >> path to the real file in the maker_opts file. That has resolved this >> error, but now I am getting the following: >> STATUS: Parsing control files... >> ERROR: Cannot get initialization lock. >> >> Probably not relevant any more, but: No trailing commas, no ':' >> characters in the path. >> >> Thanks for your help! >> Christoph >> >> Am 16.10.2012 16:19, schrieb Carson Holt: >>> No. that's just a recommendation for the maker2chado script to work. >>> >>> Is es.mod the actual HMM or a soft link to the HMM? >>> After the error message their should be a list of the actual files. >>> Does >>> it print that? >>> Are there trailing commas in you maker_opts.ctl file after the hmm file >>> name in the control files? >>> Are their ':' characters in the path name to the file? >>> >>> --Carson >>> >>> >>> >>> On 12-10-16 10:02 AM, "Christoph Hahn" wrote: >>> >>>> Thanks for your help Carson! >>>> >>>> I tried maker2.26, but the error still remains. I also tried to change >>>> the TMP= option in the maker_opts file, but no change either.. >>>> >>>> STATUS: Parsing control files... >>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>> >>>> When building the maker2.26 I saw that it is complaining about one more >>>> (recommended) dependency (I must have overlooked that before): >>>> Checking prerequisites... >>>> recommends: >>>> * DBD::Pg is not installed >>>> >>>> I am having problems to get that one.. Is that causing the error? >>>> >>>> much obliged, >>>> Christoph >>>> >>>> Am 15.10.2012 20:05, schrieb Carson Holt: >>>>> It looks like either an NFS issue or perhaps an issue with the >>>>> location >>>>> provided for the TMP= in the maker_opts.ctl file. >>>>> >>>>> First I would suggest trying maker 2.26 to see if the issue still >>>>> happens >>>>> there (there are a few updates to how NFS locks are handled in 2.26). >>>>> >>>>> Second if you are setting TMP make sure your disk is not full. Or if >>>>> TMP >>>>> was set to a tmpfs location (in memory filesystem), it could be a full >>>>> memory issue in which case you could just set TMP to somewhere else. >>>>> >>>>> Let me know if you still see the issue in 2.26 or after changing the >>>>> location of TMP. >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> On 12-10-12 1:09 PM, "Christoph Hahn" wrote: >>>>> >>>>>> Hi Carson, >>>>>> >>>>>> Thanks for your reply. I managed to install maker2.25 properly now, >>>>>> but >>>>>> when running it I am getting strange errors (two kinds): >>>>>> type one: >>>>>> STATUS: Parsing control files... >>>>>> ERROR: The HMM file[s] provided for gmhmm do not exist: >>>>>> >>>>>> type two: >>>>>> STATUS: Parsing control files... >>>>>> ERROR: Cannot get initialization lock. >>>>>> >>>>>> I have divided the draft assembly to several files, each containing a >>>>>> subset of contigs on which I am running a maker instance. I am >>>>>> getting >>>>>> the above named error type one for the majority of the instances, >>>>>> although the HMM file it is complaining about (es.mod) is actually in >>>>>> the path it is naming (not shown above). In a small number of >>>>>> instances >>>>>> I am getting the type two error mentioned above. >>>>>> >>>>>> Can you help me? >>>>>> >>>>>> much obliged, >>>>>> Christoph >>>>>> >>>>>> >>>>>> >>>>>> On 09.10.2012 18:27, Carson Holt wrote: >>>>>>> You may need to reconfigure your cpan preferences. >>>>>>> >>>>>>> You can do this manually by editing the ~/.cpan/CPAN/MyConfig.pm >>>>>>> file, >>>>>>> or >>>>>>> by starting cpan from your command line (command is cpan). Then run >>>>>>> 'o >>>>>>> conf init' inside the cpan ui. >>>>>>> >>>>>>> --Carson >>>>>>> >>>>>>> >>>>>>> On 12-10-09 7:07 AM, "Christoph Hahn" >>>>>>> wrote: >>>>>>> >>>>>>>> Hello maker-team, >>>>>>>> >>>>>>>> I am trying to install maker 2.25 on a cluster, without root >>>>>>>> privileges >>>>>>>> (I managed before, but now after migration to a new cluster I cant >>>>>>>> seem >>>>>>>> to get it to work). All necessary prerequisite programs are >>>>>>>> installed >>>>>>>> and in the path (I followed the steps in the INSTALL file that >>>>>>>> comes >>>>>>>> with maker2.25). When I do: perl Build.PL, I get: >>>>>>>> Checking prerequisites... >>>>>>>> requires: >>>>>>>> ! DBD::SQLite is not installed >>>>>>>> ! IO::All is not installed >>>>>>>> ! Inline::C is not installed >>>>>>>> ! Perl::Unsafe::Signals is not installed >>>>>>>> ! Proc::ProcessTable is not installed >>>>>>>> ! Want is not installed >>>>>>>> ! forks is not installed >>>>>>>> ! forks::shared is not installed >>>>>>>> build_requires: >>>>>>>> ! LWP::Simple is not installed >>>>>>>> recommends: >>>>>>>> * DBD::Pg is not installed >>>>>>>> >>>>>>>> ERRORS/WARNINGS FOUND IN PREREQUISITES. You may wish to install >>>>>>>> the >>>>>>>> versions >>>>>>>> of the modules indicated above before proceeding with this >>>>>>>> installation >>>>>>>> >>>>>>>> Run 'Build installdeps' to install missing prerequisites. >>>>>>>> >>>>>>>> MAKER supports distributed parallelization via MPI. >>>>>>>> Would you like to configure MAKER for MPI (This >>>>>>>> requires that you have an MPI client installed)? [N ]n >>>>>>>> >>>>>>>> WARNING: Apache cannot be located. The optional web based >>>>>>>> interface to MAKER will not be available to you. >>>>>>>> >>>>>>>> Created MYMETA.yml and MYMETA.json >>>>>>>> Creating new 'Build' script for 'MAKER' version '2.25' >>>>>>>> >>>>>>>> >>>>>>>> The file 'Build' has been created for you to finish installing >>>>>>>> MAKER. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> == >>>>>>>> == >>>>>>>> == >>>>>>>> ==== >>>>>>>> STATUS MAKER 2.25 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ==================================================================== >>>>>>>> == >>>>>>>> == >>>>>>>> == >>>>>>>> ==== >>>>>>>> PERL Dependencies: MISSING >>>>>>>> ! forks >>>>>>>> ! IO::All >>>>>>>> ! forks::shared >>>>>>>> ! Want >>>>>>>> ! DBD::SQLite >>>>>>>> ! Proc::ProcessTable >>>>>>>> ! Inline::C >>>>>>>> ! Perl::Unsafe::Signals >>>>>>>> >>>>>>>> External Programs: VERIFIED >>>>>>>> External C Libraries: VERIFIED >>>>>>>> MPI SUPPORT: DISABLED >>>>>>>> MWAS Web Interface: DISABLED >>>>>>>> MAKER PACKAGE: MISSING PREREQUISITES >>>>>>>> >>>>>>>> >>>>>>>> Important Commands: >>>>>>>> ./Build installdeps #installs missing PERL >>>>>>>> dependencies >>>>>>>> ./Build installexes #installs all missing external >>>>>>>> programs >>>>>>>> ./Build install #installs MAKER >>>>>>>> ./Build status #Shows this status menu >>>>>>>> >>>>>>>> Other Commands: >>>>>>>> ./Build repeatmasker #installs RepeatMasker (asks for >>>>>>>> RepBase) >>>>>>>> ./Build blast #installs BLAST (NCBI BLAST+) >>>>>>>> ./Build exonerate #installs Exonerate (v2 on UNIX >>>>>>>> / >>>>>>>> v1 >>>>>>>> on >>>>>>>> Mac OSX) >>>>>>>> ./Build snap #installs SNAP >>>>>>>> ./Build augustus #installs Augustus >>>>>>>> ./Build apollo #installs Apollo >>>>>>>> ./Build gbrowse #installs GBrowse (must be root) >>>>>>>> ./Build jbrowse #installs JBrowse (MAKER copy, >>>>>>>> not >>>>>>>> web >>>>>>>> accecible) >>>>>>>> ./Build mpich2 #installs MPICH2 (but manual >>>>>>>> install >>>>>>>> recommended) >>>>>>>> >>>>>>>> So it seems some Perl Dependencies are missing. As indicated in >>>>>>>> INSTALL >>>>>>>> I followed the quick and dirty installation for Bioperl, so I am >>>>>>>> not >>>>>>>> sure what I did wrong/missed out. >>>>>>>> >>>>>>>> When I run ./Build installdeps, I get: >>>>>>>> You do not have write access to install missing Modules. >>>>>>>> I can try and install these locally (i.e. only for MAKER) >>>>>>>> in the .../maker/perl/lib directory, or you can run >>>>>>>> './Build installdeps' as root or using sudo and try again. >>>>>>>> Do want MAKER to try and build a local installation? [N ]y >>>>>>>> mkdir /usit/titan: Permission denied at >>>>>>>> /cluster/software/VERSIONS/perlmodules-5.10_1/lib/perl5/CPAN/FTP.pm >>>>>>>> line >>>>>>>> 59 >>>>>>>> >>>>>>>> Sorry to bother you with this minor things!! Any help would me much >>>>>>>> appreciated! >>>>>>>> >>>>>>>> much obliged, >>>>>>>> Christoph >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> maker-devel mailing list >>>>>>>> maker-devel at box290.bluehost.com >>>>>>>> >>>>>>>> >>>>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab. >>>>>>>> or >>>>>>>> g >> > From mikael.durling at slu.se Tue Oct 16 10:58:55 2012 From: mikael.durling at slu.se (=?iso-8859-1?Q?Mikael_Brandstr=F6m_Durling?=) Date: Tue, 16 Oct 2012 18:58:55 +0200 Subject: [maker-devel] Format of repeat_gff gff3 file Message-ID: Hi, I would like to mask my fungal genome from two different sources (ie. repbase and an inhouse repeat library). However, I suppose the that if I supply a library as rmlib in maker_opts, it will be mutually exclusive to the model_org option, in the same way as -spec and -lib options to RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the model_org option as fungi, and would like to provide maker with additional masking as a gff file. I tried by running RepeatMasker with my inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker package to obtain a gff3 file. This file was supplied to maker as rm_gff (see below for a sample from the file). The run fail with backtraces like the one paseted below. How should this gff file be formatted for maker to understand it? I see that in maker produced gff files, there are additional information found in the id of the hits. Is this required? Maybe it's easier to modify maker to make two rounds of RepeatMasker calls if both model_org and rmlib are specified? Thanks for any input, Mikael ------------- EXCEPTION: Bio::Root::Exception ------------- MSG: Must have defined a valid name for Hit STACK: Error::throw STACK: Bio::Root::Root::throw /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 STACK: Bio::Search::Hit::GenericHit::new /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:149 STACK: Bio::Search::Hit::PhatHit::Base::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm:127 STACK: Bio::Search::Hit::PhatHit::gff3::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/gff3.pm:23 STACK: GFFDB::_load_hits /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:1026 STACK: GFFDB::phathits_on_chunk /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:651 STACK: Process::MpiChunk::_go /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:752 STACK: Process::MpiChunk::run /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:331 STACK: main::node_thread /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 STACK: threads::new /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm:799 STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 ----------------------------------------------------------- Cannot restore overloading on HASH(0x2580810) (package Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at /opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm line 2256. Compilation failed in require at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. BEGIN failed--compilation aborted at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. Perl exited with active threads: 1 running and unjoined 0 finished and unjoined 0 running and detached ERROR: Could not open '/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.output/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fungi.rb.out' ERROR: Failed while doing repeat masking ERROR: Chunk failed at level:0, tier_type:1 FAILED CONTIG:scf_55419 The gff3-file looks like this: ##gff-version 3 ##sequence-region scf_42697 1 3949 scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 ##sequence-region scf_82496 1 2757 scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_family-1046 2478 4915 ##sequence-region scf_82727 1 4159 scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 23 scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_family-64 15 50 ##sequence-region scf_82785 1 4084 scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 24 scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_family-690 419 786 ##sequence-region scf_86740 1 4293 scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_family-262 1 25 scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_family-262 2 60 scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_family-262 13 60 ##sequence-region scf_86782 1 8564 scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_family-480 1027 1129 ##sequence-region scf_86808 1 4495 scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family-690 1 969 scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 74 ##sequence-region scf_86815 1 4139 scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-262 825 918 scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_family-262 526 4459 ##sequence-region scf_86823 1 2528 scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 178 scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 35 scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 45 scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 41 scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 96 scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 40 scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_family-262 881 920 ##sequence-region scf_86857 1 2778 From mike.thon at gmail.com Wed Oct 17 04:48:59 2012 From: mike.thon at gmail.com (Michael Thon) Date: Wed, 17 Oct 2012 12:48:59 +0200 Subject: [maker-devel] Format of repeat_gff gff3 file In-Reply-To: References: Message-ID: Hi - 2 ideas: 1) There is a utility included with RepeatMasker that enables you to extract sequences from it. its in the utils directory. You can run it like this: ./queryRepeatDatabase.pl -species Fungi -clade that will extract all repeat sequences belonging to Fungi or its descendants. Maybe the simplest thing for you to do is extract the sequences that you want and cat them together with your in house sequences to provide MAKER with a single file of reference sequences. 2) some of the MAKER parameters take a comma separated list of files. I don't know if this applied to rm_lib though. Mike On Oct 16, 2012, at 6:58 PM, Mikael Brandstr?m Durling wrote: > Hi, > > I would like to mask my fungal genome from two different sources (ie. repbase and an inhouse repeat library). However, I suppose the that if I supply a library as rmlib in maker_opts, it will be mutually exclusive to the model_org option, in the same way as -spec and -lib options to RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the model_org option as fungi, and would like to provide maker with additional masking as a gff file. I tried by running RepeatMasker with my inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker package to obtain a gff3 file. This file was supplied to maker as rm_gff (see below for a sample from the file). The run fail with backtraces like the one paseted below. How should this gff file be formatted for maker to understand it? I see that in maker produced gff files, there are additional information found in the id of the hits. Is this required? > > Maybe it's easier to modify maker to make two rounds of RepeatMasker calls if both model_org and rmlib are specified? > > Thanks for any input, > Mikael > > > ------------- EXCEPTION: Bio::Root::Exception ------------- > MSG: Must have defined a valid name for Hit > STACK: Error::throw > STACK: Bio::Root::Root::throw /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 > STACK: Bio::Search::Hit::GenericHit::new /opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:149 > STACK: Bio::Search::Hit::PhatHit::Base::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm:127 > STACK: Bio::Search::Hit::PhatHit::gff3::new /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Bio/Search/Hit/PhatHit/gff3.pm:23 > STACK: GFFDB::_load_hits /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:1026 > STACK: GFFDB::phathits_on_chunk /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/GFFDB.pm:651 > STACK: Process::MpiChunk::_go /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:752 > STACK: Process::MpiChunk::run /net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib/Process/MpiChunk.pm:331 > STACK: main::node_thread /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 > STACK: threads::new /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm:799 > STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 > ----------------------------------------------------------- > Cannot restore overloading on HASH(0x2580810) (package Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at /opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at /proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16.1/x86_64-linux/forks.pm line 2256. > Compilation failed in require at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. > BEGIN failed--compilation aborted at /proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. > Perl exited with active threads: > 1 running and unjoined > 0 finished and unjoined > 0 running and detached > ERROR: Could not open '/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.output/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fungi.rb.out' > ERROR: Failed while doing repeat masking > ERROR: Chunk failed at level:0, tier_type:1 > FAILED CONTIG:scf_55419 > > > The gff3-file looks like this: > ##gff-version 3 > ##sequence-region scf_42697 1 3949 > scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 > scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 > scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 > ##sequence-region scf_82496 1 2757 > scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_family-1046 2478 4915 > ##sequence-region scf_82727 1 4159 > scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 > scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 23 > scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_family-64 15 50 > ##sequence-region scf_82785 1 4084 > scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 24 > scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_family-690 419 786 > ##sequence-region scf_86740 1 4293 > scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_family-262 1 25 > scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_family-262 2 60 > scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_family-262 13 60 > ##sequence-region scf_86782 1 8564 > scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_family-480 1027 1129 > ##sequence-region scf_86808 1 4495 > scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family-690 1 969 > scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 74 > ##sequence-region scf_86815 1 4139 > scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-262 825 918 > scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_family-262 526 4459 > ##sequence-region scf_86823 1 2528 > scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 > scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 > scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 178 > scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 35 > scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 45 > scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 41 > scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 96 > scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 40 > scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_family-262 881 920 > ##sequence-region scf_86857 1 2778 > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Fri Oct 19 08:00:12 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 19 Oct 2012 10:00:12 -0400 Subject: [maker-devel] Format of repeat_gff gff3 file In-Reply-To: Message-ID: This command line should add IDs to the end of the GFF3 to let it pass through without the error message. cat file.gff | perl -ane '$id; if(!/^\#/){@F = split(/\t/, $_); chomp $F[-1];$id++; $F[-1] .= "\;ID=$id"; $_ = join("\t", @F)."\n"} print $_' MAKER will use the GFF3 first if provided, then run the species specific library, and then any model organism specified. So if there is overlap and one must be excluded, then they will be kept in that same order of precedence. Thanks, Carson On 12-10-16 12:58 PM, "Mikael Brandstr?m Durling" wrote: >Hi, > >I would like to mask my fungal genome from two different sources (ie. >repbase and an inhouse repeat library). However, I suppose the that if I >supply a library as rmlib in maker_opts, it will be mutually exclusive to >the model_org option, in the same way as -spec and -lib options to >RepeatMasker (I hope I am wrong here...)). To circumvent this, I give the >model_org option as fungi, and would like to provide maker with >additional masking as a gff file. I tried by running RepeatMasker with my >inhouse library, and then used rmOutToGFF3.pl from the RepeatMasker >package to obtain a gff3 file. This file was supplied to maker as rm_gff >(see below for a sample from the file). The run fail with backtraces like >the one paseted below. How should this gff file be formatted for maker to >understand it? I see that in maker produced gff files, there are >additional information found in the id of the hits. Is this required? > >Maybe it's easier to modify maker to make two rounds of RepeatMasker >calls if both model_org and rmlib are specified? > >Thanks for any input, >Mikael > > >------------- EXCEPTION: Bio::Root::Exception ------------- >MSG: Must have defined a valid name for Hit >STACK: Error::throw >STACK: Bio::Root::Root::throw >/opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Root/Root.pm:472 >STACK: Bio::Search::Hit::GenericHit::new >/opt/sw/bioperl/2.1.8/lib/site_perl/5.16.1/Bio/Search/Hit/GenericHit.pm:14 >9 >STACK: Bio::Search::Hit::PhatHit::Base::new >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Bio/Search/Hit/PhatHit/Base.pm:127 >STACK: Bio::Search::Hit::PhatHit::gff3::new >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Bio/Search/Hit/PhatHit/gff3.pm:23 >STACK: GFFDB::_load_hits >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/GFFDB.pm:1026 >STACK: GFFDB::phathits_on_chunk >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/GFFDB.pm:651 >STACK: Process::MpiChunk::_go >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Process/MpiChunk.pm:752 >STACK: Process::MpiChunk::run >/net/gridnas4/volume4/proj1/mykopat-gbrowse/software/maker/2.26/bin/../lib >/Process/MpiChunk.pm:331 >STACK: main::node_thread >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker:1307 >STACK: threads::new >/proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16. >1/x86_64-linux/forks.pm:799 >STACK: /proj/mykopat-gbrowse/software/maker/2.26/bin//maker:803 >----------------------------------------------------------- >Cannot restore overloading on HASH(0x2580810) (package >Bio::Root::Exception) (even after a "require Bio::Root::Exception;") at >/opt/sw/bioperl/2.1.8/lib/5.16.1/x86_64-linux/Storable.pm line 417, at >/proj/mykopat-gbrowse/software/maker/2.26/perl_modules/lib/site_perl/5.16. >1/x86_64-linux/forks.pm line 2256. >Compilation failed in require at >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. >BEGIN failed--compilation aborted at >/proj/mykopat-gbrowse/software/maker/2.26/bin//maker line 11. >Perl exited with active threads: > 1 running and unjoined > 0 finished and unjoined > 0 running and detached >ERROR: Could not open >'/net/gridnas4/volume4/proj1/mykopat-gbrowse/genomes/CrosV1/CrosV1.maker.o >utput/CrosV1_datastore/05/2B/scf_55419//theVoid.scf_55419/scf_55419.0.fung >i.rb.out' >ERROR: Failed while doing repeat masking >ERROR: Chunk failed at level:0, tier_type:1 >FAILED CONTIG:scf_55419 > > >The gff3-file looks like this: >##gff-version 3 >##sequence-region scf_42697 1 3949 >scf_42697 RepeatMasker dispersed_repeat 186 256 22 + . Target=AT_rich 1 71 >scf_42697 RepeatMasker dispersed_repeat 351 378 28 + . Target=AT_rich 1 28 >scf_42697 RepeatMasker dispersed_repeat 560 602 22 + . Target=AT_rich 1 43 >##sequence-region scf_82496 1 2757 >scf_82496 RepeatMasker dispersed_repeat 1 2385 13046 + . Target=rnd-4_fami >ly-1046 2478 4915 >##sequence-region scf_82727 1 4159 >scf_82727 RepeatMasker dispersed_repeat 212 240 29 + . Target=AT_rich 1 29 >scf_82727 RepeatMasker dispersed_repeat 3974 3996 23 + . Target=AT_rich 1 >23 >scf_82727 RepeatMasker dispersed_repeat 4124 4159 264 - . Target=rnd-4_fam >ily-64 15 50 >##sequence-region scf_82785 1 4084 >scf_82785 RepeatMasker dispersed_repeat 2166 2189 24 + . Target=AT_rich 1 >24 >scf_82785 RepeatMasker dispersed_repeat 3498 3865 660 + . Target=rnd-4_fam >ily-690 419 786 >##sequence-region scf_86740 1 4293 >scf_86740 RepeatMasker dispersed_repeat 290 313 369 + . Target=rnd-4_famil >y-262 1 25 >scf_86740 RepeatMasker dispersed_repeat 314 371 270 + . Target=rnd-4_famil >y-262 2 60 >scf_86740 RepeatMasker dispersed_repeat 359 406 309 - . Target=rnd-4_famil >y-262 13 60 >##sequence-region scf_86782 1 8564 >scf_86782 RepeatMasker dispersed_repeat 6987 7085 326 - . Target=rnd-4_fam >ily-480 1027 1129 >##sequence-region scf_86808 1 4495 >scf_86808 RepeatMasker dispersed_repeat 6 974 4027 - . Target=rnd-4_family >-690 1 969 >scf_86808 RepeatMasker dispersed_repeat 4224 4294 216 + . Target=T-rich 5 >74 >##sequence-region scf_86815 1 4139 >scf_86815 RepeatMasker dispersed_repeat 1 94 645 + . Target=rnd-4_family-2 >62 825 918 >scf_86815 RepeatMasker dispersed_repeat 137 4139 27862 + . Target=rnd-4_fa >mily-262 526 4459 >##sequence-region scf_86823 1 2528 >scf_86823 RepeatMasker dispersed_repeat 82 266 205 + . Target=A-rich 1 173 >scf_86823 RepeatMasker dispersed_repeat 564 641 29 + . Target=AT_rich 1 78 >scf_86823 RepeatMasker dispersed_repeat 1168 1347 218 + . Target=A-rich 2 >178 >scf_86823 RepeatMasker dispersed_repeat 1352 1386 28 + . Target=AT_rich 1 >35 >scf_86823 RepeatMasker dispersed_repeat 1698 1742 38 + . Target=AT_rich 1 >45 >scf_86823 RepeatMasker dispersed_repeat 2087 2127 20 + . Target=AT_rich 1 >41 >scf_86823 RepeatMasker dispersed_repeat 2301 2396 26 + . Target=AT_rich 1 >96 >scf_86823 RepeatMasker dispersed_repeat 2433 2472 26 + . Target=AT_rich 1 >40 >scf_86823 RepeatMasker dispersed_repeat 2489 2528 225 - . Target=rnd-4_fam >ily-262 881 920 >##sequence-region scf_86857 1 2778 > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From parulk at caltech.edu Mon Oct 22 15:46:18 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Mon, 22 Oct 2012 14:46:18 -0700 (PDT) Subject: [maker-devel] gff3_preds2models script Message-ID: <3207.131.215.15.234.1350942378.squirrel@webmail.caltech.edu> Hello, I want to add gene structure(gene/mRNA/exon) to gff3 file. I am using gff3_preds2models for this purpose. However I get following error **WARNING: No top level feature found for ID WHL22.100252 I have attached the sample input gff3 file and the list of ids Thanks and regards, Parul Kudtarkar -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org -------------- next part -------------- A non-text attachment was scrubbed... Name: sample.gff3 Type: application/octet-stream Size: 1874 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: list Type: application/octet-stream Size: 77 bytes Desc: not available URL: From fourie.joubert at up.ac.za Wed Oct 24 01:07:13 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 09:07:13 +0200 Subject: [maker-devel] Segfault during build_fasta_index Message-ID: <508793A1.1000704@up.ac.za> Hi I am getting a segfault in maker somewhere during the build_fasta_index step. I suspect it is happening after some updates I made to bioperl. Has anyone else experienced something similar, and is there a recommended bioperl version? Thanks and regards! Fourie -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 06:28:10 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 08:28:10 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <508793A1.1000704@up.ac.za> Message-ID: Yes. Don't use the live version of BioPerl. Use the CPAN version. The live version has a bug. Thanks, Carson On 12-10-24 3:07 AM, "Fourie Joubert" wrote: >Hi > >I am getting a segfault in maker somewhere during the build_fasta_index >step. > >I suspect it is happening after some updates I made to bioperl. > >Has anyone else experienced something similar, and is there a >recommended bioperl version? > >Thanks and regards! > >Fourie > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > > >_______________________________________________ >maker-devel mailing list >maker-devel at box290.bluehost.com >http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From fourie.joubert at up.ac.za Wed Oct 24 08:05:12 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:05:12 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087F598.20303@up.ac.za> Hi I have installed from CPAN (C/CJ/CJFIELDS/BioPerl-1.6.901.tar.gz), but the error persists... Does anyone know if the versions in CPAN may have been replaced recently? Best regards! Foorie On 10/24/2012 02:28 PM, Carson Holt wrote: > Yes. Don't use the live version of BioPerl. Use the CPAN version. The > live version has a bug. > > Thanks, > Carson > > > On 12-10-24 3:07 AM, "Fourie Joubert" wrote: > >> Hi >> >> I am getting a segfault in maker somewhere during the build_fasta_index >> step. >> >> I suspect it is happening after some updates I made to bioperl. >> >> Has anyone else experienced something similar, and is there a >> recommended bioperl version? >> >> Thanks and regards! >> >> Fourie >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:10:32 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:10:32 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087F598.20303@up.ac.za> Message-ID: Run this command --> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' What does it print? --Carson On 12-10-24 10:05 AM, "Fourie Joubert" wrote: >Hi > >I have installed from CPAN (C/CJ/CJFIELDS/BioPerl-1.6.901.tar.gz), but >the error persists... > >Does anyone know if the versions in CPAN may have been replaced recently? > >Best regards! > >Foorie > > > > > > >On 10/24/2012 02:28 PM, Carson Holt wrote: >> Yes. Don't use the live version of BioPerl. Use the CPAN version. The >> live version has a bug. >> >> Thanks, >> Carson >> >> >> On 12-10-24 3:07 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> I am getting a segfault in maker somewhere during the build_fasta_index >>> step. >>> >>> I suspect it is happening after some updates I made to bioperl. >>> >>> Has anyone else experienced something similar, and is there a >>> recommended bioperl version? >>> >>> Thanks and regards! >>> >>> Fourie >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:24:43 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:24:43 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FA2B.5020702@up.ac.za> Hi 1.006901 Regards! Fourie On 10/24/2012 04:10 PM, Carson Holt wrote: > perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:26:41 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:26:41 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FA2B.5020702@up.ac.za> Message-ID: Try completely deleting the mpi_blastdb directory under the MAKER output folder before restarting. Perhaps also try reinstalling DB_File via CPAN. --Carson On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >Hi > >1.006901 > >Regards! > >Fourie > > > >On 10/24/2012 04:10 PM, Carson Holt wrote: >> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:30:15 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:30:15 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FB77.7060802@up.ac.za> Hi Darn - did both, but unfortunately still segfaults... Best regards! Fourie On 10/24/2012 04:26 PM, Carson Holt wrote: > Try completely deleting the mpi_blastdb directory under the MAKER output > folder before restarting. Perhaps also try reinstalling DB_File via CPAN. > > --Carson > > On 12-10-24 10:24 AM, "Fourie Joubert" wrote: > >> Hi >> >> 1.006901 >> >> Regards! >> >> Fourie >> >> >> >> On 10/24/2012 04:10 PM, Carson Holt wrote: >>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:33:07 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:33:07 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FB77.7060802@up.ac.za> Message-ID: I'm going to write you a test script that just creates a fasta index to see if it seg faults. I'll send it to you later. For now could you run maker with the --debug flag set, apture the STDERR and send it to me. Thanks, Carson On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >Hi > >Darn - did both, but unfortunately still segfaults... > >Best regards! > >Fourie > > >On 10/24/2012 04:26 PM, Carson Holt wrote: >> Try completely deleting the mpi_blastdb directory under the MAKER output >> folder before restarting. Perhaps also try reinstalling DB_File via >>CPAN. >> >> --Carson >> >> On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> 1.006901 >>> >>> Regards! >>> >>> Fourie >>> >>> >>> >>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 08:41:15 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 16:41:15 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5087FE0B.7030206@up.ac.za> Hi Really weird - as soon as I add the debug flag - no more segfault... Best regards! Fourie On 10/24/2012 04:33 PM, Carson Holt wrote: > I'm going to write you a test script that just creates a fasta index to > see if it seg faults. I'll send it to you later. For now could you run > maker with the --debug flag set, apture the STDERR and send it to me. > > Thanks, > Carson > > > On 12-10-24 10:30 AM, "Fourie Joubert" wrote: > >> Hi >> >> Darn - did both, but unfortunately still segfaults... >> >> Best regards! >> >> Fourie >> >> >> On 10/24/2012 04:26 PM, Carson Holt wrote: >>> Try completely deleting the mpi_blastdb directory under the MAKER output >>> folder before restarting. Perhaps also try reinstalling DB_File via >>> CPAN. >>> >>> --Carson >>> >>> On 12-10-24 10:24 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> 1.006901 >>>> >>>> Regards! >>>> >>>> Fourie >>>> >>>> >>>> >>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>> perl -MBio::Root::Version -e 'print "$Bio::Root::Version::VERSION\n"' >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 08:56:51 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 10:56:51 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5087FE0B.7030206@up.ac.za> Message-ID: Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA variable. It's set by both BioPerl and a couple of MAKER modules. If it gets set twice, sometimes you get bugs. Try upgrading to MAKER 2.26 I have a statement in that that will make sure it won't get set wrong. Otherwise I can give you instructions on editing the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER modules. Thanks, Carson On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >Hi > >Really weird - as soon as I add the debug flag - no more segfault... > >Best regards! > >Fourie > > > > >On 10/24/2012 04:33 PM, Carson Holt wrote: >> I'm going to write you a test script that just creates a fasta index to >> see if it seg faults. I'll send it to you later. For now could you run >> maker with the --debug flag set, apture the STDERR and send it to me. >> >> Thanks, >> Carson >> >> >> On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> Darn - did both, but unfortunately still segfaults... >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>output >>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>> CPAN. >>>> >>>> --Carson >>>> >>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> 1.006901 >>>>> >>>>> Regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> >>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>> perl -MBio::Root::Version -e 'print >>>>>>"$Bio::Root::Version::VERSION\n"' >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Wed Oct 24 09:19:08 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Wed, 24 Oct 2012 17:19:08 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <508806EC.6050600@up.ac.za> Hi Carson We are indeed running 2.26. Instructions on editing the statements would be great. Many thanks for all the help, and sorry for the bother! Best regards! Fourie On 10/24/2012 04:56 PM, Carson Holt wrote: > Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA > variable. It's set by both BioPerl and a couple of MAKER modules. If it > gets set twice, sometimes you get bugs. > > Try upgrading to MAKER 2.26 I have a statement in that that will make sure > it won't get set wrong. Otherwise I can give you instructions on editing > the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER > modules. > > Thanks, > Carson > > > On 12-10-24 10:41 AM, "Fourie Joubert" wrote: > >> Hi >> >> Really weird - as soon as I add the debug flag - no more segfault... >> >> Best regards! >> >> Fourie >> >> >> >> >> On 10/24/2012 04:33 PM, Carson Holt wrote: >>> I'm going to write you a test script that just creates a fasta index to >>> see if it seg faults. I'll send it to you later. For now could you run >>> maker with the --debug flag set, apture the STDERR and send it to me. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-24 10:30 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> Darn - did both, but unfortunately still segfaults... >>>> >>>> Best regards! >>>> >>>> Fourie >>>> >>>> >>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>> output >>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>> CPAN. >>>>> >>>>> --Carson >>>>> >>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> 1.006901 >>>>>> >>>>>> Regards! >>>>>> >>>>>> Fourie >>>>>> >>>>>> >>>>>> >>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>> perl -MBio::Root::Version -e 'print >>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>> -- >>>>>> -------------- >>>>>> Prof Fourie Joubert >>>>>> Bioinformatics and Computational Biology Unit >>>>>> Department of Biochemistry >>>>>> University of Pretoria >>>>>> fourie.joubert at up.ac.za >>>>>> http://www.bi.up.ac.za >>>>>> Tel. +27-12-420-5825 >>>>>> Fax. +27-12-420-5800 >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> -- >>>>>> - >>>>>> This message and attachments are subject to a disclaimer. Please >>>>>> refer >>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>> details. >>>>>> >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Wed Oct 24 09:50:03 2012 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 24 Oct 2012 11:50:03 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <508806EC.6050600@up.ac.za> Message-ID: Find these lines in these files in MAKER .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); And the Bio::DB::Fasta file in BioPerl. Use this command to identify the location of Bio::DB::Fasta --> perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File NDBM_File SDBM_File); Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); Use an editor like emacs or vi to make the changes. Thanks, Carson On 12-10-24 11:19 AM, "Fourie Joubert" wrote: >Hi Carson > >We are indeed running 2.26. > >Instructions on editing the statements would be great. > >Many thanks for all the help, and sorry for the bother! > >Best regards! > >Fourie > > > > >On 10/24/2012 04:56 PM, Carson Holt wrote: >> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >> variable. It's set by both BioPerl and a couple of MAKER modules. If it >> gets set twice, sometimes you get bugs. >> >> Try upgrading to MAKER 2.26 I have a statement in that that will make >>sure >> it won't get set wrong. Otherwise I can give you instructions on >>editing >> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >> modules. >> >> Thanks, >> Carson >> >> >> On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >> >>> Hi >>> >>> Really weird - as soon as I add the debug flag - no more segfault... >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> >>> >>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>> I'm going to write you a test script that just creates a fasta index >>>>to >>>> see if it seg faults. I'll send it to you later. For now could you >>>>run >>>> maker with the --debug flag set, apture the STDERR and send it to me. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> Darn - did both, but unfortunately still segfaults... >>>>> >>>>> Best regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>> output >>>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>>> CPAN. >>>>>> >>>>>> --Carson >>>>>> >>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> 1.006901 >>>>>>> >>>>>>> Regards! >>>>>>> >>>>>>> Fourie >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>> -- >>>>>>> -------------- >>>>>>> Prof Fourie Joubert >>>>>>> Bioinformatics and Computational Biology Unit >>>>>>> Department of Biochemistry >>>>>>> University of Pretoria >>>>>>> fourie.joubert at up.ac.za >>>>>>> http://www.bi.up.ac.za >>>>>>> Tel. +27-12-420-5825 >>>>>>> Fax. +27-12-420-5800 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>> -- >>>>>>> - >>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>> refer >>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>> details. >>>>>>> >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From fourie.joubert at up.ac.za Thu Oct 25 00:51:05 2012 From: fourie.joubert at up.ac.za (Fourie Joubert) Date: Thu, 25 Oct 2012 08:51:05 +0200 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: References: Message-ID: <5088E159.3050502@up.ac.za> Hi Carson All happy now - much appreciated! Best regards! Fourie On 10/24/2012 05:50 PM, Carson Holt wrote: > Find these lines in these files in MAKER > > .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > > > And the Bio::DB::Fasta file in BioPerl. Use this command to identify the > location of Bio::DB::Fasta --> > perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' > > .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File > NDBM_File SDBM_File); > > > Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); > > Use an editor like emacs or vi to make the changes. > > Thanks, > Carson > > > > On 12-10-24 11:19 AM, "Fourie Joubert" wrote: > >> Hi Carson >> >> We are indeed running 2.26. >> >> Instructions on editing the statements would be great. >> >> Many thanks for all the help, and sorry for the bother! >> >> Best regards! >> >> Fourie >> >> >> >> >> On 10/24/2012 04:56 PM, Carson Holt wrote: >>> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >>> variable. It's set by both BioPerl and a couple of MAKER modules. If it >>> gets set twice, sometimes you get bugs. >>> >>> Try upgrading to MAKER 2.26 I have a statement in that that will make >>> sure >>> it won't get set wrong. Otherwise I can give you instructions on >>> editing >>> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >>> modules. >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-24 10:41 AM, "Fourie Joubert" wrote: >>> >>>> Hi >>>> >>>> Really weird - as soon as I add the debug flag - no more segfault... >>>> >>>> Best regards! >>>> >>>> Fourie >>>> >>>> >>>> >>>> >>>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>>> I'm going to write you a test script that just creates a fasta index >>>>> to >>>>> see if it seg faults. I'll send it to you later. For now could you >>>>> run >>>>> maker with the --debug flag set, apture the STDERR and send it to me. >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>> wrote: >>>>> >>>>>> Hi >>>>>> >>>>>> Darn - did both, but unfortunately still segfaults... >>>>>> >>>>>> Best regards! >>>>>> >>>>>> Fourie >>>>>> >>>>>> >>>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>>> output >>>>>>> folder before restarting. Perhaps also try reinstalling DB_File via >>>>>>> CPAN. >>>>>>> >>>>>>> --Carson >>>>>>> >>>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>>> wrote: >>>>>>> >>>>>>>> Hi >>>>>>>> >>>>>>>> 1.006901 >>>>>>>> >>>>>>>> Regards! >>>>>>>> >>>>>>>> Fourie >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>>> -- >>>>>>>> -------------- >>>>>>>> Prof Fourie Joubert >>>>>>>> Bioinformatics and Computational Biology Unit >>>>>>>> Department of Biochemistry >>>>>>>> University of Pretoria >>>>>>>> fourie.joubert at up.ac.za >>>>>>>> http://www.bi.up.ac.za >>>>>>>> Tel. +27-12-420-5825 >>>>>>>> Fax. +27-12-420-5800 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -------------------------------------------------------------------- >>>>>>>> -- >>>>>>>> -- >>>>>>>> - >>>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>>> refer >>>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>>> details. >>>>>>>> >>>>>> -- >>>>>> -------------- >>>>>> Prof Fourie Joubert >>>>>> Bioinformatics and Computational Biology Unit >>>>>> Department of Biochemistry >>>>>> University of Pretoria >>>>>> fourie.joubert at up.ac.za >>>>>> http://www.bi.up.ac.za >>>>>> Tel. +27-12-420-5825 >>>>>> Fax. +27-12-420-5800 >>>>>> >>>>>> >>>>>> >>>>>> ---------------------------------------------------------------------- >>>>>> -- >>>>>> - >>>>>> This message and attachments are subject to a disclaimer. Please >>>>>> refer >>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>> details. >>>>>> >>>> -- >>>> -------------- >>>> Prof Fourie Joubert >>>> Bioinformatics and Computational Biology Unit >>>> Department of Biochemistry >>>> University of Pretoria >>>> fourie.joubert at up.ac.za >>>> http://www.bi.up.ac.za >>>> Tel. +27-12-420-5825 >>>> Fax. +27-12-420-5800 >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> - >>>> This message and attachments are subject to a disclaimer. Please refer >>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>> details. >>>> >> >> -- >> -------------- >> Prof Fourie Joubert >> Bioinformatics and Computational Biology Unit >> Department of Biochemistry >> University of Pretoria >> fourie.joubert at up.ac.za >> http://www.bi.up.ac.za >> Tel. +27-12-420-5825 >> Fax. +27-12-420-5800 >> >> ------------------------------------------------------------------------- >> This message and attachments are subject to a disclaimer. Please refer >> to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. >> > -- -------------- Prof Fourie Joubert Bioinformatics and Computational Biology Unit Department of Biochemistry University of Pretoria fourie.joubert at up.ac.za http://www.bi.up.ac.za Tel. +27-12-420-5825 Fax. +27-12-420-5800 ------------------------------------------------------------------------- This message and attachments are subject to a disclaimer. Please refer to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. From carsonhh at gmail.com Thu Oct 25 06:57:31 2012 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 25 Oct 2012 08:57:31 -0400 Subject: [maker-devel] Segfault during build_fasta_index In-Reply-To: <5088E159.3050502@up.ac.za> Message-ID: Glad I could help. Thanks, Carson On 12-10-25 2:51 AM, "Fourie Joubert" wrote: >Hi Carson > >All happy now - much appreciated! > >Best regards! > >Fourie > >On 10/24/2012 05:50 PM, Carson Holt wrote: >> Find these lines in these files in MAKER >> >> .../maker/lib/ds_utility.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> .../maker/lib/runlog.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> >> >> And the Bio::DB::Fasta file in BioPerl. Use this command to identify >>the >> location of Bio::DB::Fasta --> >> perl -MBio::DB::Fasta -e 'print $INC{"Bio/DB/Fasta.pm"}."\n"' >> >> .../Bio/DB/Fasta.pm: @AnyDBM_File::ISA = qw(DB_File GDBM_File >> NDBM_File SDBM_File); >> >> >> Change them all to this --> @AnyDBM_File::ISA = qw(DB_File); >> >> Use an editor like emacs or vi to make the changes. >> >> Thanks, >> Carson >> >> >> >> On 12-10-24 11:19 AM, "Fourie Joubert" wrote: >> >>> Hi Carson >>> >>> We are indeed running 2.26. >>> >>> Instructions on editing the statements would be great. >>> >>> Many thanks for all the help, and sorry for the bother! >>> >>> Best regards! >>> >>> Fourie >>> >>> >>> >>> >>> On 10/24/2012 04:56 PM, Carson Holt wrote: >>>> Yes. I've seen that before. It's an issue with the @AnyDBM_File::ISA >>>> variable. It's set by both BioPerl and a couple of MAKER modules. If >>>>it >>>> gets set twice, sometimes you get bugs. >>>> >>>> Try upgrading to MAKER 2.26 I have a statement in that that will make >>>> sure >>>> it won't get set wrong. Otherwise I can give you instructions on >>>> editing >>>> the BEGIN statements in both BioPerl's Bio::DB::Fasta and the 2 MAKER >>>> modules. >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> On 12-10-24 10:41 AM, "Fourie Joubert" >>>>wrote: >>>> >>>>> Hi >>>>> >>>>> Really weird - as soon as I add the debug flag - no more segfault... >>>>> >>>>> Best regards! >>>>> >>>>> Fourie >>>>> >>>>> >>>>> >>>>> >>>>> On 10/24/2012 04:33 PM, Carson Holt wrote: >>>>>> I'm going to write you a test script that just creates a fasta index >>>>>> to >>>>>> see if it seg faults. I'll send it to you later. For now could you >>>>>> run >>>>>> maker with the --debug flag set, apture the STDERR and send it to >>>>>>me. >>>>>> >>>>>> Thanks, >>>>>> Carson >>>>>> >>>>>> >>>>>> On 12-10-24 10:30 AM, "Fourie Joubert" >>>>>> wrote: >>>>>> >>>>>>> Hi >>>>>>> >>>>>>> Darn - did both, but unfortunately still segfaults... >>>>>>> >>>>>>> Best regards! >>>>>>> >>>>>>> Fourie >>>>>>> >>>>>>> >>>>>>> On 10/24/2012 04:26 PM, Carson Holt wrote: >>>>>>>> Try completely deleting the mpi_blastdb directory under the MAKER >>>>>>>> output >>>>>>>> folder before restarting. Perhaps also try reinstalling DB_File >>>>>>>>via >>>>>>>> CPAN. >>>>>>>> >>>>>>>> --Carson >>>>>>>> >>>>>>>> On 12-10-24 10:24 AM, "Fourie Joubert" >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi >>>>>>>>> >>>>>>>>> 1.006901 >>>>>>>>> >>>>>>>>> Regards! >>>>>>>>> >>>>>>>>> Fourie >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 10/24/2012 04:10 PM, Carson Holt wrote: >>>>>>>>>> perl -MBio::Root::Version -e 'print >>>>>>>>>> "$Bio::Root::Version::VERSION\n"' >>>>>>>>> -- >>>>>>>>> -------------- >>>>>>>>> Prof Fourie Joubert >>>>>>>>> Bioinformatics and Computational Biology Unit >>>>>>>>> Department of Biochemistry >>>>>>>>> University of Pretoria >>>>>>>>> fourie.joubert at up.ac.za >>>>>>>>> http://www.bi.up.ac.za >>>>>>>>> Tel. +27-12-420-5825 >>>>>>>>> Fax. +27-12-420-5800 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>------------------------------------------------------------------ >>>>>>>>>-- >>>>>>>>> -- >>>>>>>>> -- >>>>>>>>> - >>>>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>>>> refer >>>>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>>>> details. >>>>>>>>> >>>>>>> -- >>>>>>> -------------- >>>>>>> Prof Fourie Joubert >>>>>>> Bioinformatics and Computational Biology Unit >>>>>>> Department of Biochemistry >>>>>>> University of Pretoria >>>>>>> fourie.joubert at up.ac.za >>>>>>> http://www.bi.up.ac.za >>>>>>> Tel. +27-12-420-5825 >>>>>>> Fax. +27-12-420-5800 >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>-------------------------------------------------------------------- >>>>>>>-- >>>>>>> -- >>>>>>> - >>>>>>> This message and attachments are subject to a disclaimer. Please >>>>>>> refer >>>>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>>>> details. >>>>>>> >>>>> -- >>>>> -------------- >>>>> Prof Fourie Joubert >>>>> Bioinformatics and Computational Biology Unit >>>>> Department of Biochemistry >>>>> University of Pretoria >>>>> fourie.joubert at up.ac.za >>>>> http://www.bi.up.ac.za >>>>> Tel. +27-12-420-5825 >>>>> Fax. +27-12-420-5800 >>>>> >>>>> >>>>> >>>>>---------------------------------------------------------------------- >>>>>-- >>>>> - >>>>> This message and attachments are subject to a disclaimer. Please >>>>>refer >>>>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>>> details. >>>>> >>> >>> -- >>> -------------- >>> Prof Fourie Joubert >>> Bioinformatics and Computational Biology Unit >>> Department of Biochemistry >>> University of Pretoria >>> fourie.joubert at up.ac.za >>> http://www.bi.up.ac.za >>> Tel. +27-12-420-5825 >>> Fax. +27-12-420-5800 >>> >>> >>>------------------------------------------------------------------------ >>>- >>> This message and attachments are subject to a disclaimer. Please refer >>> to www.it.up.ac.za/documentation/governance/disclaimer/ for full >>>details. >>> >> > > >-- >-------------- >Prof Fourie Joubert >Bioinformatics and Computational Biology Unit >Department of Biochemistry >University of Pretoria >fourie.joubert at up.ac.za >http://www.bi.up.ac.za >Tel. +27-12-420-5825 >Fax. +27-12-420-5800 > >------------------------------------------------------------------------- >This message and attachments are subject to a disclaimer. Please refer >to www.it.up.ac.za/documentation/governance/disclaimer/ for full details. > From daniel.standage at gmail.com Thu Oct 25 13:30:44 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Thu, 25 Oct 2012 15:30:44 -0400 Subject: [maker-devel] Strange error at blastn step Message-ID: Greetings! I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. #--------- command -------------# Widget::blastn: /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn #-------------------------------# deleted:0 hits ERROR: Could not obtain lock to format database FATAL ERROR ERROR: Failed while doing blastn of ESTs!! ERROR: Chunk failed at level 8 !! FAILED CONTIG:scaffold_866 Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 11:52:43 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 13:52:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx #-------------------------------# deleted:-10 hits formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx #-------------------------------# deleted:-6 hits WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. stop here:comp59088_c1_seq7 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:scaffold_0 --Next Contig-- #--------------------------------------------------------------------- Now starting the contig!! SeqID: scaffold_1 Length: 5805686 #--------------------------------------------------------------------- My first thought based on the message is that *blastdbcmd* could not find the sequence in the database. I verified this was the case--I could not extract sequence *comp59088_c1_seq7* from the database Maker had created under /tmp. However, after removing the index files and re-running * makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully extracted the sequence. I was initially happy with this finding, but upon closer inspection it looks like Maker does not use *blastdbcmd* to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a > pretty short contig (about 3kb). However, there seems to be a hiccup during > the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db > /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 > -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 > -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 > -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp > 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis > -out > /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff > old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > > Several blastn steps appeared to have completed successfully to this one > failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From myandell at genetics.utah.edu Fri Oct 26 12:02:29 2012 From: myandell at genetics.utah.edu (Mark Yandell) Date: Fri, 26 Oct 2012 18:02:29 +0000 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: , Message-ID: <7A60AB257EFF2B48B1F4C814817EA05331BA6EC8@mxb1.hg.genetics.utah.edu> Hi Daniel, I think its your fasta-file '> comp59088_c1_seq' note the space between the chevron and the id. This isn't allowed by the fasta format. cheers, --mark Mark Yandell Professor of Human Genetics H.A. & Edna Benning Presidential Endowed Chair Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 ph:801-587-7707 ________________________________________ From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Daniel Standage [daniel.standage at gmail.com] Sent: Friday, October 26, 2012 11:52 AM To: Maker Mailing List Subject: Re: [maker-devel] Strange error at blastn step I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx #-------------------------------# deleted:-10 hits formating database... #--------- command -------------# Widget::formater: /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 #-------------------------------# running blast search. #--------- command -------------# Widget::blastx: /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx #-------------------------------# deleted:-6 hits WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. stop here:comp59088_c1_seq7 ERROR: Fasta index error FATAL ERROR ERROR: Failed while polishig ESTs!! ERROR: Chunk failed at level 14 !! FAILED CONTIG:scaffold_0 --Next Contig-- #--------------------------------------------------------------------- Now starting the contig!! SeqID: scaffold_1 Length: 5805686 #--------------------------------------------------------------------- My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage > wrote: Greetings! I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. #--------- command -------------# Widget::blastn: /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn #-------------------------------# deleted:0 hits ERROR: Could not obtain lock to format database FATAL ERROR ERROR: Failed while doing blastn of ESTs!! ERROR: Chunk failed at level 8 !! FAILED CONTIG:scaffold_866 Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? Thanks! -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University From carsonhh at gmail.com Fri Oct 26 12:09:39 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:09:39 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Check to see where /tmp is located? Some clusters have it set up as a tmpfs directory and I have had problems with fasta indexes running from tmpfs mounts in the past. --Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:05 PM To: Carson Holt Subject: Re: [maker-devel] Strange error at blastn step The maker working directory is in a cluster environment with shared scratch space (I'm guessing NFS-mounted). I didn't change the temp directory setting, so it should be the local default (/tmp). I'll give the dev version a shot. Thanks. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > Could you try this development version and tell me if the error still happens? > > Use this command to download --> > <> > > Username: <> > Password: <> > > Are you running in an NFS mounted directory or are you resetting TMP to a > different location? > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 1:52 PM > To: Maker Mailing List > Subject: Re: [maker-devel] Strange error at blastn step > > I have since installed Maker on a different machine and tried it out. The test > run completed successfully, but as I commenced with the full genome > annotation, I have noticed the following error popping up frequently. > >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions >> 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes >> -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker >> .output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0 >> .0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi. >> 10.8.blastx >> #-------------------------------# >> deleted:-10 hits >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions >> 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes >> -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker >> .output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0 >> .0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi. >> 10.9.blastx >> #-------------------------------# >> deleted:-6 hits >> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >> stop here:comp59088_c1_seq7 >> ERROR: Fasta index error >> >> FATAL ERROR >> ERROR: Failed while polishig ESTs!! >> >> ERROR: Chunk failed at level 14 >> !! >> FAILED CONTIG:scaffold_0 >> >> >> >> >> --Next Contig-- >> >> #--------------------------------------------------------------------- >> Now starting the contig!! >> SeqID: scaffold_1 >> Length: 5805686 >> #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the > sequence in the database. I verified this was the case--I could not extract > sequence comp59088_c1_seq7 from the database Maker had created under /tmp. > However, after removing the index files and re-running makeblastdb with the > -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks > like Maker does not use blastdbcmd to extract sequences, but rather its own > internal code. Therefore I'm still not sure where the problem is and how I > might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage > wrote: >> Greetings! >> >> I am doing a test run of my Maker setup on a new machine, annotating a pretty >> short contig (about 3kb). However, there seems to be a hiccup during the >> blastn stage. This is the terminal message. >> >>> #--------- command -------------# >>> Widget::blastn: >>> /share/home/01854/standage/local/bin/blastn -db >>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta >>> .mpi.10.7 -query >>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments >>> 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 >>> -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 >>> -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out >>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.out >>> put/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.P >>> dom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmom >>> atic%2Efasta.mpi.10.7.blastn >>> #-------------------------------# >>> deleted:0 hits >>> ERROR: Could not obtain lock to format database >>> >>> >>> FATAL ERROR >>> ERROR: Failed while doing blastn of ESTs!! >>> >>> ERROR: Chunk failed at level 8 >>> !! >>> FAILED CONTIG:scaffold_866 >> >> Several blastn steps appeared to have completed successfully to this one >> failing. Any ideas what could be causing this? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> > > _______________________________________________ maker-devel mailing list > maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/mak > er-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:12:38 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:12:38 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: It looks like /tmp is indeed being used: the files I played with were under */tmp/maker_1YQF9o*. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a > tmpfs directory and I have had problems with fasta indexes running from > tmpfs mounts in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared > scratch space (I'm guessing NFS-mounted). I didn't change the temp > directory setting, so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > >> Could you try this development version and tell me if the error still >> happens? >> >> Use this command to download --> >> <> >> >> Username: <> >> Password: <> >> >> Are you running in an NFS mounted directory or are you resetting TMP to a >> different location? >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 1:52 PM >> To: Maker Mailing List >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I have since installed Maker on a different machine and tried it out. The >> test run completed successfully, but as I commenced with the full genome >> annotation, I have noticed the following error popping up frequently. >> >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >> #-------------------------------# >> deleted:-10 hits >> formating database... >> #--------- command -------------# >> Widget::formater: >> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >> #-------------------------------# >> running blast search. >> #--------- command -------------# >> Widget::blastx: >> /N/u/dstandag/Mason/local/bin/blastx -db >> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >> #-------------------------------# >> deleted:-6 hits >> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >> stop here:comp59088_c1_seq7 >> ERROR: Fasta index error >> >> FATAL ERROR >> ERROR: Failed while polishig ESTs!! >> >> ERROR: Chunk failed at level 14 >> !! >> FAILED CONTIG:scaffold_0 >> >> >> >> >> --Next Contig-- >> >> #--------------------------------------------------------------------- >> Now starting the contig!! >> SeqID: scaffold_1 >> Length: 5805686 >> #--------------------------------------------------------------------- >> >> >> My first thought based on the message is that *blastdbcmd* could not >> find the sequence in the database. I verified this was the case--I could >> not extract sequence *comp59088_c1_seq7* from the database Maker had >> created under /tmp. However, after removing the index files and re-running >> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >> extracted the sequence. >> >> I was initially happy with this finding, but upon closer inspection it >> looks like Maker does not use *blastdbcmd* to extract sequences, but >> rather its own internal code. Therefore I'm still not sure where the >> problem is and how I might fix it. Any insights? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >> daniel.standage at gmail.com> wrote: >> >>> Greetings! >>> >>> I am doing a test run of my Maker setup on a new machine, annotating a >>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>> the blastn stage. This is the terminal message. >>> >>> #--------- command -------------# >>> Widget::blastn: >>> /share/home/01854/standage/local/bin/blastn -db >>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>> -out >>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>> #-------------------------------# >>> deleted:0 hits >>> ERROR: Could not obtain lock to format database >>> >>> >>> FATAL ERROR >>> ERROR: Failed while doing blastn of ESTs!! >>> >>> ERROR: Chunk failed at level 8 >>> !! >>> FAILED CONTIG:scaffold_866 >>> >>> >>> Several blastn steps appeared to have completed successfully to this one >>> failing. Any ideas what could be causing this? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:14:29 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:14:29 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: The command 'df /tmp' will tell you whether /tmp is a tmpfs mount Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:12 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step It looks like /tmp is indeed being used: the files I played with were under /tmp/maker_1YQF9o. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a tmpfs > directory and I have had problems with fasta indexes running from tmpfs mounts > in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared scratch > space (I'm guessing NFS-mounted). I didn't change the temp directory setting, > so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >> Could you try this development version and tell me if the error still >> happens? >> >> Use this command to download --> >> <> >> >> Username: <> >> Password: <> >> >> Are you running in an NFS mounted directory or are you resetting TMP to a >> different location? >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 1:52 PM >> To: Maker Mailing List >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I have since installed Maker on a different machine and tried it out. The >> test run completed successfully, but as I commenced with the full genome >> annotation, I have noticed the following error popping up frequently. >> >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.make >>> r.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold >>> _0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.m >>> pi.10.8.blastx >>> #-------------------------------# >>> deleted:-10 hits >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.make >>> r.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold >>> _0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.m >>> pi.10.9.blastx >>> #-------------------------------# >>> deleted:-6 hits >>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>> stop here:comp59088_c1_seq7 >>> ERROR: Fasta index error >>> >>> FATAL ERROR >>> ERROR: Failed while polishig ESTs!! >>> >>> ERROR: Chunk failed at level 14 >>> !! >>> FAILED CONTIG:scaffold_0 >>> >>> >>> >>> >>> --Next Contig-- >>> >>> #--------------------------------------------------------------------- >>> Now starting the contig!! >>> SeqID: scaffold_1 >>> Length: 5805686 >>> #--------------------------------------------------------------------- >> >> My first thought based on the message is that blastdbcmd could not find the >> sequence in the database. I verified this was the case--I could not extract >> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >> However, after removing the index files and re-running makeblastdb with the >> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >> >> I was initially happy with this finding, but upon closer inspection it looks >> like Maker does not use blastdbcmd to extract sequences, but rather its own >> internal code. Therefore I'm still not sure where the problem is and how I >> might fix it. Any insights? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >> wrote: >>> Greetings! >>> >>> I am doing a test run of my Maker setup on a new machine, annotating a >>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>> the blastn stage. This is the terminal message. >>> >>>> #--------- command -------------# >>>> Widget::blastn: >>>> /share/home/01854/standage/local/bin/blastn -db >>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efast >>>> a.mpi.10.7 -query >>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>> -show_gis -out >>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.ou >>>> tput/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0 >>>> .Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrim >>>> momatic%2Efasta.mpi.10.7.blastn >>>> #-------------------------------# >>>> deleted:0 hits >>>> ERROR: Could not obtain lock to format database >>>> >>>> >>>> FATAL ERROR >>>> ERROR: Failed while doing blastn of ESTs!! >>>> >>>> ERROR: Chunk failed at level 8 >>>> !! >>>> FAILED CONTIG:scaffold_866 >>> >>> Several blastn steps appeared to have completed successfully to this one >>> failing. Any ideas what could be causing this? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >> >> _______________________________________________ maker-devel mailing list >> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ma >> ker-devel_yandell-lab.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry.moore at genetics.utah.edu Fri Oct 26 12:19:06 2012 From: barry.moore at genetics.utah.edu (Barry Moore) Date: Fri, 26 Oct 2012 12:19:06 -0600 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> Hi Daniel, What version or revision of MAKER are you running. This sounds like something we were seeing here last night. We traced it as far as what appeared to be soft links in /tmp being set incorrectly. The FastaDB objects had pointers to fasta files in the void for the correct fasta file, but their dir attribute pointed to a /tmp directory in where there were soft links to another (incorrect) fasta file with it's index. It would look appeared that it was looking in the /tmp index (which pointed to an incorrect fasta file) and when it failed to find what it was looking for it would re-index the fasta file in the void (the correct one) and then look again in the index in tmp. Don't know if that helps, but the errors look similar and that's as far as I got with our error here? This was on one of Mike's annotation projects, so I don't know for sure what revision he was running, but I think it was the latest. B On Oct 26, 2012, at 11:52 AM, Daniel Standage wrote: > I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Barry Moore Research Scientist Dept. of Human Genetics University of Utah Salt Lake City, UT 84112 -------------------------------------------- (801) 585-3543 -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:19:07 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:19:07 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Unfortunately, the job is no longer running and as a result I cannot connect to the compute nodes as I could while it was running. On the interactive node, it looks like it's real disk, although it looks like there are some tmpfs mounts. [dstandag at mason src] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 462824304 180235660 259078476 42% /tmp [dstandag at mason src] df Filesystem 1K-blocks Used Available Use% Mounted on login_x86_64 16497564 3077352 13420212 19% / tmpfs 16497564 0 16497564 0% /dev/shm tmpfs 10240 0 10240 0% /var/tmp /dev/sdb2 462824304 180235660 259078476 42% /tmp AFS 9000000 0 9000000 0% /afs bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs ... ... I'll see if I can launch another short job and verify this on the compute nodes. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > The command 'df /tmp' will tell you whether /tmp is a tmpfs mount > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:12 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > It looks like /tmp is indeed being used: the files I played with were > under */tmp/maker_1YQF9o*. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > >> Check to see where /tmp is located? Some clusters have it set up as a >> tmpfs directory and I have had problems with fasta indexes running from >> tmpfs mounts in the past. >> >> --Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:05 PM >> To: Carson Holt >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> The maker working directory is in a cluster environment with shared >> scratch space (I'm guessing NFS-mounted). I didn't change the temp >> directory setting, so it should be the local default (/tmp). >> >> I'll give the dev version a shot. Thanks. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >> >>> Could you try this development version and tell me if the error still >>> happens? >>> >>> Use this command to download --> >>> <> >>> >>> Username: <> >>> Password: <> >>> >>> Are you running in an NFS mounted directory or are you resetting TMP to >>> a different location? >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 1:52 PM >>> To: Maker Mailing List >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> I have since installed Maker on a different machine and tried it out. >>> The test run completed successfully, but as I commenced with the full >>> genome annotation, I have noticed the following error popping up frequently. >>> >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>> #-------------------------------# >>> deleted:-10 hits >>> formating database... >>> #--------- command -------------# >>> Widget::formater: >>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>> #-------------------------------# >>> running blast search. >>> #--------- command -------------# >>> Widget::blastx: >>> /N/u/dstandag/Mason/local/bin/blastx -db >>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>> #-------------------------------# >>> deleted:-6 hits >>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>> stop here:comp59088_c1_seq7 >>> ERROR: Fasta index error >>> >>> FATAL ERROR >>> ERROR: Failed while polishig ESTs!! >>> >>> ERROR: Chunk failed at level 14 >>> !! >>> FAILED CONTIG:scaffold_0 >>> >>> >>> >>> >>> --Next Contig-- >>> >>> #--------------------------------------------------------------------- >>> Now starting the contig!! >>> SeqID: scaffold_1 >>> Length: 5805686 >>> #--------------------------------------------------------------------- >>> >>> >>> My first thought based on the message is that *blastdbcmd* could not >>> find the sequence in the database. I verified this was the case--I could >>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>> created under /tmp. However, after removing the index files and re-running >>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>> extracted the sequence. >>> >>> I was initially happy with this finding, but upon closer inspection it >>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>> rather its own internal code. Therefore I'm still not sure where the >>> problem is and how I might fix it. Any insights? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>> daniel.standage at gmail.com> wrote: >>> >>>> Greetings! >>>> >>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>> the blastn stage. This is the terminal message. >>>> >>>> #--------- command -------------# >>>> Widget::blastn: >>>> /share/home/01854/standage/local/bin/blastn -db >>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>>> -out >>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>> #-------------------------------# >>>> deleted:0 hits >>>> ERROR: Could not obtain lock to format database >>>> >>>> >>>> FATAL ERROR >>>> ERROR: Failed while doing blastn of ESTs!! >>>> >>>> ERROR: Chunk failed at level 8 >>>> !! >>>> FAILED CONTIG:scaffold_866 >>>> >>>> >>>> Several blastn steps appeared to have completed successfully to this >>>> one failing. Any ideas what could be causing this? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:20:55 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:20:55 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> References: <611E35F6-5B03-4A4A-B93A-6A37C1EDEF8B@genetics.utah.edu> Message-ID: Running 2.10. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:19 PM, Barry Moore wrote: > Hi Daniel, > > What version or revision of MAKER are you running. This sounds like > something we were seeing here last night. We traced it as far as what > appeared to be soft links in /tmp being set incorrectly. The FastaDB > objects had pointers to fasta files in the void for the correct fasta file, > but their dir attribute pointed to a /tmp directory in where there were > soft links to another (incorrect) fasta file with it's index. It would > look appeared that it was looking in the /tmp index (which pointed to an > incorrect fasta file) and when it failed to find what it was looking for it > would re-index the fasta file in the void (the correct one) and then look > again in the index in tmp. Don't know if that helps, but the errors look > similar and that's as far as I got with our error here? This was on one of > Mike's annotation projects, so I don't know for sure what revision he was > running, but I think it was the latest. > > B > > On Oct 26, 2012, at 11:52 AM, Daniel Standage wrote: > > I have since installed Maker on a different machine and tried it out. The > test run completed successfully, but as I commenced with the full genome > annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query > /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 > -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 > -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db > /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query > /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 > -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 > -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > > My first thought based on the message is that *blastdbcmd* could not find > the sequence in the database. I verified this was the case--I could not > extract sequence *comp59088_c1_seq7* from the database Maker had created > under /tmp. However, after removing the index files and re-running * > makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully > extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it > looks like Maker does not use *blastdbcmd* to extract sequences, but > rather its own internal code. Therefore I'm still not sure where the > problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < > daniel.standage at gmail.com> wrote: > >> Greetings! >> >> I am doing a test run of my Maker setup on a new machine, annotating a >> pretty short contig (about 3kb). However, there seems to be a hiccup during >> the blastn stage. This is the terminal message. >> >> #--------- command -------------# >> Widget::blastn: >> /share/home/01854/standage/local/bin/blastn -db >> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >> -out >> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >> #-------------------------------# >> deleted:0 hits >> ERROR: Could not obtain lock to format database >> >> >> FATAL ERROR >> ERROR: Failed while doing blastn of ESTs!! >> >> ERROR: Chunk failed at level 8 >> !! >> FAILED CONTIG:scaffold_866 >> >> >> Several blastn steps appeared to have completed successfully to this one >> failing. Any ideas what could be causing this? >> >> Thanks! >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > Barry Moore > Research Scientist > Dept. of Human Genetics > University of Utah > Salt Lake City, UT 84112 > -------------------------------------------- > (801) 585-3543 > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:24:43 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:24:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Ok. Try the developer release and see if it still happens. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:19 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Unfortunately, the job is no longer running and as a result I cannot connect to the compute nodes as I could while it was running. On the interactive node, it looks like it's real disk, although it looks like there are some tmpfs mounts. [dstandag at mason src] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb2 462824304 180235660 259078476 42% /tmp [dstandag at mason src] df Filesystem 1K-blocks Used Available Use% Mounted on login_x86_64 16497564 3077352 13420212 19% / tmpfs 16497564 0 16497564 0% /dev/shm tmpfs 10240 0 10240 0% /var/tmp /dev/sdb2 462824304 180235660 259078476 42% /tmp AFS 9000000 0 9000000 0% /afs bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs ... ... I'll see if I can launch another short job and verify this on the compute nodes. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > The command 'df /tmp' will tell you whether /tmp is a tmpfs mount > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:12 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > It looks like /tmp is indeed being used: the files I played with were under > /tmp/maker_1YQF9o. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >> directory and I have had problems with fasta indexes running from tmpfs >> mounts in the past. >> >> --Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:05 PM >> To: Carson Holt >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> The maker working directory is in a cluster environment with shared scratch >> space (I'm guessing NFS-mounted). I didn't change the temp directory setting, >> so it should be the local default (/tmp). >> >> I'll give the dev version a shot. Thanks. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>> Could you try this development version and tell me if the error still >>> happens? >>> >>> Use this command to download --> >>> <> >>> >>> Username: <> >>> Password: <> >>> >>> Are you running in an NFS mounted directory or are you resetting TMP to a >>> different location? >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 1:52 PM >>> To: Maker Mailing List >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> I have since installed Maker on a different machine and tried it out. The >>> test run completed successfully, but as I commenced with the full genome >>> annotation, I have noticed the following error popping up frequently. >>> >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.mak >>>> er.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffo >>>> ld_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efa >>>> a.mpi.10.8.blastx >>>> #-------------------------------# >>>> deleted:-10 hits >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.mak >>>> er.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffo >>>> ld_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efa >>>> a.mpi.10.9.blastx >>>> #-------------------------------# >>>> deleted:-6 hits >>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>> stop here:comp59088_c1_seq7 >>>> ERROR: Fasta index error >>>> >>>> FATAL ERROR >>>> ERROR: Failed while polishig ESTs!! >>>> >>>> ERROR: Chunk failed at level 14 >>>> !! >>>> FAILED CONTIG:scaffold_0 >>>> >>>> >>>> >>>> >>>> --Next Contig-- >>>> >>>> #--------------------------------------------------------------------- >>>> Now starting the contig!! >>>> SeqID: scaffold_1 >>>> Length: 5805686 >>>> #--------------------------------------------------------------------- >>> >>> My first thought based on the message is that blastdbcmd could not find the >>> sequence in the database. I verified this was the case--I could not extract >>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>> However, after removing the index files and re-running makeblastdb with the >>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>> >>> I was initially happy with this finding, but upon closer inspection it looks >>> like Maker does not use blastdbcmd to extract sequences, but rather its own >>> internal code. Therefore I'm still not sure where the problem is and how I >>> might fix it. Any insights? >>> >>> Thanks! >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>> wrote: >>>> Greetings! >>>> >>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>> the blastn stage. This is the terminal message. >>>> >>>>> #--------- command -------------# >>>>> Widget::blastn: >>>>> /share/home/01854/standage/local/bin/blastn -db >>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efas >>>>> ta.mpi.10.7 -query >>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>> -show_gis -out >>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.o >>>>> utput/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866 >>>>> .0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ET >>>>> rimmomatic%2Efasta.mpi.10.7.blastn >>>>> #-------------------------------# >>>>> deleted:0 hits >>>>> ERROR: Could not obtain lock to format database >>>>> >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while doing blastn of ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 8 >>>>> !! >>>>> FAILED CONTIG:scaffold_866 >>>> >>>> Several blastn steps appeared to have completed successfully to this one >>>> failing. Any ideas what could be causing this? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>> >>> _______________________________________________ maker-devel mailing list >>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m >>> aker-devel_yandell-lab.org >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:29:18 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:29:18 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot > connect to the compute nodes as I could while it was running. On the > interactive node, it looks like it's real disk, although it looks like > there are some tmpfs mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: > >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were >> under */tmp/maker_1YQF9o*. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >> >>> Check to see where /tmp is located? Some clusters have it set up as a >>> tmpfs directory and I have had problems with fasta indexes running from >>> tmpfs mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared >>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>> directory setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>> >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to >>>> a different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. >>>> The test run completed successfully, but as I commenced with the full >>>> genome annotation, I have noticed the following error popping up frequently. >>>> >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>> #-------------------------------# >>>> deleted:-10 hits >>>> formating database... >>>> #--------- command -------------# >>>> Widget::formater: >>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>> #-------------------------------# >>>> running blast search. >>>> #--------- command -------------# >>>> Widget::blastx: >>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>> #-------------------------------# >>>> deleted:-6 hits >>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>> stop here:comp59088_c1_seq7 >>>> ERROR: Fasta index error >>>> >>>> FATAL ERROR >>>> ERROR: Failed while polishig ESTs!! >>>> >>>> ERROR: Chunk failed at level 14 >>>> !! >>>> FAILED CONTIG:scaffold_0 >>>> >>>> >>>> >>>> >>>> --Next Contig-- >>>> >>>> #--------------------------------------------------------------------- >>>> Now starting the contig!! >>>> SeqID: scaffold_1 >>>> Length: 5805686 >>>> #--------------------------------------------------------------------- >>>> >>>> >>>> My first thought based on the message is that *blastdbcmd* could not >>>> find the sequence in the database. I verified this was the case--I could >>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>> created under /tmp. However, after removing the index files and re-running >>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>> extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>> rather its own internal code. Therefore I'm still not sure where the >>>> problem is and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>> daniel.standage at gmail.com> wrote: >>>> >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup during >>>>> the blastn stage. This is the terminal message. >>>>> >>>>> #--------- command -------------# >>>>> Widget::blastn: >>>>> /share/home/01854/standage/local/bin/blastn -db >>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis >>>>> -out >>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>> #-------------------------------# >>>>> deleted:0 hits >>>>> ERROR: Could not obtain lock to format database >>>>> >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while doing blastn of ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 8 >>>>> !! >>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> >>>>> Several blastn steps appeared to have completed successfully to this >>>>> one failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>> _______________________________________________ maker-devel mailing >>>> list maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> >>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:32:35 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:32:35 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: If running './Build install_deps' to get the prereqs automatically, you can say yes to the local version question if you want those prereqs to be installed just for MAKER rather than globally to whatever perl you are using. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:29 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot connect > to the compute nodes as I could while it was running. On the interactive node, > it looks like it's real disk, although it looks like there are some tmpfs > mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were under >> /tmp/maker_1YQF9o. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >>> directory and I have had problems with fasta indexes running from tmpfs >>> mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared scratch >>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>> setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>> different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. The >>>> test run completed successfully, but as I commenced with the full genome >>>> annotation, I have noticed the following error popping up frequently. >>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>> >>>> My first thought based on the message is that blastdbcmd could not find the >>>> sequence in the database. I verified this was the case--I could not extract >>>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>>> However, after removing the index files and re-running makeblastdb with the >>>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>> its own internal code. Therefore I'm still not sure where the problem is >>>> and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>> wrote: >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>> during the blastn stage. This is the terminal message. >>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efa >>>>>> sta.mpi.10.7 -query >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>> -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker. >>>>>> output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_8 >>>>>> 66.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity% >>>>>> 2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> Several blastn steps appeared to have completed successfully to this one >>>>> failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>> >>>> _______________________________________________ maker-devel mailing list >>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ >>>> maker-devel_yandell-lab.org >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Fri Oct 26 12:47:43 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Fri, 26 Oct 2012 14:47:43 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I've got a test run (with the dev version) waiting in the queue. But just to be clear, if the temp directory is indeed the problem, I'm assuming that would that be fixed by setting a different temp directory on the same disk as the scratch disk? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:32 PM, Carson Holt wrote: > If running './Build install_deps' to get the prereqs automatically, you > can say yes to the local version question if you want those prereqs to be > installed just for MAKER rather than globally to whatever perl you are > using. > > Thanks, > Carson > > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot >> connect to the compute nodes as I could while it was running. On the >> interactive node, it looks like it's real disk, although it looks like >> there are some tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were >>> under */tmp/maker_1YQF9o*. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared >>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>> directory setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>> to a different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. >>>>> The test run completed successfully, but as I commenced with the full >>>>> genome annotation, I have noticed the following error popping up frequently. >>>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>>> >>>>> >>>>> My first thought based on the message is that *blastdbcmd* could not >>>>> find the sequence in the database. I verified this was the case--I could >>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>> created under /tmp. However, after removing the index files and re-running >>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>>> rather its own internal code. Therefore I'm still not sure where the >>>>> problem is and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>> daniel.standage at gmail.com> wrote: >>>>> >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>> true -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this >>>>>> one failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>> _______________________________________________ maker-devel mailing >>>>> list maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 12:56:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 14:56:30 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: Yes. That is correct. Also I've made one more update to the development release with respect to indexes for your test run. Could you run 'svn update' inside the devel maker directory. Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:47 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step I've got a test run (with the dev version) waiting in the queue. But just to be clear, if the temp directory is indeed the problem, I'm assuming that would that be fixed by setting a different temp directory on the same disk as the scratch disk? -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:32 PM, Carson Holt wrote: > If running './Build install_deps' to get the prereqs automatically, you can > say yes to the local version question if you want those prereqs to be > installed just for MAKER rather than globally to whatever perl you are using. > > Thanks, > Carson > > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot connect >> to the compute nodes as I could while it was running. On the interactive >> node, it looks like it's real disk, although it looks like there are some >> tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were under >>> /tmp/maker_1YQF9o. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared scratch >>>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>>> setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>>> different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. The >>>>> test run completed successfully, but as I commenced with the full genome >>>>> annotation, I have noticed the following error popping up frequently. >>>>> >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.m >>>>>> aker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/sc >>>>>> affold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E4 >>>>>> 7%2Efaa.mpi.10.8.blastx >>>>>> #-------------------------------# >>>>>> deleted:-10 hits >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.m >>>>>> aker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/sc >>>>>> affold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E4 >>>>>> 7%2Efaa.mpi.10.9.blastx >>>>>> #-------------------------------# >>>>>> deleted:-6 hits >>>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>>> stop here:comp59088_c1_seq7 >>>>>> ERROR: Fasta index error >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while polishig ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 14 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --Next Contig-- >>>>>> >>>>>> #--------------------------------------------------------------------- >>>>>> Now starting the contig!! >>>>>> SeqID: scaffold_1 >>>>>> Length: 5805686 >>>>>> #--------------------------------------------------------------------- >>>>> >>>>> My first thought based on the message is that blastdbcmd could not find >>>>> the sequence in the database. I verified this was the case--I could not >>>>> extract sequence comp59088_c1_seq7 from the database Maker had created >>>>> under /tmp. However, after removing the index files and re-running >>>>> makeblastdb with the -parse_seqids option set, blastdbcmd successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>>> its own internal code. Therefore I'm still not sure where the problem is >>>>> and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>>> wrote: >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>>> #--------- command -------------# >>>>>>> Widget::blastn: >>>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Ef >>>>>>> asta.mpi.10.7 -query >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>>> -show_gis -out >>>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker >>>>>>> .output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold >>>>>>> _866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrini >>>>>>> ty%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>>> #-------------------------------# >>>>>>> deleted:0 hits >>>>>>> ERROR: Could not obtain lock to format database >>>>>>> >>>>>>> >>>>>>> FATAL ERROR >>>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>>> >>>>>>> ERROR: Chunk failed at level 8 >>>>>>> !! >>>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this one >>>>>> failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>> >>>>> _______________________________________________ maker-devel mailing list >>>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo >>>>> /maker-devel_yandell-lab.org >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jason.stajich at gmail.com Fri Oct 26 14:49:57 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Fri, 26 Oct 2012 13:49:57 -0700 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Right but is tmp on a disk or a tmpfs df -h /tmp would give you the first hint at whether it is a local drive or something else. On Oct 26, 2012, at 11:12 AM, Daniel Standage wrote: > It looks like /tmp is indeed being used: the files I played with were under /tmp/maker_1YQF9o. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: > Check to see where /tmp is located? Some clusters have it set up as a tmpfs directory and I have had problems with fasta indexes running from tmpfs mounts in the past. > > --Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:05 PM > To: Carson Holt > > Subject: Re: [maker-devel] Strange error at blastn step > > The maker working directory is in a cluster environment with shared scratch space (I'm guessing NFS-mounted). I didn't change the temp directory setting, so it should be the local default (/tmp). > > I'll give the dev version a shot. Thanks. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: > Could you try this development version and tell me if the error still happens? > > Use this command to download --> > <> > > Username: <> > Password: <> > > Are you running in an NFS mounted directory or are you resetting TMP to a different location? > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 1:52 PM > To: Maker Mailing List > Subject: Re: [maker-devel] Strange error at blastn step > > I have since installed Maker on a different machine and tried it out. The test run completed successfully, but as I commenced with the full genome annotation, I have noticed the following error popping up frequently. > > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx > #-------------------------------# > deleted:-10 hits > formating database... > #--------- command -------------# > Widget::formater: > /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 > #-------------------------------# > running blast search. > #--------- command -------------# > Widget::blastx: > /N/u/dstandag/Mason/local/bin/blastx -db /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx > #-------------------------------# > deleted:-6 hits > WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. > stop here:comp59088_c1_seq7 > ERROR: Fasta index error > > FATAL ERROR > ERROR: Failed while polishig ESTs!! > > ERROR: Chunk failed at level 14 > !! > FAILED CONTIG:scaffold_0 > > > > > --Next Contig-- > > #--------------------------------------------------------------------- > Now starting the contig!! > SeqID: scaffold_1 > Length: 5805686 > #--------------------------------------------------------------------- > > My first thought based on the message is that blastdbcmd could not find the sequence in the database. I verified this was the case--I could not extract sequence comp59088_c1_seq7 from the database Maker had created under /tmp. However, after removing the index files and re-running makeblastdb with the -parse_seqids option set, blastdbcmd successfully extracted the sequence. > > I was initially happy with this finding, but upon closer inspection it looks like Maker does not use blastdbcmd to extract sequences, but rather its own internal code. Therefore I'm still not sure where the problem is and how I might fix it. Any insights? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage wrote: > Greetings! > > I am doing a test run of my Maker setup on a new machine, annotating a pretty short contig (about 3kb). However, there seems to be a hiccup during the blastn stage. This is the terminal message. > > #--------- command -------------# > Widget::blastn: > /share/home/01854/standage/local/bin/blastn -db /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true -show_gis -out /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn > #-------------------------------# > deleted:0 hits > ERROR: Could not obtain lock to format database > > > FATAL ERROR > ERROR: Failed while doing blastn of ESTs!! > > ERROR: Chunk failed at level 8 > !! > FAILED CONTIG:scaffold_866 > > Several blastn steps appeared to have completed successfully to this one failing. Any ideas what could be causing this? > > Thanks! > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Oct 26 16:59:30 2012 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Oct 2012 18:59:30 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: Message-ID: From: Carson Holt Date: Friday, 26 October, 2012 6:10 PM To: Daniel Standage Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step I've been going over the indexing code using different scenarios and I may have isolated a candidate for what is causing this. Could you do one more 'svn update' inside the maker devel directory before running a test job? Thanks, Carson From: Daniel Standage Date: Friday, 26 October, 2012 2:29 PM To: Carson Holt Cc: "maker-devel at yandell-lab.org" Subject: Re: [maker-devel] Strange error at blastn step Got this from the compute node. Looks like native disk space to me. [dstandag at mason ~] df /tmp Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 478573472 12319684 441943620 3% /tmp Installing a bundle of Perl prereqs for development version, will try that soon. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > Ok. Try the developer release and see if it still happens. > > Thanks, > Carson > > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:19 PM > > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] Strange error at blastn step > > Unfortunately, the job is no longer running and as a result I cannot connect > to the compute nodes as I could while it was running. On the interactive node, > it looks like it's real disk, although it looks like there are some tmpfs > mounts. > > [dstandag at mason src] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sdb2 462824304 180235660 259078476 42% /tmp > [dstandag at mason src] df > Filesystem 1K-blocks Used Available Use% Mounted on > login_x86_64 16497564 3077352 13420212 19% / > tmpfs 16497564 0 16497564 0% /dev/shm > tmpfs 10240 0 10240 0% /var/tmp > /dev/sdb2 462824304 180235660 259078476 42% /tmp > AFS 9000000 0 9000000 0% /afs > bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 > bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 > bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 > bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 > bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u > bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft > bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs > ... > ... > > I'll see if I can launch another short job and verify this on the compute > nodes. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:12 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> It looks like /tmp is indeed being used: the files I played with were under >> /tmp/maker_1YQF9o. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> Check to see where /tmp is located? Some clusters have it set up as a tmpfs >>> directory and I have had problems with fasta indexes running from tmpfs >>> mounts in the past. >>> >>> --Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:05 PM >>> To: Carson Holt >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> The maker working directory is in a cluster environment with shared scratch >>> space (I'm guessing NFS-mounted). I didn't change the temp directory >>> setting, so it should be the local default (/tmp). >>> >>> I'll give the dev version a shot. Thanks. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> Could you try this development version and tell me if the error still >>>> happens? >>>> >>>> Use this command to download --> >>>> <> >>>> >>>> Username: <> >>>> Password: <> >>>> >>>> Are you running in an NFS mounted directory or are you resetting TMP to a >>>> different location? >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 1:52 PM >>>> To: Maker Mailing List >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> I have since installed Maker on a different machine and tried it out. The >>>> test run completed successfully, but as I commenced with the full genome >>>> annotation, I have noticed the following error popping up frequently. >>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.ma >>>>> ker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaf >>>>> fold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2 >>>>> Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>> >>>> My first thought based on the message is that blastdbcmd could not find the >>>> sequence in the database. I verified this was the case--I could not extract >>>> sequence comp59088_c1_seq7 from the database Maker had created under /tmp. >>>> However, after removing the index files and re-running makeblastdb with the >>>> -parse_seqids option set, blastdbcmd successfully extracted the sequence. >>>> >>>> I was initially happy with this finding, but upon closer inspection it >>>> looks like Maker does not use blastdbcmd to extract sequences, but rather >>>> its own internal code. Therefore I'm still not sure where the problem is >>>> and how I might fix it. Any insights? >>>> >>>> Thanks! >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage >>>> wrote: >>>>> Greetings! >>>>> >>>>> I am doing a test run of my Maker setup on a new machine, annotating a >>>>> pretty short contig (about 3kb). However, there seems to be a hiccup >>>>> during the blastn stage. This is the terminal message. >>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efa >>>>>> sta.mpi.10.7 -query >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend >>>>>> 3 -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 -searchsp >>>>>> 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking true >>>>>> -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker. >>>>>> output/maker.bogus_datastore/scaffold_866/theVoid.scaffold_866/scaffold_8 >>>>>> 66.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity% >>>>>> 2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>> >>>>> Several blastn steps appeared to have completed successfully to this one >>>>> failing. Any ideas what could be causing this? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>> >>>> _______________________________________________ maker-devel mailing list >>>> maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/ >>>> maker-devel_yandell-lab.org >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From kokwei86 at gmail.com Sun Oct 28 12:50:19 2012 From: kokwei86 at gmail.com (kokwei) Date: Mon, 29 Oct 2012 02:50:19 +0800 Subject: [maker-devel] Query on genemark results Message-ID: <508D7E6B.3060001@gmail.com> Hi, I have the same problems as posted by Andr? Gomes on 10/4/10. I still have the same problems even though using the current available version of maker (maker 2.1 and maker 2.26 beta version). I have tried to do the gene prediction on eukaryotic genome using 3 ab-initio gene predictors (SNAP, Augustus and GeneMark-ES) using Maker. >From the fasta_merge output, I have 3 separate files of gene models ($prefix.all.maker.augustus_masked.proteins.fasta, $prefix.all.maker.snap_masked.proteins.fasta and $prefix.all.maker.genemark.proteins.fasta) and one $prefix.all.maker.proteins.fasta (I presume this should be the consensus gene models from 3 predictors' results, right?). From the consensus gene models file, I don't see even one result from genemark but all from snap/augustus only. Is that normal? Also from the file naming and gene model label in final gff file, it's showing that genemark is not masked like those of augustus and snap? Is that true? Why only augustus and snap masked but not genemark? Please assist and thanks for your helps. Kok Wei From daniel.standage at gmail.com Mon Oct 29 07:39:05 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Mon, 29 Oct 2012 09:39:05 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: Carson, I was able to run the first svn update before the test job ran, but I didn't get the message about this second update until after it has already executed and failed. I got about 372 lines of such. STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_datastore To access files for individual sequences use the datastore index: /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_master_datastore_index.log STATUS: Now running MAKER... WARNING: Cannot find >scaffold_0, trying to re-index the fasta. stop here: scaffold_0 ERROR: Fasta index error at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 239. Process::MpiChunk::_prepare('Process::MpiChunk=HASH(0x3c8b300)', 'HASH(0x3c80820)', 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 73 Process::MpiTiers::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c808b0)', 'HASH(0x3c8ec40)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 79 Process::MpiTiers::_prepare('Process::MpiTiers=HASH(0x3c71f30)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 56 Process::MpiTiers::new('Process::MpiTiers', 'HASH(0x3c80640)', 0, 'Process::MpiChunk') called at /N/u/dstandag/Mason/local/src/maker-dev/bin/maker line 627 --> rank=NA, hostname=c4 ERROR: Failed in tier preparation WARNING: You must always set a rank before running MpiTiers FATAL: argument `seq_id` does not exist in MpiTier object at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 86. Process::MpiChunk::_initialize_vars('Process::MpiChunk=HASH(0x3cc93d0)', 'HASH(0x3cc9400)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 47 Process::MpiChunk::new('Process::MpiChunk', 'HASH(0x3c80820)', 0, 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 407 Process::MpiChunk::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c8f498)', 'HASH(0x3cc8f20)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 3811 Process::MpiChunk::_go('Process::MpiChunk=HASH(0x3c8b300)', 'load', 'HASH(0x3c80820)', 0, 0) called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 310 Process::MpiChunk::_loader('Process::MpiChunk=HASH(0x3c8b300)', 'HASH(0x3c80820)', 0, 0, 'Process::MpiTiers=HASH(0x3c71f30)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 364 Process::MpiTiers::__ANON__() called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 eval {...} called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 Error::subs::try('CODE(0x3c8f9c0)', 'HASH(0x3c8fb28)') called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm line 375 I'll try the next update and run the test again. I see a lot of references in there to MPI, and just thought I would make it clear that although I am running in a cluster environment, I am using the default serial version of Maker, not the parallel MPI version. Also, after this failed, I tried changing the TMP directory so that it was located on the same NFS mount as the scratch disk to which the output was written. This did not seem to have any affect, and I saw the same issues with the EST sequences unable to be found. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Fri, Oct 26, 2012 at 6:59 PM, Carson Holt wrote: > > > From: Carson Holt > Date: Friday, 26 October, 2012 6:10 PM > To: Daniel Standage > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > I've been going over the indexing code using different scenarios and I may > have isolated a candidate for what is causing this. Could you do one more > 'svn update' inside the maker devel directory before running a test job? > > Thanks, > Carson > > From: Daniel Standage > Date: Friday, 26 October, 2012 2:29 PM > To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > > Subject: Re: [maker-devel] Strange error at blastn step > > Got this from the compute node. Looks like native disk space to me. > > [dstandag at mason ~] df /tmp > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/sda1 478573472 12319684 441943620 3% /tmp > > Installing a bundle of Perl prereqs for development version, will try that > soon. > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: > >> Ok. Try the developer release and see if it still happens. >> >> Thanks, >> Carson >> >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:19 PM >> >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Unfortunately, the job is no longer running and as a result I cannot >> connect to the compute nodes as I could while it was running. On the >> interactive node, it looks like it's real disk, although it looks like >> there are some tmpfs mounts. >> >> [dstandag at mason src] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> [dstandag at mason src] df >> Filesystem 1K-blocks Used Available Use% Mounted on >> login_x86_64 16497564 3077352 13420212 19% / >> tmpfs 16497564 0 16497564 0% /dev/shm >> tmpfs 10240 0 10240 0% /var/tmp >> /dev/sdb2 462824304 180235660 259078476 42% /tmp >> AFS 9000000 0 9000000 0% /afs >> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >> ... >> ... >> >> I'll see if I can launch another short job and verify this on the compute >> nodes. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >> >>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:12 PM >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> It looks like /tmp is indeed being used: the files I played with were >>> under */tmp/maker_1YQF9o*. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>> >>>> Check to see where /tmp is located? Some clusters have it set up as a >>>> tmpfs directory and I have had problems with fasta indexes running from >>>> tmpfs mounts in the past. >>>> >>>> --Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:05 PM >>>> To: Carson Holt >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> The maker working directory is in a cluster environment with shared >>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>> directory setting, so it should be the local default (/tmp). >>>> >>>> I'll give the dev version a shot. Thanks. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>> >>>>> Could you try this development version and tell me if the error still >>>>> happens? >>>>> >>>>> Use this command to download --> >>>>> <> >>>>> >>>>> Username: <> >>>>> Password: <> >>>>> >>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>> to a different location? >>>>> >>>>> Thanks, >>>>> Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>> To: Maker Mailing List >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> I have since installed Maker on a different machine and tried it out. >>>>> The test run completed successfully, but as I commenced with the full >>>>> genome annotation, I have noticed the following error popping up frequently. >>>>> >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>> #-------------------------------# >>>>> deleted:-10 hits >>>>> formating database... >>>>> #--------- command -------------# >>>>> Widget::formater: >>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>> #-------------------------------# >>>>> running blast search. >>>>> #--------- command -------------# >>>>> Widget::blastx: >>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>> #-------------------------------# >>>>> deleted:-6 hits >>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>> stop here:comp59088_c1_seq7 >>>>> ERROR: Fasta index error >>>>> >>>>> FATAL ERROR >>>>> ERROR: Failed while polishig ESTs!! >>>>> >>>>> ERROR: Chunk failed at level 14 >>>>> !! >>>>> FAILED CONTIG:scaffold_0 >>>>> >>>>> >>>>> >>>>> >>>>> --Next Contig-- >>>>> >>>>> #--------------------------------------------------------------------- >>>>> Now starting the contig!! >>>>> SeqID: scaffold_1 >>>>> Length: 5805686 >>>>> #--------------------------------------------------------------------- >>>>> >>>>> >>>>> My first thought based on the message is that *blastdbcmd* could not >>>>> find the sequence in the database. I verified this was the case--I could >>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>> created under /tmp. However, after removing the index files and re-running >>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>> extracted the sequence. >>>>> >>>>> I was initially happy with this finding, but upon closer inspection it >>>>> looks like Maker does not use *blastdbcmd* to extract sequences, but >>>>> rather its own internal code. Therefore I'm still not sure where the >>>>> problem is and how I might fix it. Any insights? >>>>> >>>>> Thanks! >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>> daniel.standage at gmail.com> wrote: >>>>> >>>>>> Greetings! >>>>>> >>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>> during the blastn stage. This is the terminal message. >>>>>> >>>>>> #--------- command -------------# >>>>>> Widget::blastn: >>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>> true -show_gis -out >>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>> #-------------------------------# >>>>>> deleted:0 hits >>>>>> ERROR: Could not obtain lock to format database >>>>>> >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 8 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_866 >>>>>> >>>>>> >>>>>> Several blastn steps appeared to have completed successfully to this >>>>>> one failing. Any ideas what could be causing this? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>> _______________________________________________ maker-devel mailing >>>>> list maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>> >>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel.standage at gmail.com Mon Oct 29 07:43:46 2012 From: daniel.standage at gmail.com (Daniel Standage) Date: Mon, 29 Oct 2012 09:43:46 -0400 Subject: [maker-devel] Strange error at blastn step In-Reply-To: References: Message-ID: I just ran svn update and tried again with the development version of Maker. Same long list of errors as in the last message. -- Daniel S. Standage Ph.D. Candidate Bioinformatics and Computational Biology Program Department of Genetics, Development, and Cell Biology Iowa State University On Mon, Oct 29, 2012 at 9:39 AM, Daniel Standage wrote: > Carson, > > I was able to run the first svn update before the test job ran, but I > didn't get the message about this second update until after it has already > executed and failed. I got about 372 lines of such. > > STATUS: Parsing control files... > STATUS: Processing and indexing input FASTA files... > STATUS: Setting up database for any GFF3 input... > A data structure will be created for you at: > > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_datastore > > To access files for individual sequences use the datastore index: > > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/DELETEME.maker.pdom.1.mason.maker.output/DELETEME.maker.pdom.1.mason_master_datastore_index.log > > STATUS: Now running MAKER... > WARNING: Cannot find >scaffold_0, trying to re-index the fasta. > stop here: scaffold_0 > ERROR: Fasta index error > at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 239. > Process::MpiChunk::_prepare('Process::MpiChunk=HASH(0x3c8b300)', > 'HASH(0x3c80820)', 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 73 > Process::MpiTiers::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c808b0)', 'HASH(0x3c8ec40)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 79 > Process::MpiTiers::_prepare('Process::MpiTiers=HASH(0x3c71f30)') > called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 56 > Process::MpiTiers::new('Process::MpiTiers', 'HASH(0x3c80640)', 0, > 'Process::MpiChunk') called at > /N/u/dstandag/Mason/local/src/maker-dev/bin/maker line 627 > --> rank=NA, hostname=c4 > ERROR: Failed in tier preparation > WARNING: You must always set a rank before running MpiTiers > FATAL: argument `seq_id` does not exist in MpiTier object > at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 86. > > Process::MpiChunk::_initialize_vars('Process::MpiChunk=HASH(0x3cc93d0)', > 'HASH(0x3cc9400)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 47 > Process::MpiChunk::new('Process::MpiChunk', 'HASH(0x3c80820)', 0, > 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 407 > Process::MpiChunk::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c8f498)', 'HASH(0x3cc8f20)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 3811 > Process::MpiChunk::_go('Process::MpiChunk=HASH(0x3c8b300)', > 'load', 'HASH(0x3c80820)', 0, 0) called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm > line 310 > Process::MpiChunk::_loader('Process::MpiChunk=HASH(0x3c8b300)', > 'HASH(0x3c80820)', 0, 0, 'Process::MpiTiers=HASH(0x3c71f30)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 364 > Process::MpiTiers::__ANON__() called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 415 > eval {...} called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Error.pm line 407 > Error::subs::try('CODE(0x3c8f9c0)', 'HASH(0x3c8fb28)') called at > /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm > line 375 > > > I'll try the next update and run the test again. I see a lot of references > in there to MPI, and just thought I would make it clear that although I am > running in a cluster environment, I am using the default serial version of > Maker, not the parallel MPI version. > > Also, after this failed, I tried changing the TMP directory so that it was > located on the same NFS mount as the scratch disk to which the output was > written. This did not seem to have any affect, and I saw the same issues > with the EST sequences unable to be found. > > > -- > Daniel S. Standage > Ph.D. Candidate > Bioinformatics and Computational Biology Program > Department of Genetics, Development, and Cell Biology > Iowa State University > > > > On Fri, Oct 26, 2012 at 6:59 PM, Carson Holt wrote: > >> >> >> From: Carson Holt >> Date: Friday, 26 October, 2012 6:10 PM >> To: Daniel Standage >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> I've been going over the indexing code using different scenarios and I >> may have isolated a candidate for what is causing this. Could you do one >> more 'svn update' inside the maker devel directory before running a test >> job? >> >> Thanks, >> Carson >> >> From: Daniel Standage >> Date: Friday, 26 October, 2012 2:29 PM >> To: Carson Holt >> Cc: "maker-devel at yandell-lab.org" >> >> Subject: Re: [maker-devel] Strange error at blastn step >> >> Got this from the compute node. Looks like native disk space to me. >> >> [dstandag at mason ~] df /tmp >> Filesystem 1K-blocks Used Available Use% Mounted on >> /dev/sda1 478573472 12319684 441943620 3% /tmp >> >> Installing a bundle of Perl prereqs for development version, will try >> that soon. >> >> -- >> Daniel S. Standage >> Ph.D. Candidate >> Bioinformatics and Computational Biology Program >> Department of Genetics, Development, and Cell Biology >> Iowa State University >> >> >> >> On Fri, Oct 26, 2012 at 2:24 PM, Carson Holt wrote: >> >>> Ok. Try the developer release and see if it still happens. >>> >>> Thanks, >>> Carson >>> >>> >>> From: Daniel Standage >>> Date: Friday, 26 October, 2012 2:19 PM >>> >>> To: Carson Holt >>> Cc: "maker-devel at yandell-lab.org" >>> >>> Subject: Re: [maker-devel] Strange error at blastn step >>> >>> Unfortunately, the job is no longer running and as a result I cannot >>> connect to the compute nodes as I could while it was running. On the >>> interactive node, it looks like it's real disk, although it looks like >>> there are some tmpfs mounts. >>> >>> [dstandag at mason src] df /tmp >>> Filesystem 1K-blocks Used Available Use% Mounted on >>> /dev/sdb2 462824304 180235660 259078476 42% /tmp >>> [dstandag at mason src] df >>> Filesystem 1K-blocks Used Available Use% Mounted on >>> login_x86_64 16497564 3077352 13420212 19% / >>> tmpfs 16497564 0 16497564 0% /dev/shm >>> tmpfs 10240 0 10240 0% /var/tmp >>> /dev/sdb2 462824304 180235660 259078476 42% /tmp >>> AFS 9000000 0 9000000 0% /afs >>> bl-nas1:/vol/hd00 3435973856 1775658144 1660315712 52% /N/hd00 >>> bl-nas1:/vol/hd01 3435973856 1684116928 1751856928 50% /N/hd01 >>> bl-nas2:/vol/hd02 3435973856 1856598656 1579375200 55% /N/hd02 >>> bl-nas2:/vol/hd03 3435973856 2747626240 688347616 80% /N/hd03 >>> bl-nas1:/vol/hdln 81920 3424 78496 5% /N/u >>> bl-nas2:/vol/soft 1258291200 837003424 421287776 67% /N/soft >>> bl-nas1:/vol/logs 419430400 67163328 352267072 17% /N/logs >>> ... >>> ... >>> >>> I'll see if I can launch another short job and verify this on the >>> compute nodes. >>> >>> -- >>> Daniel S. Standage >>> Ph.D. Candidate >>> Bioinformatics and Computational Biology Program >>> Department of Genetics, Development, and Cell Biology >>> Iowa State University >>> >>> >>> >>> On Fri, Oct 26, 2012 at 2:14 PM, Carson Holt wrote: >>> >>>> The command 'df /tmp' will tell you whether /tmp is a tmpfs mount >>>> >>>> Thanks, >>>> Carson >>>> >>>> >>>> From: Daniel Standage >>>> Date: Friday, 26 October, 2012 2:12 PM >>>> To: Carson Holt >>>> Cc: "maker-devel at yandell-lab.org" >>>> >>>> Subject: Re: [maker-devel] Strange error at blastn step >>>> >>>> It looks like /tmp is indeed being used: the files I played with were >>>> under */tmp/maker_1YQF9o*. >>>> >>>> -- >>>> Daniel S. Standage >>>> Ph.D. Candidate >>>> Bioinformatics and Computational Biology Program >>>> Department of Genetics, Development, and Cell Biology >>>> Iowa State University >>>> >>>> >>>> >>>> On Fri, Oct 26, 2012 at 2:09 PM, Carson Holt wrote: >>>> >>>>> Check to see where /tmp is located? Some clusters have it set up as a >>>>> tmpfs directory and I have had problems with fasta indexes running from >>>>> tmpfs mounts in the past. >>>>> >>>>> --Carson >>>>> >>>>> >>>>> From: Daniel Standage >>>>> Date: Friday, 26 October, 2012 2:05 PM >>>>> To: Carson Holt >>>>> >>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>> >>>>> The maker working directory is in a cluster environment with shared >>>>> scratch space (I'm guessing NFS-mounted). I didn't change the temp >>>>> directory setting, so it should be the local default (/tmp). >>>>> >>>>> I'll give the dev version a shot. Thanks. >>>>> >>>>> -- >>>>> Daniel S. Standage >>>>> Ph.D. Candidate >>>>> Bioinformatics and Computational Biology Program >>>>> Department of Genetics, Development, and Cell Biology >>>>> Iowa State University >>>>> >>>>> >>>>> >>>>> On Fri, Oct 26, 2012 at 1:57 PM, Carson Holt wrote: >>>>> >>>>>> Could you try this development version and tell me if the error still >>>>>> happens? >>>>>> >>>>>> Use this command to download --> >>>>>> <> >>>>>> >>>>>> Username: <> >>>>>> Password: <> >>>>>> >>>>>> Are you running in an NFS mounted directory or are you resetting TMP >>>>>> to a different location? >>>>>> >>>>>> Thanks, >>>>>> Carson >>>>>> >>>>>> >>>>>> From: Daniel Standage >>>>>> Date: Friday, 26 October, 2012 1:52 PM >>>>>> To: Maker Mailing List >>>>>> Subject: Re: [maker-devel] Strange error at blastn step >>>>>> >>>>>> I have since installed Maker on a different machine and tried it out. >>>>>> The test run completed successfully, but as I commenced with the full >>>>>> genome annotation, I have noticed the following error popping up frequently. >>>>>> >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.8.blastx >>>>>> #-------------------------------# >>>>>> deleted:-10 hits >>>>>> formating database... >>>>>> #--------- command -------------# >>>>>> Widget::formater: >>>>>> /N/u/dstandag/Mason/local/bin/makeblastdb -dbtype prot -in >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 >>>>>> #-------------------------------# >>>>>> running blast search. >>>>>> #--------- command -------------# >>>>>> Widget::blastx: >>>>>> /N/u/dstandag/Mason/local/bin/blastx -db >>>>>> /tmp/maker_1YQF9o/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9 -query >>>>>> /tmp/maker_1YQF9o/rank0/scaffold_0.0 -num_alignments 100000 >>>>>> -num_descriptions 100000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 >>>>>> -num_threads 16 -seg yes -soft_masking true -lcase_masking -show_gis -out >>>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.1.mason.maker.output/maker.pdom.1.mason_datastore/scaffold_0/theVoid.scaffold_0/scaffold_0.0.Amel3%2E2_Dmel5%2E47%2Efaa.blastx.temp_dir/Amel3%2E2_Dmel5%2E47%2Efaa.mpi.10.9.blastx >>>>>> #-------------------------------# >>>>>> deleted:-6 hits >>>>>> WARNING: Cannot find> comp59088_c1_seq7, trying to re-index the fasta. >>>>>> stop here:comp59088_c1_seq7 >>>>>> ERROR: Fasta index error >>>>>> >>>>>> FATAL ERROR >>>>>> ERROR: Failed while polishig ESTs!! >>>>>> >>>>>> ERROR: Chunk failed at level 14 >>>>>> !! >>>>>> FAILED CONTIG:scaffold_0 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> --Next Contig-- >>>>>> >>>>>> #--------------------------------------------------------------------- >>>>>> Now starting the contig!! >>>>>> SeqID: scaffold_1 >>>>>> Length: 5805686 >>>>>> #--------------------------------------------------------------------- >>>>>> >>>>>> >>>>>> My first thought based on the message is that *blastdbcmd* could not >>>>>> find the sequence in the database. I verified this was the case--I could >>>>>> not extract sequence *comp59088_c1_seq7* from the database Maker had >>>>>> created under /tmp. However, after removing the index files and re-running >>>>>> *makeblastdb* with the *-parse_seqids* option set, *blastdbcmd* successfully >>>>>> extracted the sequence. >>>>>> >>>>>> I was initially happy with this finding, but upon closer inspection >>>>>> it looks like Maker does not use *blastdbcmd* to extract sequences, >>>>>> but rather its own internal code. Therefore I'm still not sure where the >>>>>> problem is and how I might fix it. Any insights? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> -- >>>>>> Daniel S. Standage >>>>>> Ph.D. Candidate >>>>>> Bioinformatics and Computational Biology Program >>>>>> Department of Genetics, Development, and Cell Biology >>>>>> Iowa State University >>>>>> >>>>>> >>>>>> >>>>>> On Thu, Oct 25, 2012 at 3:30 PM, Daniel Standage < >>>>>> daniel.standage at gmail.com> wrote: >>>>>> >>>>>>> Greetings! >>>>>>> >>>>>>> I am doing a test run of my Maker setup on a new machine, annotating >>>>>>> a pretty short contig (about 3kb). However, there seems to be a hiccup >>>>>>> during the blastn stage. This is the terminal message. >>>>>>> >>>>>>> #--------- command -------------# >>>>>>> Widget::blastn: >>>>>>> /share/home/01854/standage/local/bin/blastn -db >>>>>>> /tmp/2881473.1.development/maker_i6ZSJi/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7 >>>>>>> -query /tmp/2881473.1.development/maker_i6ZSJi/rank0/scaffold_866.0 >>>>>>> -num_alignments 100000 -num_descriptions 100000 -evalue 1e-10 -gapextend 3 >>>>>>> -word_size 15 -reward 1 -penalty -3 -gapopen 3 -dbsize 1000 >>>>>>> -searchsp 500000000 -num_threads 16 -lcase_masking -dust yes -soft_masking >>>>>>> true -show_gis -out >>>>>>> /scratch/01854/standage/PdomGenomic/Annotation/scripts/maker.bogus.maker.output/maker.bogus_datastore/scaffold_866/theVoid.scaff >>>>>>> old_866/scaffold_866.0.Pdom%2ETrinity%2ETrimmomatic%2Efasta.blastn.temp_dir/Pdom%2ETrinity%2ETrimmomatic%2Efasta.mpi.10.7.blastn >>>>>>> #-------------------------------# >>>>>>> deleted:0 hits >>>>>>> ERROR: Could not obtain lock to format database >>>>>>> >>>>>>> >>>>>>> FATAL ERROR >>>>>>> ERROR: Failed while doing blastn of ESTs!! >>>>>>> >>>>>>> ERROR: Chunk failed at level 8 >>>>>>> !! >>>>>>> FAILED CONTIG:scaffold_866 >>>>>>> >>>>>>> >>>>>>> Several blastn steps appeared to have completed successfully to this >>>>>>> one failing. Any ideas what could be causing this? >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> -- >>>>>>> Daniel S. Standage >>>>>>> Ph.D. Candidate >>>>>>> Bioinformatics and Computational Biology Program >>>>>>> Department of Genetics, Development, and Cell Biology >>>>>>> Iowa State University >>>>>>> >>>>>>> >>>>>> _______________________________________________ maker-devel mailing >>>>>> list maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>>>> >>>>> >>>>> >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From parulk at caltech.edu Tue Oct 30 15:18:06 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Tue, 30 Oct 2012 14:18:06 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: Message-ID: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Hello Carson and maker community, Thank you very much for your guidelines on using the maker-pipeline. Yes, green sea urchin genome that we are trying to annotate. We are running the on scaffolds and most of these scaffolds are small in size(very first genome assembly). We would typically expect 20,000 genes in this genome. So we are running maker using EST and proteins from the genome and out-groups to generate training dataset for SNAP and Augustus. Depending on the resulting predictions we may bootstrap the predicted genes once again using EST and proteins. Do you have any further suggestions? Also could you point how to convert training set generated for SNAP to be used as training set for Augustus as well? Would maker give equal weightage to SNAP and Augustus predictions for generating gene model? Thanks and regards, Parul Kudtarkar > One thing you seem to be missing is protein evidence. > > Is this a sea urchin (I looked up some of the ESTs)? If so, I would recommend adding all proteins from the Strongylocentrotus purpuratus genome, then throw in another Deuterstome of your choice. Perhaps you should also add a couple of outgroup organisms like Nematostella vectensis > (cnidaria) and a protostome of your choice. Be careful if adding adding to many protostome outgroups (i.e. C. elegans and Drosophila) because a big part of their evolution is gene loss (so distant cnidaria often match > deuterstomes better than most protostomes do). > > You could take the maker results when protein data is included and use it > to retrain SNAP again. > > Even a 22 kb contig is still really short. Is this genome primarily constituted by short contigs like this? I would recommend running CEGMA once on this genome to get an appropriate estimate of how recoverable the > genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). Cegma > will give you an estimate for genome completeness as well as estimates of > what percentage of genes will be found in their entirety and what percent > will be partial genes. This is important to do if your genome is fragmented as it will give you a reasonable expectation of what you can expected to recover (as short contigs don't annotate very well - you tend > to loose a lot). > > Thanks, > Carson > > > On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: > >>Hi Carson, >>Thanks. I have attached another contig which is 22 kb, with as many as 3 exons EST alignments. Could you please recommend additional training. We are currently running maker on the entire contig set and eventually merge >>all the gff3 contig predictions. The using suggested parameter/methods we >>would like to get a consensus gene-set with minimal false >>positives/negatives. >>Thanks, >>Parul >>> The contig in question is really too small to get much out of it (only 5 >>kb). There was only one single exon EST alignments and a couple of predictions with no evidence support. Anything smaller than 10 kb is mostly useless for annotation purposes. You would really need a few 100kb >>> length or longer contigs to glean enough information for optimizing your >>parameters. >>> The general suggestions for any maker run are to use proteins from a >>closely related organism or a couple of closely related organisms for the >>> protein= option in maker. Also leave single_exon set to 0, except for >>certain eukaryotes that have a bias for single exon transcripts (i.e. some >>> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >>predictors tend to over-predict by a wide margin (lots of false >>> positives). >>> Additional training would really depend on what your other contigs look >>like. Do you have any large contigs? I could look at one of those and give suggestions but the provided contig is just too short to glean much. >>> Thanks, >>> Carson >>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>Hello, >>>>Please advice on the aforementioned query? >>>>Thanks, >>>>Parul Kudtarkar >>>>---------------------------- Original Message >>>> ---------------------------- >>>>Subject: [maker-devel] Conensus gene model >>>>From: "Parul Kudtarkar" >>>>Date: Fri, October 12, 2012 2:46 pm >>>>To: maker-devel at yandell-lab.org >>>>------------------------------------------------------------------------ -- >>Hi, >>>>We are using snap(training set[hmm file] generated using est,protein >>>> and >>contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>> file >>>>as input). The output that we get is combination of various >>>>gene-predictors and evidences. I have attached sample result file. What >>would you recommend to get consensus result set? Bootstrapping the resulting gff3 file (rerunning maker)? >>>>Thanks, >>>>Parul Kudtarkar >>>>-- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org_______________________________________________ >>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -- >>>>Scientific Programmer >>>>Center for Computational Regulatory Genomics >>>>Beckman Institute, >>>>California Institute of Technology >>>>http://www.spbase.org_______________________________________________ >>maker-devel mailing list >>>>maker-devel at box290.bluehost.com >>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>-- >>Scientific Programmer >>Center for Computational Regulatory Genomics >>Beckman Institute, >>California Institute of Technology >>http://www.spbase.org > > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org From jason.stajich at gmail.com Tue Oct 30 18:52:43 2012 From: jason.stajich at gmail.com (Jason Stajich) Date: Tue, 30 Oct 2012 17:52:43 -0700 Subject: [maker-devel] Conensus gene model In-Reply-To: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> References: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Message-ID: Paul - I think I've posted on this before here if you are asking how to go from SNAP training to Augustus training. http://sourceforge.net/mailarchive/message.php?msg_id=29361270 I do this type of training a lot - here some pointers. I often train by generating models using cegma on the genome and get these 400 or so good models as my training set. when I have EST or RNA-Seq I use PASA to generate the best set of annotations. For CEGMA - then I run this script that comes with MAKER: cegma2zff output.cegma.gff genome.fa Then I follow the SNAP directions fathom genome.ann genome.dna -categorize 1000 fathom uni.ann uni.dna -export 1000 -plus mkdir MYGENOME cd MYGENOME forge ../export.ann ../export.dna --OPTIONS cd ../MYGENOME hmm-assembler.pl MYGENOME MYGENOME > MYGENOME.snap.hmm I then also make the augustus training data like this running in the directory that has the export.ann and export.dna files: perl gene_prediction/zff2augustus_gbk.pl > train.gb using this script: https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl I also make ZFF from GFF with this script if I got the RNA-Seq aligned and best models from PASA and incorporate all these data in to my SNAP training set, and then export again back to gbk for the augustus training. https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/pasatraining2zff.pl Then you just need to run the Augustus training (autoAugTrain.pl) on the train.gb file. Jason On Oct 30, 2012, at 2:18 PM, Parul Kudtarkar wrote: > Hello Carson and maker community, > > Thank you very much for your guidelines on using the maker-pipeline. Yes, > green sea urchin genome that we are trying to annotate. > We are running the on scaffolds and most of these scaffolds are small in > size(very first genome assembly). We would typically expect 20,000 genes > in this genome. So we are running maker using EST and proteins from the > genome and out-groups to generate training dataset for SNAP and Augustus. > Depending on the resulting predictions we may bootstrap the predicted > genes once again using EST and proteins. > > Do you have any further suggestions? Also could you point how to convert > training set generated for SNAP to be used as training set for Augustus as > well? Would maker give equal weightage to SNAP and Augustus predictions > for generating gene model? > > Thanks and regards, > Parul Kudtarkar > >> One thing you seem to be missing is protein evidence. >> >> Is this a sea urchin (I looked up some of the ESTs)? If so, I would > recommend adding all proteins from the Strongylocentrotus purpuratus > genome, then throw in another Deuterstome of your choice. Perhaps you > should also add a couple of outgroup organisms like Nematostella > vectensis >> (cnidaria) and a protostome of your choice. Be careful if adding adding > to many protostome outgroups (i.e. C. elegans and Drosophila) because a > big part of their evolution is gene loss (so distant cnidaria often > match >> deuterstomes better than most protostomes do). >> >> You could take the maker results when protein data is included and use > it >> to retrain SNAP again. >> >> Even a 22 kb contig is still really short. Is this genome primarily > constituted by short contigs like this? I would recommend running CEGMA > once on this genome to get an appropriate estimate of how recoverable > the >> genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). > Cegma >> will give you an estimate for genome completeness as well as estimates > of >> what percentage of genes will be found in their entirety and what > percent >> will be partial genes. This is important to do if your genome is > fragmented as it will give you a reasonable expectation of what you can > expected to recover (as short contigs don't annotate very well - you > tend >> to loose a lot). >> >> Thanks, >> Carson >> >> >> On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >> >>> Hi Carson, >>> Thanks. I have attached another contig which is 22 kb, with as many as 3 > exons EST alignments. Could you please recommend additional training. We > are currently running maker on the entire contig set and eventually > merge >>> all the gff3 contig predictions. The using suggested parameter/methods > we >>> would like to get a consensus gene-set with minimal false >>> positives/negatives. >>> Thanks, >>> Parul >>>> The contig in question is really too small to get much out of it (only 5 >>> kb). There was only one single exon EST alignments and a couple of > predictions with no evidence support. Anything smaller than 10 kb is > mostly useless for annotation purposes. You would really need a few > 100kb >>>> length or longer contigs to glean enough information for optimizing your >>> parameters. >>>> The general suggestions for any maker run are to use proteins from a >>> closely related organism or a couple of closely related organisms for the >>>> protein= option in maker. Also leave single_exon set to 0, except for >>> certain eukaryotes that have a bias for single exon transcripts (i.e. some >>>> fungi and oomycetes). And leave keep_preds set to 0 because ab initio >>> predictors tend to over-predict by a wide margin (lots of false >>>> positives). >>>> Additional training would really depend on what your other contigs > look >>> like. Do you have any large contigs? I could look at one of those and > give suggestions but the provided contig is just too short to glean > much. >>>> Thanks, >>>> Carson >>>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>> Hello, >>>>> Please advice on the aforementioned query? >>>>> Thanks, >>>>> Parul Kudtarkar >>>>> ---------------------------- Original Message >>>>> ---------------------------- >>>>> Subject: [maker-devel] Conensus gene model >>>>> From: "Parul Kudtarkar" >>>>> Date: Fri, October 12, 2012 2:46 pm >>>>> To: maker-devel at yandell-lab.org >>>>> ------------------------------------------------------------------------ > -- >>> Hi, >>>>> We are using snap(training set[hmm file] generated using est,protein >>>>> and >>> contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>>> file >>>>> as input). The output that we get is combination of various >>>>> gene-predictors and evidences. I have attached sample result file. > What >>> would you recommend to get consensus result set? Bootstrapping the > resulting gff3 file (rerunning maker)? >>>>> Thanks, >>>>> Parul Kudtarkar >>>>> -- >>>>> Scientific Programmer >>>>> Center for Computational Regulatory Genomics >>>>> Beckman Institute, >>>>> California Institute of Technology >>>>> http://www.spbase.org_______________________________________________ >>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- >>>>> Scientific Programmer >>>>> Center for Computational Regulatory Genomics >>>>> Beckman Institute, >>>>> California Institute of Technology >>>>> http://www.spbase.org_______________________________________________ >>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> -- >>> Scientific Programmer >>> Center for Computational Regulatory Genomics >>> Beckman Institute, >>> California Institute of Technology >>> http://www.spbase.org >> >> >> > > > -- > Scientific Programmer > Center for Computational Regulatory Genomics > Beckman Institute, > California Institute of Technology > http://www.spbase.org > > > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org Jason Stajich jason.stajich at gmail.com jason at bioperl.org From parulk at caltech.edu Wed Oct 31 18:04:33 2012 From: parulk at caltech.edu (Parul Kudtarkar) Date: Wed, 31 Oct 2012 17:04:33 -0700 (PDT) Subject: [maker-devel] Conensus gene model In-Reply-To: References: <1250.131.215.15.234.1351631886.squirrel@webmail.caltech.edu> Message-ID: <1640.131.215.15.234.1351728273.squirrel@webmail.caltech.edu> Hi Jason, thanks for directions on generating training-set for augustus. Also as alignment evidence if we are providing protein sequences from the primary organism as well as other closely related species is there an option to give the primary protein file precedence over others? At the moment I have all the proteins(from primary organism as well as related species) into a single file as protein option in maker_opts.ctl Thanks and regards, Parul Kudtarkar > Paul - > > I think I've posted on this before here if you are asking how to go from > SNAP training to Augustus training. > http://sourceforge.net/mailarchive/message.php?msg_id=29361270 > > I do this type of training a lot - here some pointers. > > I often train by generating models using cegma on the genome and get these > 400 or so good models as my training set. when I have EST or RNA-Seq I > use PASA to generate the best set of annotations. > > For CEGMA - then I run this script that comes with MAKER: > cegma2zff output.cegma.gff genome.fa > > Then I follow the SNAP directions > > fathom genome.ann genome.dna -categorize 1000 > fathom uni.ann uni.dna -export 1000 -plus > mkdir MYGENOME > cd MYGENOME > forge ../export.ann ../export.dna --OPTIONS > cd ../MYGENOME > hmm-assembler.pl MYGENOME MYGENOME > MYGENOME.snap.hmm > > I then also make the augustus training data like this running in the > directory that has the export.ann and export.dna files: > perl gene_prediction/zff2augustus_gbk.pl > train.gb > > using this script: > https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/zff2augustus_gbk.pl > > I also make ZFF from GFF with this script if I got the RNA-Seq aligned and > best models from PASA and incorporate all these data in to my SNAP > training set, and then export again back to gbk for the augustus training. > https://github.com/hyphaltip/genome-scripts/blob/master/gene_prediction/pasatraining2zff.pl > > Then you just need to run the Augustus training (autoAugTrain.pl) on the > train.gb file. > > Jason > > On Oct 30, 2012, at 2:18 PM, Parul Kudtarkar wrote: > >> Hello Carson and maker community, >> >> Thank you very much for your guidelines on using the maker-pipeline. >> Yes, >> green sea urchin genome that we are trying to annotate. >> We are running the on scaffolds and most of these scaffolds are small in >> size(very first genome assembly). We would typically expect 20,000 genes >> in this genome. So we are running maker using EST and proteins from the >> genome and out-groups to generate training dataset for SNAP and >> Augustus. >> Depending on the resulting predictions we may bootstrap the predicted >> genes once again using EST and proteins. >> >> Do you have any further suggestions? Also could you point how to convert >> training set generated for SNAP to be used as training set for Augustus >> as >> well? Would maker give equal weightage to SNAP and Augustus predictions >> for generating gene model? >> >> Thanks and regards, >> Parul Kudtarkar >> >>> One thing you seem to be missing is protein evidence. >>> >>> Is this a sea urchin (I looked up some of the ESTs)? If so, I would >> recommend adding all proteins from the Strongylocentrotus purpuratus >> genome, then throw in another Deuterstome of your choice. Perhaps you >> should also add a couple of outgroup organisms like Nematostella >> vectensis >>> (cnidaria) and a protostome of your choice. Be careful if adding >>> adding >> to many protostome outgroups (i.e. C. elegans and Drosophila) because a >> big part of their evolution is gene loss (so distant cnidaria often >> match >>> deuterstomes better than most protostomes do). >>> >>> You could take the maker results when protein data is included and use >> it >>> to retrain SNAP again. >>> >>> Even a 22 kb contig is still really short. Is this genome primarily >> constituted by short contigs like this? I would recommend running CEGMA >> once on this genome to get an appropriate estimate of how recoverable >> the >>> genes are going to be (http://korflab.ucdavis.edu/datasets/cegma/). >> Cegma >>> will give you an estimate for genome completeness as well as estimates >> of >>> what percentage of genes will be found in their entirety and what >> percent >>> will be partial genes. This is important to do if your genome is >> fragmented as it will give you a reasonable expectation of what you can >> expected to recover (as short contigs don't annotate very well - you >> tend >>> to loose a lot). >>> >>> Thanks, >>> Carson >>> >>> >>> On 12-10-15 3:45 PM, "Parul Kudtarkar" wrote: >>> >>>> Hi Carson, >>>> Thanks. I have attached another contig which is 22 kb, with as many as >>>> 3 >> exons EST alignments. Could you please recommend additional training. We >> are currently running maker on the entire contig set and eventually >> merge >>>> all the gff3 contig predictions. The using suggested parameter/methods >> we >>>> would like to get a consensus gene-set with minimal false >>>> positives/negatives. >>>> Thanks, >>>> Parul >>>>> The contig in question is really too small to get much out of it >>>>> (only 5 >>>> kb). There was only one single exon EST alignments and a couple of >> predictions with no evidence support. Anything smaller than 10 kb is >> mostly useless for annotation purposes. You would really need a few >> 100kb >>>>> length or longer contigs to glean enough information for optimizing >>>>> your >>>> parameters. >>>>> The general suggestions for any maker run are to use proteins from a >>>> closely related organism or a couple of closely related organisms for >>>> the >>>>> protein= option in maker. Also leave single_exon set to 0, except >>>>> for >>>> certain eukaryotes that have a bias for single exon transcripts (i.e. >>>> some >>>>> fungi and oomycetes). And leave keep_preds set to 0 because ab >>>>> initio >>>> predictors tend to over-predict by a wide margin (lots of false >>>>> positives). >>>>> Additional training would really depend on what your other contigs >> look >>>> like. Do you have any large contigs? I could look at one of those >>>> and >> give suggestions but the provided contig is just too short to glean >> much. >>>>> Thanks, >>>>> Carson >>>>> On 12-10-15 1:41 PM, "Parul Kudtarkar" wrote: >>>>>> Hello, >>>>>> Please advice on the aforementioned query? >>>>>> Thanks, >>>>>> Parul Kudtarkar >>>>>> ---------------------------- Original Message >>>>>> ---------------------------- >>>>>> Subject: [maker-devel] Conensus gene model >>>>>> From: "Parul Kudtarkar" >>>>>> Date: Fri, October 12, 2012 2:46 pm >>>>>> To: maker-devel at yandell-lab.org >>>>>> ------------------------------------------------------------------------ >> -- >>>> Hi, >>>>>> We are using snap(training set[hmm file] generated using est,protein >>>>>> and >>>> contig file), agustus,genemarkE(we ran it outside maker and have gff3 >>>>>> file >>>>>> as input). The output that we get is combination of various >>>>>> gene-predictors and evidences. I have attached sample result file. >> What >>>> would you recommend to get consensus result set? Bootstrapping the >> resulting gff3 file (rerunning maker)? >>>>>> Thanks, >>>>>> Parul Kudtarkar >>>>>> -- >>>>>> Scientific Programmer >>>>>> Center for Computational Regulatory Genomics >>>>>> Beckman Institute, >>>>>> California Institute of Technology >>>>>> http://www.spbase.org_______________________________________________ >>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> -- >>>>>> Scientific Programmer >>>>>> Center for Computational Regulatory Genomics >>>>>> Beckman Institute, >>>>>> California Institute of Technology >>>>>> http://www.spbase.org_______________________________________________ >>>> maker-devel mailing list >>>>>> maker-devel at box290.bluehost.com >>>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> -- >>>> Scientific Programmer >>>> Center for Computational Regulatory Genomics >>>> Beckman Institute, >>>> California Institute of Technology >>>> http://www.spbase.org >>> >>> >>> >> >> >> -- >> Scientific Programmer >> Center for Computational Regulatory Genomics >> Beckman Institute, >> California Institute of Technology >> http://www.spbase.org >> >> >> >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > Jason Stajich > jason.stajich at gmail.com > jason at bioperl.org > > -- Scientific Programmer Center for Computational Regulatory Genomics Beckman Institute, California Institute of Technology http://www.spbase.org