From qwzhang0601 at gmail.com Thu Feb 2 10:16:51 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Thu, 2 Feb 2017 11:16:51 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> Message-ID: Thank for you suggestions. So it does not matter if there are redundancies of protein sequence from different sources? I am trying to annotating a *rodent *genome, and planned to collect protein sequences of human, mouse, rat from bout UniProt and NCBI (besides I also have RNA-seq data). I choose these species, because they are close to the species that I am working on and they are well annotated. But I saw someone said that if we choose protein sequence from one lineage, the genes that are missing in the lineage will not be detected. And in the following paper, the authors claim they used the entire SwissProt database as the input. How do you think about this? Should I include protein sequences from more species (like all Eukaryota)? I think it can help us identify more genes, but on the other hand won't this also give us more false positives? This paper used the entire SwissProt database as the input. Insights into the evolution of longevity from the bowhead whale genome. 2015. *Cell Rep* *10*(1): 112-122. Thanks Best Quanwei 2017-01-31 15:57 GMT-05:00 Michael Campbell : > Hi Quanwei, > > (1) When I use uniprot I use SWISS-prot and not tremble. > (2) I don?t merge files together. I just pass them all to MAKER as a comma > separated list. > > Thanks, > Mike > > > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang > wrote: > > > > I wonder what's the best way to collect protein sequences for gene > annotation of a de novo genome assembly. > > (1) My first choice is to get protein sequences of human and mouse from > UniProt. At this step, I am not clear whether I should download the > reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., > TrEMBL). > > (2) On ther other hand, I also get protein sequences from NCBI, should I > just simply merge those fasta files. Does it matter if there are > redundancies? And also, if I get protein sequences from different sources, > they may not have the same quality. Do I need to do something before I > integrate protein sequences from different sources? > > > > Many thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Thu Feb 2 13:24:04 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Thu, 2 Feb 2017 14:24:04 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> Message-ID: <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> It is nice to have an outgroup of some kind. In your case Human would serve that function. The issue that you might have with distantly related proteins is funny blast alignments that may lead to merging genes. You generally don?t get many false positives because the alignment parameters require a pretty good match, which is unlikely to happen by chance. You could limit swiss-prot to mammals if you wanted to. Sometimes I?ll try different combinations of evidence on a few large scaffolds and look at the results in a browser to get a feel for what is going to work best. Thanks, Mike > On Feb 2, 2017, at 11:16 AM, Quanwei Zhang wrote: > > > Thank for you suggestions. So it does not matter if there are redundancies of protein sequence from different sources? > I am trying to annotating a rodent genome, and planned to collect protein sequences of human, mouse, rat from bout UniProt and NCBI (besides I also have RNA-seq data). I choose these species, because they are close to the species that I am working on and they are well annotated. But I saw someone said that if we choose protein sequence from one lineage, the genes that are missing in the lineage will not be detected. And in the following paper, the authors claim they used the entire SwissProt database as the input. How do you think about this? Should I include protein sequences from more species (like all Eukaryota)? I think it can help us identify more genes, but on the other hand won't this also give us more false positives? > > This paper used the entire SwissProt database as the input. > Insights into the evolution of longevity from the bowhead whale genome. 2015. Cell Rep 10(1): 112-122. > > Thanks > > Best > Quanwei > > 2017-01-31 15:57 GMT-05:00 Michael Campbell >: > Hi Quanwei, > > (1) When I use uniprot I use SWISS-prot and not tremble. > (2) I don?t merge files together. I just pass them all to MAKER as a comma separated list. > > Thanks, > Mike > > > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang > wrote: > > > > I wonder what's the best way to collect protein sequences for gene annotation of a de novo genome assembly. > > (1) My first choice is to get protein sequences of human and mouse from UniProt. At this step, I am not clear whether I should download the reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., TrEMBL). > > (2) On ther other hand, I also get protein sequences from NCBI, should I just simply merge those fasta files. Does it matter if there are redundancies? And also, if I get protein sequences from different sources, they may not have the same quality. Do I need to do something before I integrate protein sequences from different sources? > > > > Many thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Thu Feb 2 15:05:53 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Thu, 2 Feb 2017 16:05:53 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> Message-ID: Thank you for your suggestions. I need to have some tests. Best Quanwei 2017-02-02 14:24 GMT-05:00 Michael Campbell : > It is nice to have an outgroup of some kind. In your case Human would > serve that function. The issue that you might have with distantly related > proteins is funny blast alignments that may lead to merging genes. You > generally don?t get many false positives because the alignment parameters > require a pretty good match, which is unlikely to happen by chance. You > could limit swiss-prot to mammals if you wanted to. > > Sometimes I?ll try different combinations of evidence on a few large > scaffolds and look at the results in a browser to get a feel for what is > going to work best. > > Thanks, > Mike > > On Feb 2, 2017, at 11:16 AM, Quanwei Zhang wrote: > > > Thank for you suggestions. So it does not matter if there are redundancies > of protein sequence from different sources? > I am trying to annotating a *rodent *genome, and planned to collect > protein sequences of human, mouse, rat from bout UniProt and NCBI (besides > I also have RNA-seq data). I choose these species, because they are close > to the species that I am working on and they are well annotated. But I saw > someone said that if we choose protein sequence from one lineage, the genes > that are missing in the lineage will not be detected. And in the following > paper, the authors claim they used the entire SwissProt database as the > input. How do you think about this? Should I include protein sequences from > more species (like all Eukaryota)? I think it can help us identify more > genes, but on the other hand won't this also give us more false positives? > > This paper used the entire SwissProt database as the input. > Insights into the evolution of longevity from the bowhead whale genome. > 2015. *Cell Rep* *10*(1): 112-122. > > Thanks > > Best > Quanwei > > 2017-01-31 15:57 GMT-05:00 Michael Campbell >: > >> Hi Quanwei, >> >> (1) When I use uniprot I use SWISS-prot and not tremble. >> (2) I don?t merge files together. I just pass them all to MAKER as a >> comma separated list. >> >> Thanks, >> Mike >> >> > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang >> wrote: >> > >> > I wonder what's the best way to collect protein sequences for gene >> annotation of a de novo genome assembly. >> > (1) My first choice is to get protein sequences of human and mouse from >> UniProt. At this step, I am not clear whether I should download the >> reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., >> TrEMBL). >> > (2) On ther other hand, I also get protein sequences from NCBI, should >> I just simply merge those fasta files. Does it matter if there are >> redundancies? And also, if I get protein sequences from different sources, >> they may not have the same quality. Do I need to do something before I >> integrate protein sequences from different sources? >> > >> > Many thanks >> > >> > Best >> > Quanwei >> > _______________________________________________ >> > maker-devel mailing list >> > maker-devel at box290.bluehost.com >> > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 3 10:04:45 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 3 Feb 2017 11:04:45 -0500 Subject: [maker-devel] augustus failed in maker2 Message-ID: I am trying to add augustus to the prediction. The maker2 run well without augustus, and augustus seems installed correctly. But it returns me the following errors. Any ideas or suggestions? What I did is just to set "augustus_species=human" in the "maker_opts.ctl" file, and I am running maker on a HPC with linux system. preparing ab-inits ERROR: Augustus failed --> rank=NA, hostname=n530 ERROR: Failed while preparing ab-inits ERROR: Chunk failed at level:0, tier_type:2 FAILED CONTIG:CasCan_contig_0 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:CasCan_contig_0 Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 3 11:15:39 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 3 Feb 2017 10:15:39 -0700 Subject: [maker-devel] augustus failed in maker2 In-Reply-To: References: Message-ID: Look a little further back into the STERR log to see if there are other errors further back. The issue is probably your Augustus installation. Try running it by itself (outside of MAKER) on the same dataset, and it may help identify on what is happening. Thanks, Carson > On Feb 3, 2017, at 9:04 AM, Quanwei Zhang wrote: > > I am trying to add augustus to the prediction. The maker2 run well without augustus, and augustus seems installed correctly. But it returns me the following errors. Any ideas or suggestions? What I did is just to set "augustus_species=human" in the "maker_opts.ctl" file, and I am running maker on a HPC with linux system. > > preparing ab-inits > ERROR: Augustus failed > --> rank=NA, hostname=n530 > ERROR: Failed while preparing ab-inits > ERROR: Chunk failed at level:0, tier_type:2 > FAILED CONTIG:CasCan_contig_0 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:CasCan_contig_0 > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From mnaymik at tgen.org Mon Feb 6 15:27:22 2017 From: mnaymik at tgen.org (Marcus Naymik) Date: Mon, 6 Feb 2017 14:27:22 -0700 Subject: [maker-devel] Can't Install Proc::Signal Message-ID: I can't find Proc::Signal in cpan or anywhere. How can I install it? -- *This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisa.postma at gmail.com Sun Feb 5 07:15:08 2017 From: alisa.postma at gmail.com (Alisa Postma) Date: Sun, 5 Feb 2017 15:15:08 +0200 Subject: [maker-devel] Problem with fgenesh protein IDs Message-ID: Good afternoon I am having problems with my all.maker.proteins.fasta file. I ran maker with ab initio gene predictions from augustus and genemark and I added my fgenesh gff file to the pred_gff option in the ctl file. However, my maker protein fasta file does not give the fgenesh proteins IDs similar to that for augustus and genemark. For example: >FGENESH protein AED:0.06 eAED:0.06 QI:0|0|0|1|1|1|3|0|433 vs. >maker-scaffold74-augustus-gene-0.179-mRNA-1 protein AED:0.07 eAED:0.07 QI:123|1|1|1|1|1|5|793|326 I would appreciate any advice on this matter. Kind regards Alisa -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 6 22:05:33 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 6 Feb 2017 21:05:33 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: References: Message-ID: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Proc::Signal is part of MAKER?s internal libraries. If you are getting an error saying it is missing, then you have a problem with your installation. Either it was not installed successfully, or you modified the location of files or executables post installation. Make sure you do not move the bin directory or its contents. If you need to be somewhere else, it is best to use softlinks instead. Thanks, Carson > On Feb 6, 2017, at 2:27 PM, Marcus Naymik wrote: > > I can't find Proc::Signal in cpan or anywhere. How can I install it? > > This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaymik at tgen.org Tue Feb 7 11:00:37 2017 From: mnaymik at tgen.org (Marcus Naymik) Date: Tue, 7 Feb 2017 10:00:37 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> References: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Message-ID: Ah, It was listed on as a prereq on your wiki and was trying to install all those first. I was just a little confused. Thanks! Marcus On Mon, Feb 6, 2017 at 9:05 PM, Carson Holt wrote: > Proc::Signal is part of MAKER?s internal libraries. If you are getting an > error saying it is missing, then you have a problem with your installation. > > Either it was not installed successfully, or you modified the location of > files or executables post installation. > > Make sure you do not move the bin directory or its contents. If you need > to be somewhere else, it is best to use softlinks instead. > > Thanks, > Carson > > On Feb 6, 2017, at 2:27 PM, Marcus Naymik wrote: > > I can't find Proc::Signal in cpan or anywhere. How can I install it? > > *This electronic message is intended to be for the use only of the named > recipient, and may contain information that is confidential or privileged, > including patient health information. If you are not the intended > recipient, you are hereby notified that any disclosure, copying, > distribution or use of the contents of this message is strictly prohibited. > If you have received this message in error or are not the named recipient, > please notify us immediately by contacting the sender at the electronic > mail address noted above, and delete and destroy all copies of this > message. Thank you.* > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- *This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 7 11:11:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 7 Feb 2017 10:11:46 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: References: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Message-ID: Use the MAKER Tutorial for GMOD Online Training 2014 It is the most up to date ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014 Thanks, Carson > On Feb 7, 2017, at 10:00 AM, Marcus Naymik wrote: > > Ah, It was listed on as a prereq on your wiki and was trying to install all those first. I was just a little confused. Thanks! > > Marcus > > On Mon, Feb 6, 2017 at 9:05 PM, Carson Holt > wrote: > Proc::Signal is part of MAKER?s internal libraries. If you are getting an error saying it is missing, then you have a problem with your installation. > > Either it was not installed successfully, or you modified the location of files or executables post installation. > > Make sure you do not move the bin directory or its contents. If you need to be somewhere else, it is best to use softlinks instead. > > Thanks, > Carson > >> On Feb 6, 2017, at 2:27 PM, Marcus Naymik > wrote: >> >> I can't find Proc::Signal in cpan or anywhere. How can I install it? >> >> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 7 11:12:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 7 Feb 2017 10:12:46 -0700 Subject: [maker-devel] Problem with fgenesh protein IDs In-Reply-To: References: Message-ID: The issue is probably with the format of your GFF3. I can take a look if you want. ?Carson > On Feb 5, 2017, at 6:15 AM, Alisa Postma wrote: > > Good afternoon > > I am having problems with my all.maker.proteins.fasta file. > > I ran maker with ab initio gene predictions from augustus and genemark and I added my fgenesh gff file to the pred_gff option in the ctl file. > > However, my maker protein fasta file does not give the fgenesh proteins IDs similar to that for augustus and genemark. > > For example: > >FGENESH protein AED:0.06 eAED:0.06 QI:0|0|0|1|1|1|3|0|433 > > vs. > > > >maker-scaffold74-augustus-gene-0.179-mRNA-1 protein AED:0.07 eAED:0.07 QI:123|1|1|1|1|1|5|793|326 > > I would appreciate any advice on this matter. > > Kind regards > > Alisa > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Wed Feb 8 09:45:11 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Wed, 8 Feb 2017 10:45:11 -0500 Subject: [maker-devel] make use of other gene finders in maker Message-ID: Hello: I am trying to annotate the new genome of a rodent species. I saw a clear pipeline to train the SNAP gene finder. But not clear what I should do if I also want to include augustus and/or GeneMark. Do I need to (could I and how) train it in Maker? In one of recently published paper, they mention the augustus was trained in Maker, but I am not clear how to do it? I wonder what's you opinions on using other gene finders in Maker. Thanks "Unique features of a global human ectoparasite identified through sequencing of the bed bug genome." Nat Commun, 2016. In this paper, the training step was described as below. *Three rounds of training of the Augustus and SNAP gene predictors within Maker were used to bootstrap to a high-quality training set.* Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Wed Feb 8 09:56:08 2017 From: dence at genetics.utah.edu (Daniel Ence) Date: Wed, 8 Feb 2017 15:56:08 +0000 Subject: [maker-devel] make use of other gene finders in maker In-Reply-To: References: Message-ID: <02D8BB85-1B0B-4028-A3F6-182F4F3318F7@genetics.utah.edu> Hi Quanwei, MAKER will run Augustus, SNAP, genemark or Fgenesh internally. You can also provide predictions from other gene predictors in valid GFF3 format in the ?pred_gff" field in the maker_opts control file. Genemark is self training on your genome. You provide a path to the ?es.mod? file that Genemark makes in the ?gmhmm? field. For augustus, the training is more involved, but is described here: http://www.molecularevolution.org/molevolfiles/exercises/augustus/training.html Augustus, snap, Fgenesh, and genemark are all complementary in some respects, but Augustus usually provides the most predictions that MAKER ends up selecting to promote to models. Hope that helps, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Feb 8, 2017, at 10:45 AM, Quanwei Zhang > wrote: Hello: I am trying to annotate the new genome of a rodent species. I saw a clear pipeline to train the SNAP gene finder. But not clear what I should do if I also want to include augustus and/or GeneMark. Do I need to (could I and how) train it in Maker? In one of recently published paper, they mention the augustus was trained in Maker, but I am not clear how to do it? I wonder what's you opinions on using other gene finders in Maker. Thanks "Unique features of a global human ectoparasite identified through sequencing of the bed bug genome." Nat Commun, 2016. In this paper, the training step was described as below. Three rounds of training of the Augustus and SNAP gene predictors within Maker were used to bootstrap to a high-quality training set. Best Quanwei _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 8 11:18:49 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 8 Feb 2017 12:18:49 -0500 Subject: [maker-devel] MAKER and OpenMPI Message-ID: Hello everyone, I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 8 11:22:19 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 10:22:19 -0700 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: References: Message-ID: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> You might have two different MPI flavors installed. Since the library and executable files have the same name, this kind of mismatch can happen in that case. Check the locations of MPI libraries given to MAKER during install, and the location of the mpiexec being used when you run. There is likely a mismatch with parts of one MPI flavor being used for one but not the other. ?Carson > On Feb 8, 2017, at 10:18 AM, Seth Munholland wrote: > > Hello everyone, > > I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 8 11:27:15 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 8 Feb 2017 12:27:15 -0500 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> References: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> Message-ID: That's for sure what it is, but I never installed a second flavour. I was commited to MPICH from the get go and am trying to figure out where OpenMPI came from. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 8, 2017 at 12:22 PM, Carson Holt wrote: > You might have two different MPI flavors installed. Since the library and > executable files have the same name, this kind of mismatch can happen in > that case. Check the locations of MPI libraries given to MAKER during > install, and the location of the mpiexec being used when you run. There is > likely a mismatch with parts of one MPI flavor being used for one but not > the other. > > ?Carson > > > > On Feb 8, 2017, at 10:18 AM, Seth Munholland wrote: > > Hello everyone, > > I'm setting up a cluster and configured MPICH on it, but after installing > MAKER my mpiexec gave me an error that comes from an improperly configured > OpenMPI. The weird thing is I haven't installed anything else associated > to MPI, particularly not OpenMPI. Did I get server gremlins or is there > any way MAKER's installation somehow is looking for OpenMPI? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 8 11:32:22 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 10:32:22 -0700 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: References: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> Message-ID: <863A3CDC-24F3-4A91-8D3E-45387262DF46@gmail.com> When you install MAKER, you have to provide the location to mpi.h and mpicc. You may have to reinstall, and give the correct location for one or both (don?t just trust the location that shows up, it may be the wrong one). So you will need to track down both of those for MPICH (depending on where you installed it to). Then do ?which -a mpiexec? to get all locations for every mpiexec found in your path. You will need to ensure the one at the top is the one you want. If it?s not, then you need to reconfigure your PATH or provide the full path to the mpiexec you want each time you launch it. ?Carson > On Feb 8, 2017, at 10:27 AM, Seth Munholland wrote: > > That's for sure what it is, but I never installed a second flavour. I was commited to MPICH from the get go and am trying to figure out where OpenMPI came from. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 8, 2017 at 12:22 PM, Carson Holt > wrote: > You might have two different MPI flavors installed. Since the library and executable files have the same name, this kind of mismatch can happen in that case. Check the locations of MPI libraries given to MAKER during install, and the location of the mpiexec being used when you run. There is likely a mismatch with parts of one MPI flavor being used for one but not the other. > > ?Carson > > > >> On Feb 8, 2017, at 10:18 AM, Seth Munholland > wrote: >> >> Hello everyone, >> >> I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjfi2sb3 at gmail.com Wed Feb 8 01:22:57 2017 From: mjfi2sb3 at gmail.com (Salim Bougouffa) Date: Wed, 08 Feb 2017 07:22:57 +0000 Subject: [maker-devel] Masking is causing problems with exon/CDS boundary predictions Message-ID: Hi, I am having trouble with MAKER/AUGUSTUS annotation. One of the recurrent ones is the example I attach in artemis screenshot. In red is the gene of interest. There are three GFF files. Top is augustus standalone that I executed on the non-masked scaffold. The second is maker annotation and third is the augustus (from maker) on the masked query. The gene clearly has four exons and three CDS but the masking seem to somehow cause augustus to get the boundaries wrong for the first and third CDS and skip the second CDS entirely. How do I fix this problem? Best, /SB [image: artemis.png] -- ____________________________ Sent from Inbox Mobile -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: artemis.png Type: image/png Size: 128354 bytes Desc: not available URL: From carsonhh at gmail.com Wed Feb 8 13:31:16 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 12:31:16 -0700 Subject: [maker-devel] Masking is causing problems with exon/CDS boundary predictions In-Reply-To: References: Message-ID: <4B4ACC1F-541E-4182-9DE6-5C22CEF2C91D@gmail.com> What do your evidence alignments look like? i.e. assembled mRNA-seq and protein homology? Not the mRNA-seq pileup (because MAKER can?t see that). Masking is applied before evidence seeding and the first ab initio run. It is actually then removed for evidence polishing around splice sites, and the hint based rerun of Augustus. So evidence is allowed to extend through masked regions and Augustus can make it part of the model if it wants on the second run. Since it didn?t the question becomes, what does the transcript and protein homology alignments from the maker run look like. ?Carson > On Feb 8, 2017, at 12:22 AM, Salim Bougouffa wrote: > > Hi, > > I am having trouble with MAKER/AUGUSTUS annotation. > > One of the recurrent ones is the example I attach in artemis screenshot. In red is the gene of interest. There are three GFF files. Top is augustus standalone that I executed on the non-masked scaffold. The second is maker annotation and third is the augustus (from maker) on the masked query. > > > The gene clearly has four exons and three CDS but the masking seem to somehow cause augustus to get the boundaries wrong for the first and third CDS and skip the second CDS entirely. > > How do I fix this problem? > > Best, > /SB > > -- > ____________________________ > Sent from Inbox Mobile > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 09:50:24 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 10:50:24 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? Message-ID: Hello: I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 10:03:41 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 11:03:41 -0500 Subject: [maker-devel] training of gene finders using whole assembly or longest contigs? Message-ID: Hello: I am training the gene finders using the whole assembly. But it seems very time consuming. Besides, I have to repeat the training process several times. Although I am running it on 25 nodes on a server, it may still take 3 (or even more) weeks for the training. I wonder how you guys train the SNAP. Do you use the whole assembly or just select the longest contigs for the training. If I only use longest contigs (like top 20% longest), will it be good enough as that get by using the whole assembly? Or should I randomly select 20% contigs for the training, for which we will have similar length distribution as the whole assembly? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alessandro.Rossoni at uni-duesseldorf.de Fri Feb 10 09:50:07 2017 From: Alessandro.Rossoni at uni-duesseldorf.de (Alessandro Rossoni) Date: Fri, 10 Feb 2017 16:50:07 +0100 Subject: [maker-devel] Updating Reference Gene Models - Legacy - Strategy Message-ID: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> Dear makers of MAKER, first of all - thank you for this awesome program! In the context of my project, I have been running MAKER on a set of novel genomes and it worked very well :) During the last days, I realized that the reference sequences of species A that I have been using as starting point for gene model prediction for species B, C and D are a bit flawed. How do I know that? Through visualization of novel RNA-Seq data that was mapped to the "old" gene models of species A, I came to the conclusion that the "old" gene models are not really accurate. In fact, between 52?62% of the reads map within annotated exonic regions of the genome and up to 47% map within intergenic regions. Intron/Exon boarders are pretty messed up and there are a lot more transcribed sequences in species A than previously thought. Hence, I would love to update the gene models of species A including the new RNA-Seq evidence and hope to get more accurate gene models out of it. The more accurate gene models of species A would be then used to predict genes for species B, C and D (which are hopefully going to benefit from the more accurate input). However, I there is a little understanding issue on how to set the parameters of the maker_opts.ctl. My plan is to produce new RNA-Seq based gene models for species A using Cufflinks, Stringtie, Breaker, Trinity and Velvet. I would pass the output to the maker_opts.ctl as: model_gff=Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff #the new gene models est=Species_A_reference_ests.fasta #the old/flawed gene models protein=swissprot.fasta Question 1: is this correct so far? But what sequences do I use for training the ab-initio predictors? snaphmm= augustus_species= Question 2: Do I use the "old/flawed" sequences that I know are not really good? I am not sure what sequences to use for training. Any help on this issue would be amazing! Best, Ale -- Alessandro W. Rossoni, M.Sc. Institute for Plant Biochemistry Heinrich-Heine-University -- http:///www.plant-biochemistry.hhu.de/ E-Mail: alessandro.rossoni at hhu-duesseldorf.de From carsonhh at gmail.com Fri Feb 10 10:36:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:36:46 -0700 Subject: [maker-devel] Updating Reference Gene Models - Legacy - Strategy In-Reply-To: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> References: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> Message-ID: If the old models are poor, then I suggest you do new training using BUSCO, CEGMA, or the est2genome or protein2genome options within MAKER ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors Also this thread ?> https://groups.google.com/forum/#!topic/maker-devel/FWMSTdqWQqI model_gff is for existing gene models you want to keep. So none of these should go there ?> Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff model_gff will always make it into the final annotation set even without any evidence support. By putting those files there, you are basically turning every feature in each of those files into a final gene model no matter how bad it is. Also if the original models are poor, don?t put them there either. You can doing reciprocal best blast hits with final models to old models to see how they match each other in the end. Will take a little data processing to make it work though. For all transcript based files, you should provide those to est_gff since they are evidence alignments and not model predictions. For Breaker.gff, that should be pred_gff since it is a prediction model. With Trinity, I suggest you provide the fasta file and allow MAKER to align and filter things rather than a GFF3. The problem with using GFF3 is you are basically short circuiting upstream prioritization and filtering saying ?take this evidence as is.? Also providing same evidence from multiple sources is a bad idea. By purposely making the evidence dataset more noisy, you are forcing lower accuracy. My suggesting would be not to use Cufflinks (it will introduce a very high false positive rate). Provide Trinity input as fasta (also make sure you use jaccard_clip option was used when assembling). And you will have to manually review models with and without Stringtie data to see if it hurts more than it helps. Provide Breaker.gff to pred_gff, but still allow maker to run Augustus itself internally (otherwise you won?t be able to use protein evidence as hints). Thanks, Carson > On Feb 10, 2017, at 8:50 AM, Alessandro Rossoni wrote: > > Dear makers of MAKER, > first of all - thank you for this awesome program! In the context of my project, I have been running MAKER on a set of novel genomes and it worked very well :) > > During the last days, I realized that the reference sequences of species A that I have been using as starting point for gene model prediction for species B, C and D are a bit flawed. How do I know that? Through visualization of novel RNA-Seq data that was mapped to the "old" gene models of species A, I came to the conclusion that the "old" gene models are not really accurate. In fact, between 52?62% of the reads map within annotated exonic regions of the genome and up to 47% map within intergenic regions. Intron/Exon boarders are pretty messed up and there are a lot more transcribed sequences in species A than previously thought. > > Hence, I would love to update the gene models of species A including the new RNA-Seq evidence and hope to get more accurate gene models out of it. The more accurate gene models of species A would be then used to predict genes for species B, C and D (which are hopefully going to benefit from the more accurate input). However, I there is a little understanding issue on how to set the parameters of the maker_opts.ctl. > > My plan is to produce new RNA-Seq based gene models for species A using Cufflinks, Stringtie, Breaker, Trinity and Velvet. I would pass the output to the maker_opts.ctl as: > > model_gff=Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff #the new gene models > est=Species_A_reference_ests.fasta #the old/flawed gene models > protein=swissprot.fasta > > Question 1: is this correct so far? > > But what sequences do I use for training the ab-initio predictors? > snaphmm= > augustus_species= > > Question 2: Do I use the "old/flawed" sequences that I know are not really good? I am not sure what sequences to use for training. > > Any help on this issue would be amazing! > Best, > Ale > > > -- > Alessandro W. Rossoni, M.Sc. > Institute for Plant Biochemistry > Heinrich-Heine-University > > -- > http:///www.plant-biochemistry.hhu.de/ > E-Mail: alessandro.rossoni at hhu-duesseldorf.de > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 10 10:39:31 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:39:31 -0700 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: MAKER is restartable. As long as you run each time in the same location, it can reuse existing alignments from the previous run. You also only need to train on ~10MB of the genome depending on gene density. Target size should be 300-400 genes. If you follow this GMOD wiki, this is demonstrated (you can also watch video - link as top of page - to see it being done) ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors ?Carson > On Feb 10, 2017, at 8:50 AM, Quanwei Zhang wrote: > > Hello: > > I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 10 10:42:13 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:42:13 -0700 Subject: [maker-devel] training of gene finders using whole assembly or longest contigs? In-Reply-To: References: Message-ID: Example of training here ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors You can also search the devel mailing list archives here ?> https://groups.google.com/forum/#!forum/maker-devel There are lots and lots of threads that go into detail on training. Note more than 2 rounds of training is not beneficial, and can actually make performance worse (there is an overtraining paradox). ?Carson > On Feb 10, 2017, at 9:03 AM, Quanwei Zhang wrote: > > Hello: > > I am training the gene finders using the whole assembly. But it seems very time consuming. Besides, I have to repeat the training process several times. Although I am running it on 25 nodes on a server, it may still take 3 (or even more) weeks for the training. I wonder how you guys train the SNAP. Do you use the whole assembly or just select the longest contigs for the training. If I only use longest contigs (like top 20% longest), will it be good enough as that get by using the whole assembly? Or should I randomly select 20% contigs for the training, for which we will have similar length distribution as the whole assembly? > > Thanks > > Best > Quanwei > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 11:04:55 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 12:04:55 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: Great. Many thanks. So I can select part of my genome assembly for the training. Which the following ways do you think is better? (a) Select the longest contigs. (b) Randomly select contigs with any length? Thank you! Best Quanwei 2017-02-10 11:39 GMT-05:00 Carson Holt : > MAKER is restartable. As long as you run each time in the same location, > it can reuse existing alignments from the previous run. You also only need > to train on ~10MB of the genome depending on gene density. Target size > should be 300-400 genes. > > If you follow this GMOD wiki, this is demonstrated (you can also watch > video - link as top of page - to see it being done) ?> > http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/ > MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ > ab_initio_Gene_Predictors > > ?Carson > > > > On Feb 10, 2017, at 8:50 AM, Quanwei Zhang wrote: > > Hello: > > I am annotating a new genome using Maker. I have RNA-seq assembly and > protein sequences (from other organisms) in fasta format. Since I need to > train gene finders, so I have to run Maker several times. I think the > aligning process between the transcript assembly (protein sequences) and > the genome assembly may be time consuming. So I wonder whether I can save > such alignment in the first run, and then make use of such alignment in the > following runs? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Fri Feb 10 11:07:51 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Fri, 10 Feb 2017 12:07:51 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: The longer ones will have more complete genes on them. If you get a set scaffolds that has about 1,000 genes you probably have enough for training. Mike > On Feb 10, 2017, at 12:04 PM, Quanwei Zhang wrote: > > Great. Many thanks. So I can select part of my genome assembly for the training. Which the following ways do you think is better? (a) Select the longest contigs. (b) Randomly select contigs with any length? > > Thank you! > > Best > Quanwei > > 2017-02-10 11:39 GMT-05:00 Carson Holt >: > MAKER is restartable. As long as you run each time in the same location, it can reuse existing alignments from the previous run. You also only need to train on ~10MB of the genome depending on gene density. Target size should be 300-400 genes. > > If you follow this GMOD wiki, this is demonstrated (you can also watch video - link as top of page - to see it being done) ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors > > ?Carson > > > >> On Feb 10, 2017, at 8:50 AM, Quanwei Zhang > wrote: >> >> Hello: >> >> I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? >> >> Thanks >> >> Best >> Quanwei >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 22:53:02 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 23:53:02 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" Message-ID: Hello: I want to assign gene function to the predicted genes. I collected UniProt/SwissProt human, mouse and rat protein sequences (including both canonical and isoforms). Then use "makeblastdb" to build the database. And then run "blastp -db ... -max_hsps_per_subject 1" following the example in the protocol. But it returns me an error: Unknown argument: "max_hsps_per_subject". Why this happens? Is it because I am using a different version of blastp? USAGE blastp [-h] [-help] [-import_search_strategy filename] [-export_search_strategy filename] [-task task_name] [-db database_name] [-dbsize num_letters] [-gilist filename] [-seqidlist filename] [-negative_gilist filename] [-entrez_query entrez_query] [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm] [-subject subject_input_file] [-subject_loc range] [-query input_file] [-out output_file] [-evalue evalue] [-word_size int_value] [-gapopen open_penalty] [-gapextend extend_penalty] [-qcov_hsp_perc float_value] [-max_hsps int_value] [-xdrop_ungap float_value] [-xdrop_gap float_value] [-xdrop_gap_final float_value] [-searchsp int_value] [-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking] [-matrix matrix_name] [-threshold float_value] [-culling_limit int_value] [-best_hit_overhang float_value] [-best_hit_score_edge float_value] [-window_size int_value] [-lcase_masking] [-query_loc range] [-parse_deflines] [-outfmt format] [-show_gis] [-num_descriptions int_value] [-num_alignments int_value] [-line_length line_length] [-html] [-max_target_seqs num_sequences] [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo] [-use_sw_tback] [-version] Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Mon Feb 13 08:02:30 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Mon, 13 Feb 2017 09:02:30 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" In-Reply-To: References: Message-ID: Hi Quanwei, Different versions/implementations of blast do have parameters. Based on the usage you posted my guess is that the parameter you want is ?-max_hsps? Thanks, Mike > On Feb 10, 2017, at 11:53 PM, Quanwei Zhang wrote: > > Hello: > > I want to assign gene function to the predicted genes. I collected UniProt/SwissProt human, mouse and rat protein sequences (including both canonical and isoforms). Then use "makeblastdb" to build the database. And then run "blastp -db ... -max_hsps_per_subject 1" following the example in the protocol. But it returns me an error: Unknown argument: "max_hsps_per_subject". Why this happens? Is it because I am using a different version of blastp? > > USAGE > blastp [-h] [-help] [-import_search_strategy filename] > [-export_search_strategy filename] [-task task_name] [-db database_name] > [-dbsize num_letters] [-gilist filename] [-seqidlist filename] > [-negative_gilist filename] [-entrez_query entrez_query] > [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm] > [-subject subject_input_file] [-subject_loc range] [-query input_file] > [-out output_file] [-evalue evalue] [-word_size int_value] > [-gapopen open_penalty] [-gapextend extend_penalty] > [-qcov_hsp_perc float_value] [-max_hsps int_value] > [-xdrop_ungap float_value] [-xdrop_gap float_value] > [-xdrop_gap_final float_value] [-searchsp int_value] > [-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking] > [-matrix matrix_name] [-threshold float_value] [-culling_limit int_value] > [-best_hit_overhang float_value] [-best_hit_score_edge float_value] > [-window_size int_value] [-lcase_masking] [-query_loc range] > [-parse_deflines] [-outfmt format] [-show_gis] > [-num_descriptions int_value] [-num_alignments int_value] > [-line_length line_length] [-html] [-max_target_seqs num_sequences] > [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo] > [-use_sw_tback] [-version] > > Thanks > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From qwzhang0601 at gmail.com Mon Feb 13 09:02:24 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 10:02:24 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" In-Reply-To: References: Message-ID: Thanks. Yes, I should use "-max_hsps". Best Quanwei 2017-02-13 9:02 GMT-05:00 Michael Campbell : > Hi Quanwei, > > Different versions/implementations of blast do have parameters. Based on > the usage you posted my guess is that the parameter you want is ?-max_hsps? > > Thanks, > Mike > > On Feb 10, 2017, at 11:53 PM, Quanwei Zhang > wrote: > > > > Hello: > > > > I want to assign gene function to the predicted genes. I collected > UniProt/SwissProt human, mouse and rat protein sequences (including both > canonical and isoforms). Then use "makeblastdb" to build the database. And > then run "blastp -db ... -max_hsps_per_subject 1" following the example in > the protocol. But it returns me an error: Unknown argument: > "max_hsps_per_subject". Why this happens? Is it because I am using a > different version of blastp? > > > > USAGE > > blastp [-h] [-help] [-import_search_strategy filename] > > [-export_search_strategy filename] [-task task_name] [-db > database_name] > > [-dbsize num_letters] [-gilist filename] [-seqidlist filename] > > [-negative_gilist filename] [-entrez_query entrez_query] > > [-db_soft_mask filtering_algorithm] [-db_hard_mask > filtering_algorithm] > > [-subject subject_input_file] [-subject_loc range] [-query > input_file] > > [-out output_file] [-evalue evalue] [-word_size int_value] > > [-gapopen open_penalty] [-gapextend extend_penalty] > > [-qcov_hsp_perc float_value] [-max_hsps int_value] > > [-xdrop_ungap float_value] [-xdrop_gap float_value] > > [-xdrop_gap_final float_value] [-searchsp int_value] > > [-sum_stats bool_value] [-seg SEG_options] [-soft_masking > soft_masking] > > [-matrix matrix_name] [-threshold float_value] [-culling_limit > int_value] > > [-best_hit_overhang float_value] [-best_hit_score_edge float_value] > > [-window_size int_value] [-lcase_masking] [-query_loc range] > > [-parse_deflines] [-outfmt format] [-show_gis] > > [-num_descriptions int_value] [-num_alignments int_value] > > [-line_length line_length] [-html] [-max_target_seqs num_sequences] > > [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats > compo] > > [-use_sw_tback] [-version] > > > > Thanks > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 09:16:35 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 10:16:35 -0500 Subject: [maker-devel] failed to assign putative gene function Message-ID: Hello: I am trying to add putative gene function to the predicted gene models. Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. I used canonical and isoform proteins of human, mouse and rat with the script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose context is as below. maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 7e-164 464 snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 3e-80 236 After that, I am trying to add the protein homology data to the Maker gff3 and fasta files with maker_functional_gff and maker_functional_fasta, but get the reports as below. Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 39. Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 39. Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 45. Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 45. Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B ..... I am not sure how to deal with this. I followed the command given in the protocol. Any suggestions? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 10:09:48 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 11:09:48 -0500 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: I just found at the bottom of the output file it include information like below. But I did not get those GFF3 files with gene homolog information added. >snap_masked-CasCan_contig_27993-processed-gene-0.6-mRNA-1 transcript Name:"Protein of unknown function" offset:0 AED:0.47 eAED:0.47 QI:0|0|0|1|1|1|2|0|84 ATGAAAGACATTGGTACCCCAGAGGCATGGCAGATAATGATGTCCCTCAAGTCTGGACTC TTGGCAGAGATCACATGGGCTTTAGACACCATTAACATTCTACTGTATGATGACAGCAGC ATTATGACCTTCAACCTCAGTCAGTTCCCAGGATTGCTAGAGCTCTTTGAGTATGAGGTG GGTGACCGAAGACAGAGAACTCTACTGGACTCTGGGAGATTCAGTGAAGTGTCTGGTCCA ACCCCTACAGAG Thanks Best Quanwei 2017-02-13 10:16 GMT-05:00 Quanwei Zhang : > Hello: > > I am trying to add putative gene function to the predicted gene models. > Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. > I used canonical and isoform proteins of human, mouse and rat with the > script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose > context is as below. > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN > 69.97 303 91 0 1 303 1 303 7e-164 464 > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 > sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 > 3e-80 236 > > After that, I am trying to add the protein homology data to the Maker gff3 > and fasta files with maker_functional_gff and maker_functional_fasta, but > get the reports as below. > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 39. > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 39. > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 45. > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 45. > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > ..... > > I am not sure how to deal with this. I followed the command given in the > protocol. Any suggestions? > > Thanks > > Best > Quanwei > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 11:03:13 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 12:03:13 -0500 Subject: [maker-devel] Which version of InterProScan should I use Message-ID: I am planning to add domain information to my predicted gene models. But not sure whether the scripts provided by Maker is compatible with the latest version of InterProScan, or should I use a old version. I am using maker2.31.9. It seems the latest version is 5.22-61.0 released on 1/23/17. ftp://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/ Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcui at age.mpg.de Tue Feb 14 06:38:27 2017 From: rcui at age.mpg.de (Ray Cui) Date: Tue, 14 Feb 2017 13:38:27 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints Message-ID: Hello, I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Tue Feb 14 08:45:29 2017 From: dence at genetics.utah.edu (Daniel Ence) Date: Tue, 14 Feb 2017 14:45:29 +0000 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: Message-ID: Hi Ray, I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. ~Daniel On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: Hello, I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcui at age.mpg.de Tue Feb 14 09:44:19 2017 From: rcui at age.mpg.de (Ray Cui) Date: Tue, 14 Feb 2017 16:44:19 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> Message-ID: Hi Daniel, thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel wrote: > Hi Ray, > > I think you?re on the right track with training Genemark with RNAseq data. > It should only change the training steps, which are external to MAKER, but > not how MAKER runs Genemark. You?ll still give MAKER the path to the > ?es.mod" file made by Genemark. > > For the 2nd question, in the MAKER beta 3, MAKER creates a control file > for EVM, in which you set your weights for the various inputs, and then > MAKER runs EVM alongside all the other gene predictors and chooses the > model that is best supported by the evidence. > > ~Daniel > > > > On Feb 14, 2017, at 7:38 AM, Ray Cui wrote: > > Hello, > > I have sucessfully installed Maker beta 3, working with both > Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio > predictor. > When I read the GeneMark-ES manual, it says that one can use > RNAseq data to aid training. I'm wondering what would be the best way to > integrate Genemark-ET predictions into Maker. Should I run Genemark-ET > independent of Maker, then integrate the GFF at some point during the maker > process? If so, how should I edit the configuration file? Currently maker > has an option called "gmhmm". Should I then train GeneMark by myself with > RNAseq data, then feed the hmm to maker? > > And perhaps an unrelated question is that now Maker beta 3 > supports EVM. I'm wondering how EVM is used by Maker (at which step, what > does it do), and how does it differ from what Maker is designed for (both > reconciles different gene models). > > Best Regards, > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for > Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 <+49%20221%20496> > Mobile: +49 0221 37970 496 > rcui at age.mpg.de > www.age.mpg.de > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Tue Feb 14 13:51:46 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Tue, 14 Feb 2017 14:51:46 -0500 Subject: [maker-devel] Which version of InterProScan should I use In-Reply-To: References: Message-ID: <6A140E2D-603E-46C9-97FA-D59AEAEF95A9@gmail.com> I have been able to make the accessory scripts work with output from InterProScan v5 that I downloaded a couple of month ago. Mike > On Feb 13, 2017, at 12:03 PM, Quanwei Zhang wrote: > > I am planning to add domain information to my predicted gene models. But not sure whether the scripts provided by Maker is compatible with the latest version of InterProScan, or should I use a old version. I am using maker2.31.9. > > It seems the latest version is 5.22-61.0 released on 1/23/17. > ftp://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/ > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Thu Feb 16 04:44:39 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Thu, 16 Feb 2017 11:44:39 +0100 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> Message-ID: <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> Hi! Unfortunately all of the options failed on our cluster. See: Hi, Most recent Maker test with --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 Error: --> rank=2, hostname=uc1n518.localdomain [uc1n518:67009] *** Process received signal *** [uc1n518:67009] Signal: Segmentation fault (11) [uc1n518:67009] Signal code: Address not mapped (1) [uc1n518:67009] Failing at address: 0x4b0 -------------------------------------------------------------------------- mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). With: --mca btl ^openib and also this --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 Error: ### Runing Maker example STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... [uc1n514:59985] *** Process received signal *** [uc1n514:59985] Signal: Segmentation fault (11) [uc1n514:59985] Signal code: Address not mapped (1) [uc1n514:59985] Failing at address: 0x4b0 -------------------------------------------------------------------------- mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). Am 28.01.2017 um 21:53 schrieb Carson Holt: > Try adding one of the following to your mpiexec command ?> > > 1. --mca btl ^openib > 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 > 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 > > One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. > > --Carson > > >> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >> >> Hi everybody. >> >> My name is Rainer. I am an administrator for our HPC-Systems at our >> university in Konstanz, Baden-Wuertemberg/Germany. >> The procect is called bwHPC-C5. >> >> See: https://www.bwhpc-c5.de/en/index.php >> >> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >> i get errors while running a Maker job in the MPI-environment. >> >> BUILD STATUS >> >> ============================================================================== >> STATUS MAKER v2.31.9 >> ============================================================================== >> PERL Dependencies: VERIFIED >> External Programs: VERIFIED >> External C Libraries: VERIFIED >> MPI SUPPORT: ENABLED >> MWAS Web Interface: DISABLED >> MAKER PACKAGE: CONFIGURATION OK >> >> MODULES / INCLUDES / COMPILERS >> >> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >> # >> ##### (B) Dependencies: >> # >> # conflict: any other maker version >> # module load compiler/gnu/5.2 >> # module load mpi/openmpi/2.0-gnu-5.2 >> [...] >> >> MPI/MOAB SUBMIT >> >> [...] >> ### Queues ### >> #MSUB -q fat >> #MSUB -l nodes=1:ppn=16 >> #MSUB -l mem=20gb >> #MSUB -l walltime=50:00:00 >> # >> [...] >> echo " " >> echo "### Loading MAKER module:" >> echo " " >> module load bio/maker/2.31.9 >> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >> echo "MAKER_VERSION = $MAKER_VERSION" >> module list >> [...] >> echo " " >> echo "### Runing Maker example" >> echo " " >> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >> export OMPI_MCA_mpi_warn_on_fork=0 >> >> echo "LD_PRELOAD=${LD_PRELOAD}" >> # >> # "STATUS: Processing and indexing input FASTA files..." >> # >> mpiexec -mca btl ^openib -n 16 maker >> [...] >> >> >> E R R O R S >> ======= >> [...] >> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >> STATUS: Parsing control files... >> STATUS: Processing and indexing input FASTA files... >> [uc1n338:113607] *** Process received signal *** >> [uc1n338:113607] Signal: Segmentation fault (11) >> [uc1n338:113607] Signal code: Address not mapped (1) >> [uc1n338:113607] Failing at address: 0x4b0 >> [uc1n338:113608] *** Process received signal *** >> [uc1n338:113608] Signal: Segmentation fault (11) >> [uc1n338:113608] Signal code: Address not mapped (1) >> [uc1n338:113608] Failing at address: 0x4b0 >> [uc1n338:113621] *** Process received signal *** >> [uc1n338:113621] Signal: Segmentation fault (11) >> [uc1n338:113621] Signal code: Address not mapped (1) >> [uc1n338:113621] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >> -------------------------------------------------------------------------- >> [...] >> >> WHATS WRONG HERE!? >> >> Thank you for your help! >> >> All the best , >> >> Rainer >> >> -- >> Rainer Rutka >> University of Konstanz >> Communication, Information, Media Centre (KIM) >> * High-Performance-Computing (HPC) >> * KIM-Support and -Base-Services >> Room: V511 >> 78457 Konstanz, Germany >> +49 7531 88-5413 >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- Rainer Rutka Universit?t Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Raum: V511, Tel: 54 13 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From munholl at uwindsor.ca Fri Feb 17 14:11:44 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Fri, 17 Feb 2017 15:11:44 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there Message-ID: Hi Everyone, After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. However I also see $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 17 14:39:33 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 17 Feb 2017 13:39:33 -0700 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: Message-ID: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. Thanks, Carson > On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: > > Hi Everyone, > > After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl > > (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: > > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > However I also see > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 17 15:57:05 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 17 Feb 2017 16:57:05 -0500 Subject: [maker-devel] default gene model by quality_filter.pl is the same with those by setting keep_preds=0? Message-ID: Hello: I am following the Maker protocol (2014) to rescue rejected gene models. I set keep_preds=1 when I ran Maker, and then used "quality_filter.pl -d" to get the "default" Maker report. According to the annotation it will "Prints transcripts with an AED <1". But in a previous post (the link below), it was said "there are models with AED less than 1 that get rejected for other reasons. ... For example if the only evidence is a protein alignment that has deep overlapping HSPs (extremely low complexity alignment) it will be filtered out even though AED is not technically equal to 1. Also if the overlapping protein evidence is in a different reading frame than the model it is supposed to support then the AED will be less than 1 but eAED will be 1 (extended AED), and the model will be rejected." *So I wonder, whether the "default" result obtained by "quality_filter.pl " are really the same as those achieved by setting keep_preds=0? Will the script also consider other reasons besides "* *AED <1"?* https://groups.google.com/forum/#!searchin/maker-devel/The$20reason$20there$20are$20differences$20between$20the$20runs$20is$20that$20there$20are$20models$20with$20AED$20less$20%7Csort:relevance/maker-devel/97aNJkT3bgk/W0MEyqEZAAAJ Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qlian003 at ucr.edu Sat Feb 18 10:43:08 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Sat, 18 Feb 2017 08:43:08 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation Message-ID: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> Dear Maker develop team, I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? Thanks Qihua From carsonhh at gmail.com Sun Feb 19 23:39:14 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 22:39:14 -0700 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> Message-ID: <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> MAKER was support was designed with GeneMark-ES. It may or may not work with GeneMark-ET. So any MAKER related archive posts etc. will be related to the latter. With GeneMark-ES, you simply provided a genome assembly and let it run. It would then produce several files and output directories. The es.mod file was the one you provided to MAKER. I don?t know how this compares to GeneMark-ET. ?Carson > On Feb 14, 2017, at 8:44 AM, Ray Cui wrote: > > Hi Daniel, > > thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? > > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 > Mobile: +49 0221 37970 496 <> > rcui at age.mpg.de > www.age.mpg.de > > > > On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel > wrote: > Hi Ray, > > I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. > > For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. > > ~Daniel > > > >> On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: >> >> Hello, >> >> I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. >> When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? >> >> And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). >> >> Best Regards, >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 >> Mobile: +49 0221 37970 496 <> >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Sun Feb 19 23:43:49 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 22:43:49 -0700 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> Message-ID: <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> Try running just on a single node (not across nodes). If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and running MAKER with that new version. You can install it in your home directory and test from there, just make sure to add it to your path. Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. ?Carson > On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: > > Hi! > > Unfortunately all of the options failed on our cluster. > > See: > > Hi, > > Most recent Maker test with > --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 > Error: > --> rank=2, hostname=uc1n518.localdomain > [uc1n518:67009] *** Process received signal *** > [uc1n518:67009] Signal: Segmentation fault (11) > [uc1n518:67009] Signal code: Address not mapped (1) > [uc1n518:67009] Failing at address: 0x4b0 > -------------------------------------------------------------------------- > mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). > > > With: > --mca btl ^openib > and also this > --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 > Error: > ### Runing Maker example > > STATUS: Parsing control files... > STATUS: Processing and indexing input FASTA files... > [uc1n514:59985] *** Process received signal *** > [uc1n514:59985] Signal: Segmentation fault (11) > [uc1n514:59985] Signal code: Address not mapped (1) > [uc1n514:59985] Failing at address: 0x4b0 > -------------------------------------------------------------------------- > mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). > > > Am 28.01.2017 um 21:53 schrieb Carson Holt: >> Try adding one of the following to your mpiexec command ?> >> >> 1. --mca btl ^openib >> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >> >> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >> >> --Carson >> >> >>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>> >>> Hi everybody. >>> >>> My name is Rainer. I am an administrator for our HPC-Systems at our >>> university in Konstanz, Baden-Wuertemberg/Germany. >>> The procect is called bwHPC-C5. >>> >>> See: https://www.bwhpc-c5.de/en/index.php >>> >>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>> i get errors while running a Maker job in the MPI-environment. >>> >>> BUILD STATUS >>> >>> ============================================================================== >>> STATUS MAKER v2.31.9 >>> ============================================================================== >>> PERL Dependencies: VERIFIED >>> External Programs: VERIFIED >>> External C Libraries: VERIFIED >>> MPI SUPPORT: ENABLED >>> MWAS Web Interface: DISABLED >>> MAKER PACKAGE: CONFIGURATION OK >>> >>> MODULES / INCLUDES / COMPILERS >>> >>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>> # >>> ##### (B) Dependencies: >>> # >>> # conflict: any other maker version >>> # module load compiler/gnu/5.2 >>> # module load mpi/openmpi/2.0-gnu-5.2 >>> [...] >>> >>> MPI/MOAB SUBMIT >>> >>> [...] >>> ### Queues ### >>> #MSUB -q fat >>> #MSUB -l nodes=1:ppn=16 >>> #MSUB -l mem=20gb >>> #MSUB -l walltime=50:00:00 >>> # >>> [...] >>> echo " " >>> echo "### Loading MAKER module:" >>> echo " " >>> module load bio/maker/2.31.9 >>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>> echo "MAKER_VERSION = $MAKER_VERSION" >>> module list >>> [...] >>> echo " " >>> echo "### Runing Maker example" >>> echo " " >>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>> export OMPI_MCA_mpi_warn_on_fork=0 >>> >>> echo "LD_PRELOAD=${LD_PRELOAD}" >>> # >>> # "STATUS: Processing and indexing input FASTA files..." >>> # >>> mpiexec -mca btl ^openib -n 16 maker >>> [...] >>> >>> >>> E R R O R S >>> ======= >>> [...] >>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>> STATUS: Parsing control files... >>> STATUS: Processing and indexing input FASTA files... >>> [uc1n338:113607] *** Process received signal *** >>> [uc1n338:113607] Signal: Segmentation fault (11) >>> [uc1n338:113607] Signal code: Address not mapped (1) >>> [uc1n338:113607] Failing at address: 0x4b0 >>> [uc1n338:113608] *** Process received signal *** >>> [uc1n338:113608] Signal: Segmentation fault (11) >>> [uc1n338:113608] Signal code: Address not mapped (1) >>> [uc1n338:113608] Failing at address: 0x4b0 >>> [uc1n338:113621] *** Process received signal *** >>> [uc1n338:113621] Signal: Segmentation fault (11) >>> [uc1n338:113621] Signal code: Address not mapped (1) >>> [uc1n338:113621] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>> -------------------------------------------------------------------------- >>> [...] >>> >>> WHATS WRONG HERE!? >>> >>> Thank you for your help! >>> >>> All the best , >>> >>> Rainer >>> >>> -- >>> Rainer Rutka >>> University of Konstanz >>> Communication, Information, Media Centre (KIM) >>> * High-Performance-Computing (HPC) >>> * KIM-Support and -Base-Services >>> Room: V511 >>> 78457 Konstanz, Germany >>> +49 7531 88-5413 >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -- > Rainer Rutka > Universit?t Konstanz > Kommunikations-, Informations-, Medienzentrum (KIM) > Raum: V511, Tel: 54 13 > From carsonhh at gmail.com Mon Feb 20 00:05:09 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 23:05:09 -0700 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> Message-ID: <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. Thanks, Carson > On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: > > Dear Maker develop team, > > I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? > > Thanks > Qihua > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From rcui at age.mpg.de Mon Feb 20 02:59:12 2017 From: rcui at age.mpg.de (Ray Cui) Date: Mon, 20 Feb 2017 09:59:12 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> Message-ID: Dear Carson, I have now run GeneMark-ET, and it produces a trained .mod file. I think it can be then passed to Maker. Do you know what is the final constructed command line in Maker that calls genemark? Genemark-et and es use the same perl script so one probably only needs to use the --prediction and --predict_with xxx.mod options to predict genes using the species specific parameters (bypassing regular training and prediction steps) Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de On Mon, Feb 20, 2017 at 6:39 AM, Carson Holt wrote: > MAKER was support was designed with GeneMark-ES. It may or may not work > with GeneMark-ET. So any MAKER related archive posts etc. will be related > to the latter. > > With GeneMark-ES, you simply provided a genome assembly and let it run. It > would then produce several files and output directories. The es.mod file > was the one you provided to MAKER. I don?t know how this compares > to GeneMark-ET. > > ?Carson > > > > On Feb 14, 2017, at 8:44 AM, Ray Cui wrote: > > Hi Daniel, > > thanks! It seems that Genemark-ET has a "--training" flag, is that > the flag I should use when training or should I just let Genemark also > perform the prediction? > > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for > Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 <+49%20221%20496> > Mobile: +49 0221 37970 496 > rcui at age.mpg.de > www.age.mpg.de > > > > On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel wrote: > >> Hi Ray, >> >> I think you?re on the right track with training Genemark with RNAseq >> data. It should only change the training steps, which are external to >> MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to >> the ?es.mod" file made by Genemark. >> >> For the 2nd question, in the MAKER beta 3, MAKER creates a control file >> for EVM, in which you set your weights for the various inputs, and then >> MAKER runs EVM alongside all the other gene predictors and chooses the >> model that is best supported by the evidence. >> >> ~Daniel >> >> >> >> On Feb 14, 2017, at 7:38 AM, Ray Cui wrote: >> >> Hello, >> >> I have sucessfully installed Maker beta 3, working with both >> Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio >> predictor. >> When I read the GeneMark-ES manual, it says that one can use >> RNAseq data to aid training. I'm wondering what would be the best way to >> integrate Genemark-ET predictions into Maker. Should I run Genemark-ET >> independent of Maker, then integrate the GFF at some point during the maker >> process? If so, how should I edit the configuration file? Currently maker >> has an option called "gmhmm". Should I then train GeneMark by myself with >> RNAseq data, then feed the hmm to maker? >> >> And perhaps an unrelated question is that now Maker beta 3 >> supports EVM. I'm wondering how EVM is used by Maker (at which step, what >> does it do), and how does it differ from what Maker is designed for (both >> reconciles different gene models). >> >> Best Regards, >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for >> Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 <+49%20221%20496> >> Mobile: +49 0221 37970 496 >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 20 03:02:00 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 02:02:00 -0700 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> Message-ID: The names of scripts used are listed in the maker_exe.ctl file. It depends on if formatting or any flags have changed between versions. ?Carson > On Feb 20, 2017, at 1:59 AM, Ray Cui wrote: > > Dear Carson, > > I have now run GeneMark-ET, and it produces a trained .mod file. I think it can be then passed to Maker. Do you know what is the final constructed command line in Maker that calls genemark? Genemark-et and es use the same perl script so one probably only needs to use the --prediction and --predict_with xxx.mod options to predict genes using the species specific parameters (bypassing regular training and prediction steps) > > > Best Regards, > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 > Mobile: +49 0221 37970 496 <> > rcui at age.mpg.de > www.age.mpg.de > > > > On Mon, Feb 20, 2017 at 6:39 AM, Carson Holt > wrote: > MAKER was support was designed with GeneMark-ES. It may or may not work with GeneMark-ET. So any MAKER related archive posts etc. will be related to the latter. > > With GeneMark-ES, you simply provided a genome assembly and let it run. It would then produce several files and output directories. The es.mod file was the one you provided to MAKER. I don?t know how this compares to GeneMark-ET. > > ?Carson > > > >> On Feb 14, 2017, at 8:44 AM, Ray Cui > wrote: >> >> Hi Daniel, >> >> thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? >> >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 >> Mobile: +49 0221 37970 496 <> >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> >> On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel > wrote: >> Hi Ray, >> >> I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. >> >> For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. >> >> ~Daniel >> >> >> >>> On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: >>> >>> Hello, >>> >>> I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. >>> When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? >>> >>> And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). >>> >>> Best Regards, >>> Ray >>> >>> Dr. Rongfeng (Ray) Cui >>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >>> Wissenschaftlicher MA / Postdoctoral researcher >>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >>> Tel.:+49 (0)221 496 >>> Mobile: +49 0221 37970 496 <> >>> rcui at age.mpg.de >>> www.age.mpg.de >>> >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qlian003 at ucr.edu Mon Feb 20 14:34:31 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Mon, 20 Feb 2017 12:34:31 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> Message-ID: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Hi Carson, Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? Thanks Qihua > On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: > > IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. > > CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. > > If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. > > What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. > > However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. > > Thanks, > Carson > >> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >> >> Dear Maker develop team, >> >> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >> >> Thanks >> Qihua >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > From carsonhh at gmail.com Mon Feb 20 17:56:01 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 16:56:01 -0700 Subject: [maker-devel] default gene model by quality_filter.pl is the same with those by setting keep_preds=0? In-Reply-To: References: Message-ID: <14B6C165-54D2-4722-9A26-91FBEBB18B72@gmail.com> Any hint based predictions rejected for one of the non-AED filters will still be rejected regardless of keep_preds=1. However you are right in that any raw predictions that were rejected by other filters may still be kept by using this workflow, since keep_preds=1 essentially rescues all non-overlapping raw predictions that would have been rejected. ?Carson > On Feb 17, 2017, at 2:57 PM, Quanwei Zhang wrote: > > Hello: > > I am following the Maker protocol (2014) to rescue rejected gene models. I set keep_preds=1 when I ran Maker, and then used "quality_filter.pl -d" to get the "default" Maker report. According to the annotation it will "Prints transcripts with an AED <1". > > But in a previous post (the link below), it was said "there are models with AED less than 1 that get rejected for other reasons. ... For example if the only evidence is a protein alignment that has deep overlapping HSPs (extremely low complexity alignment) it will be filtered out even though AED is not technically equal to 1. Also if the overlapping protein evidence is in a different reading frame than the model it is supposed to support then the AED will be less than 1 but eAED will be 1 (extended AED), and the model will be rejected." > > So I wonder, whether the "default" result obtained by "quality_filter.pl " are really the same as those achieved by setting keep_preds=0? Will the script also consider other reasons besides "AED <1"? > > https://groups.google.com/forum/#!searchin/maker-devel/The$20reason$20there$20are$20differences$20between$20the$20runs$20is$20that$20there$20are$20models$20with$20AED$20less$20%7Csort:relevance/maker-devel/97aNJkT3bgk/W0MEyqEZAAAJ > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 20 17:56:02 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 16:56:02 -0700 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: You either uses TrEMBL or the UniProtKB Isoform sequence set. Their fasta headers are slightly different and will not be parsed correctly, For example, here is the header as formatted for the same sequence in the Swiss-prot dataset download ?> >sp|Q7Z5M8|AB12B_HUMAN Protein ABHD12B OS=Homo sapiens GN=ABHD12B PE=2 SV=1 I think you used the UniProtKB Isoform sequence dataset instead. ?Carson > On Feb 13, 2017, at 8:16 AM, Quanwei Zhang wrote: > > Hello: > > I am trying to add putative gene function to the predicted gene models. Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. I used canonical and isoform proteins of human, mouse and rat with the script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose context is as below. > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 7e-164 464 > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 3e-80 236 > > After that, I am trying to add the protein homology data to the Maker gff3 and fasta files with maker_functional_gff and maker_functional_fasta, but get the reports as below. > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 39. > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 39. > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 45. > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 45. > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > ..... > > I am not sure how to deal with this. I followed the command given in the protocol. Any suggestions? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From qwzhang0601 at gmail.com Tue Feb 21 08:30:35 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Tue, 21 Feb 2017 09:30:35 -0500 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: Thank you! I used both canonical and isoform of protein sequences from swiss-prot in the beginning. It reported the error, but later I only used the canonical protein sequences to build the database and then it worked. Best Quanwei 2017-02-20 18:56 GMT-05:00 Carson Holt : > You either uses TrEMBL or the UniProtKB Isoform sequence set. Their fasta > headers are slightly different and will not be parsed correctly, > > For example, here is the header as formatted for the same sequence in the > Swiss-prot dataset download ?> > >sp|Q7Z5M8|AB12B_HUMAN Protein ABHD12B OS=Homo sapiens GN=ABHD12B PE=2 SV=1 > > I think you used the UniProtKB Isoform sequence dataset instead. > > ?Carson > > > > > > > On Feb 13, 2017, at 8:16 AM, Quanwei Zhang > wrote: > > > > Hello: > > > > I am trying to add putative gene function to the predicted gene models. > Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. > I used canonical and isoform proteins of human, mouse and rat with the > script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose > context is as below. > > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 > sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 > 7e-164 464 > > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 > sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 > 3e-80 236 > > > > After that, I am trying to add the protein homology data to the Maker > gff3 and fasta files with maker_functional_gff and maker_functional_fasta, > but get the reports as below. > > > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform > 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 39. > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 39. > > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform > 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 45. > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 45. > > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform > 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > ..... > > > > I am not sure how to deal with this. I followed the command given in the > protocol. Any suggestions? > > > > Thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Tue Feb 21 11:17:22 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 12:17:22 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Message-ID: I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: cpan[1]> install forks::signals Warning: Cannot install forks::signals, don't know what it is. Try the command i /forks::signals/ to find objects with matching identifiers. but scrolling through the install scroll of the forks reinstall I do see it got installed properly. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: > Do a force reinstall of forks via CPAN. The error is coming from forks.pm line > 83, so the problem is the forks installation itself. > > Thanks, > Carson > > > On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: > > Hi Everyone, > > After sorting my MPICH/OpenMPI issue I have come across another: When I > try to run MAKER on the demo data via > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker > /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl > /Data/Apps/maker/data/maker_bopts.ctl > > (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it > working before opening up, if I change the -n value then the error repeats > once for each process I attempt to run via MPICH) I got the following: > > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > However I also see > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially > thought this was a Perl error. I hit up perlmonks and found that there is > an issue between forks and Storable, where Storable now has a verion number > of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these > instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and > tried again. Now I get: > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker > /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl > /Data/Apps/maker/data/maker_bopts.ctl > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > Clearly MAKER has an issue with forks, but it is installed, it is up to > date (I double checked via cpan and apt-get to be sure), and it is in a > directory that is in @INC. Should I pursue this as a MAKER error or as a > Perl error? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 21 11:19:30 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 21 Feb 2017 10:19:30 -0700 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Message-ID: <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> forks::signals is part of forks. It?s not a separate package. If it?s missing, there is a problem with the forks installation. So you need to do a force install of forks to force the reinstall. Example ?> cpan[1]> force install forks ?Carson > On Feb 21, 2017, at 10:17 AM, Seth Munholland wrote: > > I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: > > cpan[1]> install forks::signals > Warning: Cannot install forks::signals, don't know what it is. > Try the command > > i /forks::signals/ > > to find objects with matching identifiers. > > but scrolling through the install scroll of the forks reinstall I do see it got installed properly. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt > wrote: > Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. > > Thanks, > Carson > > >> On Feb 17, 2017, at 1:11 PM, Seth Munholland > wrote: >> >> Hi Everyone, >> >> After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl >> >> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: >> >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> However I also see >> $ locate forks.pm >> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >> >> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Tue Feb 21 12:05:48 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 13:05:48 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: I wasn't sure what modules were separated (figured it was wroth a shot), however, I did the force install of forks and I still get the same error. I tried uninstalling and reinstalling all the forks installations that showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then reloaded) to make sure I wasn't getting some kind of conflicting libraries/modules. and I'm back to: Can't locate forks.pm in @INC (you may need to install the forks module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. along with: $ sudo updatedb $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Now it's a maker issue, not a forks issue (if I'm reading it correctly), so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) and got: $ sudo ./Build install Installing MAKER... Configuring MAKER with MPI support Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 (unchanged) I don't believe that's an error or even a warning, but I get the same "can't locate forks" error when I try to run it. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt wrote: > forks::signals is part of forks. It?s not a separate package. If it?s > missing, there is a problem with the forks installation. So you need to do > a force install of forks to force the reinstall. > > Example ?> cpan[1]> force install forks > > ?Carson > > > On Feb 21, 2017, at 10:17 AM, Seth Munholland wrote: > > I tried this and still get the same error. Then I tried forcing a > reinstall of forks/signals and got: > > cpan[1]> install forks::signals > Warning: Cannot install forks::signals, don't know what it is. > Try the command > > i /forks::signals/ > > to find objects with matching identifiers. > > but scrolling through the install scroll of the forks reinstall I do see > it got installed properly. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: > >> Do a force reinstall of forks via CPAN. The error is coming from forks.pm line >> 83, so the problem is the forks installation itself. >> >> Thanks, >> Carson >> >> >> On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: >> >> Hi Everyone, >> >> After sorting my MPICH/OpenMPI issue I have come across another: When I >> try to run MAKER on the demo data via >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >> /Data/Apps/maker/data/maker_bopts.ctl >> >> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it >> working before opening up, if I change the -n value then the error repeats >> once for each process I attempt to run via MPICH) I got the following: >> >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> However I also see >> $ locate forks.pm >> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >> >> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially >> thought this was a Perl error. I hit up perlmonks and found that there is >> an issue between forks and Storable, where Storable now has a verion number >> of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these >> instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and >> tried again. Now I get: >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >> /Data/Apps/maker/data/maker_bopts.ctl >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> Clearly MAKER has an issue with forks, but it is installed, it is up to >> date (I double checked via cpan and apt-get to be sure), and it is in a >> directory that is in @INC. Should I pursue this as a MAKER error or as a >> Perl error? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bmoore at genetics.utah.edu Tue Feb 21 12:53:13 2017 From: bmoore at genetics.utah.edu (Barry Moore) Date: Tue, 21 Feb 2017 18:53:13 +0000 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: <78C26338-01EF-4F72-BFA4-EF46C70AFAEE@genetics.utah.edu> Hi Qihua, If you are using online version of SOBA, I would suggest you use the command line version found here https://github.com/The-Sequence-Ontology/SOBA as it is more flexible for the kinds of analyses you are talking about. If you are using ?footprint? as the --data_type argument you should get the nucleotide count for collapsed features that you are talking about. In addition I suggest you take a look at bedtools (http://bedtools.readthedocs.io/en/latest/index.html) for example bedtools merge as a flexible way to generate the kind of merged features you want and then you can always pass that output of that through SOBAcl for counting, graphing and reporting. Finally, if you want a great deal of flexibility in generating your own manipulation and reporting of GFF3 files that is beyond the scope of SOBA and/or BedTools, I suggest you take a look at the GAL library (https://github.com/The-Sequence-Ontology/GAL) if you don?t mind writing some perl code. Regards, Barry On Feb 20, 2017, at 1:34 PM, Qihua Liang > wrote: Hi Carson, Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? Thanks Qihua On Feb 19, 2017, at 10:05 PM, Carson Holt > wrote: IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. Thanks, Carson On Feb 18, 2017, at 9:43 AM, Qihua Liang > wrote: Dear Maker develop team, I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? Thanks Qihua _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 21 15:34:07 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 21 Feb 2017 14:34:07 -0700 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: MAKER merges overlapping RepeatMasker results into a single longer feature. ?Carson > On Feb 20, 2017, at 1:34 PM, Qihua Liang wrote: > > Hi Carson, > > Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. > > I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. > > Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. > In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. > > But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. > > When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? > Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? > > Thanks > Qihua > > >> On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: >> >> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. >> >> CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. >> >> If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. >> >> What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. >> >> However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. >> >> Thanks, >> Carson >> >>> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >>> >>> Dear Maker develop team, >>> >>> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >>> >>> Thanks >>> Qihua >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > From munholl at uwindsor.ca Tue Feb 21 16:26:23 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 17:26:23 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: I found my errors. 1) In my machine file; the first entry was not the main node, so locate was looking on a different node than Maker 2) forks (and other modules) were not properly installed on every node in the cluster. I used CPAN to install them on each cluster and it is now working. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 1:05 PM, Seth Munholland wrote: > I wasn't sure what modules were separated (figured it was wroth a shot), > however, I did the force install of forks and I still get the same error. > I tried uninstalling and reinstalling all the forks installations that > showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then > reloaded) to make sure I wasn't getting some kind of conflicting > libraries/modules. and I'm back to: > > Can't locate forks.pm in @INC (you may need to install the forks module) > (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > > along with: > > $ sudo updatedb > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Now it's a maker issue, not a forks issue (if I'm reading it correctly), > so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) > and got: > > $ sudo ./Build install > Installing MAKER... > Configuring MAKER with MPI support > Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 > (unchanged) > > I don't believe that's an error or even a warning, but I get the same > "can't locate forks" error when I try to run it. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt wrote: > >> forks::signals is part of forks. It?s not a separate package. If it?s >> missing, there is a problem with the forks installation. So you need to do >> a force install of forks to force the reinstall. >> >> Example ?> cpan[1]> force install forks >> >> ?Carson >> >> >> On Feb 21, 2017, at 10:17 AM, Seth Munholland >> wrote: >> >> I tried this and still get the same error. Then I tried forcing a >> reinstall of forks/signals and got: >> >> cpan[1]> install forks::signals >> Warning: Cannot install forks::signals, don't know what it is. >> Try the command >> >> i /forks::signals/ >> >> to find objects with matching identifiers. >> >> but scrolling through the install scroll of the forks reinstall I do see >> it got installed properly. >> >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> >> On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: >> >>> Do a force reinstall of forks via CPAN. The error is coming from >>> forks.pm line 83, so the problem is the forks installation itself. >>> >>> Thanks, >>> Carson >>> >>> >>> On Feb 17, 2017, at 1:11 PM, Seth Munholland >>> wrote: >>> >>> Hi Everyone, >>> >>> After sorting my MPICH/OpenMPI issue I have come across another: When I >>> try to run MAKER on the demo data via >>> >>> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >>> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >>> /Data/Apps/maker/data/maker_bopts.ctl >>> >>> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it >>> working before opening up, if I change the -n value then the error repeats >>> once for each process I attempt to run via MPICH) I got the following: >>> >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> >>> However I also see >>> $ locate forks.pm >>> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >>> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >>> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >>> >>> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially >>> thought this was a Perl error. I hit up perlmonks and found that there is >>> an issue between forks and Storable, where Storable now has a verion number >>> of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these >>> instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and >>> tried again. Now I get: >>> >>> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >>> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >>> /Data/Apps/maker/data/maker_bopts.ctl >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> >>> Clearly MAKER has an issue with forks, but it is installed, it is up to >>> date (I double checked via cpan and apt-get to be sure), and it is in a >>> directory that is in @INC. Should I pursue this as a MAKER error or as a >>> Perl error? >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Wed Feb 22 07:11:32 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Wed, 22 Feb 2017 14:11:32 +0100 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> Message-ID: <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> @Robert Kraus: FYI Am 20.02.2017 um 06:43 schrieb Carson Holt: > Try running just on a single node (not across nodes). THATS WHAT I DID. > If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and > running MAKER with that new version. You can install it in your home directory and test from there, > just make sure to add it to your path. Shure it is. > Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. > If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. OK, send the infos please. Thank you very much! > ?Carson > > > >> On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: >> >> Hi! >> >> Unfortunately all of the options failed on our cluster. >> >> See: >> >> Hi, >> >> Most recent Maker test with >> --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >> Error: >> --> rank=2, hostname=uc1n518.localdomain >> [uc1n518:67009] *** Process received signal *** >> [uc1n518:67009] Signal: Segmentation fault (11) >> [uc1n518:67009] Signal code: Address not mapped (1) >> [uc1n518:67009] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). >> >> >> With: >> --mca btl ^openib >> and also this >> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >> Error: >> ### Runing Maker example >> >> STATUS: Parsing control files... >> STATUS: Processing and indexing input FASTA files... >> [uc1n514:59985] *** Process received signal *** >> [uc1n514:59985] Signal: Segmentation fault (11) >> [uc1n514:59985] Signal code: Address not mapped (1) >> [uc1n514:59985] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). >> >> >> Am 28.01.2017 um 21:53 schrieb Carson Holt: >>> Try adding one of the following to your mpiexec command ?> >>> >>> 1. --mca btl ^openib >>> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>> >>> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >>> >>> --Carson >>> >>> >>>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>>> >>>> Hi everybody. >>>> >>>> My name is Rainer. I am an administrator for our HPC-Systems at our >>>> university in Konstanz, Baden-Wuertemberg/Germany. >>>> The procect is called bwHPC-C5. >>>> >>>> See: https://www.bwhpc-c5.de/en/index.php >>>> >>>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>>> i get errors while running a Maker job in the MPI-environment. >>>> >>>> BUILD STATUS >>>> >>>> ============================================================================== >>>> STATUS MAKER v2.31.9 >>>> ============================================================================== >>>> PERL Dependencies: VERIFIED >>>> External Programs: VERIFIED >>>> External C Libraries: VERIFIED >>>> MPI SUPPORT: ENABLED >>>> MWAS Web Interface: DISABLED >>>> MAKER PACKAGE: CONFIGURATION OK >>>> >>>> MODULES / INCLUDES / COMPILERS >>>> >>>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>>> # >>>> ##### (B) Dependencies: >>>> # >>>> # conflict: any other maker version >>>> # module load compiler/gnu/5.2 >>>> # module load mpi/openmpi/2.0-gnu-5.2 >>>> [...] >>>> >>>> MPI/MOAB SUBMIT >>>> >>>> [...] >>>> ### Queues ### >>>> #MSUB -q fat >>>> #MSUB -l nodes=1:ppn=16 >>>> #MSUB -l mem=20gb >>>> #MSUB -l walltime=50:00:00 >>>> # >>>> [...] >>>> echo " " >>>> echo "### Loading MAKER module:" >>>> echo " " >>>> module load bio/maker/2.31.9 >>>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>>> echo "MAKER_VERSION = $MAKER_VERSION" >>>> module list >>>> [...] >>>> echo " " >>>> echo "### Runing Maker example" >>>> echo " " >>>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>>> export OMPI_MCA_mpi_warn_on_fork=0 >>>> >>>> echo "LD_PRELOAD=${LD_PRELOAD}" >>>> # >>>> # "STATUS: Processing and indexing input FASTA files..." >>>> # >>>> mpiexec -mca btl ^openib -n 16 maker >>>> [...] >>>> >>>> >>>> E R R O R S >>>> ======= >>>> [...] >>>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>>> STATUS: Parsing control files... >>>> STATUS: Processing and indexing input FASTA files... >>>> [uc1n338:113607] *** Process received signal *** >>>> [uc1n338:113607] Signal: Segmentation fault (11) >>>> [uc1n338:113607] Signal code: Address not mapped (1) >>>> [uc1n338:113607] Failing at address: 0x4b0 >>>> [uc1n338:113608] *** Process received signal *** >>>> [uc1n338:113608] Signal: Segmentation fault (11) >>>> [uc1n338:113608] Signal code: Address not mapped (1) >>>> [uc1n338:113608] Failing at address: 0x4b0 >>>> [uc1n338:113621] *** Process received signal *** >>>> [uc1n338:113621] Signal: Segmentation fault (11) >>>> [uc1n338:113621] Signal code: Address not mapped (1) >>>> [uc1n338:113621] Failing at address: 0x4b0 >>>> -------------------------------------------------------------------------- >>>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>>> -------------------------------------------------------------------------- >>>> [...] >>>> >>>> WHATS WRONG HERE!? >>>> >>>> Thank you for your help! >>>> >>>> All the best , >>>> >>>> Rainer >>>> >>>> -- >>>> Rainer Rutka >>>> University of Konstanz >>>> Communication, Information, Media Centre (KIM) >>>> * High-Performance-Computing (HPC) >>>> * KIM-Support and -Base-Services >>>> Room: V511 >>>> 78457 Konstanz, Germany >>>> +49 7531 88-5413 >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> -- >> Rainer Rutka >> Universit?t Konstanz >> Kommunikations-, Informations-, Medienzentrum (KIM) >> Raum: V511, Tel: 54 13 >> > -- Rainer Rutka Universit?t Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Raum: V511, Tel: 54 13 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From cjfields at illinois.edu Wed Feb 22 07:30:22 2017 From: cjfields at illinois.edu (Fields, Christopher J) Date: Wed, 22 Feb 2017 13:30:22 +0000 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: <6E5122C0-A952-42D6-B205-74B5FF8255F2@illinois.edu> Just a note: when we install MAKER we generally install a clean version of perl if one isn?t already present, making sure it is on the NFS share for the cluster (which is accessible via all nodes). This works around most of the issues you describe. chris From: maker-devel > on behalf of Seth Munholland > Date: Tuesday, February 21, 2017 at 4:26 PM To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] MAKER can't find forks in @INC, but it is there I found my errors. 1) In my machine file; the first entry was not the main node, so locate was looking on a different node than Maker 2) forks (and other modules) were not properly installed on every node in the cluster. I used CPAN to install them on each cluster and it is now working. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 1:05 PM, Seth Munholland > wrote: I wasn't sure what modules were separated (figured it was wroth a shot), however, I did the force install of forks and I still get the same error. I tried uninstalling and reinstalling all the forks installations that showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then reloaded) to make sure I wasn't getting some kind of conflicting libraries/modules. and I'm back to: Can't locate forks.pm in @INC (you may need to install the forks module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. along with: $ sudo updatedb $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Now it's a maker issue, not a forks issue (if I'm reading it correctly), so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) and got: $ sudo ./Build install Installing MAKER... Configuring MAKER with MPI support Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 (unchanged) I don't believe that's an error or even a warning, but I get the same "can't locate forks" error when I try to run it. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt > wrote: forks::signals is part of forks. It?s not a separate package. If it?s missing, there is a problem with the forks installation. So you need to do a force install of forks to force the reinstall. Example ?> cpan[1]> force install forks ?Carson On Feb 21, 2017, at 10:17 AM, Seth Munholland > wrote: I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: cpan[1]> install forks::signals Warning: Cannot install forks::signals, don't know what it is. Try the command i /forks::signals/ to find objects with matching identifiers. but scrolling through the install scroll of the forks reinstall I do see it got installed properly. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt > wrote: Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. Thanks, Carson On Feb 17, 2017, at 1:11 PM, Seth Munholland > wrote: Hi Everyone, After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. However I also see $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 10:16:17 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 09:16:17 -0700 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> Message-ID: <24606FBA-4742-4F94-9447-208A385644C1@gmail.com> If OpenMPI fails on a single node, it means you have a compilation issue, which indicates a problem with your installation. This sometimes happens if you compiled on one node and run on another (if could either be MAEKR or OpenMPI itself that was compiled on another node). A few options you will need if trying intel MPI: -binding pin=disable #requires to disable processor affinity (otherwise MAKER calls to BLAST and other programs which are parallelized independent of MPI may not work) Environmental variables to set: export I_MPI_PIN_DOMAIN=node #otherwise MAKER calls to BLAST and other programs which are parallelized independent of MPI may not work export I_MPI_FABRICS='shm:tcp? #avoid potential complication with OpenFabrics libraries (they block system calls because of how they use registered memory, i.e. MAKER calling BLAST would fail) export I_MPI_HYDRA_IFACE=ib0 #set to eth0 if you don?t have an infiniband over ip inerface (required because of the above I_MPI_FABRICS change) Also make sure to compile on the node you run. You can try expanding to other nodes after that. ?Carson > On Feb 22, 2017, at 6:11 AM, Rainer Rutka wrote: > > @Robert Kraus: FYI > > Am 20.02.2017 um 06:43 schrieb Carson Holt: >> Try running just on a single node (not across nodes). > THATS WHAT I DID. > > >> If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and >> running MAKER with that new version. You can install it in your home directory and test from there, >> just make sure to add it to your path. > Shure it is. > >> Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. > > If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. > > OK, send the infos please. > > Thank you very much! > >> ?Carson >> >> >> >>> On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: >>> >>> Hi! >>> >>> Unfortunately all of the options failed on our cluster. >>> >>> See: >>> >>> Hi, >>> >>> Most recent Maker test with >>> --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>> Error: >>> --> rank=2, hostname=uc1n518.localdomain >>> [uc1n518:67009] *** Process received signal *** >>> [uc1n518:67009] Signal: Segmentation fault (11) >>> [uc1n518:67009] Signal code: Address not mapped (1) >>> [uc1n518:67009] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). >>> >>> >>> With: >>> --mca btl ^openib >>> and also this >>> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>> Error: >>> ### Runing Maker example >>> >>> STATUS: Parsing control files... >>> STATUS: Processing and indexing input FASTA files... >>> [uc1n514:59985] *** Process received signal *** >>> [uc1n514:59985] Signal: Segmentation fault (11) >>> [uc1n514:59985] Signal code: Address not mapped (1) >>> [uc1n514:59985] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). >>> >>> >>> Am 28.01.2017 um 21:53 schrieb Carson Holt: >>>> Try adding one of the following to your mpiexec command ?> >>>> >>>> 1. --mca btl ^openib >>>> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>>> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>>> >>>> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >>>> >>>> --Carson >>>> >>>> >>>>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>>>> >>>>> Hi everybody. >>>>> >>>>> My name is Rainer. I am an administrator for our HPC-Systems at our >>>>> university in Konstanz, Baden-Wuertemberg/Germany. >>>>> The procect is called bwHPC-C5. >>>>> >>>>> See: https://www.bwhpc-c5.de/en/index.php >>>>> >>>>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>>>> i get errors while running a Maker job in the MPI-environment. >>>>> >>>>> BUILD STATUS >>>>> >>>>> ============================================================================== >>>>> STATUS MAKER v2.31.9 >>>>> ============================================================================== >>>>> PERL Dependencies: VERIFIED >>>>> External Programs: VERIFIED >>>>> External C Libraries: VERIFIED >>>>> MPI SUPPORT: ENABLED >>>>> MWAS Web Interface: DISABLED >>>>> MAKER PACKAGE: CONFIGURATION OK >>>>> >>>>> MODULES / INCLUDES / COMPILERS >>>>> >>>>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>>>> # >>>>> ##### (B) Dependencies: >>>>> # >>>>> # conflict: any other maker version >>>>> # module load compiler/gnu/5.2 >>>>> # module load mpi/openmpi/2.0-gnu-5.2 >>>>> [...] >>>>> >>>>> MPI/MOAB SUBMIT >>>>> >>>>> [...] >>>>> ### Queues ### >>>>> #MSUB -q fat >>>>> #MSUB -l nodes=1:ppn=16 >>>>> #MSUB -l mem=20gb >>>>> #MSUB -l walltime=50:00:00 >>>>> # >>>>> [...] >>>>> echo " " >>>>> echo "### Loading MAKER module:" >>>>> echo " " >>>>> module load bio/maker/2.31.9 >>>>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>>>> echo "MAKER_VERSION = $MAKER_VERSION" >>>>> module list >>>>> [...] >>>>> echo " " >>>>> echo "### Runing Maker example" >>>>> echo " " >>>>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>>>> export OMPI_MCA_mpi_warn_on_fork=0 >>>>> >>>>> echo "LD_PRELOAD=${LD_PRELOAD}" >>>>> # >>>>> # "STATUS: Processing and indexing input FASTA files..." >>>>> # >>>>> mpiexec -mca btl ^openib -n 16 maker >>>>> [...] >>>>> >>>>> >>>>> E R R O R S >>>>> ======= >>>>> [...] >>>>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>>>> STATUS: Parsing control files... >>>>> STATUS: Processing and indexing input FASTA files... >>>>> [uc1n338:113607] *** Process received signal *** >>>>> [uc1n338:113607] Signal: Segmentation fault (11) >>>>> [uc1n338:113607] Signal code: Address not mapped (1) >>>>> [uc1n338:113607] Failing at address: 0x4b0 >>>>> [uc1n338:113608] *** Process received signal *** >>>>> [uc1n338:113608] Signal: Segmentation fault (11) >>>>> [uc1n338:113608] Signal code: Address not mapped (1) >>>>> [uc1n338:113608] Failing at address: 0x4b0 >>>>> [uc1n338:113621] *** Process received signal *** >>>>> [uc1n338:113621] Signal: Segmentation fault (11) >>>>> [uc1n338:113621] Signal code: Address not mapped (1) >>>>> [uc1n338:113621] Failing at address: 0x4b0 >>>>> -------------------------------------------------------------------------- >>>>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>>>> -------------------------------------------------------------------------- >>>>> [...] >>>>> >>>>> WHATS WRONG HERE!? >>>>> >>>>> Thank you for your help! >>>>> >>>>> All the best , >>>>> >>>>> Rainer >>>>> >>>>> -- >>>>> Rainer Rutka >>>>> University of Konstanz >>>>> Communication, Information, Media Centre (KIM) >>>>> * High-Performance-Computing (HPC) >>>>> * KIM-Support and -Base-Services >>>>> Room: V511 >>>>> 78457 Konstanz, Germany >>>>> +49 7531 88-5413 >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> >>> >>> -- >>> Rainer Rutka >>> Universit?t Konstanz >>> Kommunikations-, Informations-, Medienzentrum (KIM) >>> Raum: V511, Tel: 54 13 >>> >> > > -- > Rainer Rutka > Universit?t Konstanz > Kommunikations-, Informations-, Medienzentrum (KIM) > Raum: V511, Tel: 54 13 > From munholl at uwindsor.ca Wed Feb 22 12:03:54 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 13:03:54 -0500 Subject: [maker-devel] Failed while polishing ESTs Message-ID: After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: polishig ESTs running est2genome search. #--------- command -------------# Widget::exonerate::est2genome: /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate #-------------------------------# Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! --> rank=NA, hostname=beanblade2 ERROR: Failed while polishig ESTs ERROR: Chunk failed at level:2, tier_type:3 FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:contig-dpp-500-500 The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: polishig ESTs running est2genome search. #--------- command -------------# Widget::exonerate::est2genome: /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate #-------------------------------# Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! --> rank=NA, hostname=beanblade4 ERROR: Failed while polishig ESTs ERROR: Chunk failed at level:2, tier_type:3 FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:contig-dpp-500-500 So the error seems to be pointing at something happening when wrapping up the ESTs. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 12:16:41 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 11:16:41 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: Message-ID: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. And make sure to delete any run directories before retrying. ?Carson > On Feb 22, 2017, at 11:03 AM, Seth Munholland wrote: > > After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade2 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade4 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > So the error seems to be pointing at something happening when wrapping up the ESTs. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 12:57:00 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 13:57:00 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: It is the test data, the dpp set to be specific. I already checked all the installs to be sure they configured and compile without error and are up to date. I've been deleting between each run. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: > So you are running the test data job correct? So if you get any error with > the test data job, something is wrong with your installation. > > Make sure BioPerl is up to data (and the same everywhere). If using your > own installation of Perl, make sure it had BerkleyDB support when you > installed it (if you are using something like perlbrew , you may not have > BerkleyDB support and this may affect BioPerl indexing). Also try > reinstalling exonertate. > > And make sure to delete any run directories before retrying. > > ?Carson > > > On Feb 22, 2017, at 11:03 AM, Seth Munholland wrote: > > After my clustering issue I figured I would go node by node and run MAKER > locally on the test data set to be sure that it was working properly before > trying it as a full MPI run. After adjusting all my exes to point to the > NFS versions I am consistently getting the same error on each node when it > comes time to run exonerate: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q > /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna > --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent > 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp- > mRNA-5.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade2 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > The only results I find on google was an issue with an improper character > in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and > since they all point to dpp-mRNA-5 I backed up the provided est and removed > it. The response was: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q > /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna > --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent > 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp- > mRNA-4.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade4 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > So the error seems to be pointing at something happening when wrapping up > the ESTs. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 13:00:24 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 14:00:24 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland wrote: > It is the test data, the dpp set to be specific. > > I already checked all the installs to be sure they configured and compile > without error and are up to date. > > I've been deleting between each run. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: > >> So you are running the test data job correct? So if you get any error >> with the test data job, something is wrong with your installation. >> >> Make sure BioPerl is up to data (and the same everywhere). If using your >> own installation of Perl, make sure it had BerkleyDB support when you >> installed it (if you are using something like perlbrew , you may not have >> BerkleyDB support and this may affect BioPerl indexing). Also try >> reinstalling exonertate. >> >> And make sure to delete any run directories before retrying. >> >> ?Carson >> >> >> On Feb 22, 2017, at 11:03 AM, Seth Munholland >> wrote: >> >> After my clustering issue I figured I would go node by node and run MAKER >> locally on the test data set to be sure that it was working properly before >> trying it as a full MPI run. After adjusting all my exes to point to the >> NFS versions I am consistently getting the same error on each node when it >> comes time to run exonerate: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q >> /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t >> /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna >> --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent >> 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA- >> 5.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade2 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> The only results I find on google was an issue with an improper character >> in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and >> since they all point to dpp-mRNA-5 I backed up the provided est and removed >> it. The response was: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q >> /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t >> /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna >> --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent >> 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA- >> 4.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade4 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> So the error seems to be pointing at something happening when wrapping up >> the ESTs. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 13:03:42 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 12:03:42 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: I still think you may have something wrong somewhere with some part of your installation (rogue libraries, compiler incompatibilities, etc.). Especially given your earlier issues with Perl libraries. ?Carson > On Feb 22, 2017, at 12:00 PM, Seth Munholland wrote: > > On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. > > Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: > It is the test data, the dpp set to be specific. > > I already checked all the installs to be sure they configured and compile without error and are up to date. > > I've been deleting between each run. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt > wrote: > So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. > > Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. > > And make sure to delete any run directories before retrying. > > ?Carson > > >> On Feb 22, 2017, at 11:03 AM, Seth Munholland > wrote: >> >> After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade2 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade4 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> So the error seems to be pointing at something happening when wrapping up the ESTs. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 13:26:57 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 14:26:57 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: I'll wipe it clean and do a fresh install of everything to be 100% safe. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 2:03 PM, Carson Holt wrote: > I still think you may have something wrong somewhere with some part of > your installation (rogue libraries, compiler incompatibilities, etc.). > Especially given your earlier issues with Perl libraries. > > ?Carson > > > On Feb 22, 2017, at 12:00 PM, Seth Munholland wrote: > > On a whim I decided to switch to the hsap demo data and it completed > without issue. I went back to dpp and this time is completed. I hadn't > changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and > then inverted the change. > > Mayhaps there was something not unloaded from memory in an earlier run? > It's that, ghosts, or rogue ions :/ > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: > >> It is the test data, the dpp set to be specific. >> >> I already checked all the installs to be sure they configured and compile >> without error and are up to date. >> >> I've been deleting between each run. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> >> On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: >> >>> So you are running the test data job correct? So if you get any error >>> with the test data job, something is wrong with your installation. >>> >>> Make sure BioPerl is up to data (and the same everywhere). If using your >>> own installation of Perl, make sure it had BerkleyDB support when you >>> installed it (if you are using something like perlbrew , you may not have >>> BerkleyDB support and this may affect BioPerl indexing). Also try >>> reinstalling exonertate. >>> >>> And make sure to delete any run directories before retrying. >>> >>> ?Carson >>> >>> >>> On Feb 22, 2017, at 11:03 AM, Seth Munholland >>> wrote: >>> >>> After my clustering issue I figured I would go node by node and run >>> MAKER locally on the test data set to be sure that it was working properly >>> before trying it as a full MPI run. After adjusting all my exes to point >>> to the NFS versions I am consistently getting the same error on each node >>> when it comes time to run exonerate: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q >>> /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t >>> /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T >>> dna --model est2genome --minintron 20 --maxintron 10000 --showcigar >>> --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp >>> -500-500.26386-32056.dpp-mRNA-5.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade2 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> The only results I find on google was an issue with an improper >>> character in the est file, but a cat of the dpp_est.fasta shows nothing >>> incorrect and since they all point to dpp-mRNA-5 I backed up the provided >>> est and removed it. The response was: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q >>> /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t >>> /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T >>> dna --model est2genome --minintron 20 --maxintron 10000 --showcigar >>> --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp >>> -500-500.22889-32056.dpp-mRNA-4.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade4 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> So the error seems to be pointing at something happening when wrapping >>> up the ESTs. >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 13:29:12 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 12:29:12 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: <53F1286C-945C-4F91-B851-5833BE1D1F18@gmail.com> In the most extreme case, you may even need to go as far as setting up your own perl. If you do that, make sure to unset the PERL5LIB environmental variable, so other perl libraries don?t interfere. ?Carson > On Feb 22, 2017, at 12:26 PM, Seth Munholland wrote: > > I'll wipe it clean and do a fresh install of everything to be 100% safe. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 2:03 PM, Carson Holt > wrote: > I still think you may have something wrong somewhere with some part of your installation (rogue libraries, compiler incompatibilities, etc.). Especially given your earlier issues with Perl libraries. > > ?Carson > > >> On Feb 22, 2017, at 12:00 PM, Seth Munholland > wrote: >> >> On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. >> >> Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <> >> On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: >> It is the test data, the dpp set to be specific. >> >> I already checked all the installs to be sure they configured and compile without error and are up to date. >> >> I've been deleting between each run. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <> >> On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt > wrote: >> So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. >> >> Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. >> >> And make sure to delete any run directories before retrying. >> >> ?Carson >> >> >>> On Feb 22, 2017, at 11:03 AM, Seth Munholland > wrote: >>> >>> After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade2 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade4 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> So the error seems to be pointing at something happening when wrapping up the ESTs. >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lucile.soler at bils.se Thu Feb 23 06:40:08 2017 From: lucile.soler at bils.se (lucile.soler at bils.se) Date: Thu, 23 Feb 2017 13:40:08 +0100 Subject: [maker-devel] genes longer than contig in maker Message-ID: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> Hello, I am using maker3 to annotate a lizard. The maker part went well but then when I want to do the functional annotation, it seems that for some contigs maker has created genes outside of the contig size. For instance : XXX1 maker gene 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8 XXX1 maker mRNA 2 26717 . - . ID=maker- XXX1_pilon-augustus-gene-0.8-mRNA-1;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.8|0.5|1|0.4|0.33|6|5291|105 XXX1 maker mRNA 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;_AED=0.08;_eAED=0.08;_QI=0|0.5|0.4|1|0.25|0.4|5|4442|170 and the length of the contig is 26696 Any idea of what could have happened? Have you seen that already? Let me know what more information you need to answer. Thank you very much for your help, Lucile Soler PhD in Bioinformatics Genome Annotation Platform NBIS (National Bioinformatics Infrastructure Sweden) mail:lucile.soler at bils.se -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Thu Feb 23 07:10:55 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Thu, 23 Feb 2017 14:10:55 +0100 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> Message-ID: <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> HI! Running maker with mpiexec causes this error-message: mpiexec_uc1n077.localdomain: cannot connect to local mpd (/scratch/mpd2.console_uc1n077.localdomain_kn_pop235844); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option) ### Cleaning up files ... removing unnecessary scratch files ... And yes, we don't have mpd running. Environment used is: Currently Loaded Modulefiles: 1) compiler/intel/16.0(default) 2) mpi/impi/5.1.3-intel-16.0(default) 3) bio/maker/2.31.8_impi Running maker with mpiexec using only 1 node and 8 cores. mpiexec -n 8 maker :-( Any suggestions ? -- Rainer Rutka University of Konstanz Communication, Information, Media Centre (KIM) * High-Performance-Computing (HPC) * KIM-Support and -Base-Services Room: V511 78457 Konstanz, Germany +49 7531 88-5413 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From carsonhh at gmail.com Thu Feb 23 14:22:22 2017 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 23 Feb 2017 13:22:22 -0700 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> Message-ID: It means one of two things. 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. ?Carson > On Feb 23, 2017, at 6:10 AM, Rainer Rutka wrote: > > > HI! > > Running maker with mpiexec causes this error-message: > > mpiexec_uc1n077.localdomain: cannot connect to local mpd (/scratch/mpd2.console_uc1n077.localdomain_kn_pop235844); possible causes: > 1. no mpd is running on this host > 2. an mpd is running but was started without a "console" (-n option) > ### Cleaning up files ... removing unnecessary scratch files ... > > And yes, we don't have mpd running. > > Environment used is: > > Currently Loaded Modulefiles: > 1) compiler/intel/16.0(default) > 2) mpi/impi/5.1.3-intel-16.0(default) > 3) bio/maker/2.31.8_impi > > Running maker with mpiexec using only 1 node and 8 cores. > > mpiexec -n 8 maker > > :-( > > Any suggestions ? > > -- > Rainer Rutka > University of Konstanz > Communication, Information, Media Centre (KIM) > * High-Performance-Computing (HPC) > * KIM-Support and -Base-Services > Room: V511 > 78457 Konstanz, Germany > +49 7531 88-5413 > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Thu Feb 23 17:40:24 2017 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 23 Feb 2017 16:40:24 -0700 Subject: [maker-devel] genes longer than contig in maker In-Reply-To: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> References: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> Message-ID: 1. You have two contigs with the same ID in your assembly (one longer and one shorter). 2. Your BioPerl index for retrieving the sequence is corrupt somehow. Requires delete of previous output directory before restarting. 3. You are using bad formatted GFF3 as input into maker, and it is somehow not failing right away. The fact you get output means that it was able to translate the sequence into protein with CDS etc. So it was not too short for that. Look at the contig line in the GFF3 to get the length of the contig maker is seeing to compare to the feature positions given. ?Carson > On Feb 23, 2017, at 5:40 AM, lucile.soler at bils.se wrote: > > Hello, > > I am using maker3 to annotate a lizard. > > > The maker part went well but then when I want to do the functional annotation, it seems that for some contigs maker has created genes outside of the contig size. > > For instance : > > > XXX1 maker gene 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8 > > > XXX1 maker mRNA 2 26717 . - . ID=maker- XXX1_pilon-augustus-gene-0.8-mRNA-1;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.8|0.5|1|0.4|0.33|6|5291|105 > > XXX1 maker mRNA 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;_AED=0.08;_eAED=0.08;_QI=0|0.5|0.4|1|0.25|0.4|5|4442|170 > > and the length of the contig is 26696 > > > > > Any idea of what could have happened? Have you seen that already? > > > > Let me know what more information you need to answer. > > Thank you very much for your help, > > > > > > Lucile Soler > > PhD in Bioinformatics > Genome Annotation Platform > NBIS (National Bioinformatics Infrastructure Sweden) > mail:lucile.soler at bils.se > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Fri Feb 24 02:43:21 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Fri, 24 Feb 2017 09:43:21 +0100 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> Message-ID: <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> Hi Carson. First of all THANK YOU FOR YOUR HELP. MUCH APPRECIATED. :-) Am 23.02.2017 um 21:22 schrieb Carson Holt: > It means one of two things. > 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). No, I am starting the right version of mpiexec. This is a list of all our current available MPI-Versions, corresponding to their compiler: UC:[kn at uc1n997 ~]$ module avail mpi ------------------------------------- /opt/bwhpc/common/modulefiles -------------------------------------- mpi/impi/4.1.3-gnu-4.4 mpi/openmpi/1.10-intel-15.0 mpi/impi/4.1.3-gnu-4.7 mpi/openmpi/1.10-intel-16.0 mpi/impi/4.1.3-intel-14.0 mpi/openmpi/1.8-gnu-4.5 mpi/impi/5.0.3-gnu-4.4 mpi/openmpi/1.8-gnu-4.7 mpi/impi/5.0.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.7-m32 mpi/impi/5.0.3-intel-15.0 mpi/openmpi/1.8-gnu-4.8 mpi/impi/5.1.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.9 mpi/impi/5.1.3-gnu-system mpi/openmpi/1.8-intel-14.0(default) mpi/impi/5.1.3-intel-16.0(default) mpi/openmpi/1.8-intel-15.0 mpi/openmpi/1.10-gnu-4.5 mpi/openmpi/2.0-gnu-4.7 mpi/openmpi/1.10-gnu-4.7 mpi/openmpi/2.0-gnu-4.8 mpi/openmpi/1.10-gnu-4.8 mpi/openmpi/2.0-gnu-5.2 mpi/openmpi/1.10-gnu-4.9 mpi/openmpi/2.0-intel-15.0 mpi/openmpi/1.10-gnu-5.2 mpi/openmpi/2.0-intel-16.0 mpi/openmpi/1.10-intel-14.0 Here I load the MPI-Module including all it's dependencies: UC:[kn at uc1n997 ~]$ module load mpi/impi/5.1.3-intel-16.0 Loading module dependency 'compiler/intel/16.0'. Result: UC:[kn at c1n997 ~]$ which mpiexec /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec UC:[kn at uc1n997 ~]$ > 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) Extremely old? [...] Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160601 (build id: 15562) > MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. Yes. And we don't even use MPD at our clusters :-) > So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. I do not have any MPICH available on our cluster(s) now. All of the old versions had been removed some years ago. At a glance --------------- Maker is available as a so-called module on our cluster system. It's been build on a developers' node I can access. But, the MPI-modules are built by other fellows (e.g. at the KIT in Karlsruhe/Germany) on other nodes. Please check the module-file (included in this mail) bio-maker-2.31.8_impi to see how Maker was build including the envoronments set by this module. e.g.: ./Build status verify dependencies: =================================================== STATUS MAKER v2.31.8 =================================================== PERL Dependencies: VERIFIED External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: ENABLED MWAS Web Interface: DISABLED MAKER PACKAGE: CONFIGURATION OK At least, please(!) have a look at our m.moab file (included in this mail). This is the way how we submit a Maker job to our cluster. Maybe something is wrong here? Sorry again for wasting your time. But we imperatively need the Maker software running in parallel mode. :-) -- Rainer Rutka University of Konstanz Communication, Information, Media Centre (KIM) * High-Performance-Computing (HPC) * KIM-Support and -Base-Services Room: V511 78457 Konstanz, Germany +49 7531 88-5413 -------------- next part -------------- #%Module1.0 # # Cluster: bwunicluster # Module: bio/maker/2.31.8 # Revision: knbw01 # TargetSystem: Red-Hat-Enterprise # MainLocation: /opt/bwhpc/common # Status: optional # License: GPL, Artistic License # URL: http://www.yandell-lab.org/software/maker.html # 2ndLevelSupport: bwhpc [at] uni-konstanz.de # ##### (A) Revision history: # # knbw01 20160630 r.rutka Initial revision knbw01 of module version 2.31.8 # knbw02 20161121 r.rutka Initial revision knbw02 of module version 2.31.8 # knbw03 20160222 r.rutka Initial revision knbw02 of module version 2.31.8 with impi # ##### (B) Dependencies: # # conflict: any other maker version # module load compiler/intel/16.0 # module load mpi/impi/5.1.3-intel-16.0 # # # Main environments for build # # VERSION="2.31.8_impi" && echo "Version: ${VERSION}" # MAIN_DIR="/opt/bwhpc/common" && echo ${MAIN_DIR} # SOURCE_DIR=/home/kn/kn_kn/kn_pop235844/src/bio/maker/2.31.8 # TARGET_DIR="${MAIN_DIR}/bio/maker/${VERSION}" && echo ${TARGET_DIR} # ##### (C) How to obtain software? # # You have to register at first before you can download the SW. # # http://yandell.topaz.genetics.utah.edu/cgi-bin/maker_license.cgi # # # Download and unzip the binary distributions # cd ${SOURCE_DIR} && pwd # [ ! -f maker-${VERSION/_impi}.tgz ] && wget http://yandell.topaz.genetics.utah.edu/maker_downloads/CE63/461D/BD41/4510A183DC4571D5EA619E459572/maker-2.31.8.tgz # [[ -d ${TARGET_DIR} ]] && mv ${TARGET_DIR} ${TARGET_DIR}_$(date +%Y.%m.%d:%H.%M) # ##### (D) How to build and install software? # # mkdir -vp ${TARGET_DIR/$VERSION} # cd ${TARGET_DIR/$VERSION} && pwd # tar -xvzf ${SOURCE_DIR}/maker-${VERSION/_impi}.tgz # mv maker ${VERSION} # cd ${VERSION} && pwd # export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so # cd src # # Build required ad-ons. Lot's of required packages will be installed. # # perl Build.PL # | Would you like to configure MAKER for MPI.. ? [N ] Y # | # accept the next default paths and press ENTER # # ignore the error-messages - just continue with the next step # # ./Build installdeps # installs missing PERL dependencies # | ... MAKER .. local installataion: Y # | # Accept the defaults of _all_ further questions by pressing ENTER. # | # ** This is an annoying and long procedure :-( ** # # ./Build installexes # installs missing external programs # | # If are a registered user of RepBase, then MAKER can # | # download and install RepBase for RepeatMasker for you. # | # Register at: http://www.girinst.org/ # | ... download and install RepBase ... : N # We install RepBase later! # # If you type Y, this will not work # # See Install RepBase later in this file # # | # ERROR: Exonerate can't be found. URL Error 404. :-( # | # Locate exonerate entry in locations file and modify path to (line 33): # | # http://ftp.ebi.ac.uk/pub/software/vertebrategenomics/exonerate/exonerate-2.2.0-x86_64.tar.gz # | # restart ./Build installexes # ./Build installexes # # # Install RepBase to prevent "RepBase is not insalled for RepeatMasker" error-message # # http://www.girinst.org/server/RepBase/index.php # p=$(pwd) # cd ${TARGET_DIR}/exe/RepeatMasker && pwd # # http://www.girinst.org/server/RepBase/index.php # # FASTA-Format # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/RepBase22.01.fasta.tar.gz # tar xvzf RepBase22.01.fasta.tar.gz # # # # NEU! # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/repeatmaskerlibraries/RepBaseRepeatMaskerEdition-20170127.tar.gz # # # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/repeatmaskerlibraries/repeatmaskerlibraries-20160829.tar.gz # tar xvzf RepBaseRepeatMaskerEdition-20170127.tar.gz # # # tar xvzf repeatmaskerlibraries-20160829.tar.gz # # perl ./configure # | Enter path perl ... : ENTER # | Enter path ... RepeatMasker: ENTER # | Enter path TRF programm: /opt/bwhpc/common/bio/maker/2.31.8_impi/exe/RepeatMasker/trf # | Add a search engine: 2 RMBlast # | Enter path: /opt/bwhpc/common/bio/maker/2.31.8_impi/exe/RepeatMasker/rmblast/bin # | search engine ... default: Y or ENTER # | Enter Selection: 5 (Done) # # # rm *.tar.gz # cd $p && pwd # # # Now check if anything's fine until now... # ./Build status # | # veryfy dependencies: # | # =================================================== # | # STATUS MAKER v2.31.8 # | # =================================================== # | # PERL Dependencies: VERIFIED # | # External Programs: VERIFIED # | # External C Libraries: VERIFIED # | # MPI SUPPORT: ENABLED # | # MWAS Web Interface: DISABLED # | # MAKER PACKAGE: CONFIGURATION OK # # ./Build install # # # Test # cd ${TARGET_DIR} # bin/maker -version # 2.31.8 # # # Create module file (this one): # # ${SOURCE_DIR}/modulefiles/bio-maker-2.31.8 # # # Create Moab example files (and read the instructions inside of it): # # ${SOURCE_DIR}/bwhpc-examples/bwunicluster-maker-example.moab # # # Copy Moab/bwHPC, special-scripts, examples and more... # mkdir -vp ${TARGET_DIR}/bwhpc-examples # cp ${SOURCE_DIR}/bwhpc-examples/* ${TARGET_DIR}/bwhpc-examples/. # # # Copy module file (this one): # mkdir -vp ${TARGET_DIR}/modulefiles # cp -v ${SOURCE_DIR}/modulefiles/bio-maker-${VERSION} ${TARGET_DIR}/modulefiles/ # chmod -c 644 ${TARGET_DIR}/modulefiles/* # # # Create module link: # mkdir -vp ${MAIN_DIR}/modulefiles/bio/maker # ln -fs ${TARGET_DIR}/modulefiles/bio-maker-${VERSION} ${MAIN_DIR}/modulefiles/bio/maker/${VERSION} # module avail bio/maker # # # Cleanup build and installation target dir: # # chgrp -R -h -c uc1-adm-sw ${TARGET_DIR} # chmod 777 ${TARGET_DIR}/bwhpc-examples # chmod 666 ${TARGET_DIR}/bwhpc-examples/* # rm -rf ${TARGET_DIR}/src ##### END COMMENTS ### Define procedure "set_envVAR" to work around a module-unload-load bug (optional): # Exported environment variables, which do not change their values, # disappear after an automatic unload and load sequence within a module file. # This is most likely a bug in the modules environment. Unchanged # variables are not re-exported upon load in an automatic unload-load # sequence. The workaround is to "unset" the variable in the right moment # (so the load of the unload-load-chain can set the variable again) # and, in addition, to set the environment variable explicitly. proc set_envVAR {envVAR VARcontent} { # Use global function env: global env # Unset envVAR explicitly: if { [info exists env($envVAR)] } { catch { unset env($envVAR) } } # Call overloaded module command setenv: setenv $envVAR $VARcontent # Set envVAR explicitly: set env([set envVAR]) $VARcontent } ### Define fallback values and source global functions and variables from modulerc file: set first_level_support_email "bwhpc (at) uni-konstanz.de" set global_modulerc_file "/opt/bwhpc/common/etc/modules.conf" if { [ file readable "${global_modulerc_file}" ] } { source "${global_modulerc_file}" } # Override global first_level_support_email email: set first_level_support_email "bwhpc (at) uni-konstanz.de" ### Define version, base_dir and whatis entry: set version "2.31.8_impi" set base_dir "/opt/bwhpc/common/bio/maker/$version" module-whatis "maker $version is a portable and configurable genome annotation pipeline" ### Define convenience environment variables: set maker_version "${version}" set maker_home "${base_dir}" set maker_exa_dir "${base_dir}/bwhpc-examples" set maker_bin_dir "${base_dir}/bin" set maker_bpr_url "http://www.bwhpc-c5.de/wiki/index.php/Maker" set maker_blast_bin "${base_dir}/exe/blast/bin" set maker_exonerate_bin "${base_dir}/exe/exonerate/bin" set maker_snap_bin "${base_dir}/exe/snap" set maker_repeatmasker_bin "${base_dir}/exe/RepeatMasker" set_envVAR MAKER_VERSION "${maker_version}" set_envVAR MAKER_HOME "${maker_home}" set_envVAR MAKER_EXA_DIR "${maker_exa_dir}" set_envVAR MAKER_BIN_DIR "${maker_bin_dir}" set_envVAR MAKER_BPR_URL "http://www.bwhpc-c5.de/wiki/index.php/Maker" set_envVAR MAKER_BLAST_BIN "${maker_blast_bin}" set_envVAR MAKER_EXONERATE_BIN "${maker_exonerate_bin}" set_envVAR MAKER_SNAP_BIN "${maker_snap_bin}" set_envVAR MAKER_REPEATMASKER_BIN "${maker_repeatmasker_bin}" ### Update path environments and define application specific environment vars: prepend-path PATH "${maker_bin_dir}" prepend-path PATH "${maker_blast_bin}" prepend-path PATH "${maker_exonerate_bin}" prepend-path PATH "${maker_snap_bin}" prepend-path PATH "${maker_repeatmasker_bin}" ### Define help text: proc ModulesHelp { } { global maker_home maker_exa_dir first_level_support_email maker_bpr_url maker_blast_bin maker_exonerate_bin maker_snap_bin maker_repeatmasker_bin puts stderr " DESCRIPTION MAKER is a portable and easily configurable genome annotation pipeline. Its purpose is to allow smaller eukaryotic and prokaryotic genome projects to independently annotate their genomes and to create genome databases. MAKER identifies repeats, aligns ESTs and proteins to a genome, produces ab-initio gene predictions and automatically synthesizes these data into gene annotations having evidence-based quality values. EXTERNAL PROGRAMS BLAST Command Line Applications (2.2.28) $maker_blast_bin EXONERATE Generic Sequence Comparison Tool (2.2.0) $maker_exonerate_bin SNAP Semi-HMM-based Nucleic Acid Parser (version 2006-07-28) $maker_snap_bin REPEATMASKER Mask repetitive DNA (open-4.0.5) $maker_repeatmasker_bin DOCUMENTATION * Main MAKER site http://www.yandell-lab.org/software/maker.html * MAKER User Guide, Documents, Wiki-Article http://weatherby.genetics.utah.edu/MAKER/wiki/index.php $maker_home/README * MAKER Download http://yandell.topaz.genetics.utah.edu/maker_downloads/CE63/461D/BD41/4510A183DC4571D5EA619E459572/maker-2.31.8.tgz * bwHPC examples and a moab example script $maker_exa_dir * bwHPC Best Practices Repository $maker_bpr_url CITATION n./a. +-------------------------------------------+ | THIS IS THE IMPI-VERSION BUILT WITH INTEL | +-------------------------------------------+ In case of problems, please contact: $first_level_support_email This module is available for all users. " } module load compiler/intel/16.0 module load mpi/impi/5.1.3-intel-16.0 conflict bio/maker -------------- next part -------------- #!/bin/bash #MSUB -N maker_impi-job #MSUB -j oe #MSUB -o $(JOBNAME).$(JOBID) #MSUB -m ae #MSUB -l nodes=1:ppn=1 #MSUB -l mem=20gb #MSUB -l walltime=10:00:00 # start=$(date +%s) echo " " echo "### Setting up shell environment ..." echo " " # if test -e "/etc/profile"; then source "/etc/profile"; fi; if test -e "$HOME/.bash_profile"; then source "$HOME/.bash_profile"; fi; unset LANG; export LC_ALL="C"; export MKL_NUM_THREADS=1; export OMP_NUM_THREADS=1 export USER=${USER:=`logname`} export MOAB_JOBID=${MOAB_JOBID:=`date +%s`} export MOAB_SUBMITDIR=${MOAB_SUBMITDIR:=`pwd`} export MOAB_JOBNAME=${MOAB_JOBNAME:=`basename "$0"`} export MOAB_JOBNAME=$(echo "${MOAB_JOBNAME}" | sed 's/[^a-zA-Z0-9._-]/_/g') export MOAB_NODECOUNT=${MOAB_NODECOUNT:=1} export MOAB_PROCCOUNT=${MOAB_PROCCOUNT:=1} ulimit -s 200000 echo " " echo "### Printing basic job infos to stdout ..." echo " " echo "START_TIME = `date +'%y-%m-%d %H:%M:%S %s'`" echo "HOSTNAME = ${HOSTNAME}" echo "USER = ${USER}" echo "MOAB_JOBNAME = ${MOAB_JOBNAME}" echo "MOAB_JOBID = ${MOAB_JOBID}" echo "MOAB_SUBMITDIR = ${MOAB_SUBMITDIR}" echo "MOAB_NODECOUNT = ${MOAB_NODECOUNT}" echo "MOAB_PROCCOUNT = ${MOAB_PROCCOUNT}" echo "SLURM_NODELIST = ${SLURM_NODELIST}" echo "PBS_NODEFILE = ${PBS_NODEFILE}" if test -f "${PBS_NODEFILE}"; then echo "PBS_NODEFILE (begin) ---------------------------------" NO_NODES=$(wc -l < ${PBS_NODEFILE}) cat "${PBS_NODEFILE}" sort -u $PBS_NODEFILE > mpd.hosts echo "PBS_NODEFILE (end) -----------------------------------" else NO_NODES=1 fi echo " " echo "### Creating TMP_WORK_DIR directory and changing to it ..." echo " " # Using "$TMPDIR" is strongly recommended for Maker jobs # since these jobs can create a lot of disk IO. # NEVER EVER calculate in your home directory. TMP_BASE_DIR="$TMPDIR" JOB_WORK_DIR="${MOAB_JOBNAME}.${MOAB_JOBID##*.}.$(date +%y%m%d_%H%M%S)" TMP_WORK_DIR="${TMP_BASE_DIR}/${JOB_WORK_DIR}" echo "JOB_WORK_DIR = ${JOB_WORK_DIR}" echo "TMP_BASE_DIR = ${TMP_BASE_DIR}" echo "TMP_WORK_DIR = ${TMP_WORK_DIR}" mkdir -vp "${TMP_WORK_DIR}" cd "${TMP_WORK_DIR}" echo " " echo "### Loading MAKER module:" echo " " module load bio/maker/2.31.8_impi [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.8_impi'."; exit 1; } echo "MAKER_VERSION = $MAKER_VERSION" module list echo " " echo "### Copying input examples files for job:" echo " " cp -v ${MAKER_EXA_DIR}/*.{fasta,ctl} . cp -Rv ${MAKER_EXA_DIR}/_Inline . sleep 2 echo " " echo "### Display internal Maker/bwHPC environments..." echo " " echo "MAKER_BIN_DIR = ${MAKER_BIN_DIR}" echo "MAKER_EXA_DIR = ${MAKER_EXA_DIR}" echo "" echo " " echo "### Runing Maker example" echo " " # # Environmental variables to set: # export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so echo "LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so" export OMPI_MCA_mpi_warn_on_fork=0 echo "OMPI_MCA_mpi_warn_on_fork=0" export I_MPI_CPUINFO="proc" echo "I_MPI_CPUINFO=proc" export I_MPI_PMI_LIBRARY=${MPI_LIB_DIR}/libpmi.so echo "I_MPI_PMI_LIBRARY=${MPI_LIB_DIR}/libpmi.so" export I_MPI_PIN_DOMAIN=node echo "I_MPI_PIN_DOMAIN=node" # otherwise MAKER calls to BLAST and other programs which are parallelized independent # of MPI may not work export I_MPI_FABRICS='shm:tcp' echo "I_MPI_FABRICS=shm:tcp" # avoid potential complication with OpenFabrics libraries (they block system calls because of # how they use registered memory, i.e. MAKER calling BLAST would fail) # set to eth0 if you don?t have an infiniband over # ip inerface (required because of the above I_MPI_FABRICS change) export I_MPI_HYDRA_IFACE=ib0 echo "I_MPI_HYDRA_IFACE=ib0" # The ?c nobrs. option will try and run the BLAST jobs on # nobrs. cpus on the same machine, but will not run via MPI. # -nolocal # wtf? # mpiexec -n $SLURM_NPROCS -env I_MPI_DEBUG 5 ./hello echo "starting mpiexec..." mpiexec -nc 1 maker echo "### Cleaning up files ... removing unnecessary scratch files ..." echo " " # rm -fv sleep 3 # Sleep some time so potential stale nfs handles can disappear. echo " " echo "### Compressing results and copying back result archive ..." echo " " cd "${TMP_BASE_DIR}" mkdir -vp "${MOAB_SUBMITDIR}" # if user has deleted or moved the submit dir echo "Creating result tgz-file '${MOAB_SUBMITDIR}/${JOB_WORK_DIR}.tgz' ..." tar -zcvf "${MOAB_SUBMITDIR}/${JOB_WORK_DIR}.tgz" "${JOB_WORK_DIR}" \ || { echo "ERROR: Failed to create tgz-file. Please cleanup TMP_WORK_DIR '$TMP_WORK_DIR' on host '$HOSTNAME' manually (if not done automatically by queueing system)."; exit 102; } # Remarks: # * The resulting tgz file is copied back to the submit directory. # The name of the tgz file looks similar too # "bwunicluster-maker-example.moab.275.110528_101755.tgz" echo " " echo "### Final cleanup: Remove TMP_WORK_DIR ..." echo " " rm -rvf "${TMP_WORK_DIR}" echo "END_TIME = `date +'%y-%m-%d %H:%M:%S %s'`" end=$(date +%s) echo " " echo "### Calculate duration ..." echo " " diff=$[end-start] if [ $diff -lt 60 ]; then echo "Runtime (approx.): '$diff' secs" elif [ $diff -ge 60 ]; then echo 'Runtime (approx.): '$[$diff / 60] 'min(s) '$[$diff % 60] 'secs' fi -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From carsonhh at gmail.com Fri Feb 24 11:30:32 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 24 Feb 2017 10:30:32 -0700 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> Message-ID: Specific things. 1. Do not set LD_PRELOAD. That is only for OpenMPI, but it will cause problems with other MPI's. 2. Make sure you recompiled MAKER for Intel MPI (MPI code always has to be compiled for the flavor you are using, so make sure you have a separate installation of MAKER for Intel MPI). Also validate that the mpicc and libmpi.h listed during the MAKER install belong to Intel MPI. Don?t just assume they do because you loaded the module. Manually verify the paths during MAKER?s setup. 3. The error you got previously should not even be possible with the current version of Intel MPI, which is why I say that when you called mpiexec, something else (that was not Intel MPI) was launched. Easy solution is to give the full path of mpiexec in your job, so are not relying on PATH to be unaltered in your job. Do not do ?> mpiexec -nc 1 maker Do this for example ?> /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec -nc maker 4. Build and run on the same node for your test. If you build on one node and run on another, you may be changing your environment in ways you don?t realize that break things. So if you can build and test on the same node and it works, then it fails when you test it elsewhere, then you have to track down how your environment is changing. ?Carson > On Feb 24, 2017, at 1:43 AM, Rainer Rutka wrote: > > Hi Carson. > > First of all THANK YOU FOR YOUR HELP. > MUCH APPRECIATED. :-) > > Am 23.02.2017 um 21:22 schrieb Carson Holt: >> It means one of two things. >> 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). > > No, I am starting the right version of mpiexec. > > This is a list of all our current available MPI-Versions, corresponding > to their compiler: > > UC:[kn at uc1n997 ~]$ module avail mpi > ------------------------------------- /opt/bwhpc/common/modulefiles -------------------------------------- > mpi/impi/4.1.3-gnu-4.4 mpi/openmpi/1.10-intel-15.0 > mpi/impi/4.1.3-gnu-4.7 mpi/openmpi/1.10-intel-16.0 > mpi/impi/4.1.3-intel-14.0 mpi/openmpi/1.8-gnu-4.5 > mpi/impi/5.0.3-gnu-4.4 mpi/openmpi/1.8-gnu-4.7 > mpi/impi/5.0.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.7-m32 > mpi/impi/5.0.3-intel-15.0 mpi/openmpi/1.8-gnu-4.8 > mpi/impi/5.1.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.9 > mpi/impi/5.1.3-gnu-system mpi/openmpi/1.8-intel-14.0(default) > > mpi/impi/5.1.3-intel-16.0(default) > > mpi/openmpi/1.8-intel-15.0 > mpi/openmpi/1.10-gnu-4.5 mpi/openmpi/2.0-gnu-4.7 > mpi/openmpi/1.10-gnu-4.7 mpi/openmpi/2.0-gnu-4.8 > mpi/openmpi/1.10-gnu-4.8 mpi/openmpi/2.0-gnu-5.2 > mpi/openmpi/1.10-gnu-4.9 mpi/openmpi/2.0-intel-15.0 > mpi/openmpi/1.10-gnu-5.2 mpi/openmpi/2.0-intel-16.0 > mpi/openmpi/1.10-intel-14.0 > > > Here I load the MPI-Module including all it's dependencies: > > UC:[kn at uc1n997 ~]$ module load mpi/impi/5.1.3-intel-16.0 > Loading module dependency 'compiler/intel/16.0'. > > Result: > UC:[kn at c1n997 ~]$ which mpiexec > /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec > UC:[kn at uc1n997 ~]$ > >> 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) > Extremely old? > > [...] > Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160601 (build id: 15562) > >> MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. > Yes. And we don't even use MPD at our clusters :-) > >> So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. > I do not have any MPICH available on our cluster(s) now. All of the old versions had > been removed some years ago. > > At a glance > --------------- > > Maker is available as a so-called module on our cluster system. It's been build > on a developers' node I can access. But, the MPI-modules are built by other > fellows (e.g. at the KIT in Karlsruhe/Germany) on other nodes. > > Please check the module-file (included in this mail) > > bio-maker-2.31.8_impi > > to see how Maker was build including the envoronments set by this > module. > > e.g.: > ./Build status > verify dependencies: > =================================================== > STATUS MAKER v2.31.8 > =================================================== > PERL Dependencies: VERIFIED > External Programs: VERIFIED > External C Libraries: VERIFIED > MPI SUPPORT: ENABLED > MWAS Web Interface: DISABLED > MAKER PACKAGE: CONFIGURATION OK > > > At least, please(!) have a look at our m.moab file (included in this mail). > This is the way how we submit a Maker job to our cluster. Maybe something > is wrong here? > > Sorry again for wasting your time. But we imperatively need the Maker > software running in parallel mode. > > :-) > > -- > Rainer Rutka > University of Konstanz > Communication, Information, Media Centre (KIM) > * High-Performance-Computing (HPC) > * KIM-Support and -Base-Services > Room: V511 > 78457 Konstanz, Germany > +49 7531 88-5413 > From qlian003 at ucr.edu Sat Feb 25 11:14:04 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Sat, 25 Feb 2017 09:14:04 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: <2377C5DD-569C-4248-B458-349D7AEA32F5@ucr.edu> Thank you Barry and Carson! I compared the SOBA statistics of RepeatMasker footprint and the report generated by running RepeatMasker alone, I got 2 different parentage of repeats masked. Running RepeatMasker with myTrained.lib, the repeats masked are 42%. But within Maker GFF3, the percentage of repeats masker is only ~18%. What may cause such difference here? Thanks Qihua > On Feb 21, 2017, at 1:34 PM, Carson Holt wrote: > > MAKER merges overlapping RepeatMasker results into a single longer feature. > > ?Carson > > >> On Feb 20, 2017, at 1:34 PM, Qihua Liang wrote: >> >> Hi Carson, >> >> Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. >> >> I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. >> >> Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. >> In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. >> >> But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. >> >> When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? >> Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? >> >> Thanks >> Qihua >> >> >>> On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: >>> >>> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. >>> >>> CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. >>> >>> If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. >>> >>> What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. >>> >>> However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. >>> >>> Thanks, >>> Carson >>> >>>> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >>>> >>>> Dear Maker develop team, >>>> >>>> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >>>> >>>> Thanks >>>> Qihua >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> > From qwzhang0601 at gmail.com Mon Feb 27 09:25:04 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 27 Feb 2017 10:25:04 -0500 Subject: [maker-devel] PARALLELIZED DE NOVO GENOME ANNOTATION WITHOUT MPI Message-ID: Hello: I am doing genome annotation using Maker on our high performance computational cluster (HPC). Due to some issues of MPI, I submitted the Maker jobs several times under the same directory to HPC. Followed by the example in the protocol (as shown below), when I submit the jobs I make them as background processes by "&" except the first one. Is this necessary when I submit a job to a HPC? I found it costed much much longer time than I expected (according to a testing on a smaller data set). I am not sure whether setting the process as background process lead to this issue? The example in the protocol % maker 2> maker1.error % maker 2> maker2.error & % maker 2> maker3.error & ...... BTW, will the annotation on shorter contig (e.g., 500bp) cost ~ 1/100 of the time that cost for annotation a 50000bp contig? I am using SNAP for an inito and RNA-seq assembly and protein sequences as evidence. I have more than half contigs shorter than 300bp (whose total length is only about 5% of the total length of all contigs), I want to know whether I can save about half (or only about 5%) of the time if I ignore those short contigs. Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcg at cau.edu.cn Tue Feb 28 01:43:57 2017 From: dcg at cau.edu.cn (dcg at cau.edu.cn) Date: Tue, 28 Feb 2017 15:43:57 +0800 Subject: [maker-devel] how to deal with Contigs to run maker? Message-ID: <2017022815435664227911@cau.edu.cn> Dear sir: After assemblying, I got many contigs and their order in each chromosome. What I have done is merging these contigs into each chromosomes followed by the order, with 100 Ns inserted betwwen each contigs. So that I got chr1 chr2......Then I ran the repeatmodeler, predictor to annotate it. Could my way reach a high-quality result? Should I use all the contigs to mask repeats and practice predictor? Is there any better way to do genome-wide annotation? I'm looking forward to your reply! Best wishes! Chao Chao 2017.02.28 -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Thu Feb 2 09:16:51 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Thu, 2 Feb 2017 11:16:51 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> Message-ID: Thank for you suggestions. So it does not matter if there are redundancies of protein sequence from different sources? I am trying to annotating a *rodent *genome, and planned to collect protein sequences of human, mouse, rat from bout UniProt and NCBI (besides I also have RNA-seq data). I choose these species, because they are close to the species that I am working on and they are well annotated. But I saw someone said that if we choose protein sequence from one lineage, the genes that are missing in the lineage will not be detected. And in the following paper, the authors claim they used the entire SwissProt database as the input. How do you think about this? Should I include protein sequences from more species (like all Eukaryota)? I think it can help us identify more genes, but on the other hand won't this also give us more false positives? This paper used the entire SwissProt database as the input. Insights into the evolution of longevity from the bowhead whale genome. 2015. *Cell Rep* *10*(1): 112-122. Thanks Best Quanwei 2017-01-31 15:57 GMT-05:00 Michael Campbell : > Hi Quanwei, > > (1) When I use uniprot I use SWISS-prot and not tremble. > (2) I don?t merge files together. I just pass them all to MAKER as a comma > separated list. > > Thanks, > Mike > > > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang > wrote: > > > > I wonder what's the best way to collect protein sequences for gene > annotation of a de novo genome assembly. > > (1) My first choice is to get protein sequences of human and mouse from > UniProt. At this step, I am not clear whether I should download the > reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., > TrEMBL). > > (2) On ther other hand, I also get protein sequences from NCBI, should I > just simply merge those fasta files. Does it matter if there are > redundancies? And also, if I get protein sequences from different sources, > they may not have the same quality. Do I need to do something before I > integrate protein sequences from different sources? > > > > Many thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Thu Feb 2 12:24:04 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Thu, 2 Feb 2017 14:24:04 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> Message-ID: <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> It is nice to have an outgroup of some kind. In your case Human would serve that function. The issue that you might have with distantly related proteins is funny blast alignments that may lead to merging genes. You generally don?t get many false positives because the alignment parameters require a pretty good match, which is unlikely to happen by chance. You could limit swiss-prot to mammals if you wanted to. Sometimes I?ll try different combinations of evidence on a few large scaffolds and look at the results in a browser to get a feel for what is going to work best. Thanks, Mike > On Feb 2, 2017, at 11:16 AM, Quanwei Zhang wrote: > > > Thank for you suggestions. So it does not matter if there are redundancies of protein sequence from different sources? > I am trying to annotating a rodent genome, and planned to collect protein sequences of human, mouse, rat from bout UniProt and NCBI (besides I also have RNA-seq data). I choose these species, because they are close to the species that I am working on and they are well annotated. But I saw someone said that if we choose protein sequence from one lineage, the genes that are missing in the lineage will not be detected. And in the following paper, the authors claim they used the entire SwissProt database as the input. How do you think about this? Should I include protein sequences from more species (like all Eukaryota)? I think it can help us identify more genes, but on the other hand won't this also give us more false positives? > > This paper used the entire SwissProt database as the input. > Insights into the evolution of longevity from the bowhead whale genome. 2015. Cell Rep 10(1): 112-122. > > Thanks > > Best > Quanwei > > 2017-01-31 15:57 GMT-05:00 Michael Campbell >: > Hi Quanwei, > > (1) When I use uniprot I use SWISS-prot and not tremble. > (2) I don?t merge files together. I just pass them all to MAKER as a comma separated list. > > Thanks, > Mike > > > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang > wrote: > > > > I wonder what's the best way to collect protein sequences for gene annotation of a de novo genome assembly. > > (1) My first choice is to get protein sequences of human and mouse from UniProt. At this step, I am not clear whether I should download the reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., TrEMBL). > > (2) On ther other hand, I also get protein sequences from NCBI, should I just simply merge those fasta files. Does it matter if there are redundancies? And also, if I get protein sequences from different sources, they may not have the same quality. Do I need to do something before I integrate protein sequences from different sources? > > > > Many thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Thu Feb 2 14:05:53 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Thu, 2 Feb 2017 16:05:53 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> Message-ID: Thank you for your suggestions. I need to have some tests. Best Quanwei 2017-02-02 14:24 GMT-05:00 Michael Campbell : > It is nice to have an outgroup of some kind. In your case Human would > serve that function. The issue that you might have with distantly related > proteins is funny blast alignments that may lead to merging genes. You > generally don?t get many false positives because the alignment parameters > require a pretty good match, which is unlikely to happen by chance. You > could limit swiss-prot to mammals if you wanted to. > > Sometimes I?ll try different combinations of evidence on a few large > scaffolds and look at the results in a browser to get a feel for what is > going to work best. > > Thanks, > Mike > > On Feb 2, 2017, at 11:16 AM, Quanwei Zhang wrote: > > > Thank for you suggestions. So it does not matter if there are redundancies > of protein sequence from different sources? > I am trying to annotating a *rodent *genome, and planned to collect > protein sequences of human, mouse, rat from bout UniProt and NCBI (besides > I also have RNA-seq data). I choose these species, because they are close > to the species that I am working on and they are well annotated. But I saw > someone said that if we choose protein sequence from one lineage, the genes > that are missing in the lineage will not be detected. And in the following > paper, the authors claim they used the entire SwissProt database as the > input. How do you think about this? Should I include protein sequences from > more species (like all Eukaryota)? I think it can help us identify more > genes, but on the other hand won't this also give us more false positives? > > This paper used the entire SwissProt database as the input. > Insights into the evolution of longevity from the bowhead whale genome. > 2015. *Cell Rep* *10*(1): 112-122. > > Thanks > > Best > Quanwei > > 2017-01-31 15:57 GMT-05:00 Michael Campbell >: > >> Hi Quanwei, >> >> (1) When I use uniprot I use SWISS-prot and not tremble. >> (2) I don?t merge files together. I just pass them all to MAKER as a >> comma separated list. >> >> Thanks, >> Mike >> >> > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang >> wrote: >> > >> > I wonder what's the best way to collect protein sequences for gene >> annotation of a de novo genome assembly. >> > (1) My first choice is to get protein sequences of human and mouse from >> UniProt. At this step, I am not clear whether I should download the >> reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., >> TrEMBL). >> > (2) On ther other hand, I also get protein sequences from NCBI, should >> I just simply merge those fasta files. Does it matter if there are >> redundancies? And also, if I get protein sequences from different sources, >> they may not have the same quality. Do I need to do something before I >> integrate protein sequences from different sources? >> > >> > Many thanks >> > >> > Best >> > Quanwei >> > _______________________________________________ >> > maker-devel mailing list >> > maker-devel at box290.bluehost.com >> > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 3 09:04:45 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 3 Feb 2017 11:04:45 -0500 Subject: [maker-devel] augustus failed in maker2 Message-ID: I am trying to add augustus to the prediction. The maker2 run well without augustus, and augustus seems installed correctly. But it returns me the following errors. Any ideas or suggestions? What I did is just to set "augustus_species=human" in the "maker_opts.ctl" file, and I am running maker on a HPC with linux system. preparing ab-inits ERROR: Augustus failed --> rank=NA, hostname=n530 ERROR: Failed while preparing ab-inits ERROR: Chunk failed at level:0, tier_type:2 FAILED CONTIG:CasCan_contig_0 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:CasCan_contig_0 Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 3 10:15:39 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 3 Feb 2017 10:15:39 -0700 Subject: [maker-devel] augustus failed in maker2 In-Reply-To: References: Message-ID: Look a little further back into the STERR log to see if there are other errors further back. The issue is probably your Augustus installation. Try running it by itself (outside of MAKER) on the same dataset, and it may help identify on what is happening. Thanks, Carson > On Feb 3, 2017, at 9:04 AM, Quanwei Zhang wrote: > > I am trying to add augustus to the prediction. The maker2 run well without augustus, and augustus seems installed correctly. But it returns me the following errors. Any ideas or suggestions? What I did is just to set "augustus_species=human" in the "maker_opts.ctl" file, and I am running maker on a HPC with linux system. > > preparing ab-inits > ERROR: Augustus failed > --> rank=NA, hostname=n530 > ERROR: Failed while preparing ab-inits > ERROR: Chunk failed at level:0, tier_type:2 > FAILED CONTIG:CasCan_contig_0 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:CasCan_contig_0 > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From mnaymik at tgen.org Mon Feb 6 14:27:22 2017 From: mnaymik at tgen.org (Marcus Naymik) Date: Mon, 6 Feb 2017 14:27:22 -0700 Subject: [maker-devel] Can't Install Proc::Signal Message-ID: I can't find Proc::Signal in cpan or anywhere. How can I install it? -- *This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisa.postma at gmail.com Sun Feb 5 06:15:08 2017 From: alisa.postma at gmail.com (Alisa Postma) Date: Sun, 5 Feb 2017 15:15:08 +0200 Subject: [maker-devel] Problem with fgenesh protein IDs Message-ID: Good afternoon I am having problems with my all.maker.proteins.fasta file. I ran maker with ab initio gene predictions from augustus and genemark and I added my fgenesh gff file to the pred_gff option in the ctl file. However, my maker protein fasta file does not give the fgenesh proteins IDs similar to that for augustus and genemark. For example: >FGENESH protein AED:0.06 eAED:0.06 QI:0|0|0|1|1|1|3|0|433 vs. >maker-scaffold74-augustus-gene-0.179-mRNA-1 protein AED:0.07 eAED:0.07 QI:123|1|1|1|1|1|5|793|326 I would appreciate any advice on this matter. Kind regards Alisa -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 6 21:05:33 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 6 Feb 2017 21:05:33 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: References: Message-ID: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Proc::Signal is part of MAKER?s internal libraries. If you are getting an error saying it is missing, then you have a problem with your installation. Either it was not installed successfully, or you modified the location of files or executables post installation. Make sure you do not move the bin directory or its contents. If you need to be somewhere else, it is best to use softlinks instead. Thanks, Carson > On Feb 6, 2017, at 2:27 PM, Marcus Naymik wrote: > > I can't find Proc::Signal in cpan or anywhere. How can I install it? > > This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaymik at tgen.org Tue Feb 7 10:00:37 2017 From: mnaymik at tgen.org (Marcus Naymik) Date: Tue, 7 Feb 2017 10:00:37 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> References: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Message-ID: Ah, It was listed on as a prereq on your wiki and was trying to install all those first. I was just a little confused. Thanks! Marcus On Mon, Feb 6, 2017 at 9:05 PM, Carson Holt wrote: > Proc::Signal is part of MAKER?s internal libraries. If you are getting an > error saying it is missing, then you have a problem with your installation. > > Either it was not installed successfully, or you modified the location of > files or executables post installation. > > Make sure you do not move the bin directory or its contents. If you need > to be somewhere else, it is best to use softlinks instead. > > Thanks, > Carson > > On Feb 6, 2017, at 2:27 PM, Marcus Naymik wrote: > > I can't find Proc::Signal in cpan or anywhere. How can I install it? > > *This electronic message is intended to be for the use only of the named > recipient, and may contain information that is confidential or privileged, > including patient health information. If you are not the intended > recipient, you are hereby notified that any disclosure, copying, > distribution or use of the contents of this message is strictly prohibited. > If you have received this message in error or are not the named recipient, > please notify us immediately by contacting the sender at the electronic > mail address noted above, and delete and destroy all copies of this > message. Thank you.* > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- *This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 7 10:11:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 7 Feb 2017 10:11:46 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: References: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Message-ID: Use the MAKER Tutorial for GMOD Online Training 2014 It is the most up to date ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014 Thanks, Carson > On Feb 7, 2017, at 10:00 AM, Marcus Naymik wrote: > > Ah, It was listed on as a prereq on your wiki and was trying to install all those first. I was just a little confused. Thanks! > > Marcus > > On Mon, Feb 6, 2017 at 9:05 PM, Carson Holt > wrote: > Proc::Signal is part of MAKER?s internal libraries. If you are getting an error saying it is missing, then you have a problem with your installation. > > Either it was not installed successfully, or you modified the location of files or executables post installation. > > Make sure you do not move the bin directory or its contents. If you need to be somewhere else, it is best to use softlinks instead. > > Thanks, > Carson > >> On Feb 6, 2017, at 2:27 PM, Marcus Naymik > wrote: >> >> I can't find Proc::Signal in cpan or anywhere. How can I install it? >> >> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 7 10:12:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 7 Feb 2017 10:12:46 -0700 Subject: [maker-devel] Problem with fgenesh protein IDs In-Reply-To: References: Message-ID: The issue is probably with the format of your GFF3. I can take a look if you want. ?Carson > On Feb 5, 2017, at 6:15 AM, Alisa Postma wrote: > > Good afternoon > > I am having problems with my all.maker.proteins.fasta file. > > I ran maker with ab initio gene predictions from augustus and genemark and I added my fgenesh gff file to the pred_gff option in the ctl file. > > However, my maker protein fasta file does not give the fgenesh proteins IDs similar to that for augustus and genemark. > > For example: > >FGENESH protein AED:0.06 eAED:0.06 QI:0|0|0|1|1|1|3|0|433 > > vs. > > > >maker-scaffold74-augustus-gene-0.179-mRNA-1 protein AED:0.07 eAED:0.07 QI:123|1|1|1|1|1|5|793|326 > > I would appreciate any advice on this matter. > > Kind regards > > Alisa > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Wed Feb 8 08:45:11 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Wed, 8 Feb 2017 10:45:11 -0500 Subject: [maker-devel] make use of other gene finders in maker Message-ID: Hello: I am trying to annotate the new genome of a rodent species. I saw a clear pipeline to train the SNAP gene finder. But not clear what I should do if I also want to include augustus and/or GeneMark. Do I need to (could I and how) train it in Maker? In one of recently published paper, they mention the augustus was trained in Maker, but I am not clear how to do it? I wonder what's you opinions on using other gene finders in Maker. Thanks "Unique features of a global human ectoparasite identified through sequencing of the bed bug genome." Nat Commun, 2016. In this paper, the training step was described as below. *Three rounds of training of the Augustus and SNAP gene predictors within Maker were used to bootstrap to a high-quality training set.* Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Wed Feb 8 08:56:08 2017 From: dence at genetics.utah.edu (Daniel Ence) Date: Wed, 8 Feb 2017 15:56:08 +0000 Subject: [maker-devel] make use of other gene finders in maker In-Reply-To: References: Message-ID: <02D8BB85-1B0B-4028-A3F6-182F4F3318F7@genetics.utah.edu> Hi Quanwei, MAKER will run Augustus, SNAP, genemark or Fgenesh internally. You can also provide predictions from other gene predictors in valid GFF3 format in the ?pred_gff" field in the maker_opts control file. Genemark is self training on your genome. You provide a path to the ?es.mod? file that Genemark makes in the ?gmhmm? field. For augustus, the training is more involved, but is described here: http://www.molecularevolution.org/molevolfiles/exercises/augustus/training.html Augustus, snap, Fgenesh, and genemark are all complementary in some respects, but Augustus usually provides the most predictions that MAKER ends up selecting to promote to models. Hope that helps, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Feb 8, 2017, at 10:45 AM, Quanwei Zhang > wrote: Hello: I am trying to annotate the new genome of a rodent species. I saw a clear pipeline to train the SNAP gene finder. But not clear what I should do if I also want to include augustus and/or GeneMark. Do I need to (could I and how) train it in Maker? In one of recently published paper, they mention the augustus was trained in Maker, but I am not clear how to do it? I wonder what's you opinions on using other gene finders in Maker. Thanks "Unique features of a global human ectoparasite identified through sequencing of the bed bug genome." Nat Commun, 2016. In this paper, the training step was described as below. Three rounds of training of the Augustus and SNAP gene predictors within Maker were used to bootstrap to a high-quality training set. Best Quanwei _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 8 10:18:49 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 8 Feb 2017 12:18:49 -0500 Subject: [maker-devel] MAKER and OpenMPI Message-ID: Hello everyone, I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 8 10:22:19 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 10:22:19 -0700 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: References: Message-ID: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> You might have two different MPI flavors installed. Since the library and executable files have the same name, this kind of mismatch can happen in that case. Check the locations of MPI libraries given to MAKER during install, and the location of the mpiexec being used when you run. There is likely a mismatch with parts of one MPI flavor being used for one but not the other. ?Carson > On Feb 8, 2017, at 10:18 AM, Seth Munholland wrote: > > Hello everyone, > > I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 8 10:27:15 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 8 Feb 2017 12:27:15 -0500 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> References: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> Message-ID: That's for sure what it is, but I never installed a second flavour. I was commited to MPICH from the get go and am trying to figure out where OpenMPI came from. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 8, 2017 at 12:22 PM, Carson Holt wrote: > You might have two different MPI flavors installed. Since the library and > executable files have the same name, this kind of mismatch can happen in > that case. Check the locations of MPI libraries given to MAKER during > install, and the location of the mpiexec being used when you run. There is > likely a mismatch with parts of one MPI flavor being used for one but not > the other. > > ?Carson > > > > On Feb 8, 2017, at 10:18 AM, Seth Munholland wrote: > > Hello everyone, > > I'm setting up a cluster and configured MPICH on it, but after installing > MAKER my mpiexec gave me an error that comes from an improperly configured > OpenMPI. The weird thing is I haven't installed anything else associated > to MPI, particularly not OpenMPI. Did I get server gremlins or is there > any way MAKER's installation somehow is looking for OpenMPI? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 8 10:32:22 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 10:32:22 -0700 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: References: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> Message-ID: <863A3CDC-24F3-4A91-8D3E-45387262DF46@gmail.com> When you install MAKER, you have to provide the location to mpi.h and mpicc. You may have to reinstall, and give the correct location for one or both (don?t just trust the location that shows up, it may be the wrong one). So you will need to track down both of those for MPICH (depending on where you installed it to). Then do ?which -a mpiexec? to get all locations for every mpiexec found in your path. You will need to ensure the one at the top is the one you want. If it?s not, then you need to reconfigure your PATH or provide the full path to the mpiexec you want each time you launch it. ?Carson > On Feb 8, 2017, at 10:27 AM, Seth Munholland wrote: > > That's for sure what it is, but I never installed a second flavour. I was commited to MPICH from the get go and am trying to figure out where OpenMPI came from. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 8, 2017 at 12:22 PM, Carson Holt > wrote: > You might have two different MPI flavors installed. Since the library and executable files have the same name, this kind of mismatch can happen in that case. Check the locations of MPI libraries given to MAKER during install, and the location of the mpiexec being used when you run. There is likely a mismatch with parts of one MPI flavor being used for one but not the other. > > ?Carson > > > >> On Feb 8, 2017, at 10:18 AM, Seth Munholland > wrote: >> >> Hello everyone, >> >> I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjfi2sb3 at gmail.com Wed Feb 8 00:22:57 2017 From: mjfi2sb3 at gmail.com (Salim Bougouffa) Date: Wed, 08 Feb 2017 07:22:57 +0000 Subject: [maker-devel] Masking is causing problems with exon/CDS boundary predictions Message-ID: Hi, I am having trouble with MAKER/AUGUSTUS annotation. One of the recurrent ones is the example I attach in artemis screenshot. In red is the gene of interest. There are three GFF files. Top is augustus standalone that I executed on the non-masked scaffold. The second is maker annotation and third is the augustus (from maker) on the masked query. The gene clearly has four exons and three CDS but the masking seem to somehow cause augustus to get the boundaries wrong for the first and third CDS and skip the second CDS entirely. How do I fix this problem? Best, /SB [image: artemis.png] -- ____________________________ Sent from Inbox Mobile -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: artemis.png Type: image/png Size: 128354 bytes Desc: not available URL: From carsonhh at gmail.com Wed Feb 8 12:31:16 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 12:31:16 -0700 Subject: [maker-devel] Masking is causing problems with exon/CDS boundary predictions In-Reply-To: References: Message-ID: <4B4ACC1F-541E-4182-9DE6-5C22CEF2C91D@gmail.com> What do your evidence alignments look like? i.e. assembled mRNA-seq and protein homology? Not the mRNA-seq pileup (because MAKER can?t see that). Masking is applied before evidence seeding and the first ab initio run. It is actually then removed for evidence polishing around splice sites, and the hint based rerun of Augustus. So evidence is allowed to extend through masked regions and Augustus can make it part of the model if it wants on the second run. Since it didn?t the question becomes, what does the transcript and protein homology alignments from the maker run look like. ?Carson > On Feb 8, 2017, at 12:22 AM, Salim Bougouffa wrote: > > Hi, > > I am having trouble with MAKER/AUGUSTUS annotation. > > One of the recurrent ones is the example I attach in artemis screenshot. In red is the gene of interest. There are three GFF files. Top is augustus standalone that I executed on the non-masked scaffold. The second is maker annotation and third is the augustus (from maker) on the masked query. > > > The gene clearly has four exons and three CDS but the masking seem to somehow cause augustus to get the boundaries wrong for the first and third CDS and skip the second CDS entirely. > > How do I fix this problem? > > Best, > /SB > > -- > ____________________________ > Sent from Inbox Mobile > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 08:50:24 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 10:50:24 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? Message-ID: Hello: I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 09:03:41 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 11:03:41 -0500 Subject: [maker-devel] training of gene finders using whole assembly or longest contigs? Message-ID: Hello: I am training the gene finders using the whole assembly. But it seems very time consuming. Besides, I have to repeat the training process several times. Although I am running it on 25 nodes on a server, it may still take 3 (or even more) weeks for the training. I wonder how you guys train the SNAP. Do you use the whole assembly or just select the longest contigs for the training. If I only use longest contigs (like top 20% longest), will it be good enough as that get by using the whole assembly? Or should I randomly select 20% contigs for the training, for which we will have similar length distribution as the whole assembly? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alessandro.Rossoni at uni-duesseldorf.de Fri Feb 10 08:50:07 2017 From: Alessandro.Rossoni at uni-duesseldorf.de (Alessandro Rossoni) Date: Fri, 10 Feb 2017 16:50:07 +0100 Subject: [maker-devel] Updating Reference Gene Models - Legacy - Strategy Message-ID: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> Dear makers of MAKER, first of all - thank you for this awesome program! In the context of my project, I have been running MAKER on a set of novel genomes and it worked very well :) During the last days, I realized that the reference sequences of species A that I have been using as starting point for gene model prediction for species B, C and D are a bit flawed. How do I know that? Through visualization of novel RNA-Seq data that was mapped to the "old" gene models of species A, I came to the conclusion that the "old" gene models are not really accurate. In fact, between 52?62% of the reads map within annotated exonic regions of the genome and up to 47% map within intergenic regions. Intron/Exon boarders are pretty messed up and there are a lot more transcribed sequences in species A than previously thought. Hence, I would love to update the gene models of species A including the new RNA-Seq evidence and hope to get more accurate gene models out of it. The more accurate gene models of species A would be then used to predict genes for species B, C and D (which are hopefully going to benefit from the more accurate input). However, I there is a little understanding issue on how to set the parameters of the maker_opts.ctl. My plan is to produce new RNA-Seq based gene models for species A using Cufflinks, Stringtie, Breaker, Trinity and Velvet. I would pass the output to the maker_opts.ctl as: model_gff=Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff #the new gene models est=Species_A_reference_ests.fasta #the old/flawed gene models protein=swissprot.fasta Question 1: is this correct so far? But what sequences do I use for training the ab-initio predictors? snaphmm= augustus_species= Question 2: Do I use the "old/flawed" sequences that I know are not really good? I am not sure what sequences to use for training. Any help on this issue would be amazing! Best, Ale -- Alessandro W. Rossoni, M.Sc. Institute for Plant Biochemistry Heinrich-Heine-University -- http:///www.plant-biochemistry.hhu.de/ E-Mail: alessandro.rossoni at hhu-duesseldorf.de From carsonhh at gmail.com Fri Feb 10 09:36:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:36:46 -0700 Subject: [maker-devel] Updating Reference Gene Models - Legacy - Strategy In-Reply-To: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> References: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> Message-ID: If the old models are poor, then I suggest you do new training using BUSCO, CEGMA, or the est2genome or protein2genome options within MAKER ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors Also this thread ?> https://groups.google.com/forum/#!topic/maker-devel/FWMSTdqWQqI model_gff is for existing gene models you want to keep. So none of these should go there ?> Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff model_gff will always make it into the final annotation set even without any evidence support. By putting those files there, you are basically turning every feature in each of those files into a final gene model no matter how bad it is. Also if the original models are poor, don?t put them there either. You can doing reciprocal best blast hits with final models to old models to see how they match each other in the end. Will take a little data processing to make it work though. For all transcript based files, you should provide those to est_gff since they are evidence alignments and not model predictions. For Breaker.gff, that should be pred_gff since it is a prediction model. With Trinity, I suggest you provide the fasta file and allow MAKER to align and filter things rather than a GFF3. The problem with using GFF3 is you are basically short circuiting upstream prioritization and filtering saying ?take this evidence as is.? Also providing same evidence from multiple sources is a bad idea. By purposely making the evidence dataset more noisy, you are forcing lower accuracy. My suggesting would be not to use Cufflinks (it will introduce a very high false positive rate). Provide Trinity input as fasta (also make sure you use jaccard_clip option was used when assembling). And you will have to manually review models with and without Stringtie data to see if it hurts more than it helps. Provide Breaker.gff to pred_gff, but still allow maker to run Augustus itself internally (otherwise you won?t be able to use protein evidence as hints). Thanks, Carson > On Feb 10, 2017, at 8:50 AM, Alessandro Rossoni wrote: > > Dear makers of MAKER, > first of all - thank you for this awesome program! In the context of my project, I have been running MAKER on a set of novel genomes and it worked very well :) > > During the last days, I realized that the reference sequences of species A that I have been using as starting point for gene model prediction for species B, C and D are a bit flawed. How do I know that? Through visualization of novel RNA-Seq data that was mapped to the "old" gene models of species A, I came to the conclusion that the "old" gene models are not really accurate. In fact, between 52?62% of the reads map within annotated exonic regions of the genome and up to 47% map within intergenic regions. Intron/Exon boarders are pretty messed up and there are a lot more transcribed sequences in species A than previously thought. > > Hence, I would love to update the gene models of species A including the new RNA-Seq evidence and hope to get more accurate gene models out of it. The more accurate gene models of species A would be then used to predict genes for species B, C and D (which are hopefully going to benefit from the more accurate input). However, I there is a little understanding issue on how to set the parameters of the maker_opts.ctl. > > My plan is to produce new RNA-Seq based gene models for species A using Cufflinks, Stringtie, Breaker, Trinity and Velvet. I would pass the output to the maker_opts.ctl as: > > model_gff=Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff #the new gene models > est=Species_A_reference_ests.fasta #the old/flawed gene models > protein=swissprot.fasta > > Question 1: is this correct so far? > > But what sequences do I use for training the ab-initio predictors? > snaphmm= > augustus_species= > > Question 2: Do I use the "old/flawed" sequences that I know are not really good? I am not sure what sequences to use for training. > > Any help on this issue would be amazing! > Best, > Ale > > > -- > Alessandro W. Rossoni, M.Sc. > Institute for Plant Biochemistry > Heinrich-Heine-University > > -- > http:///www.plant-biochemistry.hhu.de/ > E-Mail: alessandro.rossoni at hhu-duesseldorf.de > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 10 09:39:31 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:39:31 -0700 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: MAKER is restartable. As long as you run each time in the same location, it can reuse existing alignments from the previous run. You also only need to train on ~10MB of the genome depending on gene density. Target size should be 300-400 genes. If you follow this GMOD wiki, this is demonstrated (you can also watch video - link as top of page - to see it being done) ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors ?Carson > On Feb 10, 2017, at 8:50 AM, Quanwei Zhang wrote: > > Hello: > > I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 10 09:42:13 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:42:13 -0700 Subject: [maker-devel] training of gene finders using whole assembly or longest contigs? In-Reply-To: References: Message-ID: Example of training here ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors You can also search the devel mailing list archives here ?> https://groups.google.com/forum/#!forum/maker-devel There are lots and lots of threads that go into detail on training. Note more than 2 rounds of training is not beneficial, and can actually make performance worse (there is an overtraining paradox). ?Carson > On Feb 10, 2017, at 9:03 AM, Quanwei Zhang wrote: > > Hello: > > I am training the gene finders using the whole assembly. But it seems very time consuming. Besides, I have to repeat the training process several times. Although I am running it on 25 nodes on a server, it may still take 3 (or even more) weeks for the training. I wonder how you guys train the SNAP. Do you use the whole assembly or just select the longest contigs for the training. If I only use longest contigs (like top 20% longest), will it be good enough as that get by using the whole assembly? Or should I randomly select 20% contigs for the training, for which we will have similar length distribution as the whole assembly? > > Thanks > > Best > Quanwei > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 10:04:55 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 12:04:55 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: Great. Many thanks. So I can select part of my genome assembly for the training. Which the following ways do you think is better? (a) Select the longest contigs. (b) Randomly select contigs with any length? Thank you! Best Quanwei 2017-02-10 11:39 GMT-05:00 Carson Holt : > MAKER is restartable. As long as you run each time in the same location, > it can reuse existing alignments from the previous run. You also only need > to train on ~10MB of the genome depending on gene density. Target size > should be 300-400 genes. > > If you follow this GMOD wiki, this is demonstrated (you can also watch > video - link as top of page - to see it being done) ?> > http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/ > MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ > ab_initio_Gene_Predictors > > ?Carson > > > > On Feb 10, 2017, at 8:50 AM, Quanwei Zhang wrote: > > Hello: > > I am annotating a new genome using Maker. I have RNA-seq assembly and > protein sequences (from other organisms) in fasta format. Since I need to > train gene finders, so I have to run Maker several times. I think the > aligning process between the transcript assembly (protein sequences) and > the genome assembly may be time consuming. So I wonder whether I can save > such alignment in the first run, and then make use of such alignment in the > following runs? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Fri Feb 10 10:07:51 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Fri, 10 Feb 2017 12:07:51 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: The longer ones will have more complete genes on them. If you get a set scaffolds that has about 1,000 genes you probably have enough for training. Mike > On Feb 10, 2017, at 12:04 PM, Quanwei Zhang wrote: > > Great. Many thanks. So I can select part of my genome assembly for the training. Which the following ways do you think is better? (a) Select the longest contigs. (b) Randomly select contigs with any length? > > Thank you! > > Best > Quanwei > > 2017-02-10 11:39 GMT-05:00 Carson Holt >: > MAKER is restartable. As long as you run each time in the same location, it can reuse existing alignments from the previous run. You also only need to train on ~10MB of the genome depending on gene density. Target size should be 300-400 genes. > > If you follow this GMOD wiki, this is demonstrated (you can also watch video - link as top of page - to see it being done) ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors > > ?Carson > > > >> On Feb 10, 2017, at 8:50 AM, Quanwei Zhang > wrote: >> >> Hello: >> >> I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? >> >> Thanks >> >> Best >> Quanwei >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 21:53:02 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 23:53:02 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" Message-ID: Hello: I want to assign gene function to the predicted genes. I collected UniProt/SwissProt human, mouse and rat protein sequences (including both canonical and isoforms). Then use "makeblastdb" to build the database. And then run "blastp -db ... -max_hsps_per_subject 1" following the example in the protocol. But it returns me an error: Unknown argument: "max_hsps_per_subject". Why this happens? Is it because I am using a different version of blastp? USAGE blastp [-h] [-help] [-import_search_strategy filename] [-export_search_strategy filename] [-task task_name] [-db database_name] [-dbsize num_letters] [-gilist filename] [-seqidlist filename] [-negative_gilist filename] [-entrez_query entrez_query] [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm] [-subject subject_input_file] [-subject_loc range] [-query input_file] [-out output_file] [-evalue evalue] [-word_size int_value] [-gapopen open_penalty] [-gapextend extend_penalty] [-qcov_hsp_perc float_value] [-max_hsps int_value] [-xdrop_ungap float_value] [-xdrop_gap float_value] [-xdrop_gap_final float_value] [-searchsp int_value] [-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking] [-matrix matrix_name] [-threshold float_value] [-culling_limit int_value] [-best_hit_overhang float_value] [-best_hit_score_edge float_value] [-window_size int_value] [-lcase_masking] [-query_loc range] [-parse_deflines] [-outfmt format] [-show_gis] [-num_descriptions int_value] [-num_alignments int_value] [-line_length line_length] [-html] [-max_target_seqs num_sequences] [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo] [-use_sw_tback] [-version] Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Mon Feb 13 07:02:30 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Mon, 13 Feb 2017 09:02:30 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" In-Reply-To: References: Message-ID: Hi Quanwei, Different versions/implementations of blast do have parameters. Based on the usage you posted my guess is that the parameter you want is ?-max_hsps? Thanks, Mike > On Feb 10, 2017, at 11:53 PM, Quanwei Zhang wrote: > > Hello: > > I want to assign gene function to the predicted genes. I collected UniProt/SwissProt human, mouse and rat protein sequences (including both canonical and isoforms). Then use "makeblastdb" to build the database. And then run "blastp -db ... -max_hsps_per_subject 1" following the example in the protocol. But it returns me an error: Unknown argument: "max_hsps_per_subject". Why this happens? Is it because I am using a different version of blastp? > > USAGE > blastp [-h] [-help] [-import_search_strategy filename] > [-export_search_strategy filename] [-task task_name] [-db database_name] > [-dbsize num_letters] [-gilist filename] [-seqidlist filename] > [-negative_gilist filename] [-entrez_query entrez_query] > [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm] > [-subject subject_input_file] [-subject_loc range] [-query input_file] > [-out output_file] [-evalue evalue] [-word_size int_value] > [-gapopen open_penalty] [-gapextend extend_penalty] > [-qcov_hsp_perc float_value] [-max_hsps int_value] > [-xdrop_ungap float_value] [-xdrop_gap float_value] > [-xdrop_gap_final float_value] [-searchsp int_value] > [-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking] > [-matrix matrix_name] [-threshold float_value] [-culling_limit int_value] > [-best_hit_overhang float_value] [-best_hit_score_edge float_value] > [-window_size int_value] [-lcase_masking] [-query_loc range] > [-parse_deflines] [-outfmt format] [-show_gis] > [-num_descriptions int_value] [-num_alignments int_value] > [-line_length line_length] [-html] [-max_target_seqs num_sequences] > [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo] > [-use_sw_tback] [-version] > > Thanks > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From qwzhang0601 at gmail.com Mon Feb 13 08:02:24 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 10:02:24 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" In-Reply-To: References: Message-ID: Thanks. Yes, I should use "-max_hsps". Best Quanwei 2017-02-13 9:02 GMT-05:00 Michael Campbell : > Hi Quanwei, > > Different versions/implementations of blast do have parameters. Based on > the usage you posted my guess is that the parameter you want is ?-max_hsps? > > Thanks, > Mike > > On Feb 10, 2017, at 11:53 PM, Quanwei Zhang > wrote: > > > > Hello: > > > > I want to assign gene function to the predicted genes. I collected > UniProt/SwissProt human, mouse and rat protein sequences (including both > canonical and isoforms). Then use "makeblastdb" to build the database. And > then run "blastp -db ... -max_hsps_per_subject 1" following the example in > the protocol. But it returns me an error: Unknown argument: > "max_hsps_per_subject". Why this happens? Is it because I am using a > different version of blastp? > > > > USAGE > > blastp [-h] [-help] [-import_search_strategy filename] > > [-export_search_strategy filename] [-task task_name] [-db > database_name] > > [-dbsize num_letters] [-gilist filename] [-seqidlist filename] > > [-negative_gilist filename] [-entrez_query entrez_query] > > [-db_soft_mask filtering_algorithm] [-db_hard_mask > filtering_algorithm] > > [-subject subject_input_file] [-subject_loc range] [-query > input_file] > > [-out output_file] [-evalue evalue] [-word_size int_value] > > [-gapopen open_penalty] [-gapextend extend_penalty] > > [-qcov_hsp_perc float_value] [-max_hsps int_value] > > [-xdrop_ungap float_value] [-xdrop_gap float_value] > > [-xdrop_gap_final float_value] [-searchsp int_value] > > [-sum_stats bool_value] [-seg SEG_options] [-soft_masking > soft_masking] > > [-matrix matrix_name] [-threshold float_value] [-culling_limit > int_value] > > [-best_hit_overhang float_value] [-best_hit_score_edge float_value] > > [-window_size int_value] [-lcase_masking] [-query_loc range] > > [-parse_deflines] [-outfmt format] [-show_gis] > > [-num_descriptions int_value] [-num_alignments int_value] > > [-line_length line_length] [-html] [-max_target_seqs num_sequences] > > [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats > compo] > > [-use_sw_tback] [-version] > > > > Thanks > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 08:16:35 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 10:16:35 -0500 Subject: [maker-devel] failed to assign putative gene function Message-ID: Hello: I am trying to add putative gene function to the predicted gene models. Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. I used canonical and isoform proteins of human, mouse and rat with the script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose context is as below. maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 7e-164 464 snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 3e-80 236 After that, I am trying to add the protein homology data to the Maker gff3 and fasta files with maker_functional_gff and maker_functional_fasta, but get the reports as below. Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 39. Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 39. Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 45. Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 45. Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B ..... I am not sure how to deal with this. I followed the command given in the protocol. Any suggestions? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 09:09:48 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 11:09:48 -0500 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: I just found at the bottom of the output file it include information like below. But I did not get those GFF3 files with gene homolog information added. >snap_masked-CasCan_contig_27993-processed-gene-0.6-mRNA-1 transcript Name:"Protein of unknown function" offset:0 AED:0.47 eAED:0.47 QI:0|0|0|1|1|1|2|0|84 ATGAAAGACATTGGTACCCCAGAGGCATGGCAGATAATGATGTCCCTCAAGTCTGGACTC TTGGCAGAGATCACATGGGCTTTAGACACCATTAACATTCTACTGTATGATGACAGCAGC ATTATGACCTTCAACCTCAGTCAGTTCCCAGGATTGCTAGAGCTCTTTGAGTATGAGGTG GGTGACCGAAGACAGAGAACTCTACTGGACTCTGGGAGATTCAGTGAAGTGTCTGGTCCA ACCCCTACAGAG Thanks Best Quanwei 2017-02-13 10:16 GMT-05:00 Quanwei Zhang : > Hello: > > I am trying to add putative gene function to the predicted gene models. > Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. > I used canonical and isoform proteins of human, mouse and rat with the > script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose > context is as below. > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN > 69.97 303 91 0 1 303 1 303 7e-164 464 > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 > sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 > 3e-80 236 > > After that, I am trying to add the protein homology data to the Maker gff3 > and fasta files with maker_functional_gff and maker_functional_fasta, but > get the reports as below. > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 39. > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 39. > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 45. > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 45. > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > ..... > > I am not sure how to deal with this. I followed the command given in the > protocol. Any suggestions? > > Thanks > > Best > Quanwei > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 10:03:13 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 12:03:13 -0500 Subject: [maker-devel] Which version of InterProScan should I use Message-ID: I am planning to add domain information to my predicted gene models. But not sure whether the scripts provided by Maker is compatible with the latest version of InterProScan, or should I use a old version. I am using maker2.31.9. It seems the latest version is 5.22-61.0 released on 1/23/17. ftp://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/ Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcui at age.mpg.de Tue Feb 14 05:38:27 2017 From: rcui at age.mpg.de (Ray Cui) Date: Tue, 14 Feb 2017 13:38:27 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints Message-ID: Hello, I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Tue Feb 14 07:45:29 2017 From: dence at genetics.utah.edu (Daniel Ence) Date: Tue, 14 Feb 2017 14:45:29 +0000 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: Message-ID: Hi Ray, I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. ~Daniel On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: Hello, I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcui at age.mpg.de Tue Feb 14 08:44:19 2017 From: rcui at age.mpg.de (Ray Cui) Date: Tue, 14 Feb 2017 16:44:19 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> Message-ID: Hi Daniel, thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel wrote: > Hi Ray, > > I think you?re on the right track with training Genemark with RNAseq data. > It should only change the training steps, which are external to MAKER, but > not how MAKER runs Genemark. You?ll still give MAKER the path to the > ?es.mod" file made by Genemark. > > For the 2nd question, in the MAKER beta 3, MAKER creates a control file > for EVM, in which you set your weights for the various inputs, and then > MAKER runs EVM alongside all the other gene predictors and chooses the > model that is best supported by the evidence. > > ~Daniel > > > > On Feb 14, 2017, at 7:38 AM, Ray Cui wrote: > > Hello, > > I have sucessfully installed Maker beta 3, working with both > Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio > predictor. > When I read the GeneMark-ES manual, it says that one can use > RNAseq data to aid training. I'm wondering what would be the best way to > integrate Genemark-ET predictions into Maker. Should I run Genemark-ET > independent of Maker, then integrate the GFF at some point during the maker > process? If so, how should I edit the configuration file? Currently maker > has an option called "gmhmm". Should I then train GeneMark by myself with > RNAseq data, then feed the hmm to maker? > > And perhaps an unrelated question is that now Maker beta 3 > supports EVM. I'm wondering how EVM is used by Maker (at which step, what > does it do), and how does it differ from what Maker is designed for (both > reconciles different gene models). > > Best Regards, > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for > Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 <+49%20221%20496> > Mobile: +49 0221 37970 496 > rcui at age.mpg.de > www.age.mpg.de > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Tue Feb 14 12:51:46 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Tue, 14 Feb 2017 14:51:46 -0500 Subject: [maker-devel] Which version of InterProScan should I use In-Reply-To: References: Message-ID: <6A140E2D-603E-46C9-97FA-D59AEAEF95A9@gmail.com> I have been able to make the accessory scripts work with output from InterProScan v5 that I downloaded a couple of month ago. Mike > On Feb 13, 2017, at 12:03 PM, Quanwei Zhang wrote: > > I am planning to add domain information to my predicted gene models. But not sure whether the scripts provided by Maker is compatible with the latest version of InterProScan, or should I use a old version. I am using maker2.31.9. > > It seems the latest version is 5.22-61.0 released on 1/23/17. > ftp://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/ > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Thu Feb 16 03:44:39 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Thu, 16 Feb 2017 11:44:39 +0100 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> Message-ID: <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> Hi! Unfortunately all of the options failed on our cluster. See: Hi, Most recent Maker test with --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 Error: --> rank=2, hostname=uc1n518.localdomain [uc1n518:67009] *** Process received signal *** [uc1n518:67009] Signal: Segmentation fault (11) [uc1n518:67009] Signal code: Address not mapped (1) [uc1n518:67009] Failing at address: 0x4b0 -------------------------------------------------------------------------- mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). With: --mca btl ^openib and also this --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 Error: ### Runing Maker example STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... [uc1n514:59985] *** Process received signal *** [uc1n514:59985] Signal: Segmentation fault (11) [uc1n514:59985] Signal code: Address not mapped (1) [uc1n514:59985] Failing at address: 0x4b0 -------------------------------------------------------------------------- mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). Am 28.01.2017 um 21:53 schrieb Carson Holt: > Try adding one of the following to your mpiexec command ?> > > 1. --mca btl ^openib > 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 > 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 > > One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. > > --Carson > > >> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >> >> Hi everybody. >> >> My name is Rainer. I am an administrator for our HPC-Systems at our >> university in Konstanz, Baden-Wuertemberg/Germany. >> The procect is called bwHPC-C5. >> >> See: https://www.bwhpc-c5.de/en/index.php >> >> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >> i get errors while running a Maker job in the MPI-environment. >> >> BUILD STATUS >> >> ============================================================================== >> STATUS MAKER v2.31.9 >> ============================================================================== >> PERL Dependencies: VERIFIED >> External Programs: VERIFIED >> External C Libraries: VERIFIED >> MPI SUPPORT: ENABLED >> MWAS Web Interface: DISABLED >> MAKER PACKAGE: CONFIGURATION OK >> >> MODULES / INCLUDES / COMPILERS >> >> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >> # >> ##### (B) Dependencies: >> # >> # conflict: any other maker version >> # module load compiler/gnu/5.2 >> # module load mpi/openmpi/2.0-gnu-5.2 >> [...] >> >> MPI/MOAB SUBMIT >> >> [...] >> ### Queues ### >> #MSUB -q fat >> #MSUB -l nodes=1:ppn=16 >> #MSUB -l mem=20gb >> #MSUB -l walltime=50:00:00 >> # >> [...] >> echo " " >> echo "### Loading MAKER module:" >> echo " " >> module load bio/maker/2.31.9 >> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >> echo "MAKER_VERSION = $MAKER_VERSION" >> module list >> [...] >> echo " " >> echo "### Runing Maker example" >> echo " " >> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >> export OMPI_MCA_mpi_warn_on_fork=0 >> >> echo "LD_PRELOAD=${LD_PRELOAD}" >> # >> # "STATUS: Processing and indexing input FASTA files..." >> # >> mpiexec -mca btl ^openib -n 16 maker >> [...] >> >> >> E R R O R S >> ======= >> [...] >> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >> STATUS: Parsing control files... >> STATUS: Processing and indexing input FASTA files... >> [uc1n338:113607] *** Process received signal *** >> [uc1n338:113607] Signal: Segmentation fault (11) >> [uc1n338:113607] Signal code: Address not mapped (1) >> [uc1n338:113607] Failing at address: 0x4b0 >> [uc1n338:113608] *** Process received signal *** >> [uc1n338:113608] Signal: Segmentation fault (11) >> [uc1n338:113608] Signal code: Address not mapped (1) >> [uc1n338:113608] Failing at address: 0x4b0 >> [uc1n338:113621] *** Process received signal *** >> [uc1n338:113621] Signal: Segmentation fault (11) >> [uc1n338:113621] Signal code: Address not mapped (1) >> [uc1n338:113621] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >> -------------------------------------------------------------------------- >> [...] >> >> WHATS WRONG HERE!? >> >> Thank you for your help! >> >> All the best , >> >> Rainer >> >> -- >> Rainer Rutka >> University of Konstanz >> Communication, Information, Media Centre (KIM) >> * High-Performance-Computing (HPC) >> * KIM-Support and -Base-Services >> Room: V511 >> 78457 Konstanz, Germany >> +49 7531 88-5413 >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- Rainer Rutka Universit?t Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Raum: V511, Tel: 54 13 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From munholl at uwindsor.ca Fri Feb 17 13:11:44 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Fri, 17 Feb 2017 15:11:44 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there Message-ID: Hi Everyone, After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. However I also see $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 17 13:39:33 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 17 Feb 2017 13:39:33 -0700 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: Message-ID: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. Thanks, Carson > On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: > > Hi Everyone, > > After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl > > (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: > > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > However I also see > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 17 14:57:05 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 17 Feb 2017 16:57:05 -0500 Subject: [maker-devel] default gene model by quality_filter.pl is the same with those by setting keep_preds=0? Message-ID: Hello: I am following the Maker protocol (2014) to rescue rejected gene models. I set keep_preds=1 when I ran Maker, and then used "quality_filter.pl -d" to get the "default" Maker report. According to the annotation it will "Prints transcripts with an AED <1". But in a previous post (the link below), it was said "there are models with AED less than 1 that get rejected for other reasons. ... For example if the only evidence is a protein alignment that has deep overlapping HSPs (extremely low complexity alignment) it will be filtered out even though AED is not technically equal to 1. Also if the overlapping protein evidence is in a different reading frame than the model it is supposed to support then the AED will be less than 1 but eAED will be 1 (extended AED), and the model will be rejected." *So I wonder, whether the "default" result obtained by "quality_filter.pl " are really the same as those achieved by setting keep_preds=0? Will the script also consider other reasons besides "* *AED <1"?* https://groups.google.com/forum/#!searchin/maker-devel/The$20reason$20there$20are$20differences$20between$20the$20runs$20is$20that$20there$20are$20models$20with$20AED$20less$20%7Csort:relevance/maker-devel/97aNJkT3bgk/W0MEyqEZAAAJ Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qlian003 at ucr.edu Sat Feb 18 09:43:08 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Sat, 18 Feb 2017 08:43:08 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation Message-ID: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> Dear Maker develop team, I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? Thanks Qihua From carsonhh at gmail.com Sun Feb 19 22:39:14 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 22:39:14 -0700 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> Message-ID: <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> MAKER was support was designed with GeneMark-ES. It may or may not work with GeneMark-ET. So any MAKER related archive posts etc. will be related to the latter. With GeneMark-ES, you simply provided a genome assembly and let it run. It would then produce several files and output directories. The es.mod file was the one you provided to MAKER. I don?t know how this compares to GeneMark-ET. ?Carson > On Feb 14, 2017, at 8:44 AM, Ray Cui wrote: > > Hi Daniel, > > thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? > > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 > Mobile: +49 0221 37970 496 <> > rcui at age.mpg.de > www.age.mpg.de > > > > On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel > wrote: > Hi Ray, > > I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. > > For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. > > ~Daniel > > > >> On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: >> >> Hello, >> >> I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. >> When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? >> >> And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). >> >> Best Regards, >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 >> Mobile: +49 0221 37970 496 <> >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Sun Feb 19 22:43:49 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 22:43:49 -0700 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> Message-ID: <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> Try running just on a single node (not across nodes). If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and running MAKER with that new version. You can install it in your home directory and test from there, just make sure to add it to your path. Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. ?Carson > On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: > > Hi! > > Unfortunately all of the options failed on our cluster. > > See: > > Hi, > > Most recent Maker test with > --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 > Error: > --> rank=2, hostname=uc1n518.localdomain > [uc1n518:67009] *** Process received signal *** > [uc1n518:67009] Signal: Segmentation fault (11) > [uc1n518:67009] Signal code: Address not mapped (1) > [uc1n518:67009] Failing at address: 0x4b0 > -------------------------------------------------------------------------- > mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). > > > With: > --mca btl ^openib > and also this > --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 > Error: > ### Runing Maker example > > STATUS: Parsing control files... > STATUS: Processing and indexing input FASTA files... > [uc1n514:59985] *** Process received signal *** > [uc1n514:59985] Signal: Segmentation fault (11) > [uc1n514:59985] Signal code: Address not mapped (1) > [uc1n514:59985] Failing at address: 0x4b0 > -------------------------------------------------------------------------- > mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). > > > Am 28.01.2017 um 21:53 schrieb Carson Holt: >> Try adding one of the following to your mpiexec command ?> >> >> 1. --mca btl ^openib >> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >> >> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >> >> --Carson >> >> >>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>> >>> Hi everybody. >>> >>> My name is Rainer. I am an administrator for our HPC-Systems at our >>> university in Konstanz, Baden-Wuertemberg/Germany. >>> The procect is called bwHPC-C5. >>> >>> See: https://www.bwhpc-c5.de/en/index.php >>> >>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>> i get errors while running a Maker job in the MPI-environment. >>> >>> BUILD STATUS >>> >>> ============================================================================== >>> STATUS MAKER v2.31.9 >>> ============================================================================== >>> PERL Dependencies: VERIFIED >>> External Programs: VERIFIED >>> External C Libraries: VERIFIED >>> MPI SUPPORT: ENABLED >>> MWAS Web Interface: DISABLED >>> MAKER PACKAGE: CONFIGURATION OK >>> >>> MODULES / INCLUDES / COMPILERS >>> >>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>> # >>> ##### (B) Dependencies: >>> # >>> # conflict: any other maker version >>> # module load compiler/gnu/5.2 >>> # module load mpi/openmpi/2.0-gnu-5.2 >>> [...] >>> >>> MPI/MOAB SUBMIT >>> >>> [...] >>> ### Queues ### >>> #MSUB -q fat >>> #MSUB -l nodes=1:ppn=16 >>> #MSUB -l mem=20gb >>> #MSUB -l walltime=50:00:00 >>> # >>> [...] >>> echo " " >>> echo "### Loading MAKER module:" >>> echo " " >>> module load bio/maker/2.31.9 >>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>> echo "MAKER_VERSION = $MAKER_VERSION" >>> module list >>> [...] >>> echo " " >>> echo "### Runing Maker example" >>> echo " " >>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>> export OMPI_MCA_mpi_warn_on_fork=0 >>> >>> echo "LD_PRELOAD=${LD_PRELOAD}" >>> # >>> # "STATUS: Processing and indexing input FASTA files..." >>> # >>> mpiexec -mca btl ^openib -n 16 maker >>> [...] >>> >>> >>> E R R O R S >>> ======= >>> [...] >>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>> STATUS: Parsing control files... >>> STATUS: Processing and indexing input FASTA files... >>> [uc1n338:113607] *** Process received signal *** >>> [uc1n338:113607] Signal: Segmentation fault (11) >>> [uc1n338:113607] Signal code: Address not mapped (1) >>> [uc1n338:113607] Failing at address: 0x4b0 >>> [uc1n338:113608] *** Process received signal *** >>> [uc1n338:113608] Signal: Segmentation fault (11) >>> [uc1n338:113608] Signal code: Address not mapped (1) >>> [uc1n338:113608] Failing at address: 0x4b0 >>> [uc1n338:113621] *** Process received signal *** >>> [uc1n338:113621] Signal: Segmentation fault (11) >>> [uc1n338:113621] Signal code: Address not mapped (1) >>> [uc1n338:113621] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>> -------------------------------------------------------------------------- >>> [...] >>> >>> WHATS WRONG HERE!? >>> >>> Thank you for your help! >>> >>> All the best , >>> >>> Rainer >>> >>> -- >>> Rainer Rutka >>> University of Konstanz >>> Communication, Information, Media Centre (KIM) >>> * High-Performance-Computing (HPC) >>> * KIM-Support and -Base-Services >>> Room: V511 >>> 78457 Konstanz, Germany >>> +49 7531 88-5413 >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -- > Rainer Rutka > Universit?t Konstanz > Kommunikations-, Informations-, Medienzentrum (KIM) > Raum: V511, Tel: 54 13 > From carsonhh at gmail.com Sun Feb 19 23:05:09 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 23:05:09 -0700 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> Message-ID: <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. Thanks, Carson > On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: > > Dear Maker develop team, > > I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? > > Thanks > Qihua > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From rcui at age.mpg.de Mon Feb 20 01:59:12 2017 From: rcui at age.mpg.de (Ray Cui) Date: Mon, 20 Feb 2017 09:59:12 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> Message-ID: Dear Carson, I have now run GeneMark-ET, and it produces a trained .mod file. I think it can be then passed to Maker. Do you know what is the final constructed command line in Maker that calls genemark? Genemark-et and es use the same perl script so one probably only needs to use the --prediction and --predict_with xxx.mod options to predict genes using the species specific parameters (bypassing regular training and prediction steps) Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de On Mon, Feb 20, 2017 at 6:39 AM, Carson Holt wrote: > MAKER was support was designed with GeneMark-ES. It may or may not work > with GeneMark-ET. So any MAKER related archive posts etc. will be related > to the latter. > > With GeneMark-ES, you simply provided a genome assembly and let it run. It > would then produce several files and output directories. The es.mod file > was the one you provided to MAKER. I don?t know how this compares > to GeneMark-ET. > > ?Carson > > > > On Feb 14, 2017, at 8:44 AM, Ray Cui wrote: > > Hi Daniel, > > thanks! It seems that Genemark-ET has a "--training" flag, is that > the flag I should use when training or should I just let Genemark also > perform the prediction? > > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for > Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 <+49%20221%20496> > Mobile: +49 0221 37970 496 > rcui at age.mpg.de > www.age.mpg.de > > > > On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel wrote: > >> Hi Ray, >> >> I think you?re on the right track with training Genemark with RNAseq >> data. It should only change the training steps, which are external to >> MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to >> the ?es.mod" file made by Genemark. >> >> For the 2nd question, in the MAKER beta 3, MAKER creates a control file >> for EVM, in which you set your weights for the various inputs, and then >> MAKER runs EVM alongside all the other gene predictors and chooses the >> model that is best supported by the evidence. >> >> ~Daniel >> >> >> >> On Feb 14, 2017, at 7:38 AM, Ray Cui wrote: >> >> Hello, >> >> I have sucessfully installed Maker beta 3, working with both >> Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio >> predictor. >> When I read the GeneMark-ES manual, it says that one can use >> RNAseq data to aid training. I'm wondering what would be the best way to >> integrate Genemark-ET predictions into Maker. Should I run Genemark-ET >> independent of Maker, then integrate the GFF at some point during the maker >> process? If so, how should I edit the configuration file? Currently maker >> has an option called "gmhmm". Should I then train GeneMark by myself with >> RNAseq data, then feed the hmm to maker? >> >> And perhaps an unrelated question is that now Maker beta 3 >> supports EVM. I'm wondering how EVM is used by Maker (at which step, what >> does it do), and how does it differ from what Maker is designed for (both >> reconciles different gene models). >> >> Best Regards, >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for >> Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 <+49%20221%20496> >> Mobile: +49 0221 37970 496 >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 20 02:02:00 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 02:02:00 -0700 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> Message-ID: The names of scripts used are listed in the maker_exe.ctl file. It depends on if formatting or any flags have changed between versions. ?Carson > On Feb 20, 2017, at 1:59 AM, Ray Cui wrote: > > Dear Carson, > > I have now run GeneMark-ET, and it produces a trained .mod file. I think it can be then passed to Maker. Do you know what is the final constructed command line in Maker that calls genemark? Genemark-et and es use the same perl script so one probably only needs to use the --prediction and --predict_with xxx.mod options to predict genes using the species specific parameters (bypassing regular training and prediction steps) > > > Best Regards, > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 > Mobile: +49 0221 37970 496 <> > rcui at age.mpg.de > www.age.mpg.de > > > > On Mon, Feb 20, 2017 at 6:39 AM, Carson Holt > wrote: > MAKER was support was designed with GeneMark-ES. It may or may not work with GeneMark-ET. So any MAKER related archive posts etc. will be related to the latter. > > With GeneMark-ES, you simply provided a genome assembly and let it run. It would then produce several files and output directories. The es.mod file was the one you provided to MAKER. I don?t know how this compares to GeneMark-ET. > > ?Carson > > > >> On Feb 14, 2017, at 8:44 AM, Ray Cui > wrote: >> >> Hi Daniel, >> >> thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? >> >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 >> Mobile: +49 0221 37970 496 <> >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> >> On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel > wrote: >> Hi Ray, >> >> I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. >> >> For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. >> >> ~Daniel >> >> >> >>> On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: >>> >>> Hello, >>> >>> I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. >>> When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? >>> >>> And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). >>> >>> Best Regards, >>> Ray >>> >>> Dr. Rongfeng (Ray) Cui >>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >>> Wissenschaftlicher MA / Postdoctoral researcher >>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >>> Tel.:+49 (0)221 496 >>> Mobile: +49 0221 37970 496 <> >>> rcui at age.mpg.de >>> www.age.mpg.de >>> >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qlian003 at ucr.edu Mon Feb 20 13:34:31 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Mon, 20 Feb 2017 12:34:31 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> Message-ID: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Hi Carson, Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? Thanks Qihua > On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: > > IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. > > CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. > > If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. > > What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. > > However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. > > Thanks, > Carson > >> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >> >> Dear Maker develop team, >> >> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >> >> Thanks >> Qihua >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > From carsonhh at gmail.com Mon Feb 20 16:56:01 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 16:56:01 -0700 Subject: [maker-devel] default gene model by quality_filter.pl is the same with those by setting keep_preds=0? In-Reply-To: References: Message-ID: <14B6C165-54D2-4722-9A26-91FBEBB18B72@gmail.com> Any hint based predictions rejected for one of the non-AED filters will still be rejected regardless of keep_preds=1. However you are right in that any raw predictions that were rejected by other filters may still be kept by using this workflow, since keep_preds=1 essentially rescues all non-overlapping raw predictions that would have been rejected. ?Carson > On Feb 17, 2017, at 2:57 PM, Quanwei Zhang wrote: > > Hello: > > I am following the Maker protocol (2014) to rescue rejected gene models. I set keep_preds=1 when I ran Maker, and then used "quality_filter.pl -d" to get the "default" Maker report. According to the annotation it will "Prints transcripts with an AED <1". > > But in a previous post (the link below), it was said "there are models with AED less than 1 that get rejected for other reasons. ... For example if the only evidence is a protein alignment that has deep overlapping HSPs (extremely low complexity alignment) it will be filtered out even though AED is not technically equal to 1. Also if the overlapping protein evidence is in a different reading frame than the model it is supposed to support then the AED will be less than 1 but eAED will be 1 (extended AED), and the model will be rejected." > > So I wonder, whether the "default" result obtained by "quality_filter.pl " are really the same as those achieved by setting keep_preds=0? Will the script also consider other reasons besides "AED <1"? > > https://groups.google.com/forum/#!searchin/maker-devel/The$20reason$20there$20are$20differences$20between$20the$20runs$20is$20that$20there$20are$20models$20with$20AED$20less$20%7Csort:relevance/maker-devel/97aNJkT3bgk/W0MEyqEZAAAJ > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 20 16:56:02 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 16:56:02 -0700 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: You either uses TrEMBL or the UniProtKB Isoform sequence set. Their fasta headers are slightly different and will not be parsed correctly, For example, here is the header as formatted for the same sequence in the Swiss-prot dataset download ?> >sp|Q7Z5M8|AB12B_HUMAN Protein ABHD12B OS=Homo sapiens GN=ABHD12B PE=2 SV=1 I think you used the UniProtKB Isoform sequence dataset instead. ?Carson > On Feb 13, 2017, at 8:16 AM, Quanwei Zhang wrote: > > Hello: > > I am trying to add putative gene function to the predicted gene models. Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. I used canonical and isoform proteins of human, mouse and rat with the script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose context is as below. > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 7e-164 464 > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 3e-80 236 > > After that, I am trying to add the protein homology data to the Maker gff3 and fasta files with maker_functional_gff and maker_functional_fasta, but get the reports as below. > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 39. > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 39. > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 45. > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 45. > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > ..... > > I am not sure how to deal with this. I followed the command given in the protocol. Any suggestions? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From qwzhang0601 at gmail.com Tue Feb 21 07:30:35 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Tue, 21 Feb 2017 09:30:35 -0500 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: Thank you! I used both canonical and isoform of protein sequences from swiss-prot in the beginning. It reported the error, but later I only used the canonical protein sequences to build the database and then it worked. Best Quanwei 2017-02-20 18:56 GMT-05:00 Carson Holt : > You either uses TrEMBL or the UniProtKB Isoform sequence set. Their fasta > headers are slightly different and will not be parsed correctly, > > For example, here is the header as formatted for the same sequence in the > Swiss-prot dataset download ?> > >sp|Q7Z5M8|AB12B_HUMAN Protein ABHD12B OS=Homo sapiens GN=ABHD12B PE=2 SV=1 > > I think you used the UniProtKB Isoform sequence dataset instead. > > ?Carson > > > > > > > On Feb 13, 2017, at 8:16 AM, Quanwei Zhang > wrote: > > > > Hello: > > > > I am trying to add putative gene function to the predicted gene models. > Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. > I used canonical and isoform proteins of human, mouse and rat with the > script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose > context is as below. > > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 > sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 > 7e-164 464 > > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 > sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 > 3e-80 236 > > > > After that, I am trying to add the protein homology data to the Maker > gff3 and fasta files with maker_functional_gff and maker_functional_fasta, > but get the reports as below. > > > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform > 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 39. > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 39. > > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform > 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 45. > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 45. > > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform > 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > ..... > > > > I am not sure how to deal with this. I followed the command given in the > protocol. Any suggestions? > > > > Thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Tue Feb 21 10:17:22 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 12:17:22 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Message-ID: I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: cpan[1]> install forks::signals Warning: Cannot install forks::signals, don't know what it is. Try the command i /forks::signals/ to find objects with matching identifiers. but scrolling through the install scroll of the forks reinstall I do see it got installed properly. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: > Do a force reinstall of forks via CPAN. The error is coming from forks.pm line > 83, so the problem is the forks installation itself. > > Thanks, > Carson > > > On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: > > Hi Everyone, > > After sorting my MPICH/OpenMPI issue I have come across another: When I > try to run MAKER on the demo data via > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker > /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl > /Data/Apps/maker/data/maker_bopts.ctl > > (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it > working before opening up, if I change the -n value then the error repeats > once for each process I attempt to run via MPICH) I got the following: > > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > However I also see > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially > thought this was a Perl error. I hit up perlmonks and found that there is > an issue between forks and Storable, where Storable now has a verion number > of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these > instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and > tried again. Now I get: > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker > /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl > /Data/Apps/maker/data/maker_bopts.ctl > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > Clearly MAKER has an issue with forks, but it is installed, it is up to > date (I double checked via cpan and apt-get to be sure), and it is in a > directory that is in @INC. Should I pursue this as a MAKER error or as a > Perl error? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 21 10:19:30 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 21 Feb 2017 10:19:30 -0700 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Message-ID: <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> forks::signals is part of forks. It?s not a separate package. If it?s missing, there is a problem with the forks installation. So you need to do a force install of forks to force the reinstall. Example ?> cpan[1]> force install forks ?Carson > On Feb 21, 2017, at 10:17 AM, Seth Munholland wrote: > > I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: > > cpan[1]> install forks::signals > Warning: Cannot install forks::signals, don't know what it is. > Try the command > > i /forks::signals/ > > to find objects with matching identifiers. > > but scrolling through the install scroll of the forks reinstall I do see it got installed properly. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt > wrote: > Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. > > Thanks, > Carson > > >> On Feb 17, 2017, at 1:11 PM, Seth Munholland > wrote: >> >> Hi Everyone, >> >> After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl >> >> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: >> >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> However I also see >> $ locate forks.pm >> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >> >> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Tue Feb 21 11:05:48 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 13:05:48 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: I wasn't sure what modules were separated (figured it was wroth a shot), however, I did the force install of forks and I still get the same error. I tried uninstalling and reinstalling all the forks installations that showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then reloaded) to make sure I wasn't getting some kind of conflicting libraries/modules. and I'm back to: Can't locate forks.pm in @INC (you may need to install the forks module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. along with: $ sudo updatedb $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Now it's a maker issue, not a forks issue (if I'm reading it correctly), so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) and got: $ sudo ./Build install Installing MAKER... Configuring MAKER with MPI support Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 (unchanged) I don't believe that's an error or even a warning, but I get the same "can't locate forks" error when I try to run it. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt wrote: > forks::signals is part of forks. It?s not a separate package. If it?s > missing, there is a problem with the forks installation. So you need to do > a force install of forks to force the reinstall. > > Example ?> cpan[1]> force install forks > > ?Carson > > > On Feb 21, 2017, at 10:17 AM, Seth Munholland wrote: > > I tried this and still get the same error. Then I tried forcing a > reinstall of forks/signals and got: > > cpan[1]> install forks::signals > Warning: Cannot install forks::signals, don't know what it is. > Try the command > > i /forks::signals/ > > to find objects with matching identifiers. > > but scrolling through the install scroll of the forks reinstall I do see > it got installed properly. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: > >> Do a force reinstall of forks via CPAN. The error is coming from forks.pm line >> 83, so the problem is the forks installation itself. >> >> Thanks, >> Carson >> >> >> On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: >> >> Hi Everyone, >> >> After sorting my MPICH/OpenMPI issue I have come across another: When I >> try to run MAKER on the demo data via >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >> /Data/Apps/maker/data/maker_bopts.ctl >> >> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it >> working before opening up, if I change the -n value then the error repeats >> once for each process I attempt to run via MPICH) I got the following: >> >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> However I also see >> $ locate forks.pm >> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >> >> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially >> thought this was a Perl error. I hit up perlmonks and found that there is >> an issue between forks and Storable, where Storable now has a verion number >> of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these >> instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and >> tried again. Now I get: >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >> /Data/Apps/maker/data/maker_bopts.ctl >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> Clearly MAKER has an issue with forks, but it is installed, it is up to >> date (I double checked via cpan and apt-get to be sure), and it is in a >> directory that is in @INC. Should I pursue this as a MAKER error or as a >> Perl error? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bmoore at genetics.utah.edu Tue Feb 21 11:53:13 2017 From: bmoore at genetics.utah.edu (Barry Moore) Date: Tue, 21 Feb 2017 18:53:13 +0000 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: <78C26338-01EF-4F72-BFA4-EF46C70AFAEE@genetics.utah.edu> Hi Qihua, If you are using online version of SOBA, I would suggest you use the command line version found here https://github.com/The-Sequence-Ontology/SOBA as it is more flexible for the kinds of analyses you are talking about. If you are using ?footprint? as the --data_type argument you should get the nucleotide count for collapsed features that you are talking about. In addition I suggest you take a look at bedtools (http://bedtools.readthedocs.io/en/latest/index.html) for example bedtools merge as a flexible way to generate the kind of merged features you want and then you can always pass that output of that through SOBAcl for counting, graphing and reporting. Finally, if you want a great deal of flexibility in generating your own manipulation and reporting of GFF3 files that is beyond the scope of SOBA and/or BedTools, I suggest you take a look at the GAL library (https://github.com/The-Sequence-Ontology/GAL) if you don?t mind writing some perl code. Regards, Barry On Feb 20, 2017, at 1:34 PM, Qihua Liang > wrote: Hi Carson, Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? Thanks Qihua On Feb 19, 2017, at 10:05 PM, Carson Holt > wrote: IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. Thanks, Carson On Feb 18, 2017, at 9:43 AM, Qihua Liang > wrote: Dear Maker develop team, I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? Thanks Qihua _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 21 14:34:07 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 21 Feb 2017 14:34:07 -0700 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: MAKER merges overlapping RepeatMasker results into a single longer feature. ?Carson > On Feb 20, 2017, at 1:34 PM, Qihua Liang wrote: > > Hi Carson, > > Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. > > I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. > > Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. > In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. > > But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. > > When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? > Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? > > Thanks > Qihua > > >> On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: >> >> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. >> >> CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. >> >> If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. >> >> What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. >> >> However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. >> >> Thanks, >> Carson >> >>> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >>> >>> Dear Maker develop team, >>> >>> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >>> >>> Thanks >>> Qihua >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > From munholl at uwindsor.ca Tue Feb 21 15:26:23 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 17:26:23 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: I found my errors. 1) In my machine file; the first entry was not the main node, so locate was looking on a different node than Maker 2) forks (and other modules) were not properly installed on every node in the cluster. I used CPAN to install them on each cluster and it is now working. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 1:05 PM, Seth Munholland wrote: > I wasn't sure what modules were separated (figured it was wroth a shot), > however, I did the force install of forks and I still get the same error. > I tried uninstalling and reinstalling all the forks installations that > showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then > reloaded) to make sure I wasn't getting some kind of conflicting > libraries/modules. and I'm back to: > > Can't locate forks.pm in @INC (you may need to install the forks module) > (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > > along with: > > $ sudo updatedb > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Now it's a maker issue, not a forks issue (if I'm reading it correctly), > so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) > and got: > > $ sudo ./Build install > Installing MAKER... > Configuring MAKER with MPI support > Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 > (unchanged) > > I don't believe that's an error or even a warning, but I get the same > "can't locate forks" error when I try to run it. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt wrote: > >> forks::signals is part of forks. It?s not a separate package. If it?s >> missing, there is a problem with the forks installation. So you need to do >> a force install of forks to force the reinstall. >> >> Example ?> cpan[1]> force install forks >> >> ?Carson >> >> >> On Feb 21, 2017, at 10:17 AM, Seth Munholland >> wrote: >> >> I tried this and still get the same error. Then I tried forcing a >> reinstall of forks/signals and got: >> >> cpan[1]> install forks::signals >> Warning: Cannot install forks::signals, don't know what it is. >> Try the command >> >> i /forks::signals/ >> >> to find objects with matching identifiers. >> >> but scrolling through the install scroll of the forks reinstall I do see >> it got installed properly. >> >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> >> On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: >> >>> Do a force reinstall of forks via CPAN. The error is coming from >>> forks.pm line 83, so the problem is the forks installation itself. >>> >>> Thanks, >>> Carson >>> >>> >>> On Feb 17, 2017, at 1:11 PM, Seth Munholland >>> wrote: >>> >>> Hi Everyone, >>> >>> After sorting my MPICH/OpenMPI issue I have come across another: When I >>> try to run MAKER on the demo data via >>> >>> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >>> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >>> /Data/Apps/maker/data/maker_bopts.ctl >>> >>> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it >>> working before opening up, if I change the -n value then the error repeats >>> once for each process I attempt to run via MPICH) I got the following: >>> >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> >>> However I also see >>> $ locate forks.pm >>> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >>> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >>> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >>> >>> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially >>> thought this was a Perl error. I hit up perlmonks and found that there is >>> an issue between forks and Storable, where Storable now has a verion number >>> of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these >>> instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and >>> tried again. Now I get: >>> >>> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >>> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >>> /Data/Apps/maker/data/maker_bopts.ctl >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> >>> Clearly MAKER has an issue with forks, but it is installed, it is up to >>> date (I double checked via cpan and apt-get to be sure), and it is in a >>> directory that is in @INC. Should I pursue this as a MAKER error or as a >>> Perl error? >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Wed Feb 22 06:11:32 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Wed, 22 Feb 2017 14:11:32 +0100 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> Message-ID: <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> @Robert Kraus: FYI Am 20.02.2017 um 06:43 schrieb Carson Holt: > Try running just on a single node (not across nodes). THATS WHAT I DID. > If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and > running MAKER with that new version. You can install it in your home directory and test from there, > just make sure to add it to your path. Shure it is. > Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. > If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. OK, send the infos please. Thank you very much! > ?Carson > > > >> On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: >> >> Hi! >> >> Unfortunately all of the options failed on our cluster. >> >> See: >> >> Hi, >> >> Most recent Maker test with >> --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >> Error: >> --> rank=2, hostname=uc1n518.localdomain >> [uc1n518:67009] *** Process received signal *** >> [uc1n518:67009] Signal: Segmentation fault (11) >> [uc1n518:67009] Signal code: Address not mapped (1) >> [uc1n518:67009] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). >> >> >> With: >> --mca btl ^openib >> and also this >> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >> Error: >> ### Runing Maker example >> >> STATUS: Parsing control files... >> STATUS: Processing and indexing input FASTA files... >> [uc1n514:59985] *** Process received signal *** >> [uc1n514:59985] Signal: Segmentation fault (11) >> [uc1n514:59985] Signal code: Address not mapped (1) >> [uc1n514:59985] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). >> >> >> Am 28.01.2017 um 21:53 schrieb Carson Holt: >>> Try adding one of the following to your mpiexec command ?> >>> >>> 1. --mca btl ^openib >>> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>> >>> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >>> >>> --Carson >>> >>> >>>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>>> >>>> Hi everybody. >>>> >>>> My name is Rainer. I am an administrator for our HPC-Systems at our >>>> university in Konstanz, Baden-Wuertemberg/Germany. >>>> The procect is called bwHPC-C5. >>>> >>>> See: https://www.bwhpc-c5.de/en/index.php >>>> >>>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>>> i get errors while running a Maker job in the MPI-environment. >>>> >>>> BUILD STATUS >>>> >>>> ============================================================================== >>>> STATUS MAKER v2.31.9 >>>> ============================================================================== >>>> PERL Dependencies: VERIFIED >>>> External Programs: VERIFIED >>>> External C Libraries: VERIFIED >>>> MPI SUPPORT: ENABLED >>>> MWAS Web Interface: DISABLED >>>> MAKER PACKAGE: CONFIGURATION OK >>>> >>>> MODULES / INCLUDES / COMPILERS >>>> >>>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>>> # >>>> ##### (B) Dependencies: >>>> # >>>> # conflict: any other maker version >>>> # module load compiler/gnu/5.2 >>>> # module load mpi/openmpi/2.0-gnu-5.2 >>>> [...] >>>> >>>> MPI/MOAB SUBMIT >>>> >>>> [...] >>>> ### Queues ### >>>> #MSUB -q fat >>>> #MSUB -l nodes=1:ppn=16 >>>> #MSUB -l mem=20gb >>>> #MSUB -l walltime=50:00:00 >>>> # >>>> [...] >>>> echo " " >>>> echo "### Loading MAKER module:" >>>> echo " " >>>> module load bio/maker/2.31.9 >>>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>>> echo "MAKER_VERSION = $MAKER_VERSION" >>>> module list >>>> [...] >>>> echo " " >>>> echo "### Runing Maker example" >>>> echo " " >>>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>>> export OMPI_MCA_mpi_warn_on_fork=0 >>>> >>>> echo "LD_PRELOAD=${LD_PRELOAD}" >>>> # >>>> # "STATUS: Processing and indexing input FASTA files..." >>>> # >>>> mpiexec -mca btl ^openib -n 16 maker >>>> [...] >>>> >>>> >>>> E R R O R S >>>> ======= >>>> [...] >>>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>>> STATUS: Parsing control files... >>>> STATUS: Processing and indexing input FASTA files... >>>> [uc1n338:113607] *** Process received signal *** >>>> [uc1n338:113607] Signal: Segmentation fault (11) >>>> [uc1n338:113607] Signal code: Address not mapped (1) >>>> [uc1n338:113607] Failing at address: 0x4b0 >>>> [uc1n338:113608] *** Process received signal *** >>>> [uc1n338:113608] Signal: Segmentation fault (11) >>>> [uc1n338:113608] Signal code: Address not mapped (1) >>>> [uc1n338:113608] Failing at address: 0x4b0 >>>> [uc1n338:113621] *** Process received signal *** >>>> [uc1n338:113621] Signal: Segmentation fault (11) >>>> [uc1n338:113621] Signal code: Address not mapped (1) >>>> [uc1n338:113621] Failing at address: 0x4b0 >>>> -------------------------------------------------------------------------- >>>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>>> -------------------------------------------------------------------------- >>>> [...] >>>> >>>> WHATS WRONG HERE!? >>>> >>>> Thank you for your help! >>>> >>>> All the best , >>>> >>>> Rainer >>>> >>>> -- >>>> Rainer Rutka >>>> University of Konstanz >>>> Communication, Information, Media Centre (KIM) >>>> * High-Performance-Computing (HPC) >>>> * KIM-Support and -Base-Services >>>> Room: V511 >>>> 78457 Konstanz, Germany >>>> +49 7531 88-5413 >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> -- >> Rainer Rutka >> Universit?t Konstanz >> Kommunikations-, Informations-, Medienzentrum (KIM) >> Raum: V511, Tel: 54 13 >> > -- Rainer Rutka Universit?t Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Raum: V511, Tel: 54 13 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From cjfields at illinois.edu Wed Feb 22 06:30:22 2017 From: cjfields at illinois.edu (Fields, Christopher J) Date: Wed, 22 Feb 2017 13:30:22 +0000 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: <6E5122C0-A952-42D6-B205-74B5FF8255F2@illinois.edu> Just a note: when we install MAKER we generally install a clean version of perl if one isn?t already present, making sure it is on the NFS share for the cluster (which is accessible via all nodes). This works around most of the issues you describe. chris From: maker-devel > on behalf of Seth Munholland > Date: Tuesday, February 21, 2017 at 4:26 PM To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] MAKER can't find forks in @INC, but it is there I found my errors. 1) In my machine file; the first entry was not the main node, so locate was looking on a different node than Maker 2) forks (and other modules) were not properly installed on every node in the cluster. I used CPAN to install them on each cluster and it is now working. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 1:05 PM, Seth Munholland > wrote: I wasn't sure what modules were separated (figured it was wroth a shot), however, I did the force install of forks and I still get the same error. I tried uninstalling and reinstalling all the forks installations that showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then reloaded) to make sure I wasn't getting some kind of conflicting libraries/modules. and I'm back to: Can't locate forks.pm in @INC (you may need to install the forks module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. along with: $ sudo updatedb $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Now it's a maker issue, not a forks issue (if I'm reading it correctly), so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) and got: $ sudo ./Build install Installing MAKER... Configuring MAKER with MPI support Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 (unchanged) I don't believe that's an error or even a warning, but I get the same "can't locate forks" error when I try to run it. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt > wrote: forks::signals is part of forks. It?s not a separate package. If it?s missing, there is a problem with the forks installation. So you need to do a force install of forks to force the reinstall. Example ?> cpan[1]> force install forks ?Carson On Feb 21, 2017, at 10:17 AM, Seth Munholland > wrote: I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: cpan[1]> install forks::signals Warning: Cannot install forks::signals, don't know what it is. Try the command i /forks::signals/ to find objects with matching identifiers. but scrolling through the install scroll of the forks reinstall I do see it got installed properly. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt > wrote: Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. Thanks, Carson On Feb 17, 2017, at 1:11 PM, Seth Munholland > wrote: Hi Everyone, After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. However I also see $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 09:16:17 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 09:16:17 -0700 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> Message-ID: <24606FBA-4742-4F94-9447-208A385644C1@gmail.com> If OpenMPI fails on a single node, it means you have a compilation issue, which indicates a problem with your installation. This sometimes happens if you compiled on one node and run on another (if could either be MAEKR or OpenMPI itself that was compiled on another node). A few options you will need if trying intel MPI: -binding pin=disable #requires to disable processor affinity (otherwise MAKER calls to BLAST and other programs which are parallelized independent of MPI may not work) Environmental variables to set: export I_MPI_PIN_DOMAIN=node #otherwise MAKER calls to BLAST and other programs which are parallelized independent of MPI may not work export I_MPI_FABRICS='shm:tcp? #avoid potential complication with OpenFabrics libraries (they block system calls because of how they use registered memory, i.e. MAKER calling BLAST would fail) export I_MPI_HYDRA_IFACE=ib0 #set to eth0 if you don?t have an infiniband over ip inerface (required because of the above I_MPI_FABRICS change) Also make sure to compile on the node you run. You can try expanding to other nodes after that. ?Carson > On Feb 22, 2017, at 6:11 AM, Rainer Rutka wrote: > > @Robert Kraus: FYI > > Am 20.02.2017 um 06:43 schrieb Carson Holt: >> Try running just on a single node (not across nodes). > THATS WHAT I DID. > > >> If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and >> running MAKER with that new version. You can install it in your home directory and test from there, >> just make sure to add it to your path. > Shure it is. > >> Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. > > If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. > > OK, send the infos please. > > Thank you very much! > >> ?Carson >> >> >> >>> On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: >>> >>> Hi! >>> >>> Unfortunately all of the options failed on our cluster. >>> >>> See: >>> >>> Hi, >>> >>> Most recent Maker test with >>> --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>> Error: >>> --> rank=2, hostname=uc1n518.localdomain >>> [uc1n518:67009] *** Process received signal *** >>> [uc1n518:67009] Signal: Segmentation fault (11) >>> [uc1n518:67009] Signal code: Address not mapped (1) >>> [uc1n518:67009] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). >>> >>> >>> With: >>> --mca btl ^openib >>> and also this >>> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>> Error: >>> ### Runing Maker example >>> >>> STATUS: Parsing control files... >>> STATUS: Processing and indexing input FASTA files... >>> [uc1n514:59985] *** Process received signal *** >>> [uc1n514:59985] Signal: Segmentation fault (11) >>> [uc1n514:59985] Signal code: Address not mapped (1) >>> [uc1n514:59985] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). >>> >>> >>> Am 28.01.2017 um 21:53 schrieb Carson Holt: >>>> Try adding one of the following to your mpiexec command ?> >>>> >>>> 1. --mca btl ^openib >>>> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>>> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>>> >>>> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >>>> >>>> --Carson >>>> >>>> >>>>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>>>> >>>>> Hi everybody. >>>>> >>>>> My name is Rainer. I am an administrator for our HPC-Systems at our >>>>> university in Konstanz, Baden-Wuertemberg/Germany. >>>>> The procect is called bwHPC-C5. >>>>> >>>>> See: https://www.bwhpc-c5.de/en/index.php >>>>> >>>>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>>>> i get errors while running a Maker job in the MPI-environment. >>>>> >>>>> BUILD STATUS >>>>> >>>>> ============================================================================== >>>>> STATUS MAKER v2.31.9 >>>>> ============================================================================== >>>>> PERL Dependencies: VERIFIED >>>>> External Programs: VERIFIED >>>>> External C Libraries: VERIFIED >>>>> MPI SUPPORT: ENABLED >>>>> MWAS Web Interface: DISABLED >>>>> MAKER PACKAGE: CONFIGURATION OK >>>>> >>>>> MODULES / INCLUDES / COMPILERS >>>>> >>>>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>>>> # >>>>> ##### (B) Dependencies: >>>>> # >>>>> # conflict: any other maker version >>>>> # module load compiler/gnu/5.2 >>>>> # module load mpi/openmpi/2.0-gnu-5.2 >>>>> [...] >>>>> >>>>> MPI/MOAB SUBMIT >>>>> >>>>> [...] >>>>> ### Queues ### >>>>> #MSUB -q fat >>>>> #MSUB -l nodes=1:ppn=16 >>>>> #MSUB -l mem=20gb >>>>> #MSUB -l walltime=50:00:00 >>>>> # >>>>> [...] >>>>> echo " " >>>>> echo "### Loading MAKER module:" >>>>> echo " " >>>>> module load bio/maker/2.31.9 >>>>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>>>> echo "MAKER_VERSION = $MAKER_VERSION" >>>>> module list >>>>> [...] >>>>> echo " " >>>>> echo "### Runing Maker example" >>>>> echo " " >>>>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>>>> export OMPI_MCA_mpi_warn_on_fork=0 >>>>> >>>>> echo "LD_PRELOAD=${LD_PRELOAD}" >>>>> # >>>>> # "STATUS: Processing and indexing input FASTA files..." >>>>> # >>>>> mpiexec -mca btl ^openib -n 16 maker >>>>> [...] >>>>> >>>>> >>>>> E R R O R S >>>>> ======= >>>>> [...] >>>>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>>>> STATUS: Parsing control files... >>>>> STATUS: Processing and indexing input FASTA files... >>>>> [uc1n338:113607] *** Process received signal *** >>>>> [uc1n338:113607] Signal: Segmentation fault (11) >>>>> [uc1n338:113607] Signal code: Address not mapped (1) >>>>> [uc1n338:113607] Failing at address: 0x4b0 >>>>> [uc1n338:113608] *** Process received signal *** >>>>> [uc1n338:113608] Signal: Segmentation fault (11) >>>>> [uc1n338:113608] Signal code: Address not mapped (1) >>>>> [uc1n338:113608] Failing at address: 0x4b0 >>>>> [uc1n338:113621] *** Process received signal *** >>>>> [uc1n338:113621] Signal: Segmentation fault (11) >>>>> [uc1n338:113621] Signal code: Address not mapped (1) >>>>> [uc1n338:113621] Failing at address: 0x4b0 >>>>> -------------------------------------------------------------------------- >>>>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>>>> -------------------------------------------------------------------------- >>>>> [...] >>>>> >>>>> WHATS WRONG HERE!? >>>>> >>>>> Thank you for your help! >>>>> >>>>> All the best , >>>>> >>>>> Rainer >>>>> >>>>> -- >>>>> Rainer Rutka >>>>> University of Konstanz >>>>> Communication, Information, Media Centre (KIM) >>>>> * High-Performance-Computing (HPC) >>>>> * KIM-Support and -Base-Services >>>>> Room: V511 >>>>> 78457 Konstanz, Germany >>>>> +49 7531 88-5413 >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> >>> >>> -- >>> Rainer Rutka >>> Universit?t Konstanz >>> Kommunikations-, Informations-, Medienzentrum (KIM) >>> Raum: V511, Tel: 54 13 >>> >> > > -- > Rainer Rutka > Universit?t Konstanz > Kommunikations-, Informations-, Medienzentrum (KIM) > Raum: V511, Tel: 54 13 > From munholl at uwindsor.ca Wed Feb 22 11:03:54 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 13:03:54 -0500 Subject: [maker-devel] Failed while polishing ESTs Message-ID: After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: polishig ESTs running est2genome search. #--------- command -------------# Widget::exonerate::est2genome: /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate #-------------------------------# Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! --> rank=NA, hostname=beanblade2 ERROR: Failed while polishig ESTs ERROR: Chunk failed at level:2, tier_type:3 FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:contig-dpp-500-500 The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: polishig ESTs running est2genome search. #--------- command -------------# Widget::exonerate::est2genome: /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate #-------------------------------# Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! --> rank=NA, hostname=beanblade4 ERROR: Failed while polishig ESTs ERROR: Chunk failed at level:2, tier_type:3 FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:contig-dpp-500-500 So the error seems to be pointing at something happening when wrapping up the ESTs. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 11:16:41 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 11:16:41 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: Message-ID: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. And make sure to delete any run directories before retrying. ?Carson > On Feb 22, 2017, at 11:03 AM, Seth Munholland wrote: > > After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade2 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade4 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > So the error seems to be pointing at something happening when wrapping up the ESTs. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 11:57:00 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 13:57:00 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: It is the test data, the dpp set to be specific. I already checked all the installs to be sure they configured and compile without error and are up to date. I've been deleting between each run. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: > So you are running the test data job correct? So if you get any error with > the test data job, something is wrong with your installation. > > Make sure BioPerl is up to data (and the same everywhere). If using your > own installation of Perl, make sure it had BerkleyDB support when you > installed it (if you are using something like perlbrew , you may not have > BerkleyDB support and this may affect BioPerl indexing). Also try > reinstalling exonertate. > > And make sure to delete any run directories before retrying. > > ?Carson > > > On Feb 22, 2017, at 11:03 AM, Seth Munholland wrote: > > After my clustering issue I figured I would go node by node and run MAKER > locally on the test data set to be sure that it was working properly before > trying it as a full MPI run. After adjusting all my exes to point to the > NFS versions I am consistently getting the same error on each node when it > comes time to run exonerate: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q > /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna > --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent > 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp- > mRNA-5.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade2 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > The only results I find on google was an issue with an improper character > in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and > since they all point to dpp-mRNA-5 I backed up the provided est and removed > it. The response was: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q > /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna > --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent > 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp- > mRNA-4.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade4 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > So the error seems to be pointing at something happening when wrapping up > the ESTs. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 12:00:24 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 14:00:24 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland wrote: > It is the test data, the dpp set to be specific. > > I already checked all the installs to be sure they configured and compile > without error and are up to date. > > I've been deleting between each run. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: > >> So you are running the test data job correct? So if you get any error >> with the test data job, something is wrong with your installation. >> >> Make sure BioPerl is up to data (and the same everywhere). If using your >> own installation of Perl, make sure it had BerkleyDB support when you >> installed it (if you are using something like perlbrew , you may not have >> BerkleyDB support and this may affect BioPerl indexing). Also try >> reinstalling exonertate. >> >> And make sure to delete any run directories before retrying. >> >> ?Carson >> >> >> On Feb 22, 2017, at 11:03 AM, Seth Munholland >> wrote: >> >> After my clustering issue I figured I would go node by node and run MAKER >> locally on the test data set to be sure that it was working properly before >> trying it as a full MPI run. After adjusting all my exes to point to the >> NFS versions I am consistently getting the same error on each node when it >> comes time to run exonerate: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q >> /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t >> /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna >> --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent >> 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA- >> 5.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade2 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> The only results I find on google was an issue with an improper character >> in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and >> since they all point to dpp-mRNA-5 I backed up the provided est and removed >> it. The response was: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q >> /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t >> /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna >> --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent >> 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA- >> 4.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade4 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> So the error seems to be pointing at something happening when wrapping up >> the ESTs. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 12:03:42 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 12:03:42 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: I still think you may have something wrong somewhere with some part of your installation (rogue libraries, compiler incompatibilities, etc.). Especially given your earlier issues with Perl libraries. ?Carson > On Feb 22, 2017, at 12:00 PM, Seth Munholland wrote: > > On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. > > Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: > It is the test data, the dpp set to be specific. > > I already checked all the installs to be sure they configured and compile without error and are up to date. > > I've been deleting between each run. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt > wrote: > So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. > > Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. > > And make sure to delete any run directories before retrying. > > ?Carson > > >> On Feb 22, 2017, at 11:03 AM, Seth Munholland > wrote: >> >> After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade2 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade4 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> So the error seems to be pointing at something happening when wrapping up the ESTs. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 12:26:57 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 14:26:57 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: I'll wipe it clean and do a fresh install of everything to be 100% safe. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 2:03 PM, Carson Holt wrote: > I still think you may have something wrong somewhere with some part of > your installation (rogue libraries, compiler incompatibilities, etc.). > Especially given your earlier issues with Perl libraries. > > ?Carson > > > On Feb 22, 2017, at 12:00 PM, Seth Munholland wrote: > > On a whim I decided to switch to the hsap demo data and it completed > without issue. I went back to dpp and this time is completed. I hadn't > changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and > then inverted the change. > > Mayhaps there was something not unloaded from memory in an earlier run? > It's that, ghosts, or rogue ions :/ > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: > >> It is the test data, the dpp set to be specific. >> >> I already checked all the installs to be sure they configured and compile >> without error and are up to date. >> >> I've been deleting between each run. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> >> On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: >> >>> So you are running the test data job correct? So if you get any error >>> with the test data job, something is wrong with your installation. >>> >>> Make sure BioPerl is up to data (and the same everywhere). If using your >>> own installation of Perl, make sure it had BerkleyDB support when you >>> installed it (if you are using something like perlbrew , you may not have >>> BerkleyDB support and this may affect BioPerl indexing). Also try >>> reinstalling exonertate. >>> >>> And make sure to delete any run directories before retrying. >>> >>> ?Carson >>> >>> >>> On Feb 22, 2017, at 11:03 AM, Seth Munholland >>> wrote: >>> >>> After my clustering issue I figured I would go node by node and run >>> MAKER locally on the test data set to be sure that it was working properly >>> before trying it as a full MPI run. After adjusting all my exes to point >>> to the NFS versions I am consistently getting the same error on each node >>> when it comes time to run exonerate: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q >>> /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t >>> /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T >>> dna --model est2genome --minintron 20 --maxintron 10000 --showcigar >>> --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp >>> -500-500.26386-32056.dpp-mRNA-5.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade2 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> The only results I find on google was an issue with an improper >>> character in the est file, but a cat of the dpp_est.fasta shows nothing >>> incorrect and since they all point to dpp-mRNA-5 I backed up the provided >>> est and removed it. The response was: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q >>> /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t >>> /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T >>> dna --model est2genome --minintron 20 --maxintron 10000 --showcigar >>> --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp >>> -500-500.22889-32056.dpp-mRNA-4.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade4 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> So the error seems to be pointing at something happening when wrapping >>> up the ESTs. >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 12:29:12 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 12:29:12 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: <53F1286C-945C-4F91-B851-5833BE1D1F18@gmail.com> In the most extreme case, you may even need to go as far as setting up your own perl. If you do that, make sure to unset the PERL5LIB environmental variable, so other perl libraries don?t interfere. ?Carson > On Feb 22, 2017, at 12:26 PM, Seth Munholland wrote: > > I'll wipe it clean and do a fresh install of everything to be 100% safe. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 2:03 PM, Carson Holt > wrote: > I still think you may have something wrong somewhere with some part of your installation (rogue libraries, compiler incompatibilities, etc.). Especially given your earlier issues with Perl libraries. > > ?Carson > > >> On Feb 22, 2017, at 12:00 PM, Seth Munholland > wrote: >> >> On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. >> >> Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <> >> On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: >> It is the test data, the dpp set to be specific. >> >> I already checked all the installs to be sure they configured and compile without error and are up to date. >> >> I've been deleting between each run. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <> >> On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt > wrote: >> So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. >> >> Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. >> >> And make sure to delete any run directories before retrying. >> >> ?Carson >> >> >>> On Feb 22, 2017, at 11:03 AM, Seth Munholland > wrote: >>> >>> After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade2 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade4 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> So the error seems to be pointing at something happening when wrapping up the ESTs. >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lucile.soler at bils.se Thu Feb 23 05:40:08 2017 From: lucile.soler at bils.se (lucile.soler at bils.se) Date: Thu, 23 Feb 2017 13:40:08 +0100 Subject: [maker-devel] genes longer than contig in maker Message-ID: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> Hello, I am using maker3 to annotate a lizard. The maker part went well but then when I want to do the functional annotation, it seems that for some contigs maker has created genes outside of the contig size. For instance : XXX1 maker gene 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8 XXX1 maker mRNA 2 26717 . - . ID=maker- XXX1_pilon-augustus-gene-0.8-mRNA-1;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.8|0.5|1|0.4|0.33|6|5291|105 XXX1 maker mRNA 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;_AED=0.08;_eAED=0.08;_QI=0|0.5|0.4|1|0.25|0.4|5|4442|170 and the length of the contig is 26696 Any idea of what could have happened? Have you seen that already? Let me know what more information you need to answer. Thank you very much for your help, Lucile Soler PhD in Bioinformatics Genome Annotation Platform NBIS (National Bioinformatics Infrastructure Sweden) mail:lucile.soler at bils.se -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Thu Feb 23 06:10:55 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Thu, 23 Feb 2017 14:10:55 +0100 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> Message-ID: <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> HI! Running maker with mpiexec causes this error-message: mpiexec_uc1n077.localdomain: cannot connect to local mpd (/scratch/mpd2.console_uc1n077.localdomain_kn_pop235844); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option) ### Cleaning up files ... removing unnecessary scratch files ... And yes, we don't have mpd running. Environment used is: Currently Loaded Modulefiles: 1) compiler/intel/16.0(default) 2) mpi/impi/5.1.3-intel-16.0(default) 3) bio/maker/2.31.8_impi Running maker with mpiexec using only 1 node and 8 cores. mpiexec -n 8 maker :-( Any suggestions ? -- Rainer Rutka University of Konstanz Communication, Information, Media Centre (KIM) * High-Performance-Computing (HPC) * KIM-Support and -Base-Services Room: V511 78457 Konstanz, Germany +49 7531 88-5413 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From carsonhh at gmail.com Thu Feb 23 13:22:22 2017 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 23 Feb 2017 13:22:22 -0700 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> Message-ID: It means one of two things. 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. ?Carson > On Feb 23, 2017, at 6:10 AM, Rainer Rutka wrote: > > > HI! > > Running maker with mpiexec causes this error-message: > > mpiexec_uc1n077.localdomain: cannot connect to local mpd (/scratch/mpd2.console_uc1n077.localdomain_kn_pop235844); possible causes: > 1. no mpd is running on this host > 2. an mpd is running but was started without a "console" (-n option) > ### Cleaning up files ... removing unnecessary scratch files ... > > And yes, we don't have mpd running. > > Environment used is: > > Currently Loaded Modulefiles: > 1) compiler/intel/16.0(default) > 2) mpi/impi/5.1.3-intel-16.0(default) > 3) bio/maker/2.31.8_impi > > Running maker with mpiexec using only 1 node and 8 cores. > > mpiexec -n 8 maker > > :-( > > Any suggestions ? > > -- > Rainer Rutka > University of Konstanz > Communication, Information, Media Centre (KIM) > * High-Performance-Computing (HPC) > * KIM-Support and -Base-Services > Room: V511 > 78457 Konstanz, Germany > +49 7531 88-5413 > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Thu Feb 23 16:40:24 2017 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 23 Feb 2017 16:40:24 -0700 Subject: [maker-devel] genes longer than contig in maker In-Reply-To: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> References: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> Message-ID: 1. You have two contigs with the same ID in your assembly (one longer and one shorter). 2. Your BioPerl index for retrieving the sequence is corrupt somehow. Requires delete of previous output directory before restarting. 3. You are using bad formatted GFF3 as input into maker, and it is somehow not failing right away. The fact you get output means that it was able to translate the sequence into protein with CDS etc. So it was not too short for that. Look at the contig line in the GFF3 to get the length of the contig maker is seeing to compare to the feature positions given. ?Carson > On Feb 23, 2017, at 5:40 AM, lucile.soler at bils.se wrote: > > Hello, > > I am using maker3 to annotate a lizard. > > > The maker part went well but then when I want to do the functional annotation, it seems that for some contigs maker has created genes outside of the contig size. > > For instance : > > > XXX1 maker gene 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8 > > > XXX1 maker mRNA 2 26717 . - . ID=maker- XXX1_pilon-augustus-gene-0.8-mRNA-1;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.8|0.5|1|0.4|0.33|6|5291|105 > > XXX1 maker mRNA 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;_AED=0.08;_eAED=0.08;_QI=0|0.5|0.4|1|0.25|0.4|5|4442|170 > > and the length of the contig is 26696 > > > > > Any idea of what could have happened? Have you seen that already? > > > > Let me know what more information you need to answer. > > Thank you very much for your help, > > > > > > Lucile Soler > > PhD in Bioinformatics > Genome Annotation Platform > NBIS (National Bioinformatics Infrastructure Sweden) > mail:lucile.soler at bils.se > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Fri Feb 24 01:43:21 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Fri, 24 Feb 2017 09:43:21 +0100 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> Message-ID: <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> Hi Carson. First of all THANK YOU FOR YOUR HELP. MUCH APPRECIATED. :-) Am 23.02.2017 um 21:22 schrieb Carson Holt: > It means one of two things. > 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). No, I am starting the right version of mpiexec. This is a list of all our current available MPI-Versions, corresponding to their compiler: UC:[kn at uc1n997 ~]$ module avail mpi ------------------------------------- /opt/bwhpc/common/modulefiles -------------------------------------- mpi/impi/4.1.3-gnu-4.4 mpi/openmpi/1.10-intel-15.0 mpi/impi/4.1.3-gnu-4.7 mpi/openmpi/1.10-intel-16.0 mpi/impi/4.1.3-intel-14.0 mpi/openmpi/1.8-gnu-4.5 mpi/impi/5.0.3-gnu-4.4 mpi/openmpi/1.8-gnu-4.7 mpi/impi/5.0.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.7-m32 mpi/impi/5.0.3-intel-15.0 mpi/openmpi/1.8-gnu-4.8 mpi/impi/5.1.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.9 mpi/impi/5.1.3-gnu-system mpi/openmpi/1.8-intel-14.0(default) mpi/impi/5.1.3-intel-16.0(default) mpi/openmpi/1.8-intel-15.0 mpi/openmpi/1.10-gnu-4.5 mpi/openmpi/2.0-gnu-4.7 mpi/openmpi/1.10-gnu-4.7 mpi/openmpi/2.0-gnu-4.8 mpi/openmpi/1.10-gnu-4.8 mpi/openmpi/2.0-gnu-5.2 mpi/openmpi/1.10-gnu-4.9 mpi/openmpi/2.0-intel-15.0 mpi/openmpi/1.10-gnu-5.2 mpi/openmpi/2.0-intel-16.0 mpi/openmpi/1.10-intel-14.0 Here I load the MPI-Module including all it's dependencies: UC:[kn at uc1n997 ~]$ module load mpi/impi/5.1.3-intel-16.0 Loading module dependency 'compiler/intel/16.0'. Result: UC:[kn at c1n997 ~]$ which mpiexec /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec UC:[kn at uc1n997 ~]$ > 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) Extremely old? [...] Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160601 (build id: 15562) > MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. Yes. And we don't even use MPD at our clusters :-) > So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. I do not have any MPICH available on our cluster(s) now. All of the old versions had been removed some years ago. At a glance --------------- Maker is available as a so-called module on our cluster system. It's been build on a developers' node I can access. But, the MPI-modules are built by other fellows (e.g. at the KIT in Karlsruhe/Germany) on other nodes. Please check the module-file (included in this mail) bio-maker-2.31.8_impi to see how Maker was build including the envoronments set by this module. e.g.: ./Build status verify dependencies: =================================================== STATUS MAKER v2.31.8 =================================================== PERL Dependencies: VERIFIED External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: ENABLED MWAS Web Interface: DISABLED MAKER PACKAGE: CONFIGURATION OK At least, please(!) have a look at our m.moab file (included in this mail). This is the way how we submit a Maker job to our cluster. Maybe something is wrong here? Sorry again for wasting your time. But we imperatively need the Maker software running in parallel mode. :-) -- Rainer Rutka University of Konstanz Communication, Information, Media Centre (KIM) * High-Performance-Computing (HPC) * KIM-Support and -Base-Services Room: V511 78457 Konstanz, Germany +49 7531 88-5413 -------------- next part -------------- #%Module1.0 # # Cluster: bwunicluster # Module: bio/maker/2.31.8 # Revision: knbw01 # TargetSystem: Red-Hat-Enterprise # MainLocation: /opt/bwhpc/common # Status: optional # License: GPL, Artistic License # URL: http://www.yandell-lab.org/software/maker.html # 2ndLevelSupport: bwhpc [at] uni-konstanz.de # ##### (A) Revision history: # # knbw01 20160630 r.rutka Initial revision knbw01 of module version 2.31.8 # knbw02 20161121 r.rutka Initial revision knbw02 of module version 2.31.8 # knbw03 20160222 r.rutka Initial revision knbw02 of module version 2.31.8 with impi # ##### (B) Dependencies: # # conflict: any other maker version # module load compiler/intel/16.0 # module load mpi/impi/5.1.3-intel-16.0 # # # Main environments for build # # VERSION="2.31.8_impi" && echo "Version: ${VERSION}" # MAIN_DIR="/opt/bwhpc/common" && echo ${MAIN_DIR} # SOURCE_DIR=/home/kn/kn_kn/kn_pop235844/src/bio/maker/2.31.8 # TARGET_DIR="${MAIN_DIR}/bio/maker/${VERSION}" && echo ${TARGET_DIR} # ##### (C) How to obtain software? # # You have to register at first before you can download the SW. # # http://yandell.topaz.genetics.utah.edu/cgi-bin/maker_license.cgi # # # Download and unzip the binary distributions # cd ${SOURCE_DIR} && pwd # [ ! -f maker-${VERSION/_impi}.tgz ] && wget http://yandell.topaz.genetics.utah.edu/maker_downloads/CE63/461D/BD41/4510A183DC4571D5EA619E459572/maker-2.31.8.tgz # [[ -d ${TARGET_DIR} ]] && mv ${TARGET_DIR} ${TARGET_DIR}_$(date +%Y.%m.%d:%H.%M) # ##### (D) How to build and install software? # # mkdir -vp ${TARGET_DIR/$VERSION} # cd ${TARGET_DIR/$VERSION} && pwd # tar -xvzf ${SOURCE_DIR}/maker-${VERSION/_impi}.tgz # mv maker ${VERSION} # cd ${VERSION} && pwd # export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so # cd src # # Build required ad-ons. Lot's of required packages will be installed. # # perl Build.PL # | Would you like to configure MAKER for MPI.. ? [N ] Y # | # accept the next default paths and press ENTER # # ignore the error-messages - just continue with the next step # # ./Build installdeps # installs missing PERL dependencies # | ... MAKER .. local installataion: Y # | # Accept the defaults of _all_ further questions by pressing ENTER. # | # ** This is an annoying and long procedure :-( ** # # ./Build installexes # installs missing external programs # | # If are a registered user of RepBase, then MAKER can # | # download and install RepBase for RepeatMasker for you. # | # Register at: http://www.girinst.org/ # | ... download and install RepBase ... : N # We install RepBase later! # # If you type Y, this will not work # # See Install RepBase later in this file # # | # ERROR: Exonerate can't be found. URL Error 404. :-( # | # Locate exonerate entry in locations file and modify path to (line 33): # | # http://ftp.ebi.ac.uk/pub/software/vertebrategenomics/exonerate/exonerate-2.2.0-x86_64.tar.gz # | # restart ./Build installexes # ./Build installexes # # # Install RepBase to prevent "RepBase is not insalled for RepeatMasker" error-message # # http://www.girinst.org/server/RepBase/index.php # p=$(pwd) # cd ${TARGET_DIR}/exe/RepeatMasker && pwd # # http://www.girinst.org/server/RepBase/index.php # # FASTA-Format # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/RepBase22.01.fasta.tar.gz # tar xvzf RepBase22.01.fasta.tar.gz # # # # NEU! # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/repeatmaskerlibraries/RepBaseRepeatMaskerEdition-20170127.tar.gz # # # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/repeatmaskerlibraries/repeatmaskerlibraries-20160829.tar.gz # tar xvzf RepBaseRepeatMaskerEdition-20170127.tar.gz # # # tar xvzf repeatmaskerlibraries-20160829.tar.gz # # perl ./configure # | Enter path perl ... : ENTER # | Enter path ... RepeatMasker: ENTER # | Enter path TRF programm: /opt/bwhpc/common/bio/maker/2.31.8_impi/exe/RepeatMasker/trf # | Add a search engine: 2 RMBlast # | Enter path: /opt/bwhpc/common/bio/maker/2.31.8_impi/exe/RepeatMasker/rmblast/bin # | search engine ... default: Y or ENTER # | Enter Selection: 5 (Done) # # # rm *.tar.gz # cd $p && pwd # # # Now check if anything's fine until now... # ./Build status # | # veryfy dependencies: # | # =================================================== # | # STATUS MAKER v2.31.8 # | # =================================================== # | # PERL Dependencies: VERIFIED # | # External Programs: VERIFIED # | # External C Libraries: VERIFIED # | # MPI SUPPORT: ENABLED # | # MWAS Web Interface: DISABLED # | # MAKER PACKAGE: CONFIGURATION OK # # ./Build install # # # Test # cd ${TARGET_DIR} # bin/maker -version # 2.31.8 # # # Create module file (this one): # # ${SOURCE_DIR}/modulefiles/bio-maker-2.31.8 # # # Create Moab example files (and read the instructions inside of it): # # ${SOURCE_DIR}/bwhpc-examples/bwunicluster-maker-example.moab # # # Copy Moab/bwHPC, special-scripts, examples and more... # mkdir -vp ${TARGET_DIR}/bwhpc-examples # cp ${SOURCE_DIR}/bwhpc-examples/* ${TARGET_DIR}/bwhpc-examples/. # # # Copy module file (this one): # mkdir -vp ${TARGET_DIR}/modulefiles # cp -v ${SOURCE_DIR}/modulefiles/bio-maker-${VERSION} ${TARGET_DIR}/modulefiles/ # chmod -c 644 ${TARGET_DIR}/modulefiles/* # # # Create module link: # mkdir -vp ${MAIN_DIR}/modulefiles/bio/maker # ln -fs ${TARGET_DIR}/modulefiles/bio-maker-${VERSION} ${MAIN_DIR}/modulefiles/bio/maker/${VERSION} # module avail bio/maker # # # Cleanup build and installation target dir: # # chgrp -R -h -c uc1-adm-sw ${TARGET_DIR} # chmod 777 ${TARGET_DIR}/bwhpc-examples # chmod 666 ${TARGET_DIR}/bwhpc-examples/* # rm -rf ${TARGET_DIR}/src ##### END COMMENTS ### Define procedure "set_envVAR" to work around a module-unload-load bug (optional): # Exported environment variables, which do not change their values, # disappear after an automatic unload and load sequence within a module file. # This is most likely a bug in the modules environment. Unchanged # variables are not re-exported upon load in an automatic unload-load # sequence. The workaround is to "unset" the variable in the right moment # (so the load of the unload-load-chain can set the variable again) # and, in addition, to set the environment variable explicitly. proc set_envVAR {envVAR VARcontent} { # Use global function env: global env # Unset envVAR explicitly: if { [info exists env($envVAR)] } { catch { unset env($envVAR) } } # Call overloaded module command setenv: setenv $envVAR $VARcontent # Set envVAR explicitly: set env([set envVAR]) $VARcontent } ### Define fallback values and source global functions and variables from modulerc file: set first_level_support_email "bwhpc (at) uni-konstanz.de" set global_modulerc_file "/opt/bwhpc/common/etc/modules.conf" if { [ file readable "${global_modulerc_file}" ] } { source "${global_modulerc_file}" } # Override global first_level_support_email email: set first_level_support_email "bwhpc (at) uni-konstanz.de" ### Define version, base_dir and whatis entry: set version "2.31.8_impi" set base_dir "/opt/bwhpc/common/bio/maker/$version" module-whatis "maker $version is a portable and configurable genome annotation pipeline" ### Define convenience environment variables: set maker_version "${version}" set maker_home "${base_dir}" set maker_exa_dir "${base_dir}/bwhpc-examples" set maker_bin_dir "${base_dir}/bin" set maker_bpr_url "http://www.bwhpc-c5.de/wiki/index.php/Maker" set maker_blast_bin "${base_dir}/exe/blast/bin" set maker_exonerate_bin "${base_dir}/exe/exonerate/bin" set maker_snap_bin "${base_dir}/exe/snap" set maker_repeatmasker_bin "${base_dir}/exe/RepeatMasker" set_envVAR MAKER_VERSION "${maker_version}" set_envVAR MAKER_HOME "${maker_home}" set_envVAR MAKER_EXA_DIR "${maker_exa_dir}" set_envVAR MAKER_BIN_DIR "${maker_bin_dir}" set_envVAR MAKER_BPR_URL "http://www.bwhpc-c5.de/wiki/index.php/Maker" set_envVAR MAKER_BLAST_BIN "${maker_blast_bin}" set_envVAR MAKER_EXONERATE_BIN "${maker_exonerate_bin}" set_envVAR MAKER_SNAP_BIN "${maker_snap_bin}" set_envVAR MAKER_REPEATMASKER_BIN "${maker_repeatmasker_bin}" ### Update path environments and define application specific environment vars: prepend-path PATH "${maker_bin_dir}" prepend-path PATH "${maker_blast_bin}" prepend-path PATH "${maker_exonerate_bin}" prepend-path PATH "${maker_snap_bin}" prepend-path PATH "${maker_repeatmasker_bin}" ### Define help text: proc ModulesHelp { } { global maker_home maker_exa_dir first_level_support_email maker_bpr_url maker_blast_bin maker_exonerate_bin maker_snap_bin maker_repeatmasker_bin puts stderr " DESCRIPTION MAKER is a portable and easily configurable genome annotation pipeline. Its purpose is to allow smaller eukaryotic and prokaryotic genome projects to independently annotate their genomes and to create genome databases. MAKER identifies repeats, aligns ESTs and proteins to a genome, produces ab-initio gene predictions and automatically synthesizes these data into gene annotations having evidence-based quality values. EXTERNAL PROGRAMS BLAST Command Line Applications (2.2.28) $maker_blast_bin EXONERATE Generic Sequence Comparison Tool (2.2.0) $maker_exonerate_bin SNAP Semi-HMM-based Nucleic Acid Parser (version 2006-07-28) $maker_snap_bin REPEATMASKER Mask repetitive DNA (open-4.0.5) $maker_repeatmasker_bin DOCUMENTATION * Main MAKER site http://www.yandell-lab.org/software/maker.html * MAKER User Guide, Documents, Wiki-Article http://weatherby.genetics.utah.edu/MAKER/wiki/index.php $maker_home/README * MAKER Download http://yandell.topaz.genetics.utah.edu/maker_downloads/CE63/461D/BD41/4510A183DC4571D5EA619E459572/maker-2.31.8.tgz * bwHPC examples and a moab example script $maker_exa_dir * bwHPC Best Practices Repository $maker_bpr_url CITATION n./a. +-------------------------------------------+ | THIS IS THE IMPI-VERSION BUILT WITH INTEL | +-------------------------------------------+ In case of problems, please contact: $first_level_support_email This module is available for all users. " } module load compiler/intel/16.0 module load mpi/impi/5.1.3-intel-16.0 conflict bio/maker -------------- next part -------------- #!/bin/bash #MSUB -N maker_impi-job #MSUB -j oe #MSUB -o $(JOBNAME).$(JOBID) #MSUB -m ae #MSUB -l nodes=1:ppn=1 #MSUB -l mem=20gb #MSUB -l walltime=10:00:00 # start=$(date +%s) echo " " echo "### Setting up shell environment ..." echo " " # if test -e "/etc/profile"; then source "/etc/profile"; fi; if test -e "$HOME/.bash_profile"; then source "$HOME/.bash_profile"; fi; unset LANG; export LC_ALL="C"; export MKL_NUM_THREADS=1; export OMP_NUM_THREADS=1 export USER=${USER:=`logname`} export MOAB_JOBID=${MOAB_JOBID:=`date +%s`} export MOAB_SUBMITDIR=${MOAB_SUBMITDIR:=`pwd`} export MOAB_JOBNAME=${MOAB_JOBNAME:=`basename "$0"`} export MOAB_JOBNAME=$(echo "${MOAB_JOBNAME}" | sed 's/[^a-zA-Z0-9._-]/_/g') export MOAB_NODECOUNT=${MOAB_NODECOUNT:=1} export MOAB_PROCCOUNT=${MOAB_PROCCOUNT:=1} ulimit -s 200000 echo " " echo "### Printing basic job infos to stdout ..." echo " " echo "START_TIME = `date +'%y-%m-%d %H:%M:%S %s'`" echo "HOSTNAME = ${HOSTNAME}" echo "USER = ${USER}" echo "MOAB_JOBNAME = ${MOAB_JOBNAME}" echo "MOAB_JOBID = ${MOAB_JOBID}" echo "MOAB_SUBMITDIR = ${MOAB_SUBMITDIR}" echo "MOAB_NODECOUNT = ${MOAB_NODECOUNT}" echo "MOAB_PROCCOUNT = ${MOAB_PROCCOUNT}" echo "SLURM_NODELIST = ${SLURM_NODELIST}" echo "PBS_NODEFILE = ${PBS_NODEFILE}" if test -f "${PBS_NODEFILE}"; then echo "PBS_NODEFILE (begin) ---------------------------------" NO_NODES=$(wc -l < ${PBS_NODEFILE}) cat "${PBS_NODEFILE}" sort -u $PBS_NODEFILE > mpd.hosts echo "PBS_NODEFILE (end) -----------------------------------" else NO_NODES=1 fi echo " " echo "### Creating TMP_WORK_DIR directory and changing to it ..." echo " " # Using "$TMPDIR" is strongly recommended for Maker jobs # since these jobs can create a lot of disk IO. # NEVER EVER calculate in your home directory. TMP_BASE_DIR="$TMPDIR" JOB_WORK_DIR="${MOAB_JOBNAME}.${MOAB_JOBID##*.}.$(date +%y%m%d_%H%M%S)" TMP_WORK_DIR="${TMP_BASE_DIR}/${JOB_WORK_DIR}" echo "JOB_WORK_DIR = ${JOB_WORK_DIR}" echo "TMP_BASE_DIR = ${TMP_BASE_DIR}" echo "TMP_WORK_DIR = ${TMP_WORK_DIR}" mkdir -vp "${TMP_WORK_DIR}" cd "${TMP_WORK_DIR}" echo " " echo "### Loading MAKER module:" echo " " module load bio/maker/2.31.8_impi [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.8_impi'."; exit 1; } echo "MAKER_VERSION = $MAKER_VERSION" module list echo " " echo "### Copying input examples files for job:" echo " " cp -v ${MAKER_EXA_DIR}/*.{fasta,ctl} . cp -Rv ${MAKER_EXA_DIR}/_Inline . sleep 2 echo " " echo "### Display internal Maker/bwHPC environments..." echo " " echo "MAKER_BIN_DIR = ${MAKER_BIN_DIR}" echo "MAKER_EXA_DIR = ${MAKER_EXA_DIR}" echo "" echo " " echo "### Runing Maker example" echo " " # # Environmental variables to set: # export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so echo "LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so" export OMPI_MCA_mpi_warn_on_fork=0 echo "OMPI_MCA_mpi_warn_on_fork=0" export I_MPI_CPUINFO="proc" echo "I_MPI_CPUINFO=proc" export I_MPI_PMI_LIBRARY=${MPI_LIB_DIR}/libpmi.so echo "I_MPI_PMI_LIBRARY=${MPI_LIB_DIR}/libpmi.so" export I_MPI_PIN_DOMAIN=node echo "I_MPI_PIN_DOMAIN=node" # otherwise MAKER calls to BLAST and other programs which are parallelized independent # of MPI may not work export I_MPI_FABRICS='shm:tcp' echo "I_MPI_FABRICS=shm:tcp" # avoid potential complication with OpenFabrics libraries (they block system calls because of # how they use registered memory, i.e. MAKER calling BLAST would fail) # set to eth0 if you don?t have an infiniband over # ip inerface (required because of the above I_MPI_FABRICS change) export I_MPI_HYDRA_IFACE=ib0 echo "I_MPI_HYDRA_IFACE=ib0" # The ?c nobrs. option will try and run the BLAST jobs on # nobrs. cpus on the same machine, but will not run via MPI. # -nolocal # wtf? # mpiexec -n $SLURM_NPROCS -env I_MPI_DEBUG 5 ./hello echo "starting mpiexec..." mpiexec -nc 1 maker echo "### Cleaning up files ... removing unnecessary scratch files ..." echo " " # rm -fv sleep 3 # Sleep some time so potential stale nfs handles can disappear. echo " " echo "### Compressing results and copying back result archive ..." echo " " cd "${TMP_BASE_DIR}" mkdir -vp "${MOAB_SUBMITDIR}" # if user has deleted or moved the submit dir echo "Creating result tgz-file '${MOAB_SUBMITDIR}/${JOB_WORK_DIR}.tgz' ..." tar -zcvf "${MOAB_SUBMITDIR}/${JOB_WORK_DIR}.tgz" "${JOB_WORK_DIR}" \ || { echo "ERROR: Failed to create tgz-file. Please cleanup TMP_WORK_DIR '$TMP_WORK_DIR' on host '$HOSTNAME' manually (if not done automatically by queueing system)."; exit 102; } # Remarks: # * The resulting tgz file is copied back to the submit directory. # The name of the tgz file looks similar too # "bwunicluster-maker-example.moab.275.110528_101755.tgz" echo " " echo "### Final cleanup: Remove TMP_WORK_DIR ..." echo " " rm -rvf "${TMP_WORK_DIR}" echo "END_TIME = `date +'%y-%m-%d %H:%M:%S %s'`" end=$(date +%s) echo " " echo "### Calculate duration ..." echo " " diff=$[end-start] if [ $diff -lt 60 ]; then echo "Runtime (approx.): '$diff' secs" elif [ $diff -ge 60 ]; then echo 'Runtime (approx.): '$[$diff / 60] 'min(s) '$[$diff % 60] 'secs' fi -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From carsonhh at gmail.com Fri Feb 24 10:30:32 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 24 Feb 2017 10:30:32 -0700 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> Message-ID: Specific things. 1. Do not set LD_PRELOAD. That is only for OpenMPI, but it will cause problems with other MPI's. 2. Make sure you recompiled MAKER for Intel MPI (MPI code always has to be compiled for the flavor you are using, so make sure you have a separate installation of MAKER for Intel MPI). Also validate that the mpicc and libmpi.h listed during the MAKER install belong to Intel MPI. Don?t just assume they do because you loaded the module. Manually verify the paths during MAKER?s setup. 3. The error you got previously should not even be possible with the current version of Intel MPI, which is why I say that when you called mpiexec, something else (that was not Intel MPI) was launched. Easy solution is to give the full path of mpiexec in your job, so are not relying on PATH to be unaltered in your job. Do not do ?> mpiexec -nc 1 maker Do this for example ?> /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec -nc maker 4. Build and run on the same node for your test. If you build on one node and run on another, you may be changing your environment in ways you don?t realize that break things. So if you can build and test on the same node and it works, then it fails when you test it elsewhere, then you have to track down how your environment is changing. ?Carson > On Feb 24, 2017, at 1:43 AM, Rainer Rutka wrote: > > Hi Carson. > > First of all THANK YOU FOR YOUR HELP. > MUCH APPRECIATED. :-) > > Am 23.02.2017 um 21:22 schrieb Carson Holt: >> It means one of two things. >> 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). > > No, I am starting the right version of mpiexec. > > This is a list of all our current available MPI-Versions, corresponding > to their compiler: > > UC:[kn at uc1n997 ~]$ module avail mpi > ------------------------------------- /opt/bwhpc/common/modulefiles -------------------------------------- > mpi/impi/4.1.3-gnu-4.4 mpi/openmpi/1.10-intel-15.0 > mpi/impi/4.1.3-gnu-4.7 mpi/openmpi/1.10-intel-16.0 > mpi/impi/4.1.3-intel-14.0 mpi/openmpi/1.8-gnu-4.5 > mpi/impi/5.0.3-gnu-4.4 mpi/openmpi/1.8-gnu-4.7 > mpi/impi/5.0.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.7-m32 > mpi/impi/5.0.3-intel-15.0 mpi/openmpi/1.8-gnu-4.8 > mpi/impi/5.1.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.9 > mpi/impi/5.1.3-gnu-system mpi/openmpi/1.8-intel-14.0(default) > > mpi/impi/5.1.3-intel-16.0(default) > > mpi/openmpi/1.8-intel-15.0 > mpi/openmpi/1.10-gnu-4.5 mpi/openmpi/2.0-gnu-4.7 > mpi/openmpi/1.10-gnu-4.7 mpi/openmpi/2.0-gnu-4.8 > mpi/openmpi/1.10-gnu-4.8 mpi/openmpi/2.0-gnu-5.2 > mpi/openmpi/1.10-gnu-4.9 mpi/openmpi/2.0-intel-15.0 > mpi/openmpi/1.10-gnu-5.2 mpi/openmpi/2.0-intel-16.0 > mpi/openmpi/1.10-intel-14.0 > > > Here I load the MPI-Module including all it's dependencies: > > UC:[kn at uc1n997 ~]$ module load mpi/impi/5.1.3-intel-16.0 > Loading module dependency 'compiler/intel/16.0'. > > Result: > UC:[kn at c1n997 ~]$ which mpiexec > /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec > UC:[kn at uc1n997 ~]$ > >> 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) > Extremely old? > > [...] > Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160601 (build id: 15562) > >> MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. > Yes. And we don't even use MPD at our clusters :-) > >> So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. > I do not have any MPICH available on our cluster(s) now. All of the old versions had > been removed some years ago. > > At a glance > --------------- > > Maker is available as a so-called module on our cluster system. It's been build > on a developers' node I can access. But, the MPI-modules are built by other > fellows (e.g. at the KIT in Karlsruhe/Germany) on other nodes. > > Please check the module-file (included in this mail) > > bio-maker-2.31.8_impi > > to see how Maker was build including the envoronments set by this > module. > > e.g.: > ./Build status > verify dependencies: > =================================================== > STATUS MAKER v2.31.8 > =================================================== > PERL Dependencies: VERIFIED > External Programs: VERIFIED > External C Libraries: VERIFIED > MPI SUPPORT: ENABLED > MWAS Web Interface: DISABLED > MAKER PACKAGE: CONFIGURATION OK > > > At least, please(!) have a look at our m.moab file (included in this mail). > This is the way how we submit a Maker job to our cluster. Maybe something > is wrong here? > > Sorry again for wasting your time. But we imperatively need the Maker > software running in parallel mode. > > :-) > > -- > Rainer Rutka > University of Konstanz > Communication, Information, Media Centre (KIM) > * High-Performance-Computing (HPC) > * KIM-Support and -Base-Services > Room: V511 > 78457 Konstanz, Germany > +49 7531 88-5413 > From qlian003 at ucr.edu Sat Feb 25 10:14:04 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Sat, 25 Feb 2017 09:14:04 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: <2377C5DD-569C-4248-B458-349D7AEA32F5@ucr.edu> Thank you Barry and Carson! I compared the SOBA statistics of RepeatMasker footprint and the report generated by running RepeatMasker alone, I got 2 different parentage of repeats masked. Running RepeatMasker with myTrained.lib, the repeats masked are 42%. But within Maker GFF3, the percentage of repeats masker is only ~18%. What may cause such difference here? Thanks Qihua > On Feb 21, 2017, at 1:34 PM, Carson Holt wrote: > > MAKER merges overlapping RepeatMasker results into a single longer feature. > > ?Carson > > >> On Feb 20, 2017, at 1:34 PM, Qihua Liang wrote: >> >> Hi Carson, >> >> Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. >> >> I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. >> >> Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. >> In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. >> >> But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. >> >> When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? >> Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? >> >> Thanks >> Qihua >> >> >>> On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: >>> >>> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. >>> >>> CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. >>> >>> If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. >>> >>> What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. >>> >>> However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. >>> >>> Thanks, >>> Carson >>> >>>> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >>>> >>>> Dear Maker develop team, >>>> >>>> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >>>> >>>> Thanks >>>> Qihua >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> > From qwzhang0601 at gmail.com Mon Feb 27 08:25:04 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 27 Feb 2017 10:25:04 -0500 Subject: [maker-devel] PARALLELIZED DE NOVO GENOME ANNOTATION WITHOUT MPI Message-ID: Hello: I am doing genome annotation using Maker on our high performance computational cluster (HPC). Due to some issues of MPI, I submitted the Maker jobs several times under the same directory to HPC. Followed by the example in the protocol (as shown below), when I submit the jobs I make them as background processes by "&" except the first one. Is this necessary when I submit a job to a HPC? I found it costed much much longer time than I expected (according to a testing on a smaller data set). I am not sure whether setting the process as background process lead to this issue? The example in the protocol % maker 2> maker1.error % maker 2> maker2.error & % maker 2> maker3.error & ...... BTW, will the annotation on shorter contig (e.g., 500bp) cost ~ 1/100 of the time that cost for annotation a 50000bp contig? I am using SNAP for an inito and RNA-seq assembly and protein sequences as evidence. I have more than half contigs shorter than 300bp (whose total length is only about 5% of the total length of all contigs), I want to know whether I can save about half (or only about 5%) of the time if I ignore those short contigs. Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcg at cau.edu.cn Tue Feb 28 00:43:57 2017 From: dcg at cau.edu.cn (dcg at cau.edu.cn) Date: Tue, 28 Feb 2017 15:43:57 +0800 Subject: [maker-devel] how to deal with Contigs to run maker? Message-ID: <2017022815435664227911@cau.edu.cn> Dear sir: After assemblying, I got many contigs and their order in each chromosome. What I have done is merging these contigs into each chromosomes followed by the order, with 100 Ns inserted betwwen each contigs. So that I got chr1 chr2......Then I ran the repeatmodeler, predictor to annotate it. Could my way reach a high-quality result? Should I use all the contigs to mask repeats and practice predictor? Is there any better way to do genome-wide annotation? I'm looking forward to your reply! Best wishes! Chao Chao 2017.02.28 -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Thu Feb 2 09:16:51 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Thu, 2 Feb 2017 11:16:51 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> Message-ID: Thank for you suggestions. So it does not matter if there are redundancies of protein sequence from different sources? I am trying to annotating a *rodent *genome, and planned to collect protein sequences of human, mouse, rat from bout UniProt and NCBI (besides I also have RNA-seq data). I choose these species, because they are close to the species that I am working on and they are well annotated. But I saw someone said that if we choose protein sequence from one lineage, the genes that are missing in the lineage will not be detected. And in the following paper, the authors claim they used the entire SwissProt database as the input. How do you think about this? Should I include protein sequences from more species (like all Eukaryota)? I think it can help us identify more genes, but on the other hand won't this also give us more false positives? This paper used the entire SwissProt database as the input. Insights into the evolution of longevity from the bowhead whale genome. 2015. *Cell Rep* *10*(1): 112-122. Thanks Best Quanwei 2017-01-31 15:57 GMT-05:00 Michael Campbell : > Hi Quanwei, > > (1) When I use uniprot I use SWISS-prot and not tremble. > (2) I don?t merge files together. I just pass them all to MAKER as a comma > separated list. > > Thanks, > Mike > > > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang > wrote: > > > > I wonder what's the best way to collect protein sequences for gene > annotation of a de novo genome assembly. > > (1) My first choice is to get protein sequences of human and mouse from > UniProt. At this step, I am not clear whether I should download the > reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., > TrEMBL). > > (2) On ther other hand, I also get protein sequences from NCBI, should I > just simply merge those fasta files. Does it matter if there are > redundancies? And also, if I get protein sequences from different sources, > they may not have the same quality. Do I need to do something before I > integrate protein sequences from different sources? > > > > Many thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Thu Feb 2 12:24:04 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Thu, 2 Feb 2017 14:24:04 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> Message-ID: <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> It is nice to have an outgroup of some kind. In your case Human would serve that function. The issue that you might have with distantly related proteins is funny blast alignments that may lead to merging genes. You generally don?t get many false positives because the alignment parameters require a pretty good match, which is unlikely to happen by chance. You could limit swiss-prot to mammals if you wanted to. Sometimes I?ll try different combinations of evidence on a few large scaffolds and look at the results in a browser to get a feel for what is going to work best. Thanks, Mike > On Feb 2, 2017, at 11:16 AM, Quanwei Zhang wrote: > > > Thank for you suggestions. So it does not matter if there are redundancies of protein sequence from different sources? > I am trying to annotating a rodent genome, and planned to collect protein sequences of human, mouse, rat from bout UniProt and NCBI (besides I also have RNA-seq data). I choose these species, because they are close to the species that I am working on and they are well annotated. But I saw someone said that if we choose protein sequence from one lineage, the genes that are missing in the lineage will not be detected. And in the following paper, the authors claim they used the entire SwissProt database as the input. How do you think about this? Should I include protein sequences from more species (like all Eukaryota)? I think it can help us identify more genes, but on the other hand won't this also give us more false positives? > > This paper used the entire SwissProt database as the input. > Insights into the evolution of longevity from the bowhead whale genome. 2015. Cell Rep 10(1): 112-122. > > Thanks > > Best > Quanwei > > 2017-01-31 15:57 GMT-05:00 Michael Campbell >: > Hi Quanwei, > > (1) When I use uniprot I use SWISS-prot and not tremble. > (2) I don?t merge files together. I just pass them all to MAKER as a comma separated list. > > Thanks, > Mike > > > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang > wrote: > > > > I wonder what's the best way to collect protein sequences for gene annotation of a de novo genome assembly. > > (1) My first choice is to get protein sequences of human and mouse from UniProt. At this step, I am not clear whether I should download the reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., TrEMBL). > > (2) On ther other hand, I also get protein sequences from NCBI, should I just simply merge those fasta files. Does it matter if there are redundancies? And also, if I get protein sequences from different sources, they may not have the same quality. Do I need to do something before I integrate protein sequences from different sources? > > > > Many thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Thu Feb 2 14:05:53 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Thu, 2 Feb 2017 16:05:53 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> Message-ID: Thank you for your suggestions. I need to have some tests. Best Quanwei 2017-02-02 14:24 GMT-05:00 Michael Campbell : > It is nice to have an outgroup of some kind. In your case Human would > serve that function. The issue that you might have with distantly related > proteins is funny blast alignments that may lead to merging genes. You > generally don?t get many false positives because the alignment parameters > require a pretty good match, which is unlikely to happen by chance. You > could limit swiss-prot to mammals if you wanted to. > > Sometimes I?ll try different combinations of evidence on a few large > scaffolds and look at the results in a browser to get a feel for what is > going to work best. > > Thanks, > Mike > > On Feb 2, 2017, at 11:16 AM, Quanwei Zhang wrote: > > > Thank for you suggestions. So it does not matter if there are redundancies > of protein sequence from different sources? > I am trying to annotating a *rodent *genome, and planned to collect > protein sequences of human, mouse, rat from bout UniProt and NCBI (besides > I also have RNA-seq data). I choose these species, because they are close > to the species that I am working on and they are well annotated. But I saw > someone said that if we choose protein sequence from one lineage, the genes > that are missing in the lineage will not be detected. And in the following > paper, the authors claim they used the entire SwissProt database as the > input. How do you think about this? Should I include protein sequences from > more species (like all Eukaryota)? I think it can help us identify more > genes, but on the other hand won't this also give us more false positives? > > This paper used the entire SwissProt database as the input. > Insights into the evolution of longevity from the bowhead whale genome. > 2015. *Cell Rep* *10*(1): 112-122. > > Thanks > > Best > Quanwei > > 2017-01-31 15:57 GMT-05:00 Michael Campbell >: > >> Hi Quanwei, >> >> (1) When I use uniprot I use SWISS-prot and not tremble. >> (2) I don?t merge files together. I just pass them all to MAKER as a >> comma separated list. >> >> Thanks, >> Mike >> >> > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang >> wrote: >> > >> > I wonder what's the best way to collect protein sequences for gene >> annotation of a de novo genome assembly. >> > (1) My first choice is to get protein sequences of human and mouse from >> UniProt. At this step, I am not clear whether I should download the >> reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., >> TrEMBL). >> > (2) On ther other hand, I also get protein sequences from NCBI, should >> I just simply merge those fasta files. Does it matter if there are >> redundancies? And also, if I get protein sequences from different sources, >> they may not have the same quality. Do I need to do something before I >> integrate protein sequences from different sources? >> > >> > Many thanks >> > >> > Best >> > Quanwei >> > _______________________________________________ >> > maker-devel mailing list >> > maker-devel at box290.bluehost.com >> > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 3 09:04:45 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 3 Feb 2017 11:04:45 -0500 Subject: [maker-devel] augustus failed in maker2 Message-ID: I am trying to add augustus to the prediction. The maker2 run well without augustus, and augustus seems installed correctly. But it returns me the following errors. Any ideas or suggestions? What I did is just to set "augustus_species=human" in the "maker_opts.ctl" file, and I am running maker on a HPC with linux system. preparing ab-inits ERROR: Augustus failed --> rank=NA, hostname=n530 ERROR: Failed while preparing ab-inits ERROR: Chunk failed at level:0, tier_type:2 FAILED CONTIG:CasCan_contig_0 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:CasCan_contig_0 Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 3 10:15:39 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 3 Feb 2017 10:15:39 -0700 Subject: [maker-devel] augustus failed in maker2 In-Reply-To: References: Message-ID: Look a little further back into the STERR log to see if there are other errors further back. The issue is probably your Augustus installation. Try running it by itself (outside of MAKER) on the same dataset, and it may help identify on what is happening. Thanks, Carson > On Feb 3, 2017, at 9:04 AM, Quanwei Zhang wrote: > > I am trying to add augustus to the prediction. The maker2 run well without augustus, and augustus seems installed correctly. But it returns me the following errors. Any ideas or suggestions? What I did is just to set "augustus_species=human" in the "maker_opts.ctl" file, and I am running maker on a HPC with linux system. > > preparing ab-inits > ERROR: Augustus failed > --> rank=NA, hostname=n530 > ERROR: Failed while preparing ab-inits > ERROR: Chunk failed at level:0, tier_type:2 > FAILED CONTIG:CasCan_contig_0 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:CasCan_contig_0 > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From mnaymik at tgen.org Mon Feb 6 14:27:22 2017 From: mnaymik at tgen.org (Marcus Naymik) Date: Mon, 6 Feb 2017 14:27:22 -0700 Subject: [maker-devel] Can't Install Proc::Signal Message-ID: I can't find Proc::Signal in cpan or anywhere. How can I install it? -- *This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisa.postma at gmail.com Sun Feb 5 06:15:08 2017 From: alisa.postma at gmail.com (Alisa Postma) Date: Sun, 5 Feb 2017 15:15:08 +0200 Subject: [maker-devel] Problem with fgenesh protein IDs Message-ID: Good afternoon I am having problems with my all.maker.proteins.fasta file. I ran maker with ab initio gene predictions from augustus and genemark and I added my fgenesh gff file to the pred_gff option in the ctl file. However, my maker protein fasta file does not give the fgenesh proteins IDs similar to that for augustus and genemark. For example: >FGENESH protein AED:0.06 eAED:0.06 QI:0|0|0|1|1|1|3|0|433 vs. >maker-scaffold74-augustus-gene-0.179-mRNA-1 protein AED:0.07 eAED:0.07 QI:123|1|1|1|1|1|5|793|326 I would appreciate any advice on this matter. Kind regards Alisa -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 6 21:05:33 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 6 Feb 2017 21:05:33 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: References: Message-ID: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Proc::Signal is part of MAKER?s internal libraries. If you are getting an error saying it is missing, then you have a problem with your installation. Either it was not installed successfully, or you modified the location of files or executables post installation. Make sure you do not move the bin directory or its contents. If you need to be somewhere else, it is best to use softlinks instead. Thanks, Carson > On Feb 6, 2017, at 2:27 PM, Marcus Naymik wrote: > > I can't find Proc::Signal in cpan or anywhere. How can I install it? > > This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaymik at tgen.org Tue Feb 7 10:00:37 2017 From: mnaymik at tgen.org (Marcus Naymik) Date: Tue, 7 Feb 2017 10:00:37 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> References: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Message-ID: Ah, It was listed on as a prereq on your wiki and was trying to install all those first. I was just a little confused. Thanks! Marcus On Mon, Feb 6, 2017 at 9:05 PM, Carson Holt wrote: > Proc::Signal is part of MAKER?s internal libraries. If you are getting an > error saying it is missing, then you have a problem with your installation. > > Either it was not installed successfully, or you modified the location of > files or executables post installation. > > Make sure you do not move the bin directory or its contents. If you need > to be somewhere else, it is best to use softlinks instead. > > Thanks, > Carson > > On Feb 6, 2017, at 2:27 PM, Marcus Naymik wrote: > > I can't find Proc::Signal in cpan or anywhere. How can I install it? > > *This electronic message is intended to be for the use only of the named > recipient, and may contain information that is confidential or privileged, > including patient health information. If you are not the intended > recipient, you are hereby notified that any disclosure, copying, > distribution or use of the contents of this message is strictly prohibited. > If you have received this message in error or are not the named recipient, > please notify us immediately by contacting the sender at the electronic > mail address noted above, and delete and destroy all copies of this > message. Thank you.* > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- *This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 7 10:11:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 7 Feb 2017 10:11:46 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: References: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Message-ID: Use the MAKER Tutorial for GMOD Online Training 2014 It is the most up to date ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014 Thanks, Carson > On Feb 7, 2017, at 10:00 AM, Marcus Naymik wrote: > > Ah, It was listed on as a prereq on your wiki and was trying to install all those first. I was just a little confused. Thanks! > > Marcus > > On Mon, Feb 6, 2017 at 9:05 PM, Carson Holt > wrote: > Proc::Signal is part of MAKER?s internal libraries. If you are getting an error saying it is missing, then you have a problem with your installation. > > Either it was not installed successfully, or you modified the location of files or executables post installation. > > Make sure you do not move the bin directory or its contents. If you need to be somewhere else, it is best to use softlinks instead. > > Thanks, > Carson > >> On Feb 6, 2017, at 2:27 PM, Marcus Naymik > wrote: >> >> I can't find Proc::Signal in cpan or anywhere. How can I install it? >> >> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 7 10:12:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 7 Feb 2017 10:12:46 -0700 Subject: [maker-devel] Problem with fgenesh protein IDs In-Reply-To: References: Message-ID: The issue is probably with the format of your GFF3. I can take a look if you want. ?Carson > On Feb 5, 2017, at 6:15 AM, Alisa Postma wrote: > > Good afternoon > > I am having problems with my all.maker.proteins.fasta file. > > I ran maker with ab initio gene predictions from augustus and genemark and I added my fgenesh gff file to the pred_gff option in the ctl file. > > However, my maker protein fasta file does not give the fgenesh proteins IDs similar to that for augustus and genemark. > > For example: > >FGENESH protein AED:0.06 eAED:0.06 QI:0|0|0|1|1|1|3|0|433 > > vs. > > > >maker-scaffold74-augustus-gene-0.179-mRNA-1 protein AED:0.07 eAED:0.07 QI:123|1|1|1|1|1|5|793|326 > > I would appreciate any advice on this matter. > > Kind regards > > Alisa > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Wed Feb 8 08:45:11 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Wed, 8 Feb 2017 10:45:11 -0500 Subject: [maker-devel] make use of other gene finders in maker Message-ID: Hello: I am trying to annotate the new genome of a rodent species. I saw a clear pipeline to train the SNAP gene finder. But not clear what I should do if I also want to include augustus and/or GeneMark. Do I need to (could I and how) train it in Maker? In one of recently published paper, they mention the augustus was trained in Maker, but I am not clear how to do it? I wonder what's you opinions on using other gene finders in Maker. Thanks "Unique features of a global human ectoparasite identified through sequencing of the bed bug genome." Nat Commun, 2016. In this paper, the training step was described as below. *Three rounds of training of the Augustus and SNAP gene predictors within Maker were used to bootstrap to a high-quality training set.* Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Wed Feb 8 08:56:08 2017 From: dence at genetics.utah.edu (Daniel Ence) Date: Wed, 8 Feb 2017 15:56:08 +0000 Subject: [maker-devel] make use of other gene finders in maker In-Reply-To: References: Message-ID: <02D8BB85-1B0B-4028-A3F6-182F4F3318F7@genetics.utah.edu> Hi Quanwei, MAKER will run Augustus, SNAP, genemark or Fgenesh internally. You can also provide predictions from other gene predictors in valid GFF3 format in the ?pred_gff" field in the maker_opts control file. Genemark is self training on your genome. You provide a path to the ?es.mod? file that Genemark makes in the ?gmhmm? field. For augustus, the training is more involved, but is described here: http://www.molecularevolution.org/molevolfiles/exercises/augustus/training.html Augustus, snap, Fgenesh, and genemark are all complementary in some respects, but Augustus usually provides the most predictions that MAKER ends up selecting to promote to models. Hope that helps, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Feb 8, 2017, at 10:45 AM, Quanwei Zhang > wrote: Hello: I am trying to annotate the new genome of a rodent species. I saw a clear pipeline to train the SNAP gene finder. But not clear what I should do if I also want to include augustus and/or GeneMark. Do I need to (could I and how) train it in Maker? In one of recently published paper, they mention the augustus was trained in Maker, but I am not clear how to do it? I wonder what's you opinions on using other gene finders in Maker. Thanks "Unique features of a global human ectoparasite identified through sequencing of the bed bug genome." Nat Commun, 2016. In this paper, the training step was described as below. Three rounds of training of the Augustus and SNAP gene predictors within Maker were used to bootstrap to a high-quality training set. Best Quanwei _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 8 10:18:49 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 8 Feb 2017 12:18:49 -0500 Subject: [maker-devel] MAKER and OpenMPI Message-ID: Hello everyone, I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 8 10:22:19 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 10:22:19 -0700 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: References: Message-ID: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> You might have two different MPI flavors installed. Since the library and executable files have the same name, this kind of mismatch can happen in that case. Check the locations of MPI libraries given to MAKER during install, and the location of the mpiexec being used when you run. There is likely a mismatch with parts of one MPI flavor being used for one but not the other. ?Carson > On Feb 8, 2017, at 10:18 AM, Seth Munholland wrote: > > Hello everyone, > > I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 8 10:27:15 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 8 Feb 2017 12:27:15 -0500 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> References: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> Message-ID: That's for sure what it is, but I never installed a second flavour. I was commited to MPICH from the get go and am trying to figure out where OpenMPI came from. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 8, 2017 at 12:22 PM, Carson Holt wrote: > You might have two different MPI flavors installed. Since the library and > executable files have the same name, this kind of mismatch can happen in > that case. Check the locations of MPI libraries given to MAKER during > install, and the location of the mpiexec being used when you run. There is > likely a mismatch with parts of one MPI flavor being used for one but not > the other. > > ?Carson > > > > On Feb 8, 2017, at 10:18 AM, Seth Munholland wrote: > > Hello everyone, > > I'm setting up a cluster and configured MPICH on it, but after installing > MAKER my mpiexec gave me an error that comes from an improperly configured > OpenMPI. The weird thing is I haven't installed anything else associated > to MPI, particularly not OpenMPI. Did I get server gremlins or is there > any way MAKER's installation somehow is looking for OpenMPI? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 8 10:32:22 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 10:32:22 -0700 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: References: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> Message-ID: <863A3CDC-24F3-4A91-8D3E-45387262DF46@gmail.com> When you install MAKER, you have to provide the location to mpi.h and mpicc. You may have to reinstall, and give the correct location for one or both (don?t just trust the location that shows up, it may be the wrong one). So you will need to track down both of those for MPICH (depending on where you installed it to). Then do ?which -a mpiexec? to get all locations for every mpiexec found in your path. You will need to ensure the one at the top is the one you want. If it?s not, then you need to reconfigure your PATH or provide the full path to the mpiexec you want each time you launch it. ?Carson > On Feb 8, 2017, at 10:27 AM, Seth Munholland wrote: > > That's for sure what it is, but I never installed a second flavour. I was commited to MPICH from the get go and am trying to figure out where OpenMPI came from. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 8, 2017 at 12:22 PM, Carson Holt > wrote: > You might have two different MPI flavors installed. Since the library and executable files have the same name, this kind of mismatch can happen in that case. Check the locations of MPI libraries given to MAKER during install, and the location of the mpiexec being used when you run. There is likely a mismatch with parts of one MPI flavor being used for one but not the other. > > ?Carson > > > >> On Feb 8, 2017, at 10:18 AM, Seth Munholland > wrote: >> >> Hello everyone, >> >> I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjfi2sb3 at gmail.com Wed Feb 8 00:22:57 2017 From: mjfi2sb3 at gmail.com (Salim Bougouffa) Date: Wed, 08 Feb 2017 07:22:57 +0000 Subject: [maker-devel] Masking is causing problems with exon/CDS boundary predictions Message-ID: Hi, I am having trouble with MAKER/AUGUSTUS annotation. One of the recurrent ones is the example I attach in artemis screenshot. In red is the gene of interest. There are three GFF files. Top is augustus standalone that I executed on the non-masked scaffold. The second is maker annotation and third is the augustus (from maker) on the masked query. The gene clearly has four exons and three CDS but the masking seem to somehow cause augustus to get the boundaries wrong for the first and third CDS and skip the second CDS entirely. How do I fix this problem? Best, /SB [image: artemis.png] -- ____________________________ Sent from Inbox Mobile -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: artemis.png Type: image/png Size: 128354 bytes Desc: not available URL: From carsonhh at gmail.com Wed Feb 8 12:31:16 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 12:31:16 -0700 Subject: [maker-devel] Masking is causing problems with exon/CDS boundary predictions In-Reply-To: References: Message-ID: <4B4ACC1F-541E-4182-9DE6-5C22CEF2C91D@gmail.com> What do your evidence alignments look like? i.e. assembled mRNA-seq and protein homology? Not the mRNA-seq pileup (because MAKER can?t see that). Masking is applied before evidence seeding and the first ab initio run. It is actually then removed for evidence polishing around splice sites, and the hint based rerun of Augustus. So evidence is allowed to extend through masked regions and Augustus can make it part of the model if it wants on the second run. Since it didn?t the question becomes, what does the transcript and protein homology alignments from the maker run look like. ?Carson > On Feb 8, 2017, at 12:22 AM, Salim Bougouffa wrote: > > Hi, > > I am having trouble with MAKER/AUGUSTUS annotation. > > One of the recurrent ones is the example I attach in artemis screenshot. In red is the gene of interest. There are three GFF files. Top is augustus standalone that I executed on the non-masked scaffold. The second is maker annotation and third is the augustus (from maker) on the masked query. > > > The gene clearly has four exons and three CDS but the masking seem to somehow cause augustus to get the boundaries wrong for the first and third CDS and skip the second CDS entirely. > > How do I fix this problem? > > Best, > /SB > > -- > ____________________________ > Sent from Inbox Mobile > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 08:50:24 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 10:50:24 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? Message-ID: Hello: I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 09:03:41 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 11:03:41 -0500 Subject: [maker-devel] training of gene finders using whole assembly or longest contigs? Message-ID: Hello: I am training the gene finders using the whole assembly. But it seems very time consuming. Besides, I have to repeat the training process several times. Although I am running it on 25 nodes on a server, it may still take 3 (or even more) weeks for the training. I wonder how you guys train the SNAP. Do you use the whole assembly or just select the longest contigs for the training. If I only use longest contigs (like top 20% longest), will it be good enough as that get by using the whole assembly? Or should I randomly select 20% contigs for the training, for which we will have similar length distribution as the whole assembly? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alessandro.Rossoni at uni-duesseldorf.de Fri Feb 10 08:50:07 2017 From: Alessandro.Rossoni at uni-duesseldorf.de (Alessandro Rossoni) Date: Fri, 10 Feb 2017 16:50:07 +0100 Subject: [maker-devel] Updating Reference Gene Models - Legacy - Strategy Message-ID: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> Dear makers of MAKER, first of all - thank you for this awesome program! In the context of my project, I have been running MAKER on a set of novel genomes and it worked very well :) During the last days, I realized that the reference sequences of species A that I have been using as starting point for gene model prediction for species B, C and D are a bit flawed. How do I know that? Through visualization of novel RNA-Seq data that was mapped to the "old" gene models of species A, I came to the conclusion that the "old" gene models are not really accurate. In fact, between 52?62% of the reads map within annotated exonic regions of the genome and up to 47% map within intergenic regions. Intron/Exon boarders are pretty messed up and there are a lot more transcribed sequences in species A than previously thought. Hence, I would love to update the gene models of species A including the new RNA-Seq evidence and hope to get more accurate gene models out of it. The more accurate gene models of species A would be then used to predict genes for species B, C and D (which are hopefully going to benefit from the more accurate input). However, I there is a little understanding issue on how to set the parameters of the maker_opts.ctl. My plan is to produce new RNA-Seq based gene models for species A using Cufflinks, Stringtie, Breaker, Trinity and Velvet. I would pass the output to the maker_opts.ctl as: model_gff=Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff #the new gene models est=Species_A_reference_ests.fasta #the old/flawed gene models protein=swissprot.fasta Question 1: is this correct so far? But what sequences do I use for training the ab-initio predictors? snaphmm= augustus_species= Question 2: Do I use the "old/flawed" sequences that I know are not really good? I am not sure what sequences to use for training. Any help on this issue would be amazing! Best, Ale -- Alessandro W. Rossoni, M.Sc. Institute for Plant Biochemistry Heinrich-Heine-University -- http:///www.plant-biochemistry.hhu.de/ E-Mail: alessandro.rossoni at hhu-duesseldorf.de From carsonhh at gmail.com Fri Feb 10 09:36:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:36:46 -0700 Subject: [maker-devel] Updating Reference Gene Models - Legacy - Strategy In-Reply-To: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> References: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> Message-ID: If the old models are poor, then I suggest you do new training using BUSCO, CEGMA, or the est2genome or protein2genome options within MAKER ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors Also this thread ?> https://groups.google.com/forum/#!topic/maker-devel/FWMSTdqWQqI model_gff is for existing gene models you want to keep. So none of these should go there ?> Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff model_gff will always make it into the final annotation set even without any evidence support. By putting those files there, you are basically turning every feature in each of those files into a final gene model no matter how bad it is. Also if the original models are poor, don?t put them there either. You can doing reciprocal best blast hits with final models to old models to see how they match each other in the end. Will take a little data processing to make it work though. For all transcript based files, you should provide those to est_gff since they are evidence alignments and not model predictions. For Breaker.gff, that should be pred_gff since it is a prediction model. With Trinity, I suggest you provide the fasta file and allow MAKER to align and filter things rather than a GFF3. The problem with using GFF3 is you are basically short circuiting upstream prioritization and filtering saying ?take this evidence as is.? Also providing same evidence from multiple sources is a bad idea. By purposely making the evidence dataset more noisy, you are forcing lower accuracy. My suggesting would be not to use Cufflinks (it will introduce a very high false positive rate). Provide Trinity input as fasta (also make sure you use jaccard_clip option was used when assembling). And you will have to manually review models with and without Stringtie data to see if it hurts more than it helps. Provide Breaker.gff to pred_gff, but still allow maker to run Augustus itself internally (otherwise you won?t be able to use protein evidence as hints). Thanks, Carson > On Feb 10, 2017, at 8:50 AM, Alessandro Rossoni wrote: > > Dear makers of MAKER, > first of all - thank you for this awesome program! In the context of my project, I have been running MAKER on a set of novel genomes and it worked very well :) > > During the last days, I realized that the reference sequences of species A that I have been using as starting point for gene model prediction for species B, C and D are a bit flawed. How do I know that? Through visualization of novel RNA-Seq data that was mapped to the "old" gene models of species A, I came to the conclusion that the "old" gene models are not really accurate. In fact, between 52?62% of the reads map within annotated exonic regions of the genome and up to 47% map within intergenic regions. Intron/Exon boarders are pretty messed up and there are a lot more transcribed sequences in species A than previously thought. > > Hence, I would love to update the gene models of species A including the new RNA-Seq evidence and hope to get more accurate gene models out of it. The more accurate gene models of species A would be then used to predict genes for species B, C and D (which are hopefully going to benefit from the more accurate input). However, I there is a little understanding issue on how to set the parameters of the maker_opts.ctl. > > My plan is to produce new RNA-Seq based gene models for species A using Cufflinks, Stringtie, Breaker, Trinity and Velvet. I would pass the output to the maker_opts.ctl as: > > model_gff=Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff #the new gene models > est=Species_A_reference_ests.fasta #the old/flawed gene models > protein=swissprot.fasta > > Question 1: is this correct so far? > > But what sequences do I use for training the ab-initio predictors? > snaphmm= > augustus_species= > > Question 2: Do I use the "old/flawed" sequences that I know are not really good? I am not sure what sequences to use for training. > > Any help on this issue would be amazing! > Best, > Ale > > > -- > Alessandro W. Rossoni, M.Sc. > Institute for Plant Biochemistry > Heinrich-Heine-University > > -- > http:///www.plant-biochemistry.hhu.de/ > E-Mail: alessandro.rossoni at hhu-duesseldorf.de > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 10 09:39:31 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:39:31 -0700 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: MAKER is restartable. As long as you run each time in the same location, it can reuse existing alignments from the previous run. You also only need to train on ~10MB of the genome depending on gene density. Target size should be 300-400 genes. If you follow this GMOD wiki, this is demonstrated (you can also watch video - link as top of page - to see it being done) ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors ?Carson > On Feb 10, 2017, at 8:50 AM, Quanwei Zhang wrote: > > Hello: > > I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 10 09:42:13 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:42:13 -0700 Subject: [maker-devel] training of gene finders using whole assembly or longest contigs? In-Reply-To: References: Message-ID: Example of training here ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors You can also search the devel mailing list archives here ?> https://groups.google.com/forum/#!forum/maker-devel There are lots and lots of threads that go into detail on training. Note more than 2 rounds of training is not beneficial, and can actually make performance worse (there is an overtraining paradox). ?Carson > On Feb 10, 2017, at 9:03 AM, Quanwei Zhang wrote: > > Hello: > > I am training the gene finders using the whole assembly. But it seems very time consuming. Besides, I have to repeat the training process several times. Although I am running it on 25 nodes on a server, it may still take 3 (or even more) weeks for the training. I wonder how you guys train the SNAP. Do you use the whole assembly or just select the longest contigs for the training. If I only use longest contigs (like top 20% longest), will it be good enough as that get by using the whole assembly? Or should I randomly select 20% contigs for the training, for which we will have similar length distribution as the whole assembly? > > Thanks > > Best > Quanwei > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 10:04:55 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 12:04:55 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: Great. Many thanks. So I can select part of my genome assembly for the training. Which the following ways do you think is better? (a) Select the longest contigs. (b) Randomly select contigs with any length? Thank you! Best Quanwei 2017-02-10 11:39 GMT-05:00 Carson Holt : > MAKER is restartable. As long as you run each time in the same location, > it can reuse existing alignments from the previous run. You also only need > to train on ~10MB of the genome depending on gene density. Target size > should be 300-400 genes. > > If you follow this GMOD wiki, this is demonstrated (you can also watch > video - link as top of page - to see it being done) ?> > http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/ > MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ > ab_initio_Gene_Predictors > > ?Carson > > > > On Feb 10, 2017, at 8:50 AM, Quanwei Zhang wrote: > > Hello: > > I am annotating a new genome using Maker. I have RNA-seq assembly and > protein sequences (from other organisms) in fasta format. Since I need to > train gene finders, so I have to run Maker several times. I think the > aligning process between the transcript assembly (protein sequences) and > the genome assembly may be time consuming. So I wonder whether I can save > such alignment in the first run, and then make use of such alignment in the > following runs? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Fri Feb 10 10:07:51 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Fri, 10 Feb 2017 12:07:51 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: The longer ones will have more complete genes on them. If you get a set scaffolds that has about 1,000 genes you probably have enough for training. Mike > On Feb 10, 2017, at 12:04 PM, Quanwei Zhang wrote: > > Great. Many thanks. So I can select part of my genome assembly for the training. Which the following ways do you think is better? (a) Select the longest contigs. (b) Randomly select contigs with any length? > > Thank you! > > Best > Quanwei > > 2017-02-10 11:39 GMT-05:00 Carson Holt >: > MAKER is restartable. As long as you run each time in the same location, it can reuse existing alignments from the previous run. You also only need to train on ~10MB of the genome depending on gene density. Target size should be 300-400 genes. > > If you follow this GMOD wiki, this is demonstrated (you can also watch video - link as top of page - to see it being done) ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors > > ?Carson > > > >> On Feb 10, 2017, at 8:50 AM, Quanwei Zhang > wrote: >> >> Hello: >> >> I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? >> >> Thanks >> >> Best >> Quanwei >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 21:53:02 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 23:53:02 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" Message-ID: Hello: I want to assign gene function to the predicted genes. I collected UniProt/SwissProt human, mouse and rat protein sequences (including both canonical and isoforms). Then use "makeblastdb" to build the database. And then run "blastp -db ... -max_hsps_per_subject 1" following the example in the protocol. But it returns me an error: Unknown argument: "max_hsps_per_subject". Why this happens? Is it because I am using a different version of blastp? USAGE blastp [-h] [-help] [-import_search_strategy filename] [-export_search_strategy filename] [-task task_name] [-db database_name] [-dbsize num_letters] [-gilist filename] [-seqidlist filename] [-negative_gilist filename] [-entrez_query entrez_query] [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm] [-subject subject_input_file] [-subject_loc range] [-query input_file] [-out output_file] [-evalue evalue] [-word_size int_value] [-gapopen open_penalty] [-gapextend extend_penalty] [-qcov_hsp_perc float_value] [-max_hsps int_value] [-xdrop_ungap float_value] [-xdrop_gap float_value] [-xdrop_gap_final float_value] [-searchsp int_value] [-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking] [-matrix matrix_name] [-threshold float_value] [-culling_limit int_value] [-best_hit_overhang float_value] [-best_hit_score_edge float_value] [-window_size int_value] [-lcase_masking] [-query_loc range] [-parse_deflines] [-outfmt format] [-show_gis] [-num_descriptions int_value] [-num_alignments int_value] [-line_length line_length] [-html] [-max_target_seqs num_sequences] [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo] [-use_sw_tback] [-version] Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Mon Feb 13 07:02:30 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Mon, 13 Feb 2017 09:02:30 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" In-Reply-To: References: Message-ID: Hi Quanwei, Different versions/implementations of blast do have parameters. Based on the usage you posted my guess is that the parameter you want is ?-max_hsps? Thanks, Mike > On Feb 10, 2017, at 11:53 PM, Quanwei Zhang wrote: > > Hello: > > I want to assign gene function to the predicted genes. I collected UniProt/SwissProt human, mouse and rat protein sequences (including both canonical and isoforms). Then use "makeblastdb" to build the database. And then run "blastp -db ... -max_hsps_per_subject 1" following the example in the protocol. But it returns me an error: Unknown argument: "max_hsps_per_subject". Why this happens? Is it because I am using a different version of blastp? > > USAGE > blastp [-h] [-help] [-import_search_strategy filename] > [-export_search_strategy filename] [-task task_name] [-db database_name] > [-dbsize num_letters] [-gilist filename] [-seqidlist filename] > [-negative_gilist filename] [-entrez_query entrez_query] > [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm] > [-subject subject_input_file] [-subject_loc range] [-query input_file] > [-out output_file] [-evalue evalue] [-word_size int_value] > [-gapopen open_penalty] [-gapextend extend_penalty] > [-qcov_hsp_perc float_value] [-max_hsps int_value] > [-xdrop_ungap float_value] [-xdrop_gap float_value] > [-xdrop_gap_final float_value] [-searchsp int_value] > [-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking] > [-matrix matrix_name] [-threshold float_value] [-culling_limit int_value] > [-best_hit_overhang float_value] [-best_hit_score_edge float_value] > [-window_size int_value] [-lcase_masking] [-query_loc range] > [-parse_deflines] [-outfmt format] [-show_gis] > [-num_descriptions int_value] [-num_alignments int_value] > [-line_length line_length] [-html] [-max_target_seqs num_sequences] > [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo] > [-use_sw_tback] [-version] > > Thanks > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From qwzhang0601 at gmail.com Mon Feb 13 08:02:24 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 10:02:24 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" In-Reply-To: References: Message-ID: Thanks. Yes, I should use "-max_hsps". Best Quanwei 2017-02-13 9:02 GMT-05:00 Michael Campbell : > Hi Quanwei, > > Different versions/implementations of blast do have parameters. Based on > the usage you posted my guess is that the parameter you want is ?-max_hsps? > > Thanks, > Mike > > On Feb 10, 2017, at 11:53 PM, Quanwei Zhang > wrote: > > > > Hello: > > > > I want to assign gene function to the predicted genes. I collected > UniProt/SwissProt human, mouse and rat protein sequences (including both > canonical and isoforms). Then use "makeblastdb" to build the database. And > then run "blastp -db ... -max_hsps_per_subject 1" following the example in > the protocol. But it returns me an error: Unknown argument: > "max_hsps_per_subject". Why this happens? Is it because I am using a > different version of blastp? > > > > USAGE > > blastp [-h] [-help] [-import_search_strategy filename] > > [-export_search_strategy filename] [-task task_name] [-db > database_name] > > [-dbsize num_letters] [-gilist filename] [-seqidlist filename] > > [-negative_gilist filename] [-entrez_query entrez_query] > > [-db_soft_mask filtering_algorithm] [-db_hard_mask > filtering_algorithm] > > [-subject subject_input_file] [-subject_loc range] [-query > input_file] > > [-out output_file] [-evalue evalue] [-word_size int_value] > > [-gapopen open_penalty] [-gapextend extend_penalty] > > [-qcov_hsp_perc float_value] [-max_hsps int_value] > > [-xdrop_ungap float_value] [-xdrop_gap float_value] > > [-xdrop_gap_final float_value] [-searchsp int_value] > > [-sum_stats bool_value] [-seg SEG_options] [-soft_masking > soft_masking] > > [-matrix matrix_name] [-threshold float_value] [-culling_limit > int_value] > > [-best_hit_overhang float_value] [-best_hit_score_edge float_value] > > [-window_size int_value] [-lcase_masking] [-query_loc range] > > [-parse_deflines] [-outfmt format] [-show_gis] > > [-num_descriptions int_value] [-num_alignments int_value] > > [-line_length line_length] [-html] [-max_target_seqs num_sequences] > > [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats > compo] > > [-use_sw_tback] [-version] > > > > Thanks > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 08:16:35 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 10:16:35 -0500 Subject: [maker-devel] failed to assign putative gene function Message-ID: Hello: I am trying to add putative gene function to the predicted gene models. Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. I used canonical and isoform proteins of human, mouse and rat with the script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose context is as below. maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 7e-164 464 snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 3e-80 236 After that, I am trying to add the protein homology data to the Maker gff3 and fasta files with maker_functional_gff and maker_functional_fasta, but get the reports as below. Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 39. Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 39. Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 45. Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 45. Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B ..... I am not sure how to deal with this. I followed the command given in the protocol. Any suggestions? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 09:09:48 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 11:09:48 -0500 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: I just found at the bottom of the output file it include information like below. But I did not get those GFF3 files with gene homolog information added. >snap_masked-CasCan_contig_27993-processed-gene-0.6-mRNA-1 transcript Name:"Protein of unknown function" offset:0 AED:0.47 eAED:0.47 QI:0|0|0|1|1|1|2|0|84 ATGAAAGACATTGGTACCCCAGAGGCATGGCAGATAATGATGTCCCTCAAGTCTGGACTC TTGGCAGAGATCACATGGGCTTTAGACACCATTAACATTCTACTGTATGATGACAGCAGC ATTATGACCTTCAACCTCAGTCAGTTCCCAGGATTGCTAGAGCTCTTTGAGTATGAGGTG GGTGACCGAAGACAGAGAACTCTACTGGACTCTGGGAGATTCAGTGAAGTGTCTGGTCCA ACCCCTACAGAG Thanks Best Quanwei 2017-02-13 10:16 GMT-05:00 Quanwei Zhang : > Hello: > > I am trying to add putative gene function to the predicted gene models. > Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. > I used canonical and isoform proteins of human, mouse and rat with the > script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose > context is as below. > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN > 69.97 303 91 0 1 303 1 303 7e-164 464 > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 > sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 > 3e-80 236 > > After that, I am trying to add the protein homology data to the Maker gff3 > and fasta files with maker_functional_gff and maker_functional_fasta, but > get the reports as below. > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 39. > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 39. > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 45. > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 45. > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > ..... > > I am not sure how to deal with this. I followed the command given in the > protocol. Any suggestions? > > Thanks > > Best > Quanwei > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 10:03:13 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 12:03:13 -0500 Subject: [maker-devel] Which version of InterProScan should I use Message-ID: I am planning to add domain information to my predicted gene models. But not sure whether the scripts provided by Maker is compatible with the latest version of InterProScan, or should I use a old version. I am using maker2.31.9. It seems the latest version is 5.22-61.0 released on 1/23/17. ftp://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/ Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcui at age.mpg.de Tue Feb 14 05:38:27 2017 From: rcui at age.mpg.de (Ray Cui) Date: Tue, 14 Feb 2017 13:38:27 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints Message-ID: Hello, I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Tue Feb 14 07:45:29 2017 From: dence at genetics.utah.edu (Daniel Ence) Date: Tue, 14 Feb 2017 14:45:29 +0000 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: Message-ID: Hi Ray, I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. ~Daniel On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: Hello, I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcui at age.mpg.de Tue Feb 14 08:44:19 2017 From: rcui at age.mpg.de (Ray Cui) Date: Tue, 14 Feb 2017 16:44:19 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> Message-ID: Hi Daniel, thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel wrote: > Hi Ray, > > I think you?re on the right track with training Genemark with RNAseq data. > It should only change the training steps, which are external to MAKER, but > not how MAKER runs Genemark. You?ll still give MAKER the path to the > ?es.mod" file made by Genemark. > > For the 2nd question, in the MAKER beta 3, MAKER creates a control file > for EVM, in which you set your weights for the various inputs, and then > MAKER runs EVM alongside all the other gene predictors and chooses the > model that is best supported by the evidence. > > ~Daniel > > > > On Feb 14, 2017, at 7:38 AM, Ray Cui wrote: > > Hello, > > I have sucessfully installed Maker beta 3, working with both > Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio > predictor. > When I read the GeneMark-ES manual, it says that one can use > RNAseq data to aid training. I'm wondering what would be the best way to > integrate Genemark-ET predictions into Maker. Should I run Genemark-ET > independent of Maker, then integrate the GFF at some point during the maker > process? If so, how should I edit the configuration file? Currently maker > has an option called "gmhmm". Should I then train GeneMark by myself with > RNAseq data, then feed the hmm to maker? > > And perhaps an unrelated question is that now Maker beta 3 > supports EVM. I'm wondering how EVM is used by Maker (at which step, what > does it do), and how does it differ from what Maker is designed for (both > reconciles different gene models). > > Best Regards, > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for > Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 <+49%20221%20496> > Mobile: +49 0221 37970 496 > rcui at age.mpg.de > www.age.mpg.de > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Tue Feb 14 12:51:46 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Tue, 14 Feb 2017 14:51:46 -0500 Subject: [maker-devel] Which version of InterProScan should I use In-Reply-To: References: Message-ID: <6A140E2D-603E-46C9-97FA-D59AEAEF95A9@gmail.com> I have been able to make the accessory scripts work with output from InterProScan v5 that I downloaded a couple of month ago. Mike > On Feb 13, 2017, at 12:03 PM, Quanwei Zhang wrote: > > I am planning to add domain information to my predicted gene models. But not sure whether the scripts provided by Maker is compatible with the latest version of InterProScan, or should I use a old version. I am using maker2.31.9. > > It seems the latest version is 5.22-61.0 released on 1/23/17. > ftp://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/ > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Thu Feb 16 03:44:39 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Thu, 16 Feb 2017 11:44:39 +0100 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> Message-ID: <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> Hi! Unfortunately all of the options failed on our cluster. See: Hi, Most recent Maker test with --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 Error: --> rank=2, hostname=uc1n518.localdomain [uc1n518:67009] *** Process received signal *** [uc1n518:67009] Signal: Segmentation fault (11) [uc1n518:67009] Signal code: Address not mapped (1) [uc1n518:67009] Failing at address: 0x4b0 -------------------------------------------------------------------------- mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). With: --mca btl ^openib and also this --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 Error: ### Runing Maker example STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... [uc1n514:59985] *** Process received signal *** [uc1n514:59985] Signal: Segmentation fault (11) [uc1n514:59985] Signal code: Address not mapped (1) [uc1n514:59985] Failing at address: 0x4b0 -------------------------------------------------------------------------- mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). Am 28.01.2017 um 21:53 schrieb Carson Holt: > Try adding one of the following to your mpiexec command ?> > > 1. --mca btl ^openib > 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 > 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 > > One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. > > --Carson > > >> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >> >> Hi everybody. >> >> My name is Rainer. I am an administrator for our HPC-Systems at our >> university in Konstanz, Baden-Wuertemberg/Germany. >> The procect is called bwHPC-C5. >> >> See: https://www.bwhpc-c5.de/en/index.php >> >> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >> i get errors while running a Maker job in the MPI-environment. >> >> BUILD STATUS >> >> ============================================================================== >> STATUS MAKER v2.31.9 >> ============================================================================== >> PERL Dependencies: VERIFIED >> External Programs: VERIFIED >> External C Libraries: VERIFIED >> MPI SUPPORT: ENABLED >> MWAS Web Interface: DISABLED >> MAKER PACKAGE: CONFIGURATION OK >> >> MODULES / INCLUDES / COMPILERS >> >> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >> # >> ##### (B) Dependencies: >> # >> # conflict: any other maker version >> # module load compiler/gnu/5.2 >> # module load mpi/openmpi/2.0-gnu-5.2 >> [...] >> >> MPI/MOAB SUBMIT >> >> [...] >> ### Queues ### >> #MSUB -q fat >> #MSUB -l nodes=1:ppn=16 >> #MSUB -l mem=20gb >> #MSUB -l walltime=50:00:00 >> # >> [...] >> echo " " >> echo "### Loading MAKER module:" >> echo " " >> module load bio/maker/2.31.9 >> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >> echo "MAKER_VERSION = $MAKER_VERSION" >> module list >> [...] >> echo " " >> echo "### Runing Maker example" >> echo " " >> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >> export OMPI_MCA_mpi_warn_on_fork=0 >> >> echo "LD_PRELOAD=${LD_PRELOAD}" >> # >> # "STATUS: Processing and indexing input FASTA files..." >> # >> mpiexec -mca btl ^openib -n 16 maker >> [...] >> >> >> E R R O R S >> ======= >> [...] >> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >> STATUS: Parsing control files... >> STATUS: Processing and indexing input FASTA files... >> [uc1n338:113607] *** Process received signal *** >> [uc1n338:113607] Signal: Segmentation fault (11) >> [uc1n338:113607] Signal code: Address not mapped (1) >> [uc1n338:113607] Failing at address: 0x4b0 >> [uc1n338:113608] *** Process received signal *** >> [uc1n338:113608] Signal: Segmentation fault (11) >> [uc1n338:113608] Signal code: Address not mapped (1) >> [uc1n338:113608] Failing at address: 0x4b0 >> [uc1n338:113621] *** Process received signal *** >> [uc1n338:113621] Signal: Segmentation fault (11) >> [uc1n338:113621] Signal code: Address not mapped (1) >> [uc1n338:113621] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >> -------------------------------------------------------------------------- >> [...] >> >> WHATS WRONG HERE!? >> >> Thank you for your help! >> >> All the best , >> >> Rainer >> >> -- >> Rainer Rutka >> University of Konstanz >> Communication, Information, Media Centre (KIM) >> * High-Performance-Computing (HPC) >> * KIM-Support and -Base-Services >> Room: V511 >> 78457 Konstanz, Germany >> +49 7531 88-5413 >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- Rainer Rutka Universit?t Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Raum: V511, Tel: 54 13 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From munholl at uwindsor.ca Fri Feb 17 13:11:44 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Fri, 17 Feb 2017 15:11:44 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there Message-ID: Hi Everyone, After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. However I also see $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 17 13:39:33 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 17 Feb 2017 13:39:33 -0700 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: Message-ID: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. Thanks, Carson > On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: > > Hi Everyone, > > After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl > > (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: > > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > However I also see > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 17 14:57:05 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 17 Feb 2017 16:57:05 -0500 Subject: [maker-devel] default gene model by quality_filter.pl is the same with those by setting keep_preds=0? Message-ID: Hello: I am following the Maker protocol (2014) to rescue rejected gene models. I set keep_preds=1 when I ran Maker, and then used "quality_filter.pl -d" to get the "default" Maker report. According to the annotation it will "Prints transcripts with an AED <1". But in a previous post (the link below), it was said "there are models with AED less than 1 that get rejected for other reasons. ... For example if the only evidence is a protein alignment that has deep overlapping HSPs (extremely low complexity alignment) it will be filtered out even though AED is not technically equal to 1. Also if the overlapping protein evidence is in a different reading frame than the model it is supposed to support then the AED will be less than 1 but eAED will be 1 (extended AED), and the model will be rejected." *So I wonder, whether the "default" result obtained by "quality_filter.pl " are really the same as those achieved by setting keep_preds=0? Will the script also consider other reasons besides "* *AED <1"?* https://groups.google.com/forum/#!searchin/maker-devel/The$20reason$20there$20are$20differences$20between$20the$20runs$20is$20that$20there$20are$20models$20with$20AED$20less$20%7Csort:relevance/maker-devel/97aNJkT3bgk/W0MEyqEZAAAJ Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qlian003 at ucr.edu Sat Feb 18 09:43:08 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Sat, 18 Feb 2017 08:43:08 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation Message-ID: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> Dear Maker develop team, I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? Thanks Qihua From carsonhh at gmail.com Sun Feb 19 22:39:14 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 22:39:14 -0700 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> Message-ID: <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> MAKER was support was designed with GeneMark-ES. It may or may not work with GeneMark-ET. So any MAKER related archive posts etc. will be related to the latter. With GeneMark-ES, you simply provided a genome assembly and let it run. It would then produce several files and output directories. The es.mod file was the one you provided to MAKER. I don?t know how this compares to GeneMark-ET. ?Carson > On Feb 14, 2017, at 8:44 AM, Ray Cui wrote: > > Hi Daniel, > > thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? > > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 > Mobile: +49 0221 37970 496 <> > rcui at age.mpg.de > www.age.mpg.de > > > > On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel > wrote: > Hi Ray, > > I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. > > For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. > > ~Daniel > > > >> On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: >> >> Hello, >> >> I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. >> When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? >> >> And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). >> >> Best Regards, >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 >> Mobile: +49 0221 37970 496 <> >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Sun Feb 19 22:43:49 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 22:43:49 -0700 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> Message-ID: <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> Try running just on a single node (not across nodes). If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and running MAKER with that new version. You can install it in your home directory and test from there, just make sure to add it to your path. Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. ?Carson > On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: > > Hi! > > Unfortunately all of the options failed on our cluster. > > See: > > Hi, > > Most recent Maker test with > --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 > Error: > --> rank=2, hostname=uc1n518.localdomain > [uc1n518:67009] *** Process received signal *** > [uc1n518:67009] Signal: Segmentation fault (11) > [uc1n518:67009] Signal code: Address not mapped (1) > [uc1n518:67009] Failing at address: 0x4b0 > -------------------------------------------------------------------------- > mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). > > > With: > --mca btl ^openib > and also this > --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 > Error: > ### Runing Maker example > > STATUS: Parsing control files... > STATUS: Processing and indexing input FASTA files... > [uc1n514:59985] *** Process received signal *** > [uc1n514:59985] Signal: Segmentation fault (11) > [uc1n514:59985] Signal code: Address not mapped (1) > [uc1n514:59985] Failing at address: 0x4b0 > -------------------------------------------------------------------------- > mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). > > > Am 28.01.2017 um 21:53 schrieb Carson Holt: >> Try adding one of the following to your mpiexec command ?> >> >> 1. --mca btl ^openib >> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >> >> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >> >> --Carson >> >> >>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>> >>> Hi everybody. >>> >>> My name is Rainer. I am an administrator for our HPC-Systems at our >>> university in Konstanz, Baden-Wuertemberg/Germany. >>> The procect is called bwHPC-C5. >>> >>> See: https://www.bwhpc-c5.de/en/index.php >>> >>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>> i get errors while running a Maker job in the MPI-environment. >>> >>> BUILD STATUS >>> >>> ============================================================================== >>> STATUS MAKER v2.31.9 >>> ============================================================================== >>> PERL Dependencies: VERIFIED >>> External Programs: VERIFIED >>> External C Libraries: VERIFIED >>> MPI SUPPORT: ENABLED >>> MWAS Web Interface: DISABLED >>> MAKER PACKAGE: CONFIGURATION OK >>> >>> MODULES / INCLUDES / COMPILERS >>> >>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>> # >>> ##### (B) Dependencies: >>> # >>> # conflict: any other maker version >>> # module load compiler/gnu/5.2 >>> # module load mpi/openmpi/2.0-gnu-5.2 >>> [...] >>> >>> MPI/MOAB SUBMIT >>> >>> [...] >>> ### Queues ### >>> #MSUB -q fat >>> #MSUB -l nodes=1:ppn=16 >>> #MSUB -l mem=20gb >>> #MSUB -l walltime=50:00:00 >>> # >>> [...] >>> echo " " >>> echo "### Loading MAKER module:" >>> echo " " >>> module load bio/maker/2.31.9 >>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>> echo "MAKER_VERSION = $MAKER_VERSION" >>> module list >>> [...] >>> echo " " >>> echo "### Runing Maker example" >>> echo " " >>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>> export OMPI_MCA_mpi_warn_on_fork=0 >>> >>> echo "LD_PRELOAD=${LD_PRELOAD}" >>> # >>> # "STATUS: Processing and indexing input FASTA files..." >>> # >>> mpiexec -mca btl ^openib -n 16 maker >>> [...] >>> >>> >>> E R R O R S >>> ======= >>> [...] >>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>> STATUS: Parsing control files... >>> STATUS: Processing and indexing input FASTA files... >>> [uc1n338:113607] *** Process received signal *** >>> [uc1n338:113607] Signal: Segmentation fault (11) >>> [uc1n338:113607] Signal code: Address not mapped (1) >>> [uc1n338:113607] Failing at address: 0x4b0 >>> [uc1n338:113608] *** Process received signal *** >>> [uc1n338:113608] Signal: Segmentation fault (11) >>> [uc1n338:113608] Signal code: Address not mapped (1) >>> [uc1n338:113608] Failing at address: 0x4b0 >>> [uc1n338:113621] *** Process received signal *** >>> [uc1n338:113621] Signal: Segmentation fault (11) >>> [uc1n338:113621] Signal code: Address not mapped (1) >>> [uc1n338:113621] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>> -------------------------------------------------------------------------- >>> [...] >>> >>> WHATS WRONG HERE!? >>> >>> Thank you for your help! >>> >>> All the best , >>> >>> Rainer >>> >>> -- >>> Rainer Rutka >>> University of Konstanz >>> Communication, Information, Media Centre (KIM) >>> * High-Performance-Computing (HPC) >>> * KIM-Support and -Base-Services >>> Room: V511 >>> 78457 Konstanz, Germany >>> +49 7531 88-5413 >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -- > Rainer Rutka > Universit?t Konstanz > Kommunikations-, Informations-, Medienzentrum (KIM) > Raum: V511, Tel: 54 13 > From carsonhh at gmail.com Sun Feb 19 23:05:09 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 23:05:09 -0700 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> Message-ID: <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. Thanks, Carson > On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: > > Dear Maker develop team, > > I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? > > Thanks > Qihua > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From rcui at age.mpg.de Mon Feb 20 01:59:12 2017 From: rcui at age.mpg.de (Ray Cui) Date: Mon, 20 Feb 2017 09:59:12 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> Message-ID: Dear Carson, I have now run GeneMark-ET, and it produces a trained .mod file. I think it can be then passed to Maker. Do you know what is the final constructed command line in Maker that calls genemark? Genemark-et and es use the same perl script so one probably only needs to use the --prediction and --predict_with xxx.mod options to predict genes using the species specific parameters (bypassing regular training and prediction steps) Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de On Mon, Feb 20, 2017 at 6:39 AM, Carson Holt wrote: > MAKER was support was designed with GeneMark-ES. It may or may not work > with GeneMark-ET. So any MAKER related archive posts etc. will be related > to the latter. > > With GeneMark-ES, you simply provided a genome assembly and let it run. It > would then produce several files and output directories. The es.mod file > was the one you provided to MAKER. I don?t know how this compares > to GeneMark-ET. > > ?Carson > > > > On Feb 14, 2017, at 8:44 AM, Ray Cui wrote: > > Hi Daniel, > > thanks! It seems that Genemark-ET has a "--training" flag, is that > the flag I should use when training or should I just let Genemark also > perform the prediction? > > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for > Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 <+49%20221%20496> > Mobile: +49 0221 37970 496 > rcui at age.mpg.de > www.age.mpg.de > > > > On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel wrote: > >> Hi Ray, >> >> I think you?re on the right track with training Genemark with RNAseq >> data. It should only change the training steps, which are external to >> MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to >> the ?es.mod" file made by Genemark. >> >> For the 2nd question, in the MAKER beta 3, MAKER creates a control file >> for EVM, in which you set your weights for the various inputs, and then >> MAKER runs EVM alongside all the other gene predictors and chooses the >> model that is best supported by the evidence. >> >> ~Daniel >> >> >> >> On Feb 14, 2017, at 7:38 AM, Ray Cui wrote: >> >> Hello, >> >> I have sucessfully installed Maker beta 3, working with both >> Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio >> predictor. >> When I read the GeneMark-ES manual, it says that one can use >> RNAseq data to aid training. I'm wondering what would be the best way to >> integrate Genemark-ET predictions into Maker. Should I run Genemark-ET >> independent of Maker, then integrate the GFF at some point during the maker >> process? If so, how should I edit the configuration file? Currently maker >> has an option called "gmhmm". Should I then train GeneMark by myself with >> RNAseq data, then feed the hmm to maker? >> >> And perhaps an unrelated question is that now Maker beta 3 >> supports EVM. I'm wondering how EVM is used by Maker (at which step, what >> does it do), and how does it differ from what Maker is designed for (both >> reconciles different gene models). >> >> Best Regards, >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for >> Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 <+49%20221%20496> >> Mobile: +49 0221 37970 496 >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 20 02:02:00 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 02:02:00 -0700 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> Message-ID: The names of scripts used are listed in the maker_exe.ctl file. It depends on if formatting or any flags have changed between versions. ?Carson > On Feb 20, 2017, at 1:59 AM, Ray Cui wrote: > > Dear Carson, > > I have now run GeneMark-ET, and it produces a trained .mod file. I think it can be then passed to Maker. Do you know what is the final constructed command line in Maker that calls genemark? Genemark-et and es use the same perl script so one probably only needs to use the --prediction and --predict_with xxx.mod options to predict genes using the species specific parameters (bypassing regular training and prediction steps) > > > Best Regards, > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 > Mobile: +49 0221 37970 496 <> > rcui at age.mpg.de > www.age.mpg.de > > > > On Mon, Feb 20, 2017 at 6:39 AM, Carson Holt > wrote: > MAKER was support was designed with GeneMark-ES. It may or may not work with GeneMark-ET. So any MAKER related archive posts etc. will be related to the latter. > > With GeneMark-ES, you simply provided a genome assembly and let it run. It would then produce several files and output directories. The es.mod file was the one you provided to MAKER. I don?t know how this compares to GeneMark-ET. > > ?Carson > > > >> On Feb 14, 2017, at 8:44 AM, Ray Cui > wrote: >> >> Hi Daniel, >> >> thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? >> >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 >> Mobile: +49 0221 37970 496 <> >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> >> On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel > wrote: >> Hi Ray, >> >> I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. >> >> For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. >> >> ~Daniel >> >> >> >>> On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: >>> >>> Hello, >>> >>> I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. >>> When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? >>> >>> And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). >>> >>> Best Regards, >>> Ray >>> >>> Dr. Rongfeng (Ray) Cui >>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >>> Wissenschaftlicher MA / Postdoctoral researcher >>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >>> Tel.:+49 (0)221 496 >>> Mobile: +49 0221 37970 496 <> >>> rcui at age.mpg.de >>> www.age.mpg.de >>> >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qlian003 at ucr.edu Mon Feb 20 13:34:31 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Mon, 20 Feb 2017 12:34:31 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> Message-ID: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Hi Carson, Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? Thanks Qihua > On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: > > IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. > > CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. > > If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. > > What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. > > However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. > > Thanks, > Carson > >> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >> >> Dear Maker develop team, >> >> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >> >> Thanks >> Qihua >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > From carsonhh at gmail.com Mon Feb 20 16:56:01 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 16:56:01 -0700 Subject: [maker-devel] default gene model by quality_filter.pl is the same with those by setting keep_preds=0? In-Reply-To: References: Message-ID: <14B6C165-54D2-4722-9A26-91FBEBB18B72@gmail.com> Any hint based predictions rejected for one of the non-AED filters will still be rejected regardless of keep_preds=1. However you are right in that any raw predictions that were rejected by other filters may still be kept by using this workflow, since keep_preds=1 essentially rescues all non-overlapping raw predictions that would have been rejected. ?Carson > On Feb 17, 2017, at 2:57 PM, Quanwei Zhang wrote: > > Hello: > > I am following the Maker protocol (2014) to rescue rejected gene models. I set keep_preds=1 when I ran Maker, and then used "quality_filter.pl -d" to get the "default" Maker report. According to the annotation it will "Prints transcripts with an AED <1". > > But in a previous post (the link below), it was said "there are models with AED less than 1 that get rejected for other reasons. ... For example if the only evidence is a protein alignment that has deep overlapping HSPs (extremely low complexity alignment) it will be filtered out even though AED is not technically equal to 1. Also if the overlapping protein evidence is in a different reading frame than the model it is supposed to support then the AED will be less than 1 but eAED will be 1 (extended AED), and the model will be rejected." > > So I wonder, whether the "default" result obtained by "quality_filter.pl " are really the same as those achieved by setting keep_preds=0? Will the script also consider other reasons besides "AED <1"? > > https://groups.google.com/forum/#!searchin/maker-devel/The$20reason$20there$20are$20differences$20between$20the$20runs$20is$20that$20there$20are$20models$20with$20AED$20less$20%7Csort:relevance/maker-devel/97aNJkT3bgk/W0MEyqEZAAAJ > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 20 16:56:02 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 16:56:02 -0700 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: You either uses TrEMBL or the UniProtKB Isoform sequence set. Their fasta headers are slightly different and will not be parsed correctly, For example, here is the header as formatted for the same sequence in the Swiss-prot dataset download ?> >sp|Q7Z5M8|AB12B_HUMAN Protein ABHD12B OS=Homo sapiens GN=ABHD12B PE=2 SV=1 I think you used the UniProtKB Isoform sequence dataset instead. ?Carson > On Feb 13, 2017, at 8:16 AM, Quanwei Zhang wrote: > > Hello: > > I am trying to add putative gene function to the predicted gene models. Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. I used canonical and isoform proteins of human, mouse and rat with the script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose context is as below. > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 7e-164 464 > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 3e-80 236 > > After that, I am trying to add the protein homology data to the Maker gff3 and fasta files with maker_functional_gff and maker_functional_fasta, but get the reports as below. > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 39. > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 39. > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 45. > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 45. > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > ..... > > I am not sure how to deal with this. I followed the command given in the protocol. Any suggestions? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From qwzhang0601 at gmail.com Tue Feb 21 07:30:35 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Tue, 21 Feb 2017 09:30:35 -0500 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: Thank you! I used both canonical and isoform of protein sequences from swiss-prot in the beginning. It reported the error, but later I only used the canonical protein sequences to build the database and then it worked. Best Quanwei 2017-02-20 18:56 GMT-05:00 Carson Holt : > You either uses TrEMBL or the UniProtKB Isoform sequence set. Their fasta > headers are slightly different and will not be parsed correctly, > > For example, here is the header as formatted for the same sequence in the > Swiss-prot dataset download ?> > >sp|Q7Z5M8|AB12B_HUMAN Protein ABHD12B OS=Homo sapiens GN=ABHD12B PE=2 SV=1 > > I think you used the UniProtKB Isoform sequence dataset instead. > > ?Carson > > > > > > > On Feb 13, 2017, at 8:16 AM, Quanwei Zhang > wrote: > > > > Hello: > > > > I am trying to add putative gene function to the predicted gene models. > Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. > I used canonical and isoform proteins of human, mouse and rat with the > script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose > context is as below. > > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 > sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 > 7e-164 464 > > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 > sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 > 3e-80 236 > > > > After that, I am trying to add the protein homology data to the Maker > gff3 and fasta files with maker_functional_gff and maker_functional_fasta, > but get the reports as below. > > > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform > 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 39. > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 39. > > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform > 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 45. > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 45. > > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform > 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > ..... > > > > I am not sure how to deal with this. I followed the command given in the > protocol. Any suggestions? > > > > Thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Tue Feb 21 10:17:22 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 12:17:22 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Message-ID: I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: cpan[1]> install forks::signals Warning: Cannot install forks::signals, don't know what it is. Try the command i /forks::signals/ to find objects with matching identifiers. but scrolling through the install scroll of the forks reinstall I do see it got installed properly. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: > Do a force reinstall of forks via CPAN. The error is coming from forks.pm line > 83, so the problem is the forks installation itself. > > Thanks, > Carson > > > On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: > > Hi Everyone, > > After sorting my MPICH/OpenMPI issue I have come across another: When I > try to run MAKER on the demo data via > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker > /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl > /Data/Apps/maker/data/maker_bopts.ctl > > (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it > working before opening up, if I change the -n value then the error repeats > once for each process I attempt to run via MPICH) I got the following: > > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > However I also see > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially > thought this was a Perl error. I hit up perlmonks and found that there is > an issue between forks and Storable, where Storable now has a verion number > of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these > instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and > tried again. Now I get: > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker > /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl > /Data/Apps/maker/data/maker_bopts.ctl > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > Clearly MAKER has an issue with forks, but it is installed, it is up to > date (I double checked via cpan and apt-get to be sure), and it is in a > directory that is in @INC. Should I pursue this as a MAKER error or as a > Perl error? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 21 10:19:30 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 21 Feb 2017 10:19:30 -0700 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Message-ID: <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> forks::signals is part of forks. It?s not a separate package. If it?s missing, there is a problem with the forks installation. So you need to do a force install of forks to force the reinstall. Example ?> cpan[1]> force install forks ?Carson > On Feb 21, 2017, at 10:17 AM, Seth Munholland wrote: > > I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: > > cpan[1]> install forks::signals > Warning: Cannot install forks::signals, don't know what it is. > Try the command > > i /forks::signals/ > > to find objects with matching identifiers. > > but scrolling through the install scroll of the forks reinstall I do see it got installed properly. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt > wrote: > Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. > > Thanks, > Carson > > >> On Feb 17, 2017, at 1:11 PM, Seth Munholland > wrote: >> >> Hi Everyone, >> >> After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl >> >> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: >> >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> However I also see >> $ locate forks.pm >> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >> >> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Tue Feb 21 11:05:48 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 13:05:48 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: I wasn't sure what modules were separated (figured it was wroth a shot), however, I did the force install of forks and I still get the same error. I tried uninstalling and reinstalling all the forks installations that showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then reloaded) to make sure I wasn't getting some kind of conflicting libraries/modules. and I'm back to: Can't locate forks.pm in @INC (you may need to install the forks module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. along with: $ sudo updatedb $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Now it's a maker issue, not a forks issue (if I'm reading it correctly), so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) and got: $ sudo ./Build install Installing MAKER... Configuring MAKER with MPI support Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 (unchanged) I don't believe that's an error or even a warning, but I get the same "can't locate forks" error when I try to run it. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt wrote: > forks::signals is part of forks. It?s not a separate package. If it?s > missing, there is a problem with the forks installation. So you need to do > a force install of forks to force the reinstall. > > Example ?> cpan[1]> force install forks > > ?Carson > > > On Feb 21, 2017, at 10:17 AM, Seth Munholland wrote: > > I tried this and still get the same error. Then I tried forcing a > reinstall of forks/signals and got: > > cpan[1]> install forks::signals > Warning: Cannot install forks::signals, don't know what it is. > Try the command > > i /forks::signals/ > > to find objects with matching identifiers. > > but scrolling through the install scroll of the forks reinstall I do see > it got installed properly. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: > >> Do a force reinstall of forks via CPAN. The error is coming from forks.pm line >> 83, so the problem is the forks installation itself. >> >> Thanks, >> Carson >> >> >> On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: >> >> Hi Everyone, >> >> After sorting my MPICH/OpenMPI issue I have come across another: When I >> try to run MAKER on the demo data via >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >> /Data/Apps/maker/data/maker_bopts.ctl >> >> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it >> working before opening up, if I change the -n value then the error repeats >> once for each process I attempt to run via MPICH) I got the following: >> >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> However I also see >> $ locate forks.pm >> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >> >> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially >> thought this was a Perl error. I hit up perlmonks and found that there is >> an issue between forks and Storable, where Storable now has a verion number >> of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these >> instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and >> tried again. Now I get: >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >> /Data/Apps/maker/data/maker_bopts.ctl >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> Clearly MAKER has an issue with forks, but it is installed, it is up to >> date (I double checked via cpan and apt-get to be sure), and it is in a >> directory that is in @INC. Should I pursue this as a MAKER error or as a >> Perl error? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bmoore at genetics.utah.edu Tue Feb 21 11:53:13 2017 From: bmoore at genetics.utah.edu (Barry Moore) Date: Tue, 21 Feb 2017 18:53:13 +0000 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: <78C26338-01EF-4F72-BFA4-EF46C70AFAEE@genetics.utah.edu> Hi Qihua, If you are using online version of SOBA, I would suggest you use the command line version found here https://github.com/The-Sequence-Ontology/SOBA as it is more flexible for the kinds of analyses you are talking about. If you are using ?footprint? as the --data_type argument you should get the nucleotide count for collapsed features that you are talking about. In addition I suggest you take a look at bedtools (http://bedtools.readthedocs.io/en/latest/index.html) for example bedtools merge as a flexible way to generate the kind of merged features you want and then you can always pass that output of that through SOBAcl for counting, graphing and reporting. Finally, if you want a great deal of flexibility in generating your own manipulation and reporting of GFF3 files that is beyond the scope of SOBA and/or BedTools, I suggest you take a look at the GAL library (https://github.com/The-Sequence-Ontology/GAL) if you don?t mind writing some perl code. Regards, Barry On Feb 20, 2017, at 1:34 PM, Qihua Liang > wrote: Hi Carson, Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? Thanks Qihua On Feb 19, 2017, at 10:05 PM, Carson Holt > wrote: IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. Thanks, Carson On Feb 18, 2017, at 9:43 AM, Qihua Liang > wrote: Dear Maker develop team, I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? Thanks Qihua _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 21 14:34:07 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 21 Feb 2017 14:34:07 -0700 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: MAKER merges overlapping RepeatMasker results into a single longer feature. ?Carson > On Feb 20, 2017, at 1:34 PM, Qihua Liang wrote: > > Hi Carson, > > Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. > > I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. > > Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. > In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. > > But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. > > When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? > Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? > > Thanks > Qihua > > >> On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: >> >> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. >> >> CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. >> >> If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. >> >> What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. >> >> However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. >> >> Thanks, >> Carson >> >>> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >>> >>> Dear Maker develop team, >>> >>> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >>> >>> Thanks >>> Qihua >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > From munholl at uwindsor.ca Tue Feb 21 15:26:23 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 17:26:23 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: I found my errors. 1) In my machine file; the first entry was not the main node, so locate was looking on a different node than Maker 2) forks (and other modules) were not properly installed on every node in the cluster. I used CPAN to install them on each cluster and it is now working. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 1:05 PM, Seth Munholland wrote: > I wasn't sure what modules were separated (figured it was wroth a shot), > however, I did the force install of forks and I still get the same error. > I tried uninstalling and reinstalling all the forks installations that > showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then > reloaded) to make sure I wasn't getting some kind of conflicting > libraries/modules. and I'm back to: > > Can't locate forks.pm in @INC (you may need to install the forks module) > (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > > along with: > > $ sudo updatedb > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Now it's a maker issue, not a forks issue (if I'm reading it correctly), > so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) > and got: > > $ sudo ./Build install > Installing MAKER... > Configuring MAKER with MPI support > Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 > (unchanged) > > I don't believe that's an error or even a warning, but I get the same > "can't locate forks" error when I try to run it. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt wrote: > >> forks::signals is part of forks. It?s not a separate package. If it?s >> missing, there is a problem with the forks installation. So you need to do >> a force install of forks to force the reinstall. >> >> Example ?> cpan[1]> force install forks >> >> ?Carson >> >> >> On Feb 21, 2017, at 10:17 AM, Seth Munholland >> wrote: >> >> I tried this and still get the same error. Then I tried forcing a >> reinstall of forks/signals and got: >> >> cpan[1]> install forks::signals >> Warning: Cannot install forks::signals, don't know what it is. >> Try the command >> >> i /forks::signals/ >> >> to find objects with matching identifiers. >> >> but scrolling through the install scroll of the forks reinstall I do see >> it got installed properly. >> >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> >> On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: >> >>> Do a force reinstall of forks via CPAN. The error is coming from >>> forks.pm line 83, so the problem is the forks installation itself. >>> >>> Thanks, >>> Carson >>> >>> >>> On Feb 17, 2017, at 1:11 PM, Seth Munholland >>> wrote: >>> >>> Hi Everyone, >>> >>> After sorting my MPICH/OpenMPI issue I have come across another: When I >>> try to run MAKER on the demo data via >>> >>> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >>> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >>> /Data/Apps/maker/data/maker_bopts.ctl >>> >>> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it >>> working before opening up, if I change the -n value then the error repeats >>> once for each process I attempt to run via MPICH) I got the following: >>> >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> >>> However I also see >>> $ locate forks.pm >>> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >>> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >>> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >>> >>> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially >>> thought this was a Perl error. I hit up perlmonks and found that there is >>> an issue between forks and Storable, where Storable now has a verion number >>> of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these >>> instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and >>> tried again. Now I get: >>> >>> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >>> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >>> /Data/Apps/maker/data/maker_bopts.ctl >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> >>> Clearly MAKER has an issue with forks, but it is installed, it is up to >>> date (I double checked via cpan and apt-get to be sure), and it is in a >>> directory that is in @INC. Should I pursue this as a MAKER error or as a >>> Perl error? >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Wed Feb 22 06:11:32 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Wed, 22 Feb 2017 14:11:32 +0100 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> Message-ID: <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> @Robert Kraus: FYI Am 20.02.2017 um 06:43 schrieb Carson Holt: > Try running just on a single node (not across nodes). THATS WHAT I DID. > If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and > running MAKER with that new version. You can install it in your home directory and test from there, > just make sure to add it to your path. Shure it is. > Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. > If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. OK, send the infos please. Thank you very much! > ?Carson > > > >> On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: >> >> Hi! >> >> Unfortunately all of the options failed on our cluster. >> >> See: >> >> Hi, >> >> Most recent Maker test with >> --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >> Error: >> --> rank=2, hostname=uc1n518.localdomain >> [uc1n518:67009] *** Process received signal *** >> [uc1n518:67009] Signal: Segmentation fault (11) >> [uc1n518:67009] Signal code: Address not mapped (1) >> [uc1n518:67009] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). >> >> >> With: >> --mca btl ^openib >> and also this >> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >> Error: >> ### Runing Maker example >> >> STATUS: Parsing control files... >> STATUS: Processing and indexing input FASTA files... >> [uc1n514:59985] *** Process received signal *** >> [uc1n514:59985] Signal: Segmentation fault (11) >> [uc1n514:59985] Signal code: Address not mapped (1) >> [uc1n514:59985] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). >> >> >> Am 28.01.2017 um 21:53 schrieb Carson Holt: >>> Try adding one of the following to your mpiexec command ?> >>> >>> 1. --mca btl ^openib >>> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>> >>> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >>> >>> --Carson >>> >>> >>>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>>> >>>> Hi everybody. >>>> >>>> My name is Rainer. I am an administrator for our HPC-Systems at our >>>> university in Konstanz, Baden-Wuertemberg/Germany. >>>> The procect is called bwHPC-C5. >>>> >>>> See: https://www.bwhpc-c5.de/en/index.php >>>> >>>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>>> i get errors while running a Maker job in the MPI-environment. >>>> >>>> BUILD STATUS >>>> >>>> ============================================================================== >>>> STATUS MAKER v2.31.9 >>>> ============================================================================== >>>> PERL Dependencies: VERIFIED >>>> External Programs: VERIFIED >>>> External C Libraries: VERIFIED >>>> MPI SUPPORT: ENABLED >>>> MWAS Web Interface: DISABLED >>>> MAKER PACKAGE: CONFIGURATION OK >>>> >>>> MODULES / INCLUDES / COMPILERS >>>> >>>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>>> # >>>> ##### (B) Dependencies: >>>> # >>>> # conflict: any other maker version >>>> # module load compiler/gnu/5.2 >>>> # module load mpi/openmpi/2.0-gnu-5.2 >>>> [...] >>>> >>>> MPI/MOAB SUBMIT >>>> >>>> [...] >>>> ### Queues ### >>>> #MSUB -q fat >>>> #MSUB -l nodes=1:ppn=16 >>>> #MSUB -l mem=20gb >>>> #MSUB -l walltime=50:00:00 >>>> # >>>> [...] >>>> echo " " >>>> echo "### Loading MAKER module:" >>>> echo " " >>>> module load bio/maker/2.31.9 >>>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>>> echo "MAKER_VERSION = $MAKER_VERSION" >>>> module list >>>> [...] >>>> echo " " >>>> echo "### Runing Maker example" >>>> echo " " >>>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>>> export OMPI_MCA_mpi_warn_on_fork=0 >>>> >>>> echo "LD_PRELOAD=${LD_PRELOAD}" >>>> # >>>> # "STATUS: Processing and indexing input FASTA files..." >>>> # >>>> mpiexec -mca btl ^openib -n 16 maker >>>> [...] >>>> >>>> >>>> E R R O R S >>>> ======= >>>> [...] >>>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>>> STATUS: Parsing control files... >>>> STATUS: Processing and indexing input FASTA files... >>>> [uc1n338:113607] *** Process received signal *** >>>> [uc1n338:113607] Signal: Segmentation fault (11) >>>> [uc1n338:113607] Signal code: Address not mapped (1) >>>> [uc1n338:113607] Failing at address: 0x4b0 >>>> [uc1n338:113608] *** Process received signal *** >>>> [uc1n338:113608] Signal: Segmentation fault (11) >>>> [uc1n338:113608] Signal code: Address not mapped (1) >>>> [uc1n338:113608] Failing at address: 0x4b0 >>>> [uc1n338:113621] *** Process received signal *** >>>> [uc1n338:113621] Signal: Segmentation fault (11) >>>> [uc1n338:113621] Signal code: Address not mapped (1) >>>> [uc1n338:113621] Failing at address: 0x4b0 >>>> -------------------------------------------------------------------------- >>>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>>> -------------------------------------------------------------------------- >>>> [...] >>>> >>>> WHATS WRONG HERE!? >>>> >>>> Thank you for your help! >>>> >>>> All the best , >>>> >>>> Rainer >>>> >>>> -- >>>> Rainer Rutka >>>> University of Konstanz >>>> Communication, Information, Media Centre (KIM) >>>> * High-Performance-Computing (HPC) >>>> * KIM-Support and -Base-Services >>>> Room: V511 >>>> 78457 Konstanz, Germany >>>> +49 7531 88-5413 >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> -- >> Rainer Rutka >> Universit?t Konstanz >> Kommunikations-, Informations-, Medienzentrum (KIM) >> Raum: V511, Tel: 54 13 >> > -- Rainer Rutka Universit?t Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Raum: V511, Tel: 54 13 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From cjfields at illinois.edu Wed Feb 22 06:30:22 2017 From: cjfields at illinois.edu (Fields, Christopher J) Date: Wed, 22 Feb 2017 13:30:22 +0000 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: <6E5122C0-A952-42D6-B205-74B5FF8255F2@illinois.edu> Just a note: when we install MAKER we generally install a clean version of perl if one isn?t already present, making sure it is on the NFS share for the cluster (which is accessible via all nodes). This works around most of the issues you describe. chris From: maker-devel > on behalf of Seth Munholland > Date: Tuesday, February 21, 2017 at 4:26 PM To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] MAKER can't find forks in @INC, but it is there I found my errors. 1) In my machine file; the first entry was not the main node, so locate was looking on a different node than Maker 2) forks (and other modules) were not properly installed on every node in the cluster. I used CPAN to install them on each cluster and it is now working. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 1:05 PM, Seth Munholland > wrote: I wasn't sure what modules were separated (figured it was wroth a shot), however, I did the force install of forks and I still get the same error. I tried uninstalling and reinstalling all the forks installations that showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then reloaded) to make sure I wasn't getting some kind of conflicting libraries/modules. and I'm back to: Can't locate forks.pm in @INC (you may need to install the forks module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. along with: $ sudo updatedb $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Now it's a maker issue, not a forks issue (if I'm reading it correctly), so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) and got: $ sudo ./Build install Installing MAKER... Configuring MAKER with MPI support Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 (unchanged) I don't believe that's an error or even a warning, but I get the same "can't locate forks" error when I try to run it. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt > wrote: forks::signals is part of forks. It?s not a separate package. If it?s missing, there is a problem with the forks installation. So you need to do a force install of forks to force the reinstall. Example ?> cpan[1]> force install forks ?Carson On Feb 21, 2017, at 10:17 AM, Seth Munholland > wrote: I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: cpan[1]> install forks::signals Warning: Cannot install forks::signals, don't know what it is. Try the command i /forks::signals/ to find objects with matching identifiers. but scrolling through the install scroll of the forks reinstall I do see it got installed properly. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt > wrote: Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. Thanks, Carson On Feb 17, 2017, at 1:11 PM, Seth Munholland > wrote: Hi Everyone, After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. However I also see $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 09:16:17 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 09:16:17 -0700 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> Message-ID: <24606FBA-4742-4F94-9447-208A385644C1@gmail.com> If OpenMPI fails on a single node, it means you have a compilation issue, which indicates a problem with your installation. This sometimes happens if you compiled on one node and run on another (if could either be MAEKR or OpenMPI itself that was compiled on another node). A few options you will need if trying intel MPI: -binding pin=disable #requires to disable processor affinity (otherwise MAKER calls to BLAST and other programs which are parallelized independent of MPI may not work) Environmental variables to set: export I_MPI_PIN_DOMAIN=node #otherwise MAKER calls to BLAST and other programs which are parallelized independent of MPI may not work export I_MPI_FABRICS='shm:tcp? #avoid potential complication with OpenFabrics libraries (they block system calls because of how they use registered memory, i.e. MAKER calling BLAST would fail) export I_MPI_HYDRA_IFACE=ib0 #set to eth0 if you don?t have an infiniband over ip inerface (required because of the above I_MPI_FABRICS change) Also make sure to compile on the node you run. You can try expanding to other nodes after that. ?Carson > On Feb 22, 2017, at 6:11 AM, Rainer Rutka wrote: > > @Robert Kraus: FYI > > Am 20.02.2017 um 06:43 schrieb Carson Holt: >> Try running just on a single node (not across nodes). > THATS WHAT I DID. > > >> If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and >> running MAKER with that new version. You can install it in your home directory and test from there, >> just make sure to add it to your path. > Shure it is. > >> Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. > > If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. > > OK, send the infos please. > > Thank you very much! > >> ?Carson >> >> >> >>> On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: >>> >>> Hi! >>> >>> Unfortunately all of the options failed on our cluster. >>> >>> See: >>> >>> Hi, >>> >>> Most recent Maker test with >>> --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>> Error: >>> --> rank=2, hostname=uc1n518.localdomain >>> [uc1n518:67009] *** Process received signal *** >>> [uc1n518:67009] Signal: Segmentation fault (11) >>> [uc1n518:67009] Signal code: Address not mapped (1) >>> [uc1n518:67009] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). >>> >>> >>> With: >>> --mca btl ^openib >>> and also this >>> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>> Error: >>> ### Runing Maker example >>> >>> STATUS: Parsing control files... >>> STATUS: Processing and indexing input FASTA files... >>> [uc1n514:59985] *** Process received signal *** >>> [uc1n514:59985] Signal: Segmentation fault (11) >>> [uc1n514:59985] Signal code: Address not mapped (1) >>> [uc1n514:59985] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). >>> >>> >>> Am 28.01.2017 um 21:53 schrieb Carson Holt: >>>> Try adding one of the following to your mpiexec command ?> >>>> >>>> 1. --mca btl ^openib >>>> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>>> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>>> >>>> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >>>> >>>> --Carson >>>> >>>> >>>>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>>>> >>>>> Hi everybody. >>>>> >>>>> My name is Rainer. I am an administrator for our HPC-Systems at our >>>>> university in Konstanz, Baden-Wuertemberg/Germany. >>>>> The procect is called bwHPC-C5. >>>>> >>>>> See: https://www.bwhpc-c5.de/en/index.php >>>>> >>>>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>>>> i get errors while running a Maker job in the MPI-environment. >>>>> >>>>> BUILD STATUS >>>>> >>>>> ============================================================================== >>>>> STATUS MAKER v2.31.9 >>>>> ============================================================================== >>>>> PERL Dependencies: VERIFIED >>>>> External Programs: VERIFIED >>>>> External C Libraries: VERIFIED >>>>> MPI SUPPORT: ENABLED >>>>> MWAS Web Interface: DISABLED >>>>> MAKER PACKAGE: CONFIGURATION OK >>>>> >>>>> MODULES / INCLUDES / COMPILERS >>>>> >>>>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>>>> # >>>>> ##### (B) Dependencies: >>>>> # >>>>> # conflict: any other maker version >>>>> # module load compiler/gnu/5.2 >>>>> # module load mpi/openmpi/2.0-gnu-5.2 >>>>> [...] >>>>> >>>>> MPI/MOAB SUBMIT >>>>> >>>>> [...] >>>>> ### Queues ### >>>>> #MSUB -q fat >>>>> #MSUB -l nodes=1:ppn=16 >>>>> #MSUB -l mem=20gb >>>>> #MSUB -l walltime=50:00:00 >>>>> # >>>>> [...] >>>>> echo " " >>>>> echo "### Loading MAKER module:" >>>>> echo " " >>>>> module load bio/maker/2.31.9 >>>>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>>>> echo "MAKER_VERSION = $MAKER_VERSION" >>>>> module list >>>>> [...] >>>>> echo " " >>>>> echo "### Runing Maker example" >>>>> echo " " >>>>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>>>> export OMPI_MCA_mpi_warn_on_fork=0 >>>>> >>>>> echo "LD_PRELOAD=${LD_PRELOAD}" >>>>> # >>>>> # "STATUS: Processing and indexing input FASTA files..." >>>>> # >>>>> mpiexec -mca btl ^openib -n 16 maker >>>>> [...] >>>>> >>>>> >>>>> E R R O R S >>>>> ======= >>>>> [...] >>>>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>>>> STATUS: Parsing control files... >>>>> STATUS: Processing and indexing input FASTA files... >>>>> [uc1n338:113607] *** Process received signal *** >>>>> [uc1n338:113607] Signal: Segmentation fault (11) >>>>> [uc1n338:113607] Signal code: Address not mapped (1) >>>>> [uc1n338:113607] Failing at address: 0x4b0 >>>>> [uc1n338:113608] *** Process received signal *** >>>>> [uc1n338:113608] Signal: Segmentation fault (11) >>>>> [uc1n338:113608] Signal code: Address not mapped (1) >>>>> [uc1n338:113608] Failing at address: 0x4b0 >>>>> [uc1n338:113621] *** Process received signal *** >>>>> [uc1n338:113621] Signal: Segmentation fault (11) >>>>> [uc1n338:113621] Signal code: Address not mapped (1) >>>>> [uc1n338:113621] Failing at address: 0x4b0 >>>>> -------------------------------------------------------------------------- >>>>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>>>> -------------------------------------------------------------------------- >>>>> [...] >>>>> >>>>> WHATS WRONG HERE!? >>>>> >>>>> Thank you for your help! >>>>> >>>>> All the best , >>>>> >>>>> Rainer >>>>> >>>>> -- >>>>> Rainer Rutka >>>>> University of Konstanz >>>>> Communication, Information, Media Centre (KIM) >>>>> * High-Performance-Computing (HPC) >>>>> * KIM-Support and -Base-Services >>>>> Room: V511 >>>>> 78457 Konstanz, Germany >>>>> +49 7531 88-5413 >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> >>> >>> -- >>> Rainer Rutka >>> Universit?t Konstanz >>> Kommunikations-, Informations-, Medienzentrum (KIM) >>> Raum: V511, Tel: 54 13 >>> >> > > -- > Rainer Rutka > Universit?t Konstanz > Kommunikations-, Informations-, Medienzentrum (KIM) > Raum: V511, Tel: 54 13 > From munholl at uwindsor.ca Wed Feb 22 11:03:54 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 13:03:54 -0500 Subject: [maker-devel] Failed while polishing ESTs Message-ID: After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: polishig ESTs running est2genome search. #--------- command -------------# Widget::exonerate::est2genome: /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate #-------------------------------# Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! --> rank=NA, hostname=beanblade2 ERROR: Failed while polishig ESTs ERROR: Chunk failed at level:2, tier_type:3 FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:contig-dpp-500-500 The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: polishig ESTs running est2genome search. #--------- command -------------# Widget::exonerate::est2genome: /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate #-------------------------------# Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! --> rank=NA, hostname=beanblade4 ERROR: Failed while polishig ESTs ERROR: Chunk failed at level:2, tier_type:3 FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:contig-dpp-500-500 So the error seems to be pointing at something happening when wrapping up the ESTs. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 11:16:41 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 11:16:41 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: Message-ID: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. And make sure to delete any run directories before retrying. ?Carson > On Feb 22, 2017, at 11:03 AM, Seth Munholland wrote: > > After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade2 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade4 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > So the error seems to be pointing at something happening when wrapping up the ESTs. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 11:57:00 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 13:57:00 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: It is the test data, the dpp set to be specific. I already checked all the installs to be sure they configured and compile without error and are up to date. I've been deleting between each run. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: > So you are running the test data job correct? So if you get any error with > the test data job, something is wrong with your installation. > > Make sure BioPerl is up to data (and the same everywhere). If using your > own installation of Perl, make sure it had BerkleyDB support when you > installed it (if you are using something like perlbrew , you may not have > BerkleyDB support and this may affect BioPerl indexing). Also try > reinstalling exonertate. > > And make sure to delete any run directories before retrying. > > ?Carson > > > On Feb 22, 2017, at 11:03 AM, Seth Munholland wrote: > > After my clustering issue I figured I would go node by node and run MAKER > locally on the test data set to be sure that it was working properly before > trying it as a full MPI run. After adjusting all my exes to point to the > NFS versions I am consistently getting the same error on each node when it > comes time to run exonerate: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q > /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna > --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent > 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp- > mRNA-5.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade2 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > The only results I find on google was an issue with an improper character > in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and > since they all point to dpp-mRNA-5 I backed up the provided est and removed > it. The response was: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q > /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna > --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent > 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp- > mRNA-4.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade4 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > So the error seems to be pointing at something happening when wrapping up > the ESTs. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 12:00:24 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 14:00:24 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland wrote: > It is the test data, the dpp set to be specific. > > I already checked all the installs to be sure they configured and compile > without error and are up to date. > > I've been deleting between each run. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: > >> So you are running the test data job correct? So if you get any error >> with the test data job, something is wrong with your installation. >> >> Make sure BioPerl is up to data (and the same everywhere). If using your >> own installation of Perl, make sure it had BerkleyDB support when you >> installed it (if you are using something like perlbrew , you may not have >> BerkleyDB support and this may affect BioPerl indexing). Also try >> reinstalling exonertate. >> >> And make sure to delete any run directories before retrying. >> >> ?Carson >> >> >> On Feb 22, 2017, at 11:03 AM, Seth Munholland >> wrote: >> >> After my clustering issue I figured I would go node by node and run MAKER >> locally on the test data set to be sure that it was working properly before >> trying it as a full MPI run. After adjusting all my exes to point to the >> NFS versions I am consistently getting the same error on each node when it >> comes time to run exonerate: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q >> /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t >> /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna >> --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent >> 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA- >> 5.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade2 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> The only results I find on google was an issue with an improper character >> in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and >> since they all point to dpp-mRNA-5 I backed up the provided est and removed >> it. The response was: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q >> /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t >> /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna >> --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent >> 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA- >> 4.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade4 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> So the error seems to be pointing at something happening when wrapping up >> the ESTs. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 12:03:42 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 12:03:42 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: I still think you may have something wrong somewhere with some part of your installation (rogue libraries, compiler incompatibilities, etc.). Especially given your earlier issues with Perl libraries. ?Carson > On Feb 22, 2017, at 12:00 PM, Seth Munholland wrote: > > On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. > > Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: > It is the test data, the dpp set to be specific. > > I already checked all the installs to be sure they configured and compile without error and are up to date. > > I've been deleting between each run. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt > wrote: > So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. > > Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. > > And make sure to delete any run directories before retrying. > > ?Carson > > >> On Feb 22, 2017, at 11:03 AM, Seth Munholland > wrote: >> >> After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade2 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade4 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> So the error seems to be pointing at something happening when wrapping up the ESTs. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 12:26:57 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 14:26:57 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: I'll wipe it clean and do a fresh install of everything to be 100% safe. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 2:03 PM, Carson Holt wrote: > I still think you may have something wrong somewhere with some part of > your installation (rogue libraries, compiler incompatibilities, etc.). > Especially given your earlier issues with Perl libraries. > > ?Carson > > > On Feb 22, 2017, at 12:00 PM, Seth Munholland wrote: > > On a whim I decided to switch to the hsap demo data and it completed > without issue. I went back to dpp and this time is completed. I hadn't > changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and > then inverted the change. > > Mayhaps there was something not unloaded from memory in an earlier run? > It's that, ghosts, or rogue ions :/ > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: > >> It is the test data, the dpp set to be specific. >> >> I already checked all the installs to be sure they configured and compile >> without error and are up to date. >> >> I've been deleting between each run. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> >> On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: >> >>> So you are running the test data job correct? So if you get any error >>> with the test data job, something is wrong with your installation. >>> >>> Make sure BioPerl is up to data (and the same everywhere). If using your >>> own installation of Perl, make sure it had BerkleyDB support when you >>> installed it (if you are using something like perlbrew , you may not have >>> BerkleyDB support and this may affect BioPerl indexing). Also try >>> reinstalling exonertate. >>> >>> And make sure to delete any run directories before retrying. >>> >>> ?Carson >>> >>> >>> On Feb 22, 2017, at 11:03 AM, Seth Munholland >>> wrote: >>> >>> After my clustering issue I figured I would go node by node and run >>> MAKER locally on the test data set to be sure that it was working properly >>> before trying it as a full MPI run. After adjusting all my exes to point >>> to the NFS versions I am consistently getting the same error on each node >>> when it comes time to run exonerate: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q >>> /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t >>> /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T >>> dna --model est2genome --minintron 20 --maxintron 10000 --showcigar >>> --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp >>> -500-500.26386-32056.dpp-mRNA-5.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade2 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> The only results I find on google was an issue with an improper >>> character in the est file, but a cat of the dpp_est.fasta shows nothing >>> incorrect and since they all point to dpp-mRNA-5 I backed up the provided >>> est and removed it. The response was: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q >>> /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t >>> /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T >>> dna --model est2genome --minintron 20 --maxintron 10000 --showcigar >>> --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp >>> -500-500.22889-32056.dpp-mRNA-4.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade4 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> So the error seems to be pointing at something happening when wrapping >>> up the ESTs. >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 12:29:12 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 12:29:12 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: <53F1286C-945C-4F91-B851-5833BE1D1F18@gmail.com> In the most extreme case, you may even need to go as far as setting up your own perl. If you do that, make sure to unset the PERL5LIB environmental variable, so other perl libraries don?t interfere. ?Carson > On Feb 22, 2017, at 12:26 PM, Seth Munholland wrote: > > I'll wipe it clean and do a fresh install of everything to be 100% safe. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 2:03 PM, Carson Holt > wrote: > I still think you may have something wrong somewhere with some part of your installation (rogue libraries, compiler incompatibilities, etc.). Especially given your earlier issues with Perl libraries. > > ?Carson > > >> On Feb 22, 2017, at 12:00 PM, Seth Munholland > wrote: >> >> On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. >> >> Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <> >> On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: >> It is the test data, the dpp set to be specific. >> >> I already checked all the installs to be sure they configured and compile without error and are up to date. >> >> I've been deleting between each run. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <> >> On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt > wrote: >> So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. >> >> Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. >> >> And make sure to delete any run directories before retrying. >> >> ?Carson >> >> >>> On Feb 22, 2017, at 11:03 AM, Seth Munholland > wrote: >>> >>> After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade2 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade4 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> So the error seems to be pointing at something happening when wrapping up the ESTs. >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lucile.soler at bils.se Thu Feb 23 05:40:08 2017 From: lucile.soler at bils.se (lucile.soler at bils.se) Date: Thu, 23 Feb 2017 13:40:08 +0100 Subject: [maker-devel] genes longer than contig in maker Message-ID: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> Hello, I am using maker3 to annotate a lizard. The maker part went well but then when I want to do the functional annotation, it seems that for some contigs maker has created genes outside of the contig size. For instance : XXX1 maker gene 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8 XXX1 maker mRNA 2 26717 . - . ID=maker- XXX1_pilon-augustus-gene-0.8-mRNA-1;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.8|0.5|1|0.4|0.33|6|5291|105 XXX1 maker mRNA 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;_AED=0.08;_eAED=0.08;_QI=0|0.5|0.4|1|0.25|0.4|5|4442|170 and the length of the contig is 26696 Any idea of what could have happened? Have you seen that already? Let me know what more information you need to answer. Thank you very much for your help, Lucile Soler PhD in Bioinformatics Genome Annotation Platform NBIS (National Bioinformatics Infrastructure Sweden) mail:lucile.soler at bils.se -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Thu Feb 23 06:10:55 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Thu, 23 Feb 2017 14:10:55 +0100 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> Message-ID: <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> HI! Running maker with mpiexec causes this error-message: mpiexec_uc1n077.localdomain: cannot connect to local mpd (/scratch/mpd2.console_uc1n077.localdomain_kn_pop235844); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option) ### Cleaning up files ... removing unnecessary scratch files ... And yes, we don't have mpd running. Environment used is: Currently Loaded Modulefiles: 1) compiler/intel/16.0(default) 2) mpi/impi/5.1.3-intel-16.0(default) 3) bio/maker/2.31.8_impi Running maker with mpiexec using only 1 node and 8 cores. mpiexec -n 8 maker :-( Any suggestions ? -- Rainer Rutka University of Konstanz Communication, Information, Media Centre (KIM) * High-Performance-Computing (HPC) * KIM-Support and -Base-Services Room: V511 78457 Konstanz, Germany +49 7531 88-5413 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From carsonhh at gmail.com Thu Feb 23 13:22:22 2017 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 23 Feb 2017 13:22:22 -0700 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> Message-ID: It means one of two things. 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. ?Carson > On Feb 23, 2017, at 6:10 AM, Rainer Rutka wrote: > > > HI! > > Running maker with mpiexec causes this error-message: > > mpiexec_uc1n077.localdomain: cannot connect to local mpd (/scratch/mpd2.console_uc1n077.localdomain_kn_pop235844); possible causes: > 1. no mpd is running on this host > 2. an mpd is running but was started without a "console" (-n option) > ### Cleaning up files ... removing unnecessary scratch files ... > > And yes, we don't have mpd running. > > Environment used is: > > Currently Loaded Modulefiles: > 1) compiler/intel/16.0(default) > 2) mpi/impi/5.1.3-intel-16.0(default) > 3) bio/maker/2.31.8_impi > > Running maker with mpiexec using only 1 node and 8 cores. > > mpiexec -n 8 maker > > :-( > > Any suggestions ? > > -- > Rainer Rutka > University of Konstanz > Communication, Information, Media Centre (KIM) > * High-Performance-Computing (HPC) > * KIM-Support and -Base-Services > Room: V511 > 78457 Konstanz, Germany > +49 7531 88-5413 > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Thu Feb 23 16:40:24 2017 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 23 Feb 2017 16:40:24 -0700 Subject: [maker-devel] genes longer than contig in maker In-Reply-To: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> References: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> Message-ID: 1. You have two contigs with the same ID in your assembly (one longer and one shorter). 2. Your BioPerl index for retrieving the sequence is corrupt somehow. Requires delete of previous output directory before restarting. 3. You are using bad formatted GFF3 as input into maker, and it is somehow not failing right away. The fact you get output means that it was able to translate the sequence into protein with CDS etc. So it was not too short for that. Look at the contig line in the GFF3 to get the length of the contig maker is seeing to compare to the feature positions given. ?Carson > On Feb 23, 2017, at 5:40 AM, lucile.soler at bils.se wrote: > > Hello, > > I am using maker3 to annotate a lizard. > > > The maker part went well but then when I want to do the functional annotation, it seems that for some contigs maker has created genes outside of the contig size. > > For instance : > > > XXX1 maker gene 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8 > > > XXX1 maker mRNA 2 26717 . - . ID=maker- XXX1_pilon-augustus-gene-0.8-mRNA-1;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.8|0.5|1|0.4|0.33|6|5291|105 > > XXX1 maker mRNA 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;_AED=0.08;_eAED=0.08;_QI=0|0.5|0.4|1|0.25|0.4|5|4442|170 > > and the length of the contig is 26696 > > > > > Any idea of what could have happened? Have you seen that already? > > > > Let me know what more information you need to answer. > > Thank you very much for your help, > > > > > > Lucile Soler > > PhD in Bioinformatics > Genome Annotation Platform > NBIS (National Bioinformatics Infrastructure Sweden) > mail:lucile.soler at bils.se > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Fri Feb 24 01:43:21 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Fri, 24 Feb 2017 09:43:21 +0100 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> Message-ID: <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> Hi Carson. First of all THANK YOU FOR YOUR HELP. MUCH APPRECIATED. :-) Am 23.02.2017 um 21:22 schrieb Carson Holt: > It means one of two things. > 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). No, I am starting the right version of mpiexec. This is a list of all our current available MPI-Versions, corresponding to their compiler: UC:[kn at uc1n997 ~]$ module avail mpi ------------------------------------- /opt/bwhpc/common/modulefiles -------------------------------------- mpi/impi/4.1.3-gnu-4.4 mpi/openmpi/1.10-intel-15.0 mpi/impi/4.1.3-gnu-4.7 mpi/openmpi/1.10-intel-16.0 mpi/impi/4.1.3-intel-14.0 mpi/openmpi/1.8-gnu-4.5 mpi/impi/5.0.3-gnu-4.4 mpi/openmpi/1.8-gnu-4.7 mpi/impi/5.0.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.7-m32 mpi/impi/5.0.3-intel-15.0 mpi/openmpi/1.8-gnu-4.8 mpi/impi/5.1.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.9 mpi/impi/5.1.3-gnu-system mpi/openmpi/1.8-intel-14.0(default) mpi/impi/5.1.3-intel-16.0(default) mpi/openmpi/1.8-intel-15.0 mpi/openmpi/1.10-gnu-4.5 mpi/openmpi/2.0-gnu-4.7 mpi/openmpi/1.10-gnu-4.7 mpi/openmpi/2.0-gnu-4.8 mpi/openmpi/1.10-gnu-4.8 mpi/openmpi/2.0-gnu-5.2 mpi/openmpi/1.10-gnu-4.9 mpi/openmpi/2.0-intel-15.0 mpi/openmpi/1.10-gnu-5.2 mpi/openmpi/2.0-intel-16.0 mpi/openmpi/1.10-intel-14.0 Here I load the MPI-Module including all it's dependencies: UC:[kn at uc1n997 ~]$ module load mpi/impi/5.1.3-intel-16.0 Loading module dependency 'compiler/intel/16.0'. Result: UC:[kn at c1n997 ~]$ which mpiexec /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec UC:[kn at uc1n997 ~]$ > 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) Extremely old? [...] Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160601 (build id: 15562) > MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. Yes. And we don't even use MPD at our clusters :-) > So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. I do not have any MPICH available on our cluster(s) now. All of the old versions had been removed some years ago. At a glance --------------- Maker is available as a so-called module on our cluster system. It's been build on a developers' node I can access. But, the MPI-modules are built by other fellows (e.g. at the KIT in Karlsruhe/Germany) on other nodes. Please check the module-file (included in this mail) bio-maker-2.31.8_impi to see how Maker was build including the envoronments set by this module. e.g.: ./Build status verify dependencies: =================================================== STATUS MAKER v2.31.8 =================================================== PERL Dependencies: VERIFIED External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: ENABLED MWAS Web Interface: DISABLED MAKER PACKAGE: CONFIGURATION OK At least, please(!) have a look at our m.moab file (included in this mail). This is the way how we submit a Maker job to our cluster. Maybe something is wrong here? Sorry again for wasting your time. But we imperatively need the Maker software running in parallel mode. :-) -- Rainer Rutka University of Konstanz Communication, Information, Media Centre (KIM) * High-Performance-Computing (HPC) * KIM-Support and -Base-Services Room: V511 78457 Konstanz, Germany +49 7531 88-5413 -------------- next part -------------- #%Module1.0 # # Cluster: bwunicluster # Module: bio/maker/2.31.8 # Revision: knbw01 # TargetSystem: Red-Hat-Enterprise # MainLocation: /opt/bwhpc/common # Status: optional # License: GPL, Artistic License # URL: http://www.yandell-lab.org/software/maker.html # 2ndLevelSupport: bwhpc [at] uni-konstanz.de # ##### (A) Revision history: # # knbw01 20160630 r.rutka Initial revision knbw01 of module version 2.31.8 # knbw02 20161121 r.rutka Initial revision knbw02 of module version 2.31.8 # knbw03 20160222 r.rutka Initial revision knbw02 of module version 2.31.8 with impi # ##### (B) Dependencies: # # conflict: any other maker version # module load compiler/intel/16.0 # module load mpi/impi/5.1.3-intel-16.0 # # # Main environments for build # # VERSION="2.31.8_impi" && echo "Version: ${VERSION}" # MAIN_DIR="/opt/bwhpc/common" && echo ${MAIN_DIR} # SOURCE_DIR=/home/kn/kn_kn/kn_pop235844/src/bio/maker/2.31.8 # TARGET_DIR="${MAIN_DIR}/bio/maker/${VERSION}" && echo ${TARGET_DIR} # ##### (C) How to obtain software? # # You have to register at first before you can download the SW. # # http://yandell.topaz.genetics.utah.edu/cgi-bin/maker_license.cgi # # # Download and unzip the binary distributions # cd ${SOURCE_DIR} && pwd # [ ! -f maker-${VERSION/_impi}.tgz ] && wget http://yandell.topaz.genetics.utah.edu/maker_downloads/CE63/461D/BD41/4510A183DC4571D5EA619E459572/maker-2.31.8.tgz # [[ -d ${TARGET_DIR} ]] && mv ${TARGET_DIR} ${TARGET_DIR}_$(date +%Y.%m.%d:%H.%M) # ##### (D) How to build and install software? # # mkdir -vp ${TARGET_DIR/$VERSION} # cd ${TARGET_DIR/$VERSION} && pwd # tar -xvzf ${SOURCE_DIR}/maker-${VERSION/_impi}.tgz # mv maker ${VERSION} # cd ${VERSION} && pwd # export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so # cd src # # Build required ad-ons. Lot's of required packages will be installed. # # perl Build.PL # | Would you like to configure MAKER for MPI.. ? [N ] Y # | # accept the next default paths and press ENTER # # ignore the error-messages - just continue with the next step # # ./Build installdeps # installs missing PERL dependencies # | ... MAKER .. local installataion: Y # | # Accept the defaults of _all_ further questions by pressing ENTER. # | # ** This is an annoying and long procedure :-( ** # # ./Build installexes # installs missing external programs # | # If are a registered user of RepBase, then MAKER can # | # download and install RepBase for RepeatMasker for you. # | # Register at: http://www.girinst.org/ # | ... download and install RepBase ... : N # We install RepBase later! # # If you type Y, this will not work # # See Install RepBase later in this file # # | # ERROR: Exonerate can't be found. URL Error 404. :-( # | # Locate exonerate entry in locations file and modify path to (line 33): # | # http://ftp.ebi.ac.uk/pub/software/vertebrategenomics/exonerate/exonerate-2.2.0-x86_64.tar.gz # | # restart ./Build installexes # ./Build installexes # # # Install RepBase to prevent "RepBase is not insalled for RepeatMasker" error-message # # http://www.girinst.org/server/RepBase/index.php # p=$(pwd) # cd ${TARGET_DIR}/exe/RepeatMasker && pwd # # http://www.girinst.org/server/RepBase/index.php # # FASTA-Format # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/RepBase22.01.fasta.tar.gz # tar xvzf RepBase22.01.fasta.tar.gz # # # # NEU! # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/repeatmaskerlibraries/RepBaseRepeatMaskerEdition-20170127.tar.gz # # # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/repeatmaskerlibraries/repeatmaskerlibraries-20160829.tar.gz # tar xvzf RepBaseRepeatMaskerEdition-20170127.tar.gz # # # tar xvzf repeatmaskerlibraries-20160829.tar.gz # # perl ./configure # | Enter path perl ... : ENTER # | Enter path ... RepeatMasker: ENTER # | Enter path TRF programm: /opt/bwhpc/common/bio/maker/2.31.8_impi/exe/RepeatMasker/trf # | Add a search engine: 2 RMBlast # | Enter path: /opt/bwhpc/common/bio/maker/2.31.8_impi/exe/RepeatMasker/rmblast/bin # | search engine ... default: Y or ENTER # | Enter Selection: 5 (Done) # # # rm *.tar.gz # cd $p && pwd # # # Now check if anything's fine until now... # ./Build status # | # veryfy dependencies: # | # =================================================== # | # STATUS MAKER v2.31.8 # | # =================================================== # | # PERL Dependencies: VERIFIED # | # External Programs: VERIFIED # | # External C Libraries: VERIFIED # | # MPI SUPPORT: ENABLED # | # MWAS Web Interface: DISABLED # | # MAKER PACKAGE: CONFIGURATION OK # # ./Build install # # # Test # cd ${TARGET_DIR} # bin/maker -version # 2.31.8 # # # Create module file (this one): # # ${SOURCE_DIR}/modulefiles/bio-maker-2.31.8 # # # Create Moab example files (and read the instructions inside of it): # # ${SOURCE_DIR}/bwhpc-examples/bwunicluster-maker-example.moab # # # Copy Moab/bwHPC, special-scripts, examples and more... # mkdir -vp ${TARGET_DIR}/bwhpc-examples # cp ${SOURCE_DIR}/bwhpc-examples/* ${TARGET_DIR}/bwhpc-examples/. # # # Copy module file (this one): # mkdir -vp ${TARGET_DIR}/modulefiles # cp -v ${SOURCE_DIR}/modulefiles/bio-maker-${VERSION} ${TARGET_DIR}/modulefiles/ # chmod -c 644 ${TARGET_DIR}/modulefiles/* # # # Create module link: # mkdir -vp ${MAIN_DIR}/modulefiles/bio/maker # ln -fs ${TARGET_DIR}/modulefiles/bio-maker-${VERSION} ${MAIN_DIR}/modulefiles/bio/maker/${VERSION} # module avail bio/maker # # # Cleanup build and installation target dir: # # chgrp -R -h -c uc1-adm-sw ${TARGET_DIR} # chmod 777 ${TARGET_DIR}/bwhpc-examples # chmod 666 ${TARGET_DIR}/bwhpc-examples/* # rm -rf ${TARGET_DIR}/src ##### END COMMENTS ### Define procedure "set_envVAR" to work around a module-unload-load bug (optional): # Exported environment variables, which do not change their values, # disappear after an automatic unload and load sequence within a module file. # This is most likely a bug in the modules environment. Unchanged # variables are not re-exported upon load in an automatic unload-load # sequence. The workaround is to "unset" the variable in the right moment # (so the load of the unload-load-chain can set the variable again) # and, in addition, to set the environment variable explicitly. proc set_envVAR {envVAR VARcontent} { # Use global function env: global env # Unset envVAR explicitly: if { [info exists env($envVAR)] } { catch { unset env($envVAR) } } # Call overloaded module command setenv: setenv $envVAR $VARcontent # Set envVAR explicitly: set env([set envVAR]) $VARcontent } ### Define fallback values and source global functions and variables from modulerc file: set first_level_support_email "bwhpc (at) uni-konstanz.de" set global_modulerc_file "/opt/bwhpc/common/etc/modules.conf" if { [ file readable "${global_modulerc_file}" ] } { source "${global_modulerc_file}" } # Override global first_level_support_email email: set first_level_support_email "bwhpc (at) uni-konstanz.de" ### Define version, base_dir and whatis entry: set version "2.31.8_impi" set base_dir "/opt/bwhpc/common/bio/maker/$version" module-whatis "maker $version is a portable and configurable genome annotation pipeline" ### Define convenience environment variables: set maker_version "${version}" set maker_home "${base_dir}" set maker_exa_dir "${base_dir}/bwhpc-examples" set maker_bin_dir "${base_dir}/bin" set maker_bpr_url "http://www.bwhpc-c5.de/wiki/index.php/Maker" set maker_blast_bin "${base_dir}/exe/blast/bin" set maker_exonerate_bin "${base_dir}/exe/exonerate/bin" set maker_snap_bin "${base_dir}/exe/snap" set maker_repeatmasker_bin "${base_dir}/exe/RepeatMasker" set_envVAR MAKER_VERSION "${maker_version}" set_envVAR MAKER_HOME "${maker_home}" set_envVAR MAKER_EXA_DIR "${maker_exa_dir}" set_envVAR MAKER_BIN_DIR "${maker_bin_dir}" set_envVAR MAKER_BPR_URL "http://www.bwhpc-c5.de/wiki/index.php/Maker" set_envVAR MAKER_BLAST_BIN "${maker_blast_bin}" set_envVAR MAKER_EXONERATE_BIN "${maker_exonerate_bin}" set_envVAR MAKER_SNAP_BIN "${maker_snap_bin}" set_envVAR MAKER_REPEATMASKER_BIN "${maker_repeatmasker_bin}" ### Update path environments and define application specific environment vars: prepend-path PATH "${maker_bin_dir}" prepend-path PATH "${maker_blast_bin}" prepend-path PATH "${maker_exonerate_bin}" prepend-path PATH "${maker_snap_bin}" prepend-path PATH "${maker_repeatmasker_bin}" ### Define help text: proc ModulesHelp { } { global maker_home maker_exa_dir first_level_support_email maker_bpr_url maker_blast_bin maker_exonerate_bin maker_snap_bin maker_repeatmasker_bin puts stderr " DESCRIPTION MAKER is a portable and easily configurable genome annotation pipeline. Its purpose is to allow smaller eukaryotic and prokaryotic genome projects to independently annotate their genomes and to create genome databases. MAKER identifies repeats, aligns ESTs and proteins to a genome, produces ab-initio gene predictions and automatically synthesizes these data into gene annotations having evidence-based quality values. EXTERNAL PROGRAMS BLAST Command Line Applications (2.2.28) $maker_blast_bin EXONERATE Generic Sequence Comparison Tool (2.2.0) $maker_exonerate_bin SNAP Semi-HMM-based Nucleic Acid Parser (version 2006-07-28) $maker_snap_bin REPEATMASKER Mask repetitive DNA (open-4.0.5) $maker_repeatmasker_bin DOCUMENTATION * Main MAKER site http://www.yandell-lab.org/software/maker.html * MAKER User Guide, Documents, Wiki-Article http://weatherby.genetics.utah.edu/MAKER/wiki/index.php $maker_home/README * MAKER Download http://yandell.topaz.genetics.utah.edu/maker_downloads/CE63/461D/BD41/4510A183DC4571D5EA619E459572/maker-2.31.8.tgz * bwHPC examples and a moab example script $maker_exa_dir * bwHPC Best Practices Repository $maker_bpr_url CITATION n./a. +-------------------------------------------+ | THIS IS THE IMPI-VERSION BUILT WITH INTEL | +-------------------------------------------+ In case of problems, please contact: $first_level_support_email This module is available for all users. " } module load compiler/intel/16.0 module load mpi/impi/5.1.3-intel-16.0 conflict bio/maker -------------- next part -------------- #!/bin/bash #MSUB -N maker_impi-job #MSUB -j oe #MSUB -o $(JOBNAME).$(JOBID) #MSUB -m ae #MSUB -l nodes=1:ppn=1 #MSUB -l mem=20gb #MSUB -l walltime=10:00:00 # start=$(date +%s) echo " " echo "### Setting up shell environment ..." echo " " # if test -e "/etc/profile"; then source "/etc/profile"; fi; if test -e "$HOME/.bash_profile"; then source "$HOME/.bash_profile"; fi; unset LANG; export LC_ALL="C"; export MKL_NUM_THREADS=1; export OMP_NUM_THREADS=1 export USER=${USER:=`logname`} export MOAB_JOBID=${MOAB_JOBID:=`date +%s`} export MOAB_SUBMITDIR=${MOAB_SUBMITDIR:=`pwd`} export MOAB_JOBNAME=${MOAB_JOBNAME:=`basename "$0"`} export MOAB_JOBNAME=$(echo "${MOAB_JOBNAME}" | sed 's/[^a-zA-Z0-9._-]/_/g') export MOAB_NODECOUNT=${MOAB_NODECOUNT:=1} export MOAB_PROCCOUNT=${MOAB_PROCCOUNT:=1} ulimit -s 200000 echo " " echo "### Printing basic job infos to stdout ..." echo " " echo "START_TIME = `date +'%y-%m-%d %H:%M:%S %s'`" echo "HOSTNAME = ${HOSTNAME}" echo "USER = ${USER}" echo "MOAB_JOBNAME = ${MOAB_JOBNAME}" echo "MOAB_JOBID = ${MOAB_JOBID}" echo "MOAB_SUBMITDIR = ${MOAB_SUBMITDIR}" echo "MOAB_NODECOUNT = ${MOAB_NODECOUNT}" echo "MOAB_PROCCOUNT = ${MOAB_PROCCOUNT}" echo "SLURM_NODELIST = ${SLURM_NODELIST}" echo "PBS_NODEFILE = ${PBS_NODEFILE}" if test -f "${PBS_NODEFILE}"; then echo "PBS_NODEFILE (begin) ---------------------------------" NO_NODES=$(wc -l < ${PBS_NODEFILE}) cat "${PBS_NODEFILE}" sort -u $PBS_NODEFILE > mpd.hosts echo "PBS_NODEFILE (end) -----------------------------------" else NO_NODES=1 fi echo " " echo "### Creating TMP_WORK_DIR directory and changing to it ..." echo " " # Using "$TMPDIR" is strongly recommended for Maker jobs # since these jobs can create a lot of disk IO. # NEVER EVER calculate in your home directory. TMP_BASE_DIR="$TMPDIR" JOB_WORK_DIR="${MOAB_JOBNAME}.${MOAB_JOBID##*.}.$(date +%y%m%d_%H%M%S)" TMP_WORK_DIR="${TMP_BASE_DIR}/${JOB_WORK_DIR}" echo "JOB_WORK_DIR = ${JOB_WORK_DIR}" echo "TMP_BASE_DIR = ${TMP_BASE_DIR}" echo "TMP_WORK_DIR = ${TMP_WORK_DIR}" mkdir -vp "${TMP_WORK_DIR}" cd "${TMP_WORK_DIR}" echo " " echo "### Loading MAKER module:" echo " " module load bio/maker/2.31.8_impi [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.8_impi'."; exit 1; } echo "MAKER_VERSION = $MAKER_VERSION" module list echo " " echo "### Copying input examples files for job:" echo " " cp -v ${MAKER_EXA_DIR}/*.{fasta,ctl} . cp -Rv ${MAKER_EXA_DIR}/_Inline . sleep 2 echo " " echo "### Display internal Maker/bwHPC environments..." echo " " echo "MAKER_BIN_DIR = ${MAKER_BIN_DIR}" echo "MAKER_EXA_DIR = ${MAKER_EXA_DIR}" echo "" echo " " echo "### Runing Maker example" echo " " # # Environmental variables to set: # export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so echo "LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so" export OMPI_MCA_mpi_warn_on_fork=0 echo "OMPI_MCA_mpi_warn_on_fork=0" export I_MPI_CPUINFO="proc" echo "I_MPI_CPUINFO=proc" export I_MPI_PMI_LIBRARY=${MPI_LIB_DIR}/libpmi.so echo "I_MPI_PMI_LIBRARY=${MPI_LIB_DIR}/libpmi.so" export I_MPI_PIN_DOMAIN=node echo "I_MPI_PIN_DOMAIN=node" # otherwise MAKER calls to BLAST and other programs which are parallelized independent # of MPI may not work export I_MPI_FABRICS='shm:tcp' echo "I_MPI_FABRICS=shm:tcp" # avoid potential complication with OpenFabrics libraries (they block system calls because of # how they use registered memory, i.e. MAKER calling BLAST would fail) # set to eth0 if you don?t have an infiniband over # ip inerface (required because of the above I_MPI_FABRICS change) export I_MPI_HYDRA_IFACE=ib0 echo "I_MPI_HYDRA_IFACE=ib0" # The ?c nobrs. option will try and run the BLAST jobs on # nobrs. cpus on the same machine, but will not run via MPI. # -nolocal # wtf? # mpiexec -n $SLURM_NPROCS -env I_MPI_DEBUG 5 ./hello echo "starting mpiexec..." mpiexec -nc 1 maker echo "### Cleaning up files ... removing unnecessary scratch files ..." echo " " # rm -fv sleep 3 # Sleep some time so potential stale nfs handles can disappear. echo " " echo "### Compressing results and copying back result archive ..." echo " " cd "${TMP_BASE_DIR}" mkdir -vp "${MOAB_SUBMITDIR}" # if user has deleted or moved the submit dir echo "Creating result tgz-file '${MOAB_SUBMITDIR}/${JOB_WORK_DIR}.tgz' ..." tar -zcvf "${MOAB_SUBMITDIR}/${JOB_WORK_DIR}.tgz" "${JOB_WORK_DIR}" \ || { echo "ERROR: Failed to create tgz-file. Please cleanup TMP_WORK_DIR '$TMP_WORK_DIR' on host '$HOSTNAME' manually (if not done automatically by queueing system)."; exit 102; } # Remarks: # * The resulting tgz file is copied back to the submit directory. # The name of the tgz file looks similar too # "bwunicluster-maker-example.moab.275.110528_101755.tgz" echo " " echo "### Final cleanup: Remove TMP_WORK_DIR ..." echo " " rm -rvf "${TMP_WORK_DIR}" echo "END_TIME = `date +'%y-%m-%d %H:%M:%S %s'`" end=$(date +%s) echo " " echo "### Calculate duration ..." echo " " diff=$[end-start] if [ $diff -lt 60 ]; then echo "Runtime (approx.): '$diff' secs" elif [ $diff -ge 60 ]; then echo 'Runtime (approx.): '$[$diff / 60] 'min(s) '$[$diff % 60] 'secs' fi -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From carsonhh at gmail.com Fri Feb 24 10:30:32 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 24 Feb 2017 10:30:32 -0700 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> Message-ID: Specific things. 1. Do not set LD_PRELOAD. That is only for OpenMPI, but it will cause problems with other MPI's. 2. Make sure you recompiled MAKER for Intel MPI (MPI code always has to be compiled for the flavor you are using, so make sure you have a separate installation of MAKER for Intel MPI). Also validate that the mpicc and libmpi.h listed during the MAKER install belong to Intel MPI. Don?t just assume they do because you loaded the module. Manually verify the paths during MAKER?s setup. 3. The error you got previously should not even be possible with the current version of Intel MPI, which is why I say that when you called mpiexec, something else (that was not Intel MPI) was launched. Easy solution is to give the full path of mpiexec in your job, so are not relying on PATH to be unaltered in your job. Do not do ?> mpiexec -nc 1 maker Do this for example ?> /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec -nc maker 4. Build and run on the same node for your test. If you build on one node and run on another, you may be changing your environment in ways you don?t realize that break things. So if you can build and test on the same node and it works, then it fails when you test it elsewhere, then you have to track down how your environment is changing. ?Carson > On Feb 24, 2017, at 1:43 AM, Rainer Rutka wrote: > > Hi Carson. > > First of all THANK YOU FOR YOUR HELP. > MUCH APPRECIATED. :-) > > Am 23.02.2017 um 21:22 schrieb Carson Holt: >> It means one of two things. >> 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). > > No, I am starting the right version of mpiexec. > > This is a list of all our current available MPI-Versions, corresponding > to their compiler: > > UC:[kn at uc1n997 ~]$ module avail mpi > ------------------------------------- /opt/bwhpc/common/modulefiles -------------------------------------- > mpi/impi/4.1.3-gnu-4.4 mpi/openmpi/1.10-intel-15.0 > mpi/impi/4.1.3-gnu-4.7 mpi/openmpi/1.10-intel-16.0 > mpi/impi/4.1.3-intel-14.0 mpi/openmpi/1.8-gnu-4.5 > mpi/impi/5.0.3-gnu-4.4 mpi/openmpi/1.8-gnu-4.7 > mpi/impi/5.0.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.7-m32 > mpi/impi/5.0.3-intel-15.0 mpi/openmpi/1.8-gnu-4.8 > mpi/impi/5.1.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.9 > mpi/impi/5.1.3-gnu-system mpi/openmpi/1.8-intel-14.0(default) > > mpi/impi/5.1.3-intel-16.0(default) > > mpi/openmpi/1.8-intel-15.0 > mpi/openmpi/1.10-gnu-4.5 mpi/openmpi/2.0-gnu-4.7 > mpi/openmpi/1.10-gnu-4.7 mpi/openmpi/2.0-gnu-4.8 > mpi/openmpi/1.10-gnu-4.8 mpi/openmpi/2.0-gnu-5.2 > mpi/openmpi/1.10-gnu-4.9 mpi/openmpi/2.0-intel-15.0 > mpi/openmpi/1.10-gnu-5.2 mpi/openmpi/2.0-intel-16.0 > mpi/openmpi/1.10-intel-14.0 > > > Here I load the MPI-Module including all it's dependencies: > > UC:[kn at uc1n997 ~]$ module load mpi/impi/5.1.3-intel-16.0 > Loading module dependency 'compiler/intel/16.0'. > > Result: > UC:[kn at c1n997 ~]$ which mpiexec > /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec > UC:[kn at uc1n997 ~]$ > >> 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) > Extremely old? > > [...] > Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160601 (build id: 15562) > >> MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. > Yes. And we don't even use MPD at our clusters :-) > >> So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. > I do not have any MPICH available on our cluster(s) now. All of the old versions had > been removed some years ago. > > At a glance > --------------- > > Maker is available as a so-called module on our cluster system. It's been build > on a developers' node I can access. But, the MPI-modules are built by other > fellows (e.g. at the KIT in Karlsruhe/Germany) on other nodes. > > Please check the module-file (included in this mail) > > bio-maker-2.31.8_impi > > to see how Maker was build including the envoronments set by this > module. > > e.g.: > ./Build status > verify dependencies: > =================================================== > STATUS MAKER v2.31.8 > =================================================== > PERL Dependencies: VERIFIED > External Programs: VERIFIED > External C Libraries: VERIFIED > MPI SUPPORT: ENABLED > MWAS Web Interface: DISABLED > MAKER PACKAGE: CONFIGURATION OK > > > At least, please(!) have a look at our m.moab file (included in this mail). > This is the way how we submit a Maker job to our cluster. Maybe something > is wrong here? > > Sorry again for wasting your time. But we imperatively need the Maker > software running in parallel mode. > > :-) > > -- > Rainer Rutka > University of Konstanz > Communication, Information, Media Centre (KIM) > * High-Performance-Computing (HPC) > * KIM-Support and -Base-Services > Room: V511 > 78457 Konstanz, Germany > +49 7531 88-5413 > From qlian003 at ucr.edu Sat Feb 25 10:14:04 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Sat, 25 Feb 2017 09:14:04 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: <2377C5DD-569C-4248-B458-349D7AEA32F5@ucr.edu> Thank you Barry and Carson! I compared the SOBA statistics of RepeatMasker footprint and the report generated by running RepeatMasker alone, I got 2 different parentage of repeats masked. Running RepeatMasker with myTrained.lib, the repeats masked are 42%. But within Maker GFF3, the percentage of repeats masker is only ~18%. What may cause such difference here? Thanks Qihua > On Feb 21, 2017, at 1:34 PM, Carson Holt wrote: > > MAKER merges overlapping RepeatMasker results into a single longer feature. > > ?Carson > > >> On Feb 20, 2017, at 1:34 PM, Qihua Liang wrote: >> >> Hi Carson, >> >> Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. >> >> I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. >> >> Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. >> In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. >> >> But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. >> >> When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? >> Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? >> >> Thanks >> Qihua >> >> >>> On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: >>> >>> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. >>> >>> CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. >>> >>> If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. >>> >>> What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. >>> >>> However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. >>> >>> Thanks, >>> Carson >>> >>>> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >>>> >>>> Dear Maker develop team, >>>> >>>> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >>>> >>>> Thanks >>>> Qihua >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> > From qwzhang0601 at gmail.com Mon Feb 27 08:25:04 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 27 Feb 2017 10:25:04 -0500 Subject: [maker-devel] PARALLELIZED DE NOVO GENOME ANNOTATION WITHOUT MPI Message-ID: Hello: I am doing genome annotation using Maker on our high performance computational cluster (HPC). Due to some issues of MPI, I submitted the Maker jobs several times under the same directory to HPC. Followed by the example in the protocol (as shown below), when I submit the jobs I make them as background processes by "&" except the first one. Is this necessary when I submit a job to a HPC? I found it costed much much longer time than I expected (according to a testing on a smaller data set). I am not sure whether setting the process as background process lead to this issue? The example in the protocol % maker 2> maker1.error % maker 2> maker2.error & % maker 2> maker3.error & ...... BTW, will the annotation on shorter contig (e.g., 500bp) cost ~ 1/100 of the time that cost for annotation a 50000bp contig? I am using SNAP for an inito and RNA-seq assembly and protein sequences as evidence. I have more than half contigs shorter than 300bp (whose total length is only about 5% of the total length of all contigs), I want to know whether I can save about half (or only about 5%) of the time if I ignore those short contigs. Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcg at cau.edu.cn Tue Feb 28 00:43:57 2017 From: dcg at cau.edu.cn (dcg at cau.edu.cn) Date: Tue, 28 Feb 2017 15:43:57 +0800 Subject: [maker-devel] how to deal with Contigs to run maker? Message-ID: <2017022815435664227911@cau.edu.cn> Dear sir: After assemblying, I got many contigs and their order in each chromosome. What I have done is merging these contigs into each chromosomes followed by the order, with 100 Ns inserted betwwen each contigs. So that I got chr1 chr2......Then I ran the repeatmodeler, predictor to annotate it. Could my way reach a high-quality result? Should I use all the contigs to mask repeats and practice predictor? Is there any better way to do genome-wide annotation? I'm looking forward to your reply! Best wishes! Chao Chao 2017.02.28 -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Thu Feb 2 09:16:51 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Thu, 2 Feb 2017 11:16:51 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> Message-ID: Thank for you suggestions. So it does not matter if there are redundancies of protein sequence from different sources? I am trying to annotating a *rodent *genome, and planned to collect protein sequences of human, mouse, rat from bout UniProt and NCBI (besides I also have RNA-seq data). I choose these species, because they are close to the species that I am working on and they are well annotated. But I saw someone said that if we choose protein sequence from one lineage, the genes that are missing in the lineage will not be detected. And in the following paper, the authors claim they used the entire SwissProt database as the input. How do you think about this? Should I include protein sequences from more species (like all Eukaryota)? I think it can help us identify more genes, but on the other hand won't this also give us more false positives? This paper used the entire SwissProt database as the input. Insights into the evolution of longevity from the bowhead whale genome. 2015. *Cell Rep* *10*(1): 112-122. Thanks Best Quanwei 2017-01-31 15:57 GMT-05:00 Michael Campbell : > Hi Quanwei, > > (1) When I use uniprot I use SWISS-prot and not tremble. > (2) I don?t merge files together. I just pass them all to MAKER as a comma > separated list. > > Thanks, > Mike > > > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang > wrote: > > > > I wonder what's the best way to collect protein sequences for gene > annotation of a de novo genome assembly. > > (1) My first choice is to get protein sequences of human and mouse from > UniProt. At this step, I am not clear whether I should download the > reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., > TrEMBL). > > (2) On ther other hand, I also get protein sequences from NCBI, should I > just simply merge those fasta files. Does it matter if there are > redundancies? And also, if I get protein sequences from different sources, > they may not have the same quality. Do I need to do something before I > integrate protein sequences from different sources? > > > > Many thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Thu Feb 2 12:24:04 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Thu, 2 Feb 2017 14:24:04 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> Message-ID: <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> It is nice to have an outgroup of some kind. In your case Human would serve that function. The issue that you might have with distantly related proteins is funny blast alignments that may lead to merging genes. You generally don?t get many false positives because the alignment parameters require a pretty good match, which is unlikely to happen by chance. You could limit swiss-prot to mammals if you wanted to. Sometimes I?ll try different combinations of evidence on a few large scaffolds and look at the results in a browser to get a feel for what is going to work best. Thanks, Mike > On Feb 2, 2017, at 11:16 AM, Quanwei Zhang wrote: > > > Thank for you suggestions. So it does not matter if there are redundancies of protein sequence from different sources? > I am trying to annotating a rodent genome, and planned to collect protein sequences of human, mouse, rat from bout UniProt and NCBI (besides I also have RNA-seq data). I choose these species, because they are close to the species that I am working on and they are well annotated. But I saw someone said that if we choose protein sequence from one lineage, the genes that are missing in the lineage will not be detected. And in the following paper, the authors claim they used the entire SwissProt database as the input. How do you think about this? Should I include protein sequences from more species (like all Eukaryota)? I think it can help us identify more genes, but on the other hand won't this also give us more false positives? > > This paper used the entire SwissProt database as the input. > Insights into the evolution of longevity from the bowhead whale genome. 2015. Cell Rep 10(1): 112-122. > > Thanks > > Best > Quanwei > > 2017-01-31 15:57 GMT-05:00 Michael Campbell >: > Hi Quanwei, > > (1) When I use uniprot I use SWISS-prot and not tremble. > (2) I don?t merge files together. I just pass them all to MAKER as a comma separated list. > > Thanks, > Mike > > > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang > wrote: > > > > I wonder what's the best way to collect protein sequences for gene annotation of a de novo genome assembly. > > (1) My first choice is to get protein sequences of human and mouse from UniProt. At this step, I am not clear whether I should download the reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., TrEMBL). > > (2) On ther other hand, I also get protein sequences from NCBI, should I just simply merge those fasta files. Does it matter if there are redundancies? And also, if I get protein sequences from different sources, they may not have the same quality. Do I need to do something before I integrate protein sequences from different sources? > > > > Many thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Thu Feb 2 14:05:53 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Thu, 2 Feb 2017 16:05:53 -0500 Subject: [maker-devel] collecting protein sequences as evidences In-Reply-To: <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> References: <2E4D90C9-6D6E-4F52-A361-AFB06A61D2C2@gmail.com> <2EF296FE-85C4-4514-ADE5-CEEA1F3699E1@gmail.com> Message-ID: Thank you for your suggestions. I need to have some tests. Best Quanwei 2017-02-02 14:24 GMT-05:00 Michael Campbell : > It is nice to have an outgroup of some kind. In your case Human would > serve that function. The issue that you might have with distantly related > proteins is funny blast alignments that may lead to merging genes. You > generally don?t get many false positives because the alignment parameters > require a pretty good match, which is unlikely to happen by chance. You > could limit swiss-prot to mammals if you wanted to. > > Sometimes I?ll try different combinations of evidence on a few large > scaffolds and look at the results in a browser to get a feel for what is > going to work best. > > Thanks, > Mike > > On Feb 2, 2017, at 11:16 AM, Quanwei Zhang wrote: > > > Thank for you suggestions. So it does not matter if there are redundancies > of protein sequence from different sources? > I am trying to annotating a *rodent *genome, and planned to collect > protein sequences of human, mouse, rat from bout UniProt and NCBI (besides > I also have RNA-seq data). I choose these species, because they are close > to the species that I am working on and they are well annotated. But I saw > someone said that if we choose protein sequence from one lineage, the genes > that are missing in the lineage will not be detected. And in the following > paper, the authors claim they used the entire SwissProt database as the > input. How do you think about this? Should I include protein sequences from > more species (like all Eukaryota)? I think it can help us identify more > genes, but on the other hand won't this also give us more false positives? > > This paper used the entire SwissProt database as the input. > Insights into the evolution of longevity from the bowhead whale genome. > 2015. *Cell Rep* *10*(1): 112-122. > > Thanks > > Best > Quanwei > > 2017-01-31 15:57 GMT-05:00 Michael Campbell >: > >> Hi Quanwei, >> >> (1) When I use uniprot I use SWISS-prot and not tremble. >> (2) I don?t merge files together. I just pass them all to MAKER as a >> comma separated list. >> >> Thanks, >> Mike >> >> > On Jan 31, 2017, at 12:36 PM, Quanwei Zhang >> wrote: >> > >> > I wonder what's the best way to collect protein sequences for gene >> annotation of a de novo genome assembly. >> > (1) My first choice is to get protein sequences of human and mouse from >> UniProt. At this step, I am not clear whether I should download the >> reviewed ones (i.e., SWISS-prot) or automatically annotated ones (i.e., >> TrEMBL). >> > (2) On ther other hand, I also get protein sequences from NCBI, should >> I just simply merge those fasta files. Does it matter if there are >> redundancies? And also, if I get protein sequences from different sources, >> they may not have the same quality. Do I need to do something before I >> integrate protein sequences from different sources? >> > >> > Many thanks >> > >> > Best >> > Quanwei >> > _______________________________________________ >> > maker-devel mailing list >> > maker-devel at box290.bluehost.com >> > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 3 09:04:45 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 3 Feb 2017 11:04:45 -0500 Subject: [maker-devel] augustus failed in maker2 Message-ID: I am trying to add augustus to the prediction. The maker2 run well without augustus, and augustus seems installed correctly. But it returns me the following errors. Any ideas or suggestions? What I did is just to set "augustus_species=human" in the "maker_opts.ctl" file, and I am running maker on a HPC with linux system. preparing ab-inits ERROR: Augustus failed --> rank=NA, hostname=n530 ERROR: Failed while preparing ab-inits ERROR: Chunk failed at level:0, tier_type:2 FAILED CONTIG:CasCan_contig_0 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:CasCan_contig_0 Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 3 10:15:39 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 3 Feb 2017 10:15:39 -0700 Subject: [maker-devel] augustus failed in maker2 In-Reply-To: References: Message-ID: Look a little further back into the STERR log to see if there are other errors further back. The issue is probably your Augustus installation. Try running it by itself (outside of MAKER) on the same dataset, and it may help identify on what is happening. Thanks, Carson > On Feb 3, 2017, at 9:04 AM, Quanwei Zhang wrote: > > I am trying to add augustus to the prediction. The maker2 run well without augustus, and augustus seems installed correctly. But it returns me the following errors. Any ideas or suggestions? What I did is just to set "augustus_species=human" in the "maker_opts.ctl" file, and I am running maker on a HPC with linux system. > > preparing ab-inits > ERROR: Augustus failed > --> rank=NA, hostname=n530 > ERROR: Failed while preparing ab-inits > ERROR: Chunk failed at level:0, tier_type:2 > FAILED CONTIG:CasCan_contig_0 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:CasCan_contig_0 > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From mnaymik at tgen.org Mon Feb 6 14:27:22 2017 From: mnaymik at tgen.org (Marcus Naymik) Date: Mon, 6 Feb 2017 14:27:22 -0700 Subject: [maker-devel] Can't Install Proc::Signal Message-ID: I can't find Proc::Signal in cpan or anywhere. How can I install it? -- *This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From alisa.postma at gmail.com Sun Feb 5 06:15:08 2017 From: alisa.postma at gmail.com (Alisa Postma) Date: Sun, 5 Feb 2017 15:15:08 +0200 Subject: [maker-devel] Problem with fgenesh protein IDs Message-ID: Good afternoon I am having problems with my all.maker.proteins.fasta file. I ran maker with ab initio gene predictions from augustus and genemark and I added my fgenesh gff file to the pred_gff option in the ctl file. However, my maker protein fasta file does not give the fgenesh proteins IDs similar to that for augustus and genemark. For example: >FGENESH protein AED:0.06 eAED:0.06 QI:0|0|0|1|1|1|3|0|433 vs. >maker-scaffold74-augustus-gene-0.179-mRNA-1 protein AED:0.07 eAED:0.07 QI:123|1|1|1|1|1|5|793|326 I would appreciate any advice on this matter. Kind regards Alisa -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 6 21:05:33 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 6 Feb 2017 21:05:33 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: References: Message-ID: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Proc::Signal is part of MAKER?s internal libraries. If you are getting an error saying it is missing, then you have a problem with your installation. Either it was not installed successfully, or you modified the location of files or executables post installation. Make sure you do not move the bin directory or its contents. If you need to be somewhere else, it is best to use softlinks instead. Thanks, Carson > On Feb 6, 2017, at 2:27 PM, Marcus Naymik wrote: > > I can't find Proc::Signal in cpan or anywhere. How can I install it? > > This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From mnaymik at tgen.org Tue Feb 7 10:00:37 2017 From: mnaymik at tgen.org (Marcus Naymik) Date: Tue, 7 Feb 2017 10:00:37 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> References: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Message-ID: Ah, It was listed on as a prereq on your wiki and was trying to install all those first. I was just a little confused. Thanks! Marcus On Mon, Feb 6, 2017 at 9:05 PM, Carson Holt wrote: > Proc::Signal is part of MAKER?s internal libraries. If you are getting an > error saying it is missing, then you have a problem with your installation. > > Either it was not installed successfully, or you modified the location of > files or executables post installation. > > Make sure you do not move the bin directory or its contents. If you need > to be somewhere else, it is best to use softlinks instead. > > Thanks, > Carson > > On Feb 6, 2017, at 2:27 PM, Marcus Naymik wrote: > > I can't find Proc::Signal in cpan or anywhere. How can I install it? > > *This electronic message is intended to be for the use only of the named > recipient, and may contain information that is confidential or privileged, > including patient health information. If you are not the intended > recipient, you are hereby notified that any disclosure, copying, > distribution or use of the contents of this message is strictly prohibited. > If you have received this message in error or are not the named recipient, > please notify us immediately by contacting the sender at the electronic > mail address noted above, and delete and destroy all copies of this > message. Thank you.* > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -- *This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 7 10:11:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 7 Feb 2017 10:11:46 -0700 Subject: [maker-devel] Can't Install Proc::Signal In-Reply-To: References: <184A82E7-A16B-489E-A7D4-55B50EF28533@gmail.com> Message-ID: Use the MAKER Tutorial for GMOD Online Training 2014 It is the most up to date ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014 Thanks, Carson > On Feb 7, 2017, at 10:00 AM, Marcus Naymik wrote: > > Ah, It was listed on as a prereq on your wiki and was trying to install all those first. I was just a little confused. Thanks! > > Marcus > > On Mon, Feb 6, 2017 at 9:05 PM, Carson Holt > wrote: > Proc::Signal is part of MAKER?s internal libraries. If you are getting an error saying it is missing, then you have a problem with your installation. > > Either it was not installed successfully, or you modified the location of files or executables post installation. > > Make sure you do not move the bin directory or its contents. If you need to be somewhere else, it is best to use softlinks instead. > > Thanks, > Carson > >> On Feb 6, 2017, at 2:27 PM, Marcus Naymik > wrote: >> >> I can't find Proc::Signal in cpan or anywhere. How can I install it? >> >> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > > This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged, including patient health information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 7 10:12:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 7 Feb 2017 10:12:46 -0700 Subject: [maker-devel] Problem with fgenesh protein IDs In-Reply-To: References: Message-ID: The issue is probably with the format of your GFF3. I can take a look if you want. ?Carson > On Feb 5, 2017, at 6:15 AM, Alisa Postma wrote: > > Good afternoon > > I am having problems with my all.maker.proteins.fasta file. > > I ran maker with ab initio gene predictions from augustus and genemark and I added my fgenesh gff file to the pred_gff option in the ctl file. > > However, my maker protein fasta file does not give the fgenesh proteins IDs similar to that for augustus and genemark. > > For example: > >FGENESH protein AED:0.06 eAED:0.06 QI:0|0|0|1|1|1|3|0|433 > > vs. > > > >maker-scaffold74-augustus-gene-0.179-mRNA-1 protein AED:0.07 eAED:0.07 QI:123|1|1|1|1|1|5|793|326 > > I would appreciate any advice on this matter. > > Kind regards > > Alisa > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Wed Feb 8 08:45:11 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Wed, 8 Feb 2017 10:45:11 -0500 Subject: [maker-devel] make use of other gene finders in maker Message-ID: Hello: I am trying to annotate the new genome of a rodent species. I saw a clear pipeline to train the SNAP gene finder. But not clear what I should do if I also want to include augustus and/or GeneMark. Do I need to (could I and how) train it in Maker? In one of recently published paper, they mention the augustus was trained in Maker, but I am not clear how to do it? I wonder what's you opinions on using other gene finders in Maker. Thanks "Unique features of a global human ectoparasite identified through sequencing of the bed bug genome." Nat Commun, 2016. In this paper, the training step was described as below. *Three rounds of training of the Augustus and SNAP gene predictors within Maker were used to bootstrap to a high-quality training set.* Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Wed Feb 8 08:56:08 2017 From: dence at genetics.utah.edu (Daniel Ence) Date: Wed, 8 Feb 2017 15:56:08 +0000 Subject: [maker-devel] make use of other gene finders in maker In-Reply-To: References: Message-ID: <02D8BB85-1B0B-4028-A3F6-182F4F3318F7@genetics.utah.edu> Hi Quanwei, MAKER will run Augustus, SNAP, genemark or Fgenesh internally. You can also provide predictions from other gene predictors in valid GFF3 format in the ?pred_gff" field in the maker_opts control file. Genemark is self training on your genome. You provide a path to the ?es.mod? file that Genemark makes in the ?gmhmm? field. For augustus, the training is more involved, but is described here: http://www.molecularevolution.org/molevolfiles/exercises/augustus/training.html Augustus, snap, Fgenesh, and genemark are all complementary in some respects, but Augustus usually provides the most predictions that MAKER ends up selecting to promote to models. Hope that helps, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Feb 8, 2017, at 10:45 AM, Quanwei Zhang > wrote: Hello: I am trying to annotate the new genome of a rodent species. I saw a clear pipeline to train the SNAP gene finder. But not clear what I should do if I also want to include augustus and/or GeneMark. Do I need to (could I and how) train it in Maker? In one of recently published paper, they mention the augustus was trained in Maker, but I am not clear how to do it? I wonder what's you opinions on using other gene finders in Maker. Thanks "Unique features of a global human ectoparasite identified through sequencing of the bed bug genome." Nat Commun, 2016. In this paper, the training step was described as below. Three rounds of training of the Augustus and SNAP gene predictors within Maker were used to bootstrap to a high-quality training set. Best Quanwei _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 8 10:18:49 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 8 Feb 2017 12:18:49 -0500 Subject: [maker-devel] MAKER and OpenMPI Message-ID: Hello everyone, I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 8 10:22:19 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 10:22:19 -0700 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: References: Message-ID: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> You might have two different MPI flavors installed. Since the library and executable files have the same name, this kind of mismatch can happen in that case. Check the locations of MPI libraries given to MAKER during install, and the location of the mpiexec being used when you run. There is likely a mismatch with parts of one MPI flavor being used for one but not the other. ?Carson > On Feb 8, 2017, at 10:18 AM, Seth Munholland wrote: > > Hello everyone, > > I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 8 10:27:15 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 8 Feb 2017 12:27:15 -0500 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> References: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> Message-ID: That's for sure what it is, but I never installed a second flavour. I was commited to MPICH from the get go and am trying to figure out where OpenMPI came from. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 8, 2017 at 12:22 PM, Carson Holt wrote: > You might have two different MPI flavors installed. Since the library and > executable files have the same name, this kind of mismatch can happen in > that case. Check the locations of MPI libraries given to MAKER during > install, and the location of the mpiexec being used when you run. There is > likely a mismatch with parts of one MPI flavor being used for one but not > the other. > > ?Carson > > > > On Feb 8, 2017, at 10:18 AM, Seth Munholland wrote: > > Hello everyone, > > I'm setting up a cluster and configured MPICH on it, but after installing > MAKER my mpiexec gave me an error that comes from an improperly configured > OpenMPI. The weird thing is I haven't installed anything else associated > to MPI, particularly not OpenMPI. Did I get server gremlins or is there > any way MAKER's installation somehow is looking for OpenMPI? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 8 10:32:22 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 10:32:22 -0700 Subject: [maker-devel] MAKER and OpenMPI In-Reply-To: References: <8435CA4B-72A1-4EEC-8336-692E4358C08F@gmail.com> Message-ID: <863A3CDC-24F3-4A91-8D3E-45387262DF46@gmail.com> When you install MAKER, you have to provide the location to mpi.h and mpicc. You may have to reinstall, and give the correct location for one or both (don?t just trust the location that shows up, it may be the wrong one). So you will need to track down both of those for MPICH (depending on where you installed it to). Then do ?which -a mpiexec? to get all locations for every mpiexec found in your path. You will need to ensure the one at the top is the one you want. If it?s not, then you need to reconfigure your PATH or provide the full path to the mpiexec you want each time you launch it. ?Carson > On Feb 8, 2017, at 10:27 AM, Seth Munholland wrote: > > That's for sure what it is, but I never installed a second flavour. I was commited to MPICH from the get go and am trying to figure out where OpenMPI came from. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 8, 2017 at 12:22 PM, Carson Holt > wrote: > You might have two different MPI flavors installed. Since the library and executable files have the same name, this kind of mismatch can happen in that case. Check the locations of MPI libraries given to MAKER during install, and the location of the mpiexec being used when you run. There is likely a mismatch with parts of one MPI flavor being used for one but not the other. > > ?Carson > > > >> On Feb 8, 2017, at 10:18 AM, Seth Munholland > wrote: >> >> Hello everyone, >> >> I'm setting up a cluster and configured MPICH on it, but after installing MAKER my mpiexec gave me an error that comes from an improperly configured OpenMPI. The weird thing is I haven't installed anything else associated to MPI, particularly not OpenMPI. Did I get server gremlins or is there any way MAKER's installation somehow is looking for OpenMPI? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjfi2sb3 at gmail.com Wed Feb 8 00:22:57 2017 From: mjfi2sb3 at gmail.com (Salim Bougouffa) Date: Wed, 08 Feb 2017 07:22:57 +0000 Subject: [maker-devel] Masking is causing problems with exon/CDS boundary predictions Message-ID: Hi, I am having trouble with MAKER/AUGUSTUS annotation. One of the recurrent ones is the example I attach in artemis screenshot. In red is the gene of interest. There are three GFF files. Top is augustus standalone that I executed on the non-masked scaffold. The second is maker annotation and third is the augustus (from maker) on the masked query. The gene clearly has four exons and three CDS but the masking seem to somehow cause augustus to get the boundaries wrong for the first and third CDS and skip the second CDS entirely. How do I fix this problem? Best, /SB [image: artemis.png] -- ____________________________ Sent from Inbox Mobile -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: artemis.png Type: image/png Size: 128354 bytes Desc: not available URL: From carsonhh at gmail.com Wed Feb 8 12:31:16 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 8 Feb 2017 12:31:16 -0700 Subject: [maker-devel] Masking is causing problems with exon/CDS boundary predictions In-Reply-To: References: Message-ID: <4B4ACC1F-541E-4182-9DE6-5C22CEF2C91D@gmail.com> What do your evidence alignments look like? i.e. assembled mRNA-seq and protein homology? Not the mRNA-seq pileup (because MAKER can?t see that). Masking is applied before evidence seeding and the first ab initio run. It is actually then removed for evidence polishing around splice sites, and the hint based rerun of Augustus. So evidence is allowed to extend through masked regions and Augustus can make it part of the model if it wants on the second run. Since it didn?t the question becomes, what does the transcript and protein homology alignments from the maker run look like. ?Carson > On Feb 8, 2017, at 12:22 AM, Salim Bougouffa wrote: > > Hi, > > I am having trouble with MAKER/AUGUSTUS annotation. > > One of the recurrent ones is the example I attach in artemis screenshot. In red is the gene of interest. There are three GFF files. Top is augustus standalone that I executed on the non-masked scaffold. The second is maker annotation and third is the augustus (from maker) on the masked query. > > > The gene clearly has four exons and three CDS but the masking seem to somehow cause augustus to get the boundaries wrong for the first and third CDS and skip the second CDS entirely. > > How do I fix this problem? > > Best, > /SB > > -- > ____________________________ > Sent from Inbox Mobile > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 08:50:24 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 10:50:24 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? Message-ID: Hello: I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 09:03:41 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 11:03:41 -0500 Subject: [maker-devel] training of gene finders using whole assembly or longest contigs? Message-ID: Hello: I am training the gene finders using the whole assembly. But it seems very time consuming. Besides, I have to repeat the training process several times. Although I am running it on 25 nodes on a server, it may still take 3 (or even more) weeks for the training. I wonder how you guys train the SNAP. Do you use the whole assembly or just select the longest contigs for the training. If I only use longest contigs (like top 20% longest), will it be good enough as that get by using the whole assembly? Or should I randomly select 20% contigs for the training, for which we will have similar length distribution as the whole assembly? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From Alessandro.Rossoni at uni-duesseldorf.de Fri Feb 10 08:50:07 2017 From: Alessandro.Rossoni at uni-duesseldorf.de (Alessandro Rossoni) Date: Fri, 10 Feb 2017 16:50:07 +0100 Subject: [maker-devel] Updating Reference Gene Models - Legacy - Strategy Message-ID: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> Dear makers of MAKER, first of all - thank you for this awesome program! In the context of my project, I have been running MAKER on a set of novel genomes and it worked very well :) During the last days, I realized that the reference sequences of species A that I have been using as starting point for gene model prediction for species B, C and D are a bit flawed. How do I know that? Through visualization of novel RNA-Seq data that was mapped to the "old" gene models of species A, I came to the conclusion that the "old" gene models are not really accurate. In fact, between 52?62% of the reads map within annotated exonic regions of the genome and up to 47% map within intergenic regions. Intron/Exon boarders are pretty messed up and there are a lot more transcribed sequences in species A than previously thought. Hence, I would love to update the gene models of species A including the new RNA-Seq evidence and hope to get more accurate gene models out of it. The more accurate gene models of species A would be then used to predict genes for species B, C and D (which are hopefully going to benefit from the more accurate input). However, I there is a little understanding issue on how to set the parameters of the maker_opts.ctl. My plan is to produce new RNA-Seq based gene models for species A using Cufflinks, Stringtie, Breaker, Trinity and Velvet. I would pass the output to the maker_opts.ctl as: model_gff=Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff #the new gene models est=Species_A_reference_ests.fasta #the old/flawed gene models protein=swissprot.fasta Question 1: is this correct so far? But what sequences do I use for training the ab-initio predictors? snaphmm= augustus_species= Question 2: Do I use the "old/flawed" sequences that I know are not really good? I am not sure what sequences to use for training. Any help on this issue would be amazing! Best, Ale -- Alessandro W. Rossoni, M.Sc. Institute for Plant Biochemistry Heinrich-Heine-University -- http:///www.plant-biochemistry.hhu.de/ E-Mail: alessandro.rossoni at hhu-duesseldorf.de From carsonhh at gmail.com Fri Feb 10 09:36:46 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:36:46 -0700 Subject: [maker-devel] Updating Reference Gene Models - Legacy - Strategy In-Reply-To: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> References: <5cd4b88c1daa63257aeac3be91acd0c0@uni-duesseldorf.de> Message-ID: If the old models are poor, then I suggest you do new training using BUSCO, CEGMA, or the est2genome or protein2genome options within MAKER ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors Also this thread ?> https://groups.google.com/forum/#!topic/maker-devel/FWMSTdqWQqI model_gff is for existing gene models you want to keep. So none of these should go there ?> Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff model_gff will always make it into the final annotation set even without any evidence support. By putting those files there, you are basically turning every feature in each of those files into a final gene model no matter how bad it is. Also if the original models are poor, don?t put them there either. You can doing reciprocal best blast hits with final models to old models to see how they match each other in the end. Will take a little data processing to make it work though. For all transcript based files, you should provide those to est_gff since they are evidence alignments and not model predictions. For Breaker.gff, that should be pred_gff since it is a prediction model. With Trinity, I suggest you provide the fasta file and allow MAKER to align and filter things rather than a GFF3. The problem with using GFF3 is you are basically short circuiting upstream prioritization and filtering saying ?take this evidence as is.? Also providing same evidence from multiple sources is a bad idea. By purposely making the evidence dataset more noisy, you are forcing lower accuracy. My suggesting would be not to use Cufflinks (it will introduce a very high false positive rate). Provide Trinity input as fasta (also make sure you use jaccard_clip option was used when assembling). And you will have to manually review models with and without Stringtie data to see if it hurts more than it helps. Provide Breaker.gff to pred_gff, but still allow maker to run Augustus itself internally (otherwise you won?t be able to use protein evidence as hints). Thanks, Carson > On Feb 10, 2017, at 8:50 AM, Alessandro Rossoni wrote: > > Dear makers of MAKER, > first of all - thank you for this awesome program! In the context of my project, I have been running MAKER on a set of novel genomes and it worked very well :) > > During the last days, I realized that the reference sequences of species A that I have been using as starting point for gene model prediction for species B, C and D are a bit flawed. How do I know that? Through visualization of novel RNA-Seq data that was mapped to the "old" gene models of species A, I came to the conclusion that the "old" gene models are not really accurate. In fact, between 52?62% of the reads map within annotated exonic regions of the genome and up to 47% map within intergenic regions. Intron/Exon boarders are pretty messed up and there are a lot more transcribed sequences in species A than previously thought. > > Hence, I would love to update the gene models of species A including the new RNA-Seq evidence and hope to get more accurate gene models out of it. The more accurate gene models of species A would be then used to predict genes for species B, C and D (which are hopefully going to benefit from the more accurate input). However, I there is a little understanding issue on how to set the parameters of the maker_opts.ctl. > > My plan is to produce new RNA-Seq based gene models for species A using Cufflinks, Stringtie, Breaker, Trinity and Velvet. I would pass the output to the maker_opts.ctl as: > > model_gff=Cufflinks.gff,Stringtie.gff,Breaker.gff,Trinity.gff,Velvet.gff #the new gene models > est=Species_A_reference_ests.fasta #the old/flawed gene models > protein=swissprot.fasta > > Question 1: is this correct so far? > > But what sequences do I use for training the ab-initio predictors? > snaphmm= > augustus_species= > > Question 2: Do I use the "old/flawed" sequences that I know are not really good? I am not sure what sequences to use for training. > > Any help on this issue would be amazing! > Best, > Ale > > > -- > Alessandro W. Rossoni, M.Sc. > Institute for Plant Biochemistry > Heinrich-Heine-University > > -- > http:///www.plant-biochemistry.hhu.de/ > E-Mail: alessandro.rossoni at hhu-duesseldorf.de > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 10 09:39:31 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:39:31 -0700 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: MAKER is restartable. As long as you run each time in the same location, it can reuse existing alignments from the previous run. You also only need to train on ~10MB of the genome depending on gene density. Target size should be 300-400 genes. If you follow this GMOD wiki, this is demonstrated (you can also watch video - link as top of page - to see it being done) ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors ?Carson > On Feb 10, 2017, at 8:50 AM, Quanwei Zhang wrote: > > Hello: > > I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 10 09:42:13 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 10 Feb 2017 09:42:13 -0700 Subject: [maker-devel] training of gene finders using whole assembly or longest contigs? In-Reply-To: References: Message-ID: Example of training here ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors You can also search the devel mailing list archives here ?> https://groups.google.com/forum/#!forum/maker-devel There are lots and lots of threads that go into detail on training. Note more than 2 rounds of training is not beneficial, and can actually make performance worse (there is an overtraining paradox). ?Carson > On Feb 10, 2017, at 9:03 AM, Quanwei Zhang wrote: > > Hello: > > I am training the gene finders using the whole assembly. But it seems very time consuming. Besides, I have to repeat the training process several times. Although I am running it on 25 nodes on a server, it may still take 3 (or even more) weeks for the training. I wonder how you guys train the SNAP. Do you use the whole assembly or just select the longest contigs for the training. If I only use longest contigs (like top 20% longest), will it be good enough as that get by using the whole assembly? Or should I randomly select 20% contigs for the training, for which we will have similar length distribution as the whole assembly? > > Thanks > > Best > Quanwei > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 10:04:55 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 12:04:55 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: Great. Many thanks. So I can select part of my genome assembly for the training. Which the following ways do you think is better? (a) Select the longest contigs. (b) Randomly select contigs with any length? Thank you! Best Quanwei 2017-02-10 11:39 GMT-05:00 Carson Holt : > MAKER is restartable. As long as you run each time in the same location, > it can reuse existing alignments from the previous run. You also only need > to train on ~10MB of the genome depending on gene density. Target size > should be 300-400 genes. > > If you follow this GMOD wiki, this is demonstrated (you can also watch > video - link as top of page - to see it being done) ?> > http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/ > MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ > ab_initio_Gene_Predictors > > ?Carson > > > > On Feb 10, 2017, at 8:50 AM, Quanwei Zhang wrote: > > Hello: > > I am annotating a new genome using Maker. I have RNA-seq assembly and > protein sequences (from other organisms) in fasta format. Since I need to > train gene finders, so I have to run Maker several times. I think the > aligning process between the transcript assembly (protein sequences) and > the genome assembly may be time consuming. So I wonder whether I can save > such alignment in the first run, and then make use of such alignment in the > following runs? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Fri Feb 10 10:07:51 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Fri, 10 Feb 2017 12:07:51 -0500 Subject: [maker-devel] Evidences in fasta format but be alignment in every run? In-Reply-To: References: Message-ID: The longer ones will have more complete genes on them. If you get a set scaffolds that has about 1,000 genes you probably have enough for training. Mike > On Feb 10, 2017, at 12:04 PM, Quanwei Zhang wrote: > > Great. Many thanks. So I can select part of my genome assembly for the training. Which the following ways do you think is better? (a) Select the longest contigs. (b) Randomly select contigs with any length? > > Thank you! > > Best > Quanwei > > 2017-02-10 11:39 GMT-05:00 Carson Holt >: > MAKER is restartable. As long as you run each time in the same location, it can reuse existing alignments from the previous run. You also only need to train on ~10MB of the genome depending on gene density. Target size should be 300-400 genes. > > If you follow this GMOD wiki, this is demonstrated (you can also watch video - link as top of page - to see it being done) ?> http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_GMOD_Online_Training_2014#Training_ab_initio_Gene_Predictors > > ?Carson > > > >> On Feb 10, 2017, at 8:50 AM, Quanwei Zhang > wrote: >> >> Hello: >> >> I am annotating a new genome using Maker. I have RNA-seq assembly and protein sequences (from other organisms) in fasta format. Since I need to train gene finders, so I have to run Maker several times. I think the aligning process between the transcript assembly (protein sequences) and the genome assembly may be time consuming. So I wonder whether I can save such alignment in the first run, and then make use of such alignment in the following runs? >> >> Thanks >> >> Best >> Quanwei >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 10 21:53:02 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 10 Feb 2017 23:53:02 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" Message-ID: Hello: I want to assign gene function to the predicted genes. I collected UniProt/SwissProt human, mouse and rat protein sequences (including both canonical and isoforms). Then use "makeblastdb" to build the database. And then run "blastp -db ... -max_hsps_per_subject 1" following the example in the protocol. But it returns me an error: Unknown argument: "max_hsps_per_subject". Why this happens? Is it because I am using a different version of blastp? USAGE blastp [-h] [-help] [-import_search_strategy filename] [-export_search_strategy filename] [-task task_name] [-db database_name] [-dbsize num_letters] [-gilist filename] [-seqidlist filename] [-negative_gilist filename] [-entrez_query entrez_query] [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm] [-subject subject_input_file] [-subject_loc range] [-query input_file] [-out output_file] [-evalue evalue] [-word_size int_value] [-gapopen open_penalty] [-gapextend extend_penalty] [-qcov_hsp_perc float_value] [-max_hsps int_value] [-xdrop_ungap float_value] [-xdrop_gap float_value] [-xdrop_gap_final float_value] [-searchsp int_value] [-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking] [-matrix matrix_name] [-threshold float_value] [-culling_limit int_value] [-best_hit_overhang float_value] [-best_hit_score_edge float_value] [-window_size int_value] [-lcase_masking] [-query_loc range] [-parse_deflines] [-outfmt format] [-show_gis] [-num_descriptions int_value] [-num_alignments int_value] [-line_length line_length] [-html] [-max_target_seqs num_sequences] [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo] [-use_sw_tback] [-version] Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Mon Feb 13 07:02:30 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Mon, 13 Feb 2017 09:02:30 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" In-Reply-To: References: Message-ID: Hi Quanwei, Different versions/implementations of blast do have parameters. Based on the usage you posted my guess is that the parameter you want is ?-max_hsps? Thanks, Mike > On Feb 10, 2017, at 11:53 PM, Quanwei Zhang wrote: > > Hello: > > I want to assign gene function to the predicted genes. I collected UniProt/SwissProt human, mouse and rat protein sequences (including both canonical and isoforms). Then use "makeblastdb" to build the database. And then run "blastp -db ... -max_hsps_per_subject 1" following the example in the protocol. But it returns me an error: Unknown argument: "max_hsps_per_subject". Why this happens? Is it because I am using a different version of blastp? > > USAGE > blastp [-h] [-help] [-import_search_strategy filename] > [-export_search_strategy filename] [-task task_name] [-db database_name] > [-dbsize num_letters] [-gilist filename] [-seqidlist filename] > [-negative_gilist filename] [-entrez_query entrez_query] > [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm] > [-subject subject_input_file] [-subject_loc range] [-query input_file] > [-out output_file] [-evalue evalue] [-word_size int_value] > [-gapopen open_penalty] [-gapextend extend_penalty] > [-qcov_hsp_perc float_value] [-max_hsps int_value] > [-xdrop_ungap float_value] [-xdrop_gap float_value] > [-xdrop_gap_final float_value] [-searchsp int_value] > [-sum_stats bool_value] [-seg SEG_options] [-soft_masking soft_masking] > [-matrix matrix_name] [-threshold float_value] [-culling_limit int_value] > [-best_hit_overhang float_value] [-best_hit_score_edge float_value] > [-window_size int_value] [-lcase_masking] [-query_loc range] > [-parse_deflines] [-outfmt format] [-show_gis] > [-num_descriptions int_value] [-num_alignments int_value] > [-line_length line_length] [-html] [-max_target_seqs num_sequences] > [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats compo] > [-use_sw_tback] [-version] > > Thanks > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From qwzhang0601 at gmail.com Mon Feb 13 08:02:24 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 10:02:24 -0500 Subject: [maker-devel] Error: Unknown argument: "max_hsps_per_subject" In-Reply-To: References: Message-ID: Thanks. Yes, I should use "-max_hsps". Best Quanwei 2017-02-13 9:02 GMT-05:00 Michael Campbell : > Hi Quanwei, > > Different versions/implementations of blast do have parameters. Based on > the usage you posted my guess is that the parameter you want is ?-max_hsps? > > Thanks, > Mike > > On Feb 10, 2017, at 11:53 PM, Quanwei Zhang > wrote: > > > > Hello: > > > > I want to assign gene function to the predicted genes. I collected > UniProt/SwissProt human, mouse and rat protein sequences (including both > canonical and isoforms). Then use "makeblastdb" to build the database. And > then run "blastp -db ... -max_hsps_per_subject 1" following the example in > the protocol. But it returns me an error: Unknown argument: > "max_hsps_per_subject". Why this happens? Is it because I am using a > different version of blastp? > > > > USAGE > > blastp [-h] [-help] [-import_search_strategy filename] > > [-export_search_strategy filename] [-task task_name] [-db > database_name] > > [-dbsize num_letters] [-gilist filename] [-seqidlist filename] > > [-negative_gilist filename] [-entrez_query entrez_query] > > [-db_soft_mask filtering_algorithm] [-db_hard_mask > filtering_algorithm] > > [-subject subject_input_file] [-subject_loc range] [-query > input_file] > > [-out output_file] [-evalue evalue] [-word_size int_value] > > [-gapopen open_penalty] [-gapextend extend_penalty] > > [-qcov_hsp_perc float_value] [-max_hsps int_value] > > [-xdrop_ungap float_value] [-xdrop_gap float_value] > > [-xdrop_gap_final float_value] [-searchsp int_value] > > [-sum_stats bool_value] [-seg SEG_options] [-soft_masking > soft_masking] > > [-matrix matrix_name] [-threshold float_value] [-culling_limit > int_value] > > [-best_hit_overhang float_value] [-best_hit_score_edge float_value] > > [-window_size int_value] [-lcase_masking] [-query_loc range] > > [-parse_deflines] [-outfmt format] [-show_gis] > > [-num_descriptions int_value] [-num_alignments int_value] > > [-line_length line_length] [-html] [-max_target_seqs num_sequences] > > [-num_threads int_value] [-ungapped] [-remote] [-comp_based_stats > compo] > > [-use_sw_tback] [-version] > > > > Thanks > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 08:16:35 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 10:16:35 -0500 Subject: [maker-devel] failed to assign putative gene function Message-ID: Hello: I am trying to add putative gene function to the predicted gene models. Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. I used canonical and isoform proteins of human, mouse and rat with the script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose context is as below. maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 7e-164 464 snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 3e-80 236 After that, I am trying to add the protein homology data to the Maker gff3 and fasta files with maker_functional_gff and maker_functional_fasta, but get the reports as below. Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 39. Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 39. Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 45. Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 45. Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B ..... I am not sure how to deal with this. I followed the command given in the protocol. Any suggestions? Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 09:09:48 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 11:09:48 -0500 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: I just found at the bottom of the output file it include information like below. But I did not get those GFF3 files with gene homolog information added. >snap_masked-CasCan_contig_27993-processed-gene-0.6-mRNA-1 transcript Name:"Protein of unknown function" offset:0 AED:0.47 eAED:0.47 QI:0|0|0|1|1|1|2|0|84 ATGAAAGACATTGGTACCCCAGAGGCATGGCAGATAATGATGTCCCTCAAGTCTGGACTC TTGGCAGAGATCACATGGGCTTTAGACACCATTAACATTCTACTGTATGATGACAGCAGC ATTATGACCTTCAACCTCAGTCAGTTCCCAGGATTGCTAGAGCTCTTTGAGTATGAGGTG GGTGACCGAAGACAGAGAACTCTACTGGACTCTGGGAGATTCAGTGAAGTGTCTGGTCCA ACCCCTACAGAG Thanks Best Quanwei 2017-02-13 10:16 GMT-05:00 Quanwei Zhang : > Hello: > > I am trying to add putative gene function to the predicted gene models. > Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. > I used canonical and isoform proteins of human, mouse and rat with the > script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose > context is as below. > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN > 69.97 303 91 0 1 303 1 303 7e-164 464 > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 > sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 > 3e-80 236 > > After that, I am trying to add the protein homology data to the Maker gff3 > and fasta files with maker_functional_gff and maker_functional_fasta, but > get the reports as below. > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 39. > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 39. > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 45. > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 45. > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 > of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > ..... > > I am not sure how to deal with this. I followed the command given in the > protocol. Any suggestions? > > Thanks > > Best > Quanwei > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Mon Feb 13 10:03:13 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 13 Feb 2017 12:03:13 -0500 Subject: [maker-devel] Which version of InterProScan should I use Message-ID: I am planning to add domain information to my predicted gene models. But not sure whether the scripts provided by Maker is compatible with the latest version of InterProScan, or should I use a old version. I am using maker2.31.9. It seems the latest version is 5.22-61.0 released on 1/23/17. ftp://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/ Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcui at age.mpg.de Tue Feb 14 05:38:27 2017 From: rcui at age.mpg.de (Ray Cui) Date: Tue, 14 Feb 2017 13:38:27 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints Message-ID: Hello, I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Tue Feb 14 07:45:29 2017 From: dence at genetics.utah.edu (Daniel Ence) Date: Tue, 14 Feb 2017 14:45:29 +0000 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: Message-ID: Hi Ray, I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. ~Daniel On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: Hello, I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rcui at age.mpg.de Tue Feb 14 08:44:19 2017 From: rcui at age.mpg.de (Ray Cui) Date: Tue, 14 Feb 2017 16:44:19 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> Message-ID: Hi Daniel, thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel wrote: > Hi Ray, > > I think you?re on the right track with training Genemark with RNAseq data. > It should only change the training steps, which are external to MAKER, but > not how MAKER runs Genemark. You?ll still give MAKER the path to the > ?es.mod" file made by Genemark. > > For the 2nd question, in the MAKER beta 3, MAKER creates a control file > for EVM, in which you set your weights for the various inputs, and then > MAKER runs EVM alongside all the other gene predictors and chooses the > model that is best supported by the evidence. > > ~Daniel > > > > On Feb 14, 2017, at 7:38 AM, Ray Cui wrote: > > Hello, > > I have sucessfully installed Maker beta 3, working with both > Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio > predictor. > When I read the GeneMark-ES manual, it says that one can use > RNAseq data to aid training. I'm wondering what would be the best way to > integrate Genemark-ET predictions into Maker. Should I run Genemark-ET > independent of Maker, then integrate the GFF at some point during the maker > process? If so, how should I edit the configuration file? Currently maker > has an option called "gmhmm". Should I then train GeneMark by myself with > RNAseq data, then feed the hmm to maker? > > And perhaps an unrelated question is that now Maker beta 3 > supports EVM. I'm wondering how EVM is used by Maker (at which step, what > does it do), and how does it differ from what Maker is designed for (both > reconciles different gene models). > > Best Regards, > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for > Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 <+49%20221%20496> > Mobile: +49 0221 37970 496 > rcui at age.mpg.de > www.age.mpg.de > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Tue Feb 14 12:51:46 2017 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Tue, 14 Feb 2017 14:51:46 -0500 Subject: [maker-devel] Which version of InterProScan should I use In-Reply-To: References: Message-ID: <6A140E2D-603E-46C9-97FA-D59AEAEF95A9@gmail.com> I have been able to make the accessory scripts work with output from InterProScan v5 that I downloaded a couple of month ago. Mike > On Feb 13, 2017, at 12:03 PM, Quanwei Zhang wrote: > > I am planning to add domain information to my predicted gene models. But not sure whether the scripts provided by Maker is compatible with the latest version of InterProScan, or should I use a old version. I am using maker2.31.9. > > It seems the latest version is 5.22-61.0 released on 1/23/17. > ftp://ftp.ebi.ac.uk/pub/databases/interpro/iprscan/5/ > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Thu Feb 16 03:44:39 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Thu, 16 Feb 2017 11:44:39 +0100 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> Message-ID: <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> Hi! Unfortunately all of the options failed on our cluster. See: Hi, Most recent Maker test with --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 Error: --> rank=2, hostname=uc1n518.localdomain [uc1n518:67009] *** Process received signal *** [uc1n518:67009] Signal: Segmentation fault (11) [uc1n518:67009] Signal code: Address not mapped (1) [uc1n518:67009] Failing at address: 0x4b0 -------------------------------------------------------------------------- mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). With: --mca btl ^openib and also this --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 Error: ### Runing Maker example STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... [uc1n514:59985] *** Process received signal *** [uc1n514:59985] Signal: Segmentation fault (11) [uc1n514:59985] Signal code: Address not mapped (1) [uc1n514:59985] Failing at address: 0x4b0 -------------------------------------------------------------------------- mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). Am 28.01.2017 um 21:53 schrieb Carson Holt: > Try adding one of the following to your mpiexec command ?> > > 1. --mca btl ^openib > 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 > 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 > > One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. > > --Carson > > >> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >> >> Hi everybody. >> >> My name is Rainer. I am an administrator for our HPC-Systems at our >> university in Konstanz, Baden-Wuertemberg/Germany. >> The procect is called bwHPC-C5. >> >> See: https://www.bwhpc-c5.de/en/index.php >> >> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >> i get errors while running a Maker job in the MPI-environment. >> >> BUILD STATUS >> >> ============================================================================== >> STATUS MAKER v2.31.9 >> ============================================================================== >> PERL Dependencies: VERIFIED >> External Programs: VERIFIED >> External C Libraries: VERIFIED >> MPI SUPPORT: ENABLED >> MWAS Web Interface: DISABLED >> MAKER PACKAGE: CONFIGURATION OK >> >> MODULES / INCLUDES / COMPILERS >> >> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >> # >> ##### (B) Dependencies: >> # >> # conflict: any other maker version >> # module load compiler/gnu/5.2 >> # module load mpi/openmpi/2.0-gnu-5.2 >> [...] >> >> MPI/MOAB SUBMIT >> >> [...] >> ### Queues ### >> #MSUB -q fat >> #MSUB -l nodes=1:ppn=16 >> #MSUB -l mem=20gb >> #MSUB -l walltime=50:00:00 >> # >> [...] >> echo " " >> echo "### Loading MAKER module:" >> echo " " >> module load bio/maker/2.31.9 >> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >> echo "MAKER_VERSION = $MAKER_VERSION" >> module list >> [...] >> echo " " >> echo "### Runing Maker example" >> echo " " >> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >> export OMPI_MCA_mpi_warn_on_fork=0 >> >> echo "LD_PRELOAD=${LD_PRELOAD}" >> # >> # "STATUS: Processing and indexing input FASTA files..." >> # >> mpiexec -mca btl ^openib -n 16 maker >> [...] >> >> >> E R R O R S >> ======= >> [...] >> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >> STATUS: Parsing control files... >> STATUS: Processing and indexing input FASTA files... >> [uc1n338:113607] *** Process received signal *** >> [uc1n338:113607] Signal: Segmentation fault (11) >> [uc1n338:113607] Signal code: Address not mapped (1) >> [uc1n338:113607] Failing at address: 0x4b0 >> [uc1n338:113608] *** Process received signal *** >> [uc1n338:113608] Signal: Segmentation fault (11) >> [uc1n338:113608] Signal code: Address not mapped (1) >> [uc1n338:113608] Failing at address: 0x4b0 >> [uc1n338:113621] *** Process received signal *** >> [uc1n338:113621] Signal: Segmentation fault (11) >> [uc1n338:113621] Signal code: Address not mapped (1) >> [uc1n338:113621] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >> -------------------------------------------------------------------------- >> [...] >> >> WHATS WRONG HERE!? >> >> Thank you for your help! >> >> All the best , >> >> Rainer >> >> -- >> Rainer Rutka >> University of Konstanz >> Communication, Information, Media Centre (KIM) >> * High-Performance-Computing (HPC) >> * KIM-Support and -Base-Services >> Room: V511 >> 78457 Konstanz, Germany >> +49 7531 88-5413 >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > -- Rainer Rutka Universit?t Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Raum: V511, Tel: 54 13 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From munholl at uwindsor.ca Fri Feb 17 13:11:44 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Fri, 17 Feb 2017 15:11:44 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there Message-ID: Hi Everyone, After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. However I also see $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Feb 17 13:39:33 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 17 Feb 2017 13:39:33 -0700 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: Message-ID: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. Thanks, Carson > On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: > > Hi Everyone, > > After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl > > (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: > > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > However I also see > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From qwzhang0601 at gmail.com Fri Feb 17 14:57:05 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Fri, 17 Feb 2017 16:57:05 -0500 Subject: [maker-devel] default gene model by quality_filter.pl is the same with those by setting keep_preds=0? Message-ID: Hello: I am following the Maker protocol (2014) to rescue rejected gene models. I set keep_preds=1 when I ran Maker, and then used "quality_filter.pl -d" to get the "default" Maker report. According to the annotation it will "Prints transcripts with an AED <1". But in a previous post (the link below), it was said "there are models with AED less than 1 that get rejected for other reasons. ... For example if the only evidence is a protein alignment that has deep overlapping HSPs (extremely low complexity alignment) it will be filtered out even though AED is not technically equal to 1. Also if the overlapping protein evidence is in a different reading frame than the model it is supposed to support then the AED will be less than 1 but eAED will be 1 (extended AED), and the model will be rejected." *So I wonder, whether the "default" result obtained by "quality_filter.pl " are really the same as those achieved by setting keep_preds=0? Will the script also consider other reasons besides "* *AED <1"?* https://groups.google.com/forum/#!searchin/maker-devel/The$20reason$20there$20are$20differences$20between$20the$20runs$20is$20that$20there$20are$20models$20with$20AED$20less$20%7Csort:relevance/maker-devel/97aNJkT3bgk/W0MEyqEZAAAJ Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From qlian003 at ucr.edu Sat Feb 18 09:43:08 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Sat, 18 Feb 2017 08:43:08 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation Message-ID: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> Dear Maker develop team, I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? Thanks Qihua From carsonhh at gmail.com Sun Feb 19 22:39:14 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 22:39:14 -0700 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> Message-ID: <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> MAKER was support was designed with GeneMark-ES. It may or may not work with GeneMark-ET. So any MAKER related archive posts etc. will be related to the latter. With GeneMark-ES, you simply provided a genome assembly and let it run. It would then produce several files and output directories. The es.mod file was the one you provided to MAKER. I don?t know how this compares to GeneMark-ET. ?Carson > On Feb 14, 2017, at 8:44 AM, Ray Cui wrote: > > Hi Daniel, > > thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? > > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 > Mobile: +49 0221 37970 496 <> > rcui at age.mpg.de > www.age.mpg.de > > > > On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel > wrote: > Hi Ray, > > I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. > > For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. > > ~Daniel > > > >> On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: >> >> Hello, >> >> I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. >> When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? >> >> And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). >> >> Best Regards, >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 >> Mobile: +49 0221 37970 496 <> >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Sun Feb 19 22:43:49 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 22:43:49 -0700 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> Message-ID: <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> Try running just on a single node (not across nodes). If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and running MAKER with that new version. You can install it in your home directory and test from there, just make sure to add it to your path. Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. ?Carson > On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: > > Hi! > > Unfortunately all of the options failed on our cluster. > > See: > > Hi, > > Most recent Maker test with > --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 > Error: > --> rank=2, hostname=uc1n518.localdomain > [uc1n518:67009] *** Process received signal *** > [uc1n518:67009] Signal: Segmentation fault (11) > [uc1n518:67009] Signal code: Address not mapped (1) > [uc1n518:67009] Failing at address: 0x4b0 > -------------------------------------------------------------------------- > mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). > > > With: > --mca btl ^openib > and also this > --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 > Error: > ### Runing Maker example > > STATUS: Parsing control files... > STATUS: Processing and indexing input FASTA files... > [uc1n514:59985] *** Process received signal *** > [uc1n514:59985] Signal: Segmentation fault (11) > [uc1n514:59985] Signal code: Address not mapped (1) > [uc1n514:59985] Failing at address: 0x4b0 > -------------------------------------------------------------------------- > mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). > > > Am 28.01.2017 um 21:53 schrieb Carson Holt: >> Try adding one of the following to your mpiexec command ?> >> >> 1. --mca btl ^openib >> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >> >> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >> >> --Carson >> >> >>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>> >>> Hi everybody. >>> >>> My name is Rainer. I am an administrator for our HPC-Systems at our >>> university in Konstanz, Baden-Wuertemberg/Germany. >>> The procect is called bwHPC-C5. >>> >>> See: https://www.bwhpc-c5.de/en/index.php >>> >>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>> i get errors while running a Maker job in the MPI-environment. >>> >>> BUILD STATUS >>> >>> ============================================================================== >>> STATUS MAKER v2.31.9 >>> ============================================================================== >>> PERL Dependencies: VERIFIED >>> External Programs: VERIFIED >>> External C Libraries: VERIFIED >>> MPI SUPPORT: ENABLED >>> MWAS Web Interface: DISABLED >>> MAKER PACKAGE: CONFIGURATION OK >>> >>> MODULES / INCLUDES / COMPILERS >>> >>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>> # >>> ##### (B) Dependencies: >>> # >>> # conflict: any other maker version >>> # module load compiler/gnu/5.2 >>> # module load mpi/openmpi/2.0-gnu-5.2 >>> [...] >>> >>> MPI/MOAB SUBMIT >>> >>> [...] >>> ### Queues ### >>> #MSUB -q fat >>> #MSUB -l nodes=1:ppn=16 >>> #MSUB -l mem=20gb >>> #MSUB -l walltime=50:00:00 >>> # >>> [...] >>> echo " " >>> echo "### Loading MAKER module:" >>> echo " " >>> module load bio/maker/2.31.9 >>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>> echo "MAKER_VERSION = $MAKER_VERSION" >>> module list >>> [...] >>> echo " " >>> echo "### Runing Maker example" >>> echo " " >>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>> export OMPI_MCA_mpi_warn_on_fork=0 >>> >>> echo "LD_PRELOAD=${LD_PRELOAD}" >>> # >>> # "STATUS: Processing and indexing input FASTA files..." >>> # >>> mpiexec -mca btl ^openib -n 16 maker >>> [...] >>> >>> >>> E R R O R S >>> ======= >>> [...] >>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>> STATUS: Parsing control files... >>> STATUS: Processing and indexing input FASTA files... >>> [uc1n338:113607] *** Process received signal *** >>> [uc1n338:113607] Signal: Segmentation fault (11) >>> [uc1n338:113607] Signal code: Address not mapped (1) >>> [uc1n338:113607] Failing at address: 0x4b0 >>> [uc1n338:113608] *** Process received signal *** >>> [uc1n338:113608] Signal: Segmentation fault (11) >>> [uc1n338:113608] Signal code: Address not mapped (1) >>> [uc1n338:113608] Failing at address: 0x4b0 >>> [uc1n338:113621] *** Process received signal *** >>> [uc1n338:113621] Signal: Segmentation fault (11) >>> [uc1n338:113621] Signal code: Address not mapped (1) >>> [uc1n338:113621] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>> -------------------------------------------------------------------------- >>> [...] >>> >>> WHATS WRONG HERE!? >>> >>> Thank you for your help! >>> >>> All the best , >>> >>> Rainer >>> >>> -- >>> Rainer Rutka >>> University of Konstanz >>> Communication, Information, Media Centre (KIM) >>> * High-Performance-Computing (HPC) >>> * KIM-Support and -Base-Services >>> Room: V511 >>> 78457 Konstanz, Germany >>> +49 7531 88-5413 >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > > -- > Rainer Rutka > Universit?t Konstanz > Kommunikations-, Informations-, Medienzentrum (KIM) > Raum: V511, Tel: 54 13 > From carsonhh at gmail.com Sun Feb 19 23:05:09 2017 From: carsonhh at gmail.com (Carson Holt) Date: Sun, 19 Feb 2017 23:05:09 -0700 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> Message-ID: <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. Thanks, Carson > On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: > > Dear Maker develop team, > > I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? > > Thanks > Qihua > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From rcui at age.mpg.de Mon Feb 20 01:59:12 2017 From: rcui at age.mpg.de (Ray Cui) Date: Mon, 20 Feb 2017 09:59:12 +0100 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> Message-ID: Dear Carson, I have now run GeneMark-ET, and it produces a trained .mod file. I think it can be then passed to Maker. Do you know what is the final constructed command line in Maker that calls genemark? Genemark-et and es use the same perl script so one probably only needs to use the --prediction and --predict_with xxx.mod options to predict genes using the species specific parameters (bypassing regular training and prediction steps) Best Regards, Ray Dr. Rongfeng (Ray) Cui Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing Wissenschaftlicher MA / Postdoctoral researcher Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne Tel.:+49 (0)221 496 Mobile: +49 0221 37970 496 rcui at age.mpg.de www.age.mpg.de On Mon, Feb 20, 2017 at 6:39 AM, Carson Holt wrote: > MAKER was support was designed with GeneMark-ES. It may or may not work > with GeneMark-ET. So any MAKER related archive posts etc. will be related > to the latter. > > With GeneMark-ES, you simply provided a genome assembly and let it run. It > would then produce several files and output directories. The es.mod file > was the one you provided to MAKER. I don?t know how this compares > to GeneMark-ET. > > ?Carson > > > > On Feb 14, 2017, at 8:44 AM, Ray Cui wrote: > > Hi Daniel, > > thanks! It seems that Genemark-ET has a "--training" flag, is that > the flag I should use when training or should I just let Genemark also > perform the prediction? > > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for > Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 <+49%20221%20496> > Mobile: +49 0221 37970 496 > rcui at age.mpg.de > www.age.mpg.de > > > > On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel wrote: > >> Hi Ray, >> >> I think you?re on the right track with training Genemark with RNAseq >> data. It should only change the training steps, which are external to >> MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to >> the ?es.mod" file made by Genemark. >> >> For the 2nd question, in the MAKER beta 3, MAKER creates a control file >> for EVM, in which you set your weights for the various inputs, and then >> MAKER runs EVM alongside all the other gene predictors and chooses the >> model that is best supported by the evidence. >> >> ~Daniel >> >> >> >> On Feb 14, 2017, at 7:38 AM, Ray Cui wrote: >> >> Hello, >> >> I have sucessfully installed Maker beta 3, working with both >> Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio >> predictor. >> When I read the GeneMark-ES manual, it says that one can use >> RNAseq data to aid training. I'm wondering what would be the best way to >> integrate Genemark-ET predictions into Maker. Should I run Genemark-ET >> independent of Maker, then integrate the GFF at some point during the maker >> process? If so, how should I edit the configuration file? Currently maker >> has an option called "gmhmm". Should I then train GeneMark by myself with >> RNAseq data, then feed the hmm to maker? >> >> And perhaps an unrelated question is that now Maker beta 3 >> supports EVM. I'm wondering how EVM is used by Maker (at which step, what >> does it do), and how does it differ from what Maker is designed for (both >> reconciles different gene models). >> >> Best Regards, >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for >> Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 <+49%20221%20496> >> Mobile: +49 0221 37970 496 >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 20 02:02:00 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 02:02:00 -0700 Subject: [maker-devel] Using GeneMark-ET with RNAseq intron hints In-Reply-To: References: <2A8AEAD2-D9C9-4F96-8A6C-A11B55FA0F26@mail.ufl.edu> <52CD5438-F990-4D5E-AED1-7E86101DE3B5@gmail.com> Message-ID: The names of scripts used are listed in the maker_exe.ctl file. It depends on if formatting or any flags have changed between versions. ?Carson > On Feb 20, 2017, at 1:59 AM, Ray Cui wrote: > > Dear Carson, > > I have now run GeneMark-ET, and it produces a trained .mod file. I think it can be then passed to Maker. Do you know what is the final constructed command line in Maker that calls genemark? Genemark-et and es use the same perl script so one probably only needs to use the --prediction and --predict_with xxx.mod options to predict genes using the species specific parameters (bypassing regular training and prediction steps) > > > Best Regards, > Ray > > Dr. Rongfeng (Ray) Cui > Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing > Wissenschaftlicher MA / Postdoctoral researcher > Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne > Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne > Tel.:+49 (0)221 496 > Mobile: +49 0221 37970 496 <> > rcui at age.mpg.de > www.age.mpg.de > > > > On Mon, Feb 20, 2017 at 6:39 AM, Carson Holt > wrote: > MAKER was support was designed with GeneMark-ES. It may or may not work with GeneMark-ET. So any MAKER related archive posts etc. will be related to the latter. > > With GeneMark-ES, you simply provided a genome assembly and let it run. It would then produce several files and output directories. The es.mod file was the one you provided to MAKER. I don?t know how this compares to GeneMark-ET. > > ?Carson > > > >> On Feb 14, 2017, at 8:44 AM, Ray Cui > wrote: >> >> Hi Daniel, >> >> thanks! It seems that Genemark-ET has a "--training" flag, is that the flag I should use when training or should I just let Genemark also perform the prediction? >> >> Ray >> >> Dr. Rongfeng (Ray) Cui >> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >> Wissenschaftlicher MA / Postdoctoral researcher >> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >> Tel.:+49 (0)221 496 >> Mobile: +49 0221 37970 496 <> >> rcui at age.mpg.de >> www.age.mpg.de >> >> >> >> On Tue, Feb 14, 2017 at 3:43 PM, Ence,daniel > wrote: >> Hi Ray, >> >> I think you?re on the right track with training Genemark with RNAseq data. It should only change the training steps, which are external to MAKER, but not how MAKER runs Genemark. You?ll still give MAKER the path to the ?es.mod" file made by Genemark. >> >> For the 2nd question, in the MAKER beta 3, MAKER creates a control file for EVM, in which you set your weights for the various inputs, and then MAKER runs EVM alongside all the other gene predictors and chooses the model that is best supported by the evidence. >> >> ~Daniel >> >> >> >>> On Feb 14, 2017, at 7:38 AM, Ray Cui > wrote: >>> >>> Hello, >>> >>> I have sucessfully installed Maker beta 3, working with both Augustus and SNAP. I also want to try adding GeneMark-ES to the ab initio predictor. >>> When I read the GeneMark-ES manual, it says that one can use RNAseq data to aid training. I'm wondering what would be the best way to integrate Genemark-ET predictions into Maker. Should I run Genemark-ET independent of Maker, then integrate the GFF at some point during the maker process? If so, how should I edit the configuration file? Currently maker has an option called "gmhmm". Should I then train GeneMark by myself with RNAseq data, then feed the hmm to maker? >>> >>> And perhaps an unrelated question is that now Maker beta 3 supports EVM. I'm wondering how EVM is used by Maker (at which step, what does it do), and how does it differ from what Maker is designed for (both reconciles different gene models). >>> >>> Best Regards, >>> Ray >>> >>> Dr. Rongfeng (Ray) Cui >>> Max-Planck-Institut f?r Biologie des Alterns / Max Planck Institute for Biology of Ageing >>> Wissenschaftlicher MA / Postdoctoral researcher >>> Office: Joseph-Stelzmann 9b, D-50931 K?ln / Cologne >>> Postal address: Postfach 41 06 23, D-50866 K?ln / Cologne >>> Tel.:+49 (0)221 496 >>> Mobile: +49 0221 37970 496 <> >>> rcui at age.mpg.de >>> www.age.mpg.de >>> >>> >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From qlian003 at ucr.edu Mon Feb 20 13:34:31 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Mon, 20 Feb 2017 12:34:31 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> Message-ID: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Hi Carson, Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? Thanks Qihua > On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: > > IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. > > CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. > > If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. > > What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. > > However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. > > Thanks, > Carson > >> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >> >> Dear Maker develop team, >> >> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >> >> Thanks >> Qihua >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > From carsonhh at gmail.com Mon Feb 20 16:56:01 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 16:56:01 -0700 Subject: [maker-devel] default gene model by quality_filter.pl is the same with those by setting keep_preds=0? In-Reply-To: References: Message-ID: <14B6C165-54D2-4722-9A26-91FBEBB18B72@gmail.com> Any hint based predictions rejected for one of the non-AED filters will still be rejected regardless of keep_preds=1. However you are right in that any raw predictions that were rejected by other filters may still be kept by using this workflow, since keep_preds=1 essentially rescues all non-overlapping raw predictions that would have been rejected. ?Carson > On Feb 17, 2017, at 2:57 PM, Quanwei Zhang wrote: > > Hello: > > I am following the Maker protocol (2014) to rescue rejected gene models. I set keep_preds=1 when I ran Maker, and then used "quality_filter.pl -d" to get the "default" Maker report. According to the annotation it will "Prints transcripts with an AED <1". > > But in a previous post (the link below), it was said "there are models with AED less than 1 that get rejected for other reasons. ... For example if the only evidence is a protein alignment that has deep overlapping HSPs (extremely low complexity alignment) it will be filtered out even though AED is not technically equal to 1. Also if the overlapping protein evidence is in a different reading frame than the model it is supposed to support then the AED will be less than 1 but eAED will be 1 (extended AED), and the model will be rejected." > > So I wonder, whether the "default" result obtained by "quality_filter.pl " are really the same as those achieved by setting keep_preds=0? Will the script also consider other reasons besides "AED <1"? > > https://groups.google.com/forum/#!searchin/maker-devel/The$20reason$20there$20are$20differences$20between$20the$20runs$20is$20that$20there$20are$20models$20with$20AED$20less$20%7Csort:relevance/maker-devel/97aNJkT3bgk/W0MEyqEZAAAJ > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Mon Feb 20 16:56:02 2017 From: carsonhh at gmail.com (Carson Holt) Date: Mon, 20 Feb 2017 16:56:02 -0700 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: You either uses TrEMBL or the UniProtKB Isoform sequence set. Their fasta headers are slightly different and will not be parsed correctly, For example, here is the header as formatted for the same sequence in the Swiss-prot dataset download ?> >sp|Q7Z5M8|AB12B_HUMAN Protein ABHD12B OS=Homo sapiens GN=ABHD12B PE=2 SV=1 I think you used the UniProtKB Isoform sequence dataset instead. ?Carson > On Feb 13, 2017, at 8:16 AM, Quanwei Zhang wrote: > > Hello: > > I am trying to add putative gene function to the predicted gene models. Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. I used canonical and isoform proteins of human, mouse and rat with the script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose context is as below. > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 7e-164 464 > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 3e-80 236 > > After that, I am trying to add the protein homology data to the Maker gff3 and fasta files with maker_functional_gff and maker_functional_fasta, but get the reports as below. > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 39. > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 39. > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line 45. > Use of uninitialized value $id in hash element at /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line 45. > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > ..... > > I am not sure how to deal with this. I followed the command given in the protocol. Any suggestions? > > Thanks > > Best > Quanwei > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From qwzhang0601 at gmail.com Tue Feb 21 07:30:35 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Tue, 21 Feb 2017 09:30:35 -0500 Subject: [maker-devel] failed to assign putative gene function In-Reply-To: References: Message-ID: Thank you! I used both canonical and isoform of protein sequences from swiss-prot in the beginning. It reported the error, but later I only used the canonical protein sequences to build the database and then it worked. Best Quanwei 2017-02-20 18:56 GMT-05:00 Carson Holt : > You either uses TrEMBL or the UniProtKB Isoform sequence set. Their fasta > headers are slightly different and will not be parsed correctly, > > For example, here is the header as formatted for the same sequence in the > Swiss-prot dataset download ?> > >sp|Q7Z5M8|AB12B_HUMAN Protein ABHD12B OS=Homo sapiens GN=ABHD12B PE=2 SV=1 > > I think you used the UniProtKB Isoform sequence dataset instead. > > ?Carson > > > > > > > On Feb 13, 2017, at 8:16 AM, Quanwei Zhang > wrote: > > > > Hello: > > > > I am trying to add putative gene function to the predicted gene models. > Firstly, I use uniProt/Swiss-Prot protein sequences to build the database. > I used canonical and isoform proteins of human, mouse and rat with the > script "makeblastdb". Then use "blastp" generated "maker2uni.blastp" whose > context is as below. > > maker-CasCan_contig_64815-snap-gene-0.0-mRNA-1 > sp|Q6P5S2|LEG1H_HUMAN 69.97 303 91 0 1 303 1 303 > 7e-164 464 > > snap_masked-CasCan_contig_14203-processed-gene-0.10-mRNA-1 > sp|Q91ZA8|NRARP_MOUSE 99.12 114 1 0 1 114 1 114 > 3e-80 236 > > > > After that, I am trying to add the protein homology data to the Maker > gff3 and fasta files with maker_functional_gff and maker_functional_fasta, > but get the reports as below. > > > > Can't parse details from FASTA header: >sp|Q7Z5M8-2|AB12B_HUMAN Isoform > 2 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 39. > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 39. > > Can't parse details from FASTA header: >sp|Q7Z5M8-4|AB12B_HUMAN Isoform > 4 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 139, <$IN> line > 45. > > Use of uninitialized value $id in hash element at > /public/apps/MAKER/2.31.9/bin/maker_functional_gff line 141, <$IN> line > 45. > > Can't parse details from FASTA header: >sp|Q7Z5M8-5|AB12B_HUMAN Isoform > 5 of Protein ABHD12B OS=Homo sapiens GN=ABHD12B > > ..... > > > > I am not sure how to deal with this. I followed the command given in the > protocol. Any suggestions? > > > > Thanks > > > > Best > > Quanwei > > _______________________________________________ > > maker-devel mailing list > > maker-devel at box290.bluehost.com > > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Tue Feb 21 10:17:22 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 12:17:22 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Message-ID: I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: cpan[1]> install forks::signals Warning: Cannot install forks::signals, don't know what it is. Try the command i /forks::signals/ to find objects with matching identifiers. but scrolling through the install scroll of the forks reinstall I do see it got installed properly. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: > Do a force reinstall of forks via CPAN. The error is coming from forks.pm line > 83, so the problem is the forks installation itself. > > Thanks, > Carson > > > On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: > > Hi Everyone, > > After sorting my MPICH/OpenMPI issue I have come across another: When I > try to run MAKER on the demo data via > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker > /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl > /Data/Apps/maker/data/maker_bopts.ctl > > (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it > working before opening up, if I change the -n value then the error repeats > once for each process I attempt to run via MPICH) I got the following: > > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks.pm in @INC (you may need to install the forks::signals > module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > However I also see > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially > thought this was a Perl error. I hit up perlmonks and found that there is > an issue between forks and Storable, where Storable now has a verion number > of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these > instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and > tried again. Now I get: > > $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker > /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl > /Data/Apps/maker/data/maker_bopts.ctl > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > Can't locate forks/signals.pm in @INC (you may need to install the > forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > forks.pm line 83. > BEGIN failed--compilation aborted at forks.pm line 84. > Compilation failed in require at /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. > > Clearly MAKER has an issue with forks, but it is installed, it is up to > date (I double checked via cpan and apt-get to be sure), and it is in a > directory that is in @INC. Should I pursue this as a MAKER error or as a > Perl error? > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 21 10:19:30 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 21 Feb 2017 10:19:30 -0700 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> Message-ID: <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> forks::signals is part of forks. It?s not a separate package. If it?s missing, there is a problem with the forks installation. So you need to do a force install of forks to force the reinstall. Example ?> cpan[1]> force install forks ?Carson > On Feb 21, 2017, at 10:17 AM, Seth Munholland wrote: > > I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: > > cpan[1]> install forks::signals > Warning: Cannot install forks::signals, don't know what it is. > Try the command > > i /forks::signals/ > > to find objects with matching identifiers. > > but scrolling through the install scroll of the forks reinstall I do see it got installed properly. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt > wrote: > Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. > > Thanks, > Carson > > >> On Feb 17, 2017, at 1:11 PM, Seth Munholland > wrote: >> >> Hi Everyone, >> >> After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl >> >> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: >> >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> However I also see >> $ locate forks.pm >> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >> >> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Tue Feb 21 11:05:48 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 13:05:48 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: I wasn't sure what modules were separated (figured it was wroth a shot), however, I did the force install of forks and I still get the same error. I tried uninstalling and reinstalling all the forks installations that showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then reloaded) to make sure I wasn't getting some kind of conflicting libraries/modules. and I'm back to: Can't locate forks.pm in @INC (you may need to install the forks module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. along with: $ sudo updatedb $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Now it's a maker issue, not a forks issue (if I'm reading it correctly), so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) and got: $ sudo ./Build install Installing MAKER... Configuring MAKER with MPI support Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 (unchanged) I don't believe that's an error or even a warning, but I get the same "can't locate forks" error when I try to run it. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt wrote: > forks::signals is part of forks. It?s not a separate package. If it?s > missing, there is a problem with the forks installation. So you need to do > a force install of forks to force the reinstall. > > Example ?> cpan[1]> force install forks > > ?Carson > > > On Feb 21, 2017, at 10:17 AM, Seth Munholland wrote: > > I tried this and still get the same error. Then I tried forcing a > reinstall of forks/signals and got: > > cpan[1]> install forks::signals > Warning: Cannot install forks::signals, don't know what it is. > Try the command > > i /forks::signals/ > > to find objects with matching identifiers. > > but scrolling through the install scroll of the forks reinstall I do see > it got installed properly. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: > >> Do a force reinstall of forks via CPAN. The error is coming from forks.pm line >> 83, so the problem is the forks installation itself. >> >> Thanks, >> Carson >> >> >> On Feb 17, 2017, at 1:11 PM, Seth Munholland wrote: >> >> Hi Everyone, >> >> After sorting my MPICH/OpenMPI issue I have come across another: When I >> try to run MAKER on the demo data via >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >> /Data/Apps/maker/data/maker_bopts.ctl >> >> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it >> working before opening up, if I change the -n value then the error repeats >> once for each process I attempt to run via MPICH) I got the following: >> >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> However I also see >> $ locate forks.pm >> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >> >> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially >> thought this was a Perl error. I hit up perlmonks and found that there is >> an issue between forks and Storable, where Storable now has a verion number >> of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these >> instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and >> tried again. Now I get: >> >> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >> /Data/Apps/maker/data/maker_bopts.ctl >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> Can't locate forks/signals.pm in @INC (you may need to install the >> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >> .) at forks.pm line 83. >> BEGIN failed--compilation aborted at forks.pm line 84. >> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >> Undefined subroutine &threads::_END called at (eval 2) line 1. >> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >> >> Clearly MAKER has an issue with forks, but it is installed, it is up to >> date (I double checked via cpan and apt-get to be sure), and it is in a >> directory that is in @INC. Should I pursue this as a MAKER error or as a >> Perl error? >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bmoore at genetics.utah.edu Tue Feb 21 11:53:13 2017 From: bmoore at genetics.utah.edu (Barry Moore) Date: Tue, 21 Feb 2017 18:53:13 +0000 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: <78C26338-01EF-4F72-BFA4-EF46C70AFAEE@genetics.utah.edu> Hi Qihua, If you are using online version of SOBA, I would suggest you use the command line version found here https://github.com/The-Sequence-Ontology/SOBA as it is more flexible for the kinds of analyses you are talking about. If you are using ?footprint? as the --data_type argument you should get the nucleotide count for collapsed features that you are talking about. In addition I suggest you take a look at bedtools (http://bedtools.readthedocs.io/en/latest/index.html) for example bedtools merge as a flexible way to generate the kind of merged features you want and then you can always pass that output of that through SOBAcl for counting, graphing and reporting. Finally, if you want a great deal of flexibility in generating your own manipulation and reporting of GFF3 files that is beyond the scope of SOBA and/or BedTools, I suggest you take a look at the GAL library (https://github.com/The-Sequence-Ontology/GAL) if you don?t mind writing some perl code. Regards, Barry On Feb 20, 2017, at 1:34 PM, Qihua Liang > wrote: Hi Carson, Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? Thanks Qihua On Feb 19, 2017, at 10:05 PM, Carson Holt > wrote: IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. Thanks, Carson On Feb 18, 2017, at 9:43 AM, Qihua Liang > wrote: Dear Maker develop team, I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? Thanks Qihua _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Feb 21 14:34:07 2017 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 21 Feb 2017 14:34:07 -0700 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: MAKER merges overlapping RepeatMasker results into a single longer feature. ?Carson > On Feb 20, 2017, at 1:34 PM, Qihua Liang wrote: > > Hi Carson, > > Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. > > I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. > > Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. > In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. > > But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. > > When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? > Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? > > Thanks > Qihua > > >> On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: >> >> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. >> >> CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. >> >> If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. >> >> What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. >> >> However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. >> >> Thanks, >> Carson >> >>> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >>> >>> Dear Maker develop team, >>> >>> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >>> >>> Thanks >>> Qihua >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> > From munholl at uwindsor.ca Tue Feb 21 15:26:23 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Tue, 21 Feb 2017 17:26:23 -0500 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: I found my errors. 1) In my machine file; the first entry was not the main node, so locate was looking on a different node than Maker 2) forks (and other modules) were not properly installed on every node in the cluster. I used CPAN to install them on each cluster and it is now working. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 1:05 PM, Seth Munholland wrote: > I wasn't sure what modules were separated (figured it was wroth a shot), > however, I did the force install of forks and I still get the same error. > I tried uninstalling and reinstalling all the forks installations that > showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then > reloaded) to make sure I wasn't getting some kind of conflicting > libraries/modules. and I'm back to: > > Can't locate forks.pm in @INC (you may need to install the forks module) > (@INC contains: /Data/Apps/maker/bin/../perl/lib > /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib > /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 > /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 > /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 > /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at > /Data/Apps/maker/bin/maker line 42. > BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. > > along with: > > $ sudo updatedb > $ locate forks.pm > /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm > /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm > /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm > > Now it's a maker issue, not a forks issue (if I'm reading it correctly), > so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) > and got: > > $ sudo ./Build install > Installing MAKER... > Configuring MAKER with MPI support > Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 > (unchanged) > > I don't believe that's an error or even a warning, but I get the same > "can't locate forks" error when I try to run it. > > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt wrote: > >> forks::signals is part of forks. It?s not a separate package. If it?s >> missing, there is a problem with the forks installation. So you need to do >> a force install of forks to force the reinstall. >> >> Example ?> cpan[1]> force install forks >> >> ?Carson >> >> >> On Feb 21, 2017, at 10:17 AM, Seth Munholland >> wrote: >> >> I tried this and still get the same error. Then I tried forcing a >> reinstall of forks/signals and got: >> >> cpan[1]> install forks::signals >> Warning: Cannot install forks::signals, don't know what it is. >> Try the command >> >> i /forks::signals/ >> >> to find objects with matching identifiers. >> >> but scrolling through the install scroll of the forks reinstall I do see >> it got installed properly. >> >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> >> On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt wrote: >> >>> Do a force reinstall of forks via CPAN. The error is coming from >>> forks.pm line 83, so the problem is the forks installation itself. >>> >>> Thanks, >>> Carson >>> >>> >>> On Feb 17, 2017, at 1:11 PM, Seth Munholland >>> wrote: >>> >>> Hi Everyone, >>> >>> After sorting my MPICH/OpenMPI issue I have come across another: When I >>> try to run MAKER on the demo data via >>> >>> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >>> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >>> /Data/Apps/maker/data/maker_bopts.ctl >>> >>> (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it >>> working before opening up, if I change the -n value then the error repeats >>> once for each process I attempt to run via MPICH) I got the following: >>> >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> >>> However I also see >>> $ locate forks.pm >>> /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm >>> /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm >>> /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm >>> >>> Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially >>> thought this was a Perl error. I hit up perlmonks and found that there is >>> an issue between forks and Storable, where Storable now has a verion number >>> of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these >>> instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and >>> tried again. Now I get: >>> >>> $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker >>> /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl >>> /Data/Apps/maker/data/maker_bopts.ctl >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> Can't locate forks/signals.pm in @INC (you may need to install the >>> forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib >>> /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib >>> /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 >>> /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 >>> /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 >>> /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base >>> .) at forks.pm line 83. >>> BEGIN failed--compilation aborted at forks.pm line 84. >>> Compilation failed in require at /Data/Apps/maker/bin/maker line 42. >>> BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. >>> Undefined subroutine &threads::_END called at (eval 2) line 1. >>> END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. >>> >>> Clearly MAKER has an issue with forks, but it is installed, it is up to >>> date (I double checked via cpan and apt-get to be sure), and it is in a >>> directory that is in @INC. Should I pursue this as a MAKER error or as a >>> Perl error? >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Wed Feb 22 06:11:32 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Wed, 22 Feb 2017 14:11:32 +0100 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> Message-ID: <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> @Robert Kraus: FYI Am 20.02.2017 um 06:43 schrieb Carson Holt: > Try running just on a single node (not across nodes). THATS WHAT I DID. > If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and > running MAKER with that new version. You can install it in your home directory and test from there, > just make sure to add it to your path. Shure it is. > Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. > If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. OK, send the infos please. Thank you very much! > ?Carson > > > >> On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: >> >> Hi! >> >> Unfortunately all of the options failed on our cluster. >> >> See: >> >> Hi, >> >> Most recent Maker test with >> --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >> Error: >> --> rank=2, hostname=uc1n518.localdomain >> [uc1n518:67009] *** Process received signal *** >> [uc1n518:67009] Signal: Segmentation fault (11) >> [uc1n518:67009] Signal code: Address not mapped (1) >> [uc1n518:67009] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). >> >> >> With: >> --mca btl ^openib >> and also this >> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >> Error: >> ### Runing Maker example >> >> STATUS: Parsing control files... >> STATUS: Processing and indexing input FASTA files... >> [uc1n514:59985] *** Process received signal *** >> [uc1n514:59985] Signal: Segmentation fault (11) >> [uc1n514:59985] Signal code: Address not mapped (1) >> [uc1n514:59985] Failing at address: 0x4b0 >> -------------------------------------------------------------------------- >> mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). >> >> >> Am 28.01.2017 um 21:53 schrieb Carson Holt: >>> Try adding one of the following to your mpiexec command ?> >>> >>> 1. --mca btl ^openib >>> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>> >>> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >>> >>> --Carson >>> >>> >>>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>>> >>>> Hi everybody. >>>> >>>> My name is Rainer. I am an administrator for our HPC-Systems at our >>>> university in Konstanz, Baden-Wuertemberg/Germany. >>>> The procect is called bwHPC-C5. >>>> >>>> See: https://www.bwhpc-c5.de/en/index.php >>>> >>>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>>> i get errors while running a Maker job in the MPI-environment. >>>> >>>> BUILD STATUS >>>> >>>> ============================================================================== >>>> STATUS MAKER v2.31.9 >>>> ============================================================================== >>>> PERL Dependencies: VERIFIED >>>> External Programs: VERIFIED >>>> External C Libraries: VERIFIED >>>> MPI SUPPORT: ENABLED >>>> MWAS Web Interface: DISABLED >>>> MAKER PACKAGE: CONFIGURATION OK >>>> >>>> MODULES / INCLUDES / COMPILERS >>>> >>>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>>> # >>>> ##### (B) Dependencies: >>>> # >>>> # conflict: any other maker version >>>> # module load compiler/gnu/5.2 >>>> # module load mpi/openmpi/2.0-gnu-5.2 >>>> [...] >>>> >>>> MPI/MOAB SUBMIT >>>> >>>> [...] >>>> ### Queues ### >>>> #MSUB -q fat >>>> #MSUB -l nodes=1:ppn=16 >>>> #MSUB -l mem=20gb >>>> #MSUB -l walltime=50:00:00 >>>> # >>>> [...] >>>> echo " " >>>> echo "### Loading MAKER module:" >>>> echo " " >>>> module load bio/maker/2.31.9 >>>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>>> echo "MAKER_VERSION = $MAKER_VERSION" >>>> module list >>>> [...] >>>> echo " " >>>> echo "### Runing Maker example" >>>> echo " " >>>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>>> export OMPI_MCA_mpi_warn_on_fork=0 >>>> >>>> echo "LD_PRELOAD=${LD_PRELOAD}" >>>> # >>>> # "STATUS: Processing and indexing input FASTA files..." >>>> # >>>> mpiexec -mca btl ^openib -n 16 maker >>>> [...] >>>> >>>> >>>> E R R O R S >>>> ======= >>>> [...] >>>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>>> STATUS: Parsing control files... >>>> STATUS: Processing and indexing input FASTA files... >>>> [uc1n338:113607] *** Process received signal *** >>>> [uc1n338:113607] Signal: Segmentation fault (11) >>>> [uc1n338:113607] Signal code: Address not mapped (1) >>>> [uc1n338:113607] Failing at address: 0x4b0 >>>> [uc1n338:113608] *** Process received signal *** >>>> [uc1n338:113608] Signal: Segmentation fault (11) >>>> [uc1n338:113608] Signal code: Address not mapped (1) >>>> [uc1n338:113608] Failing at address: 0x4b0 >>>> [uc1n338:113621] *** Process received signal *** >>>> [uc1n338:113621] Signal: Segmentation fault (11) >>>> [uc1n338:113621] Signal code: Address not mapped (1) >>>> [uc1n338:113621] Failing at address: 0x4b0 >>>> -------------------------------------------------------------------------- >>>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>>> -------------------------------------------------------------------------- >>>> [...] >>>> >>>> WHATS WRONG HERE!? >>>> >>>> Thank you for your help! >>>> >>>> All the best , >>>> >>>> Rainer >>>> >>>> -- >>>> Rainer Rutka >>>> University of Konstanz >>>> Communication, Information, Media Centre (KIM) >>>> * High-Performance-Computing (HPC) >>>> * KIM-Support and -Base-Services >>>> Room: V511 >>>> 78457 Konstanz, Germany >>>> +49 7531 88-5413 >>>> >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> >> -- >> Rainer Rutka >> Universit?t Konstanz >> Kommunikations-, Informations-, Medienzentrum (KIM) >> Raum: V511, Tel: 54 13 >> > -- Rainer Rutka Universit?t Konstanz Kommunikations-, Informations-, Medienzentrum (KIM) Raum: V511, Tel: 54 13 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From cjfields at illinois.edu Wed Feb 22 06:30:22 2017 From: cjfields at illinois.edu (Fields, Christopher J) Date: Wed, 22 Feb 2017 13:30:22 +0000 Subject: [maker-devel] MAKER can't find forks in @INC, but it is there In-Reply-To: References: <42A44E3A-2E39-459D-A5FA-FE7BAF8C73A0@gmail.com> <93355BE2-A34F-4781-9A58-BD49F7DA96D5@gmail.com> Message-ID: <6E5122C0-A952-42D6-B205-74B5FF8255F2@illinois.edu> Just a note: when we install MAKER we generally install a clean version of perl if one isn?t already present, making sure it is on the NFS share for the cluster (which is accessible via all nodes). This works around most of the issues you describe. chris From: maker-devel > on behalf of Seth Munholland > Date: Tuesday, February 21, 2017 at 4:26 PM To: Carson Holt > Cc: "maker-devel at yandell-lab.org" > Subject: Re: [maker-devel] MAKER can't find forks in @INC, but it is there I found my errors. 1) In my machine file; the first entry was not the main node, so locate was looking on a different node than Maker 2) forks (and other modules) were not properly installed on every node in the cluster. I used CPAN to install them on each cluster and it is now working. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 1:05 PM, Seth Munholland > wrote: I wasn't sure what modules were separated (figured it was wroth a shot), however, I did the force install of forks and I still get the same error. I tried uninstalling and reinstalling all the forks installations that showed up and commented out the PERL5LIB export I had in my ~/.bashrc (then reloaded) to make sure I wasn't getting some kind of conflicting libraries/modules. and I'm back to: Can't locate forks.pm in @INC (you may need to install the forks module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. along with: $ sudo updatedb $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Now it's a maker issue, not a forks issue (if I'm reading it correctly), so I reinstalled Maker (made sure to direct to mpicc and mpi.h properly) and got: $ sudo ./Build install Installing MAKER... Configuring MAKER with MPI support Skip /Data/Apps/maker/src/../perl/config-x86_64-linux-gnu-thread-multi-5.022001 (unchanged) I don't believe that's an error or even a warning, but I get the same "can't locate forks" error when I try to run it. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Tue, Feb 21, 2017 at 12:19 PM, Carson Holt > wrote: forks::signals is part of forks. It?s not a separate package. If it?s missing, there is a problem with the forks installation. So you need to do a force install of forks to force the reinstall. Example ?> cpan[1]> force install forks ?Carson On Feb 21, 2017, at 10:17 AM, Seth Munholland > wrote: I tried this and still get the same error. Then I tried forcing a reinstall of forks/signals and got: cpan[1]> install forks::signals Warning: Cannot install forks::signals, don't know what it is. Try the command i /forks::signals/ to find objects with matching identifiers. but scrolling through the install scroll of the forks reinstall I do see it got installed properly. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Fri, Feb 17, 2017 at 3:39 PM, Carson Holt > wrote: Do a force reinstall of forks via CPAN. The error is coming from forks.pm line 83, so the problem is the forks installation itself. Thanks, Carson On Feb 17, 2017, at 1:11 PM, Seth Munholland > wrote: Hi Everyone, After sorting my MPICH/OpenMPI issue I have come across another: When I try to run MAKER on the demo data via $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl (-n 4 because I have 4 nodes in my MPI cluster and I wanted to get it working before opening up, if I change the -n value then the error repeats once for each process I attempt to run via MPICH) I got the following: Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. However I also see $ locate forks.pm /home/seth/.cpan/build/forks-0.36-0/blib/lib/forks.pm /home/seth/.cpan/build/forks-0.36-0/lib/forks.pm /usr/local/lib/x86_64-linux-gnu/perl/5.22.1/forks.pm Since /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 is in INC I initially thought this was a Perl error. I hit up perlmonks and found that there is an issue between forks and Storable, where Storable now has a verion number of X.Y_Z but forks is only looking for X.Y (no _Z). I followed these instructions https://rt.cpan.org/Public/Bug/Display.html?id=102730 and tried again. Now I get: $ mpiexec -machinefile /Data/machinefile -n 4 /Data/Apps/maker/bin/maker /Data/Apps/maker/data/maker_exe.ctl /Data/Apps/maker/data/maker_opts.ctl /Data/Apps/maker/data/maker_bopts.ctl Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Can't locate forks/signals.pm in @INC (you may need to install the forks::signals module) (@INC contains: /Data/Apps/maker/bin/../perl/lib /Data/Apps/maker/bin/../lib /Data/Apps/maker/bin/../src/inc/perl/lib /Data/Apps/CEGMA_v2.5/lib /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1 /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at forks.pm line 83. BEGIN failed--compilation aborted at forks.pm line 84. Compilation failed in require at /Data/Apps/maker/bin/maker line 42. BEGIN failed--compilation aborted at /Data/Apps/maker/bin/maker line 42. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at /Data/Apps/maker/bin/maker line 42. Clearly MAKER has an issue with forks, but it is installed, it is up to date (I double checked via cpan and apt-get to be sure), and it is in a directory that is in @INC. Should I pursue this as a MAKER error or as a Perl error? Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 09:16:17 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 09:16:17 -0700 Subject: [maker-devel] Maker-Error when started with OpenMPI In-Reply-To: <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> References: <73509312-0658-4A58-90A8-6D3143EDB1C7@gmail.com> <13af0362-c623-14cc-86ba-3db99487c43c@uni-konstanz.de> <167D93E3-5D28-4240-A491-D35D895B4611@gmail.com> <05e0036d-0b1d-3604-71f1-ea716f72ea27@uni-konstanz.de> Message-ID: <24606FBA-4742-4F94-9447-208A385644C1@gmail.com> If OpenMPI fails on a single node, it means you have a compilation issue, which indicates a problem with your installation. This sometimes happens if you compiled on one node and run on another (if could either be MAEKR or OpenMPI itself that was compiled on another node). A few options you will need if trying intel MPI: -binding pin=disable #requires to disable processor affinity (otherwise MAKER calls to BLAST and other programs which are parallelized independent of MPI may not work) Environmental variables to set: export I_MPI_PIN_DOMAIN=node #otherwise MAKER calls to BLAST and other programs which are parallelized independent of MPI may not work export I_MPI_FABRICS='shm:tcp? #avoid potential complication with OpenFabrics libraries (they block system calls because of how they use registered memory, i.e. MAKER calling BLAST would fail) export I_MPI_HYDRA_IFACE=ib0 #set to eth0 if you don?t have an infiniband over ip inerface (required because of the above I_MPI_FABRICS change) Also make sure to compile on the node you run. You can try expanding to other nodes after that. ?Carson > On Feb 22, 2017, at 6:11 AM, Rainer Rutka wrote: > > @Robert Kraus: FYI > > Am 20.02.2017 um 06:43 schrieb Carson Holt: >> Try running just on a single node (not across nodes). > THATS WHAT I DID. > > >> If it still fails, you might need to try installing an updated OpenMPI version then reinstalling and >> running MAKER with that new version. You can install it in your home directory and test from there, >> just make sure to add it to your path. > Shure it is. > >> Alternatively MPICH3 and IntelMPI (with some extra configuration for IntelMPI) can be used. > > If you decide to try Intel MPI let me know, and I can provide you with the info on configuration. > > OK, send the infos please. > > Thank you very much! > >> ?Carson >> >> >> >>> On Feb 16, 2017, at 3:44 AM, Rainer Rutka wrote: >>> >>> Hi! >>> >>> Unfortunately all of the options failed on our cluster. >>> >>> See: >>> >>> Hi, >>> >>> Most recent Maker test with >>> --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>> Error: >>> --> rank=2, hostname=uc1n518.localdomain >>> [uc1n518:67009] *** Process received signal *** >>> [uc1n518:67009] Signal: Segmentation fault (11) >>> [uc1n518:67009] Signal code: Address not mapped (1) >>> [uc1n518:67009] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 1 with PID 67009 on node uc1n518 exited on signal 11 (Segmentation fault). >>> >>> >>> With: >>> --mca btl ^openib >>> and also this >>> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>> Error: >>> ### Runing Maker example >>> >>> STATUS: Parsing control files... >>> STATUS: Processing and indexing input FASTA files... >>> [uc1n514:59985] *** Process received signal *** >>> [uc1n514:59985] Signal: Segmentation fault (11) >>> [uc1n514:59985] Signal code: Address not mapped (1) >>> [uc1n514:59985] Failing at address: 0x4b0 >>> -------------------------------------------------------------------------- >>> mpiexec noticed that process rank 10 with PID 59985 on node uc1n514 exited on signal 11 (Segmentation fault). >>> >>> >>> Am 28.01.2017 um 21:53 schrieb Carson Holt: >>>> Try adding one of the following to your mpiexec command ?> >>>> >>>> 1. --mca btl ^openib >>>> 2. --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 >>>> 3. --mca btl vader,tcp,self --mca btl_tcp_if_include eth0 >>>> >>>> One or the other may fix your issue. The first causes OpenMPI to not use the infiniband communication option (infiniband libraries use registered memory in a way that causes system calls to generate segfaults). It will usually force communication to go over another adapter. The second tries to use the infiband adapter, but uses TCP over infiniband (way to indirectly bypass problem causing libraries). The third specifically forces the use of the ethernet adapter instead of infiniband adapter. >>>> >>>> --Carson >>>> >>>> >>>>> On Jan 27, 2017, at 3:31 AM, Rainer Rutka wrote: >>>>> >>>>> Hi everybody. >>>>> >>>>> My name is Rainer. I am an administrator for our HPC-Systems at our >>>>> university in Konstanz, Baden-Wuertemberg/Germany. >>>>> The procect is called bwHPC-C5. >>>>> >>>>> See: https://www.bwhpc-c5.de/en/index.php >>>>> >>>>> I try to get Maker running on our bwUniCluster since weeks. Unfortunately >>>>> i get errors while running a Maker job in the MPI-environment. >>>>> >>>>> BUILD STATUS >>>>> >>>>> ============================================================================== >>>>> STATUS MAKER v2.31.9 >>>>> ============================================================================== >>>>> PERL Dependencies: VERIFIED >>>>> External Programs: VERIFIED >>>>> External C Libraries: VERIFIED >>>>> MPI SUPPORT: ENABLED >>>>> MWAS Web Interface: DISABLED >>>>> MAKER PACKAGE: CONFIGURATION OK >>>>> >>>>> MODULES / INCLUDES / COMPILERS >>>>> >>>>> # knbw03 20170117 r.rutka Initial revision knbw02 of module version 2.31.9 >>>>> # >>>>> ##### (B) Dependencies: >>>>> # >>>>> # conflict: any other maker version >>>>> # module load compiler/gnu/5.2 >>>>> # module load mpi/openmpi/2.0-gnu-5.2 >>>>> [...] >>>>> >>>>> MPI/MOAB SUBMIT >>>>> >>>>> [...] >>>>> ### Queues ### >>>>> #MSUB -q fat >>>>> #MSUB -l nodes=1:ppn=16 >>>>> #MSUB -l mem=20gb >>>>> #MSUB -l walltime=50:00:00 >>>>> # >>>>> [...] >>>>> echo " " >>>>> echo "### Loading MAKER module:" >>>>> echo " " >>>>> module load bio/maker/2.31.9 >>>>> [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.9'."; exit 1; } >>>>> echo "MAKER_VERSION = $MAKER_VERSION" >>>>> module list >>>>> [...] >>>>> echo " " >>>>> echo "### Runing Maker example" >>>>> echo " " >>>>> export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so >>>>> export OMPI_MCA_mpi_warn_on_fork=0 >>>>> >>>>> echo "LD_PRELOAD=${LD_PRELOAD}" >>>>> # >>>>> # "STATUS: Processing and indexing input FASTA files..." >>>>> # >>>>> mpiexec -mca btl ^openib -n 16 maker >>>>> [...] >>>>> >>>>> >>>>> E R R O R S >>>>> ======= >>>>> [...] >>>>> LD_PRELOAD=/opt/bwhpc/common/mpi/openmpi/2.0.1-gnu-5.2/lib/libmpi.so >>>>> STATUS: Parsing control files... >>>>> STATUS: Processing and indexing input FASTA files... >>>>> [uc1n338:113607] *** Process received signal *** >>>>> [uc1n338:113607] Signal: Segmentation fault (11) >>>>> [uc1n338:113607] Signal code: Address not mapped (1) >>>>> [uc1n338:113607] Failing at address: 0x4b0 >>>>> [uc1n338:113608] *** Process received signal *** >>>>> [uc1n338:113608] Signal: Segmentation fault (11) >>>>> [uc1n338:113608] Signal code: Address not mapped (1) >>>>> [uc1n338:113608] Failing at address: 0x4b0 >>>>> [uc1n338:113621] *** Process received signal *** >>>>> [uc1n338:113621] Signal: Segmentation fault (11) >>>>> [uc1n338:113621] Signal code: Address not mapped (1) >>>>> [uc1n338:113621] Failing at address: 0x4b0 >>>>> -------------------------------------------------------------------------- >>>>> mpiexec noticed that process rank 2 with PID 113608 on node uc1n338 exited on signal 11 (Segmentation fault). >>>>> -------------------------------------------------------------------------- >>>>> [...] >>>>> >>>>> WHATS WRONG HERE!? >>>>> >>>>> Thank you for your help! >>>>> >>>>> All the best , >>>>> >>>>> Rainer >>>>> >>>>> -- >>>>> Rainer Rutka >>>>> University of Konstanz >>>>> Communication, Information, Media Centre (KIM) >>>>> * High-Performance-Computing (HPC) >>>>> * KIM-Support and -Base-Services >>>>> Room: V511 >>>>> 78457 Konstanz, Germany >>>>> +49 7531 88-5413 >>>>> >>>>> _______________________________________________ >>>>> maker-devel mailing list >>>>> maker-devel at box290.bluehost.com >>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>>> >>> >>> -- >>> Rainer Rutka >>> Universit?t Konstanz >>> Kommunikations-, Informations-, Medienzentrum (KIM) >>> Raum: V511, Tel: 54 13 >>> >> > > -- > Rainer Rutka > Universit?t Konstanz > Kommunikations-, Informations-, Medienzentrum (KIM) > Raum: V511, Tel: 54 13 > From munholl at uwindsor.ca Wed Feb 22 11:03:54 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 13:03:54 -0500 Subject: [maker-devel] Failed while polishing ESTs Message-ID: After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: polishig ESTs running est2genome search. #--------- command -------------# Widget::exonerate::est2genome: /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate #-------------------------------# Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! --> rank=NA, hostname=beanblade2 ERROR: Failed while polishig ESTs ERROR: Chunk failed at level:2, tier_type:3 FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:contig-dpp-500-500 The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: polishig ESTs running est2genome search. #--------- command -------------# Widget::exonerate::est2genome: /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate #-------------------------------# Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! --> rank=NA, hostname=beanblade4 ERROR: Failed while polishig ESTs ERROR: Chunk failed at level:2, tier_type:3 FAILED CONTIG:contig-dpp-500-500 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:contig-dpp-500-500 So the error seems to be pointing at something happening when wrapping up the ESTs. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 11:16:41 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 11:16:41 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: Message-ID: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. And make sure to delete any run directories before retrying. ?Carson > On Feb 22, 2017, at 11:03 AM, Seth Munholland wrote: > > After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade2 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade4 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > So the error seems to be pointing at something happening when wrapping up the ESTs. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <>_______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 11:57:00 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 13:57:00 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: It is the test data, the dpp set to be specific. I already checked all the installs to be sure they configured and compile without error and are up to date. I've been deleting between each run. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: > So you are running the test data job correct? So if you get any error with > the test data job, something is wrong with your installation. > > Make sure BioPerl is up to data (and the same everywhere). If using your > own installation of Perl, make sure it had BerkleyDB support when you > installed it (if you are using something like perlbrew , you may not have > BerkleyDB support and this may affect BioPerl indexing). Also try > reinstalling exonertate. > > And make sure to delete any run directories before retrying. > > ?Carson > > > On Feb 22, 2017, at 11:03 AM, Seth Munholland wrote: > > After my clustering issue I figured I would go node by node and run MAKER > locally on the test data set to be sure that it was working properly before > trying it as a full MPI run. After adjusting all my exes to point to the > NFS versions I am consistently getting the same error on each node when it > comes time to run exonerate: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q > /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna > --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent > 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp- > mRNA-5.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade2 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > The only results I find on google was an issue with an improper character > in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and > since they all point to dpp-mRNA-5 I backed up the provided est and removed > it. The response was: > > polishig ESTs > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /Data/Apps/exonerate/src/program/exonerate -q > /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna > --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent > 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp- > mRNA-4.e.exonerate > #-------------------------------# > Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! > --> rank=NA, hostname=beanblade4 > ERROR: Failed while polishig ESTs > ERROR: Chunk failed at level:2, tier_type:3 > FAILED CONTIG:contig-dpp-500-500 > > ERROR: Chunk failed at level:4, tier_type:0 > FAILED CONTIG:contig-dpp-500-500 > > So the error seems to be pointing at something happening when wrapping up > the ESTs. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 12:00:24 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 14:00:24 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland wrote: > It is the test data, the dpp set to be specific. > > I already checked all the installs to be sure they configured and compile > without error and are up to date. > > I've been deleting between each run. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: > >> So you are running the test data job correct? So if you get any error >> with the test data job, something is wrong with your installation. >> >> Make sure BioPerl is up to data (and the same everywhere). If using your >> own installation of Perl, make sure it had BerkleyDB support when you >> installed it (if you are using something like perlbrew , you may not have >> BerkleyDB support and this may affect BioPerl indexing). Also try >> reinstalling exonertate. >> >> And make sure to delete any run directories before retrying. >> >> ?Carson >> >> >> On Feb 22, 2017, at 11:03 AM, Seth Munholland >> wrote: >> >> After my clustering issue I figured I would go node by node and run MAKER >> locally on the test data set to be sure that it was working properly before >> trying it as a full MPI run. After adjusting all my exes to point to the >> NFS versions I am consistently getting the same error on each node when it >> comes time to run exonerate: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q >> /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t >> /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna >> --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent >> 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA- >> 5.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade2 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> The only results I find on google was an issue with an improper character >> in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and >> since they all point to dpp-mRNA-5 I backed up the provided est and removed >> it. The response was: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q >> /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t >> /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna >> --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent >> 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA- >> 4.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade4 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> So the error seems to be pointing at something happening when wrapping up >> the ESTs. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 12:03:42 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 12:03:42 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: I still think you may have something wrong somewhere with some part of your installation (rogue libraries, compiler incompatibilities, etc.). Especially given your earlier issues with Perl libraries. ?Carson > On Feb 22, 2017, at 12:00 PM, Seth Munholland wrote: > > On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. > > Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: > It is the test data, the dpp set to be specific. > > I already checked all the installs to be sure they configured and compile without error and are up to date. > > I've been deleting between each run. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt > wrote: > So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. > > Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. > > And make sure to delete any run directories before retrying. > > ?Carson > > >> On Feb 22, 2017, at 11:03 AM, Seth Munholland > wrote: >> >> After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade2 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: >> >> polishig ESTs >> running est2genome search. >> #--------- command -------------# >> Widget::exonerate::est2genome: >> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate >> #-------------------------------# >> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >> --> rank=NA, hostname=beanblade4 >> ERROR: Failed while polishig ESTs >> ERROR: Chunk failed at level:2, tier_type:3 >> FAILED CONTIG:contig-dpp-500-500 >> >> ERROR: Chunk failed at level:4, tier_type:0 >> FAILED CONTIG:contig-dpp-500-500 >> >> So the error seems to be pointing at something happening when wrapping up the ESTs. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From munholl at uwindsor.ca Wed Feb 22 12:26:57 2017 From: munholl at uwindsor.ca (Seth Munholland) Date: Wed, 22 Feb 2017 14:26:57 -0500 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: I'll wipe it clean and do a fresh install of everything to be 100% safe. Seth Munholland, B.Sc. Department of Biological Sciences Rm. 304 Biology Building University of Windsor 401 Sunset Ave. N9B 3P4 T: (519) 253-3000 Ext: 4755 On Wed, Feb 22, 2017 at 2:03 PM, Carson Holt wrote: > I still think you may have something wrong somewhere with some part of > your installation (rogue libraries, compiler incompatibilities, etc.). > Especially given your earlier issues with Perl libraries. > > ?Carson > > > On Feb 22, 2017, at 12:00 PM, Seth Munholland wrote: > > On a whim I decided to switch to the hsap demo data and it completed > without issue. I went back to dpp and this time is completed. I hadn't > changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and > then inverted the change. > > Mayhaps there was something not unloaded from memory in an earlier run? > It's that, ghosts, or rogue ions :/ > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 > > On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: > >> It is the test data, the dpp set to be specific. >> >> I already checked all the installs to be sure they configured and compile >> without error and are up to date. >> >> I've been deleting between each run. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 >> >> On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt wrote: >> >>> So you are running the test data job correct? So if you get any error >>> with the test data job, something is wrong with your installation. >>> >>> Make sure BioPerl is up to data (and the same everywhere). If using your >>> own installation of Perl, make sure it had BerkleyDB support when you >>> installed it (if you are using something like perlbrew , you may not have >>> BerkleyDB support and this may affect BioPerl indexing). Also try >>> reinstalling exonertate. >>> >>> And make sure to delete any run directories before retrying. >>> >>> ?Carson >>> >>> >>> On Feb 22, 2017, at 11:03 AM, Seth Munholland >>> wrote: >>> >>> After my clustering issue I figured I would go node by node and run >>> MAKER locally on the test data set to be sure that it was working properly >>> before trying it as a full MPI run. After adjusting all my exes to point >>> to the NFS versions I am consistently getting the same error on each node >>> when it comes time to run exonerate: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q >>> /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t >>> /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T >>> dna --model est2genome --minintron 20 --maxintron 10000 --showcigar >>> --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp >>> -500-500.26386-32056.dpp-mRNA-5.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade2 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> The only results I find on google was an issue with an improper >>> character in the est file, but a cat of the dpp_est.fasta shows nothing >>> incorrect and since they all point to dpp-mRNA-5 I backed up the provided >>> est and removed it. The response was: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q >>> /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t >>> /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T >>> dna --model est2genome --minintron 20 --maxintron 10000 --showcigar >>> --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp >>> -500-500.22889-32056.dpp-mRNA-4.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade4 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> So the error seems to be pointing at something happening when wrapping >>> up the ESTs. >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 >>> _______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >>> >>> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Feb 22 12:29:12 2017 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 22 Feb 2017 12:29:12 -0700 Subject: [maker-devel] Failed while polishing ESTs In-Reply-To: References: <44FA2A99-FF77-4CBD-A208-0C74B86AA1F3@gmail.com> Message-ID: <53F1286C-945C-4F91-B851-5833BE1D1F18@gmail.com> In the most extreme case, you may even need to go as far as setting up your own perl. If you do that, make sure to unset the PERL5LIB environmental variable, so other perl libraries don?t interfere. ?Carson > On Feb 22, 2017, at 12:26 PM, Seth Munholland wrote: > > I'll wipe it clean and do a fresh install of everything to be 100% safe. > > Seth Munholland, B.Sc. > Department of Biological Sciences > Rm. 304 Biology Building > University of Windsor > 401 Sunset Ave. N9B 3P4 > T: (519) 253-3000 Ext: 4755 <> > On Wed, Feb 22, 2017 at 2:03 PM, Carson Holt > wrote: > I still think you may have something wrong somewhere with some part of your installation (rogue libraries, compiler incompatibilities, etc.). Especially given your earlier issues with Perl libraries. > > ?Carson > > >> On Feb 22, 2017, at 12:00 PM, Seth Munholland > wrote: >> >> On a whim I decided to switch to the hsap demo data and it completed without issue. I went back to dpp and this time is completed. I hadn't changed anything but the maker_opts.ctl lines that had "dpp" to "hsap" and then inverted the change. >> >> Mayhaps there was something not unloaded from memory in an earlier run? It's that, ghosts, or rogue ions :/ >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <> >> On Wed, Feb 22, 2017 at 1:57 PM, Seth Munholland > wrote: >> It is the test data, the dpp set to be specific. >> >> I already checked all the installs to be sure they configured and compile without error and are up to date. >> >> I've been deleting between each run. >> >> Seth Munholland, B.Sc. >> Department of Biological Sciences >> Rm. 304 Biology Building >> University of Windsor >> 401 Sunset Ave. N9B 3P4 >> T: (519) 253-3000 Ext: 4755 <> >> On Wed, Feb 22, 2017 at 1:16 PM, Carson Holt > wrote: >> So you are running the test data job correct? So if you get any error with the test data job, something is wrong with your installation. >> >> Make sure BioPerl is up to data (and the same everywhere). If using your own installation of Perl, make sure it had BerkleyDB support when you installed it (if you are using something like perlbrew , you may not have BerkleyDB support and this may affect BioPerl indexing). Also try reinstalling exonertate. >> >> And make sure to delete any run directories before retrying. >> >> ?Carson >> >> >>> On Feb 22, 2017, at 11:03 AM, Seth Munholland > wrote: >>> >>> After my clustering issue I figured I would go node by node and run MAKER locally on the test data set to be sure that it was working properly before trying it as a full MPI run. After adjusting all my exes to point to the NFS versions I am consistently getting the same error on each node when it comes time to run exonerate: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_q3n9fB/0/dpp-mRNA-5.for.26386-32056.0.fasta -t /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_q3n9fB/0/contig-dpp-500-500.26386-32056.dpp-mRNA-5.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade2 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> The only results I find on google was an issue with an improper character in the est file, but a cat of the dpp_est.fasta shows nothing incorrect and since they all point to dpp-mRNA-5 I backed up the provided est and removed it. The response was: >>> >>> polishig ESTs >>> running est2genome search. >>> #--------- command -------------# >>> Widget::exonerate::est2genome: >>> /Data/Apps/exonerate/src/program/exonerate -q /tmp/maker_I7TZxt/0/dpp-mRNA-4.for.22889-32056.0.fasta -t /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.0.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_I7TZxt/0/contig-dpp-500-500.22889-32056.dpp-mRNA-4.e.exonerate >>> #-------------------------------# >>> Messed up acceptor:TT in Widget::exonerate::est2genome::get_acceptor! >>> --> rank=NA, hostname=beanblade4 >>> ERROR: Failed while polishig ESTs >>> ERROR: Chunk failed at level:2, tier_type:3 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> ERROR: Chunk failed at level:4, tier_type:0 >>> FAILED CONTIG:contig-dpp-500-500 >>> >>> So the error seems to be pointing at something happening when wrapping up the ESTs. >>> >>> Seth Munholland, B.Sc. >>> Department of Biological Sciences >>> Rm. 304 Biology Building >>> University of Windsor >>> 401 Sunset Ave. N9B 3P4 >>> T: (519) 253-3000 Ext: 4755 <>_______________________________________________ >>> maker-devel mailing list >>> maker-devel at box290.bluehost.com >>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >> >> >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lucile.soler at bils.se Thu Feb 23 05:40:08 2017 From: lucile.soler at bils.se (lucile.soler at bils.se) Date: Thu, 23 Feb 2017 13:40:08 +0100 Subject: [maker-devel] genes longer than contig in maker Message-ID: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> Hello, I am using maker3 to annotate a lizard. The maker part went well but then when I want to do the functional annotation, it seems that for some contigs maker has created genes outside of the contig size. For instance : XXX1 maker gene 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8 XXX1 maker mRNA 2 26717 . - . ID=maker- XXX1_pilon-augustus-gene-0.8-mRNA-1;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.8|0.5|1|0.4|0.33|6|5291|105 XXX1 maker mRNA 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;_AED=0.08;_eAED=0.08;_QI=0|0.5|0.4|1|0.25|0.4|5|4442|170 and the length of the contig is 26696 Any idea of what could have happened? Have you seen that already? Let me know what more information you need to answer. Thank you very much for your help, Lucile Soler PhD in Bioinformatics Genome Annotation Platform NBIS (National Bioinformatics Infrastructure Sweden) mail:lucile.soler at bils.se -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Thu Feb 23 06:10:55 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Thu, 23 Feb 2017 14:10:55 +0100 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> Message-ID: <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> HI! Running maker with mpiexec causes this error-message: mpiexec_uc1n077.localdomain: cannot connect to local mpd (/scratch/mpd2.console_uc1n077.localdomain_kn_pop235844); possible causes: 1. no mpd is running on this host 2. an mpd is running but was started without a "console" (-n option) ### Cleaning up files ... removing unnecessary scratch files ... And yes, we don't have mpd running. Environment used is: Currently Loaded Modulefiles: 1) compiler/intel/16.0(default) 2) mpi/impi/5.1.3-intel-16.0(default) 3) bio/maker/2.31.8_impi Running maker with mpiexec using only 1 node and 8 cores. mpiexec -n 8 maker :-( Any suggestions ? -- Rainer Rutka University of Konstanz Communication, Information, Media Centre (KIM) * High-Performance-Computing (HPC) * KIM-Support and -Base-Services Room: V511 78457 Konstanz, Germany +49 7531 88-5413 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From carsonhh at gmail.com Thu Feb 23 13:22:22 2017 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 23 Feb 2017 13:22:22 -0700 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> Message-ID: It means one of two things. 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. ?Carson > On Feb 23, 2017, at 6:10 AM, Rainer Rutka wrote: > > > HI! > > Running maker with mpiexec causes this error-message: > > mpiexec_uc1n077.localdomain: cannot connect to local mpd (/scratch/mpd2.console_uc1n077.localdomain_kn_pop235844); possible causes: > 1. no mpd is running on this host > 2. an mpd is running but was started without a "console" (-n option) > ### Cleaning up files ... removing unnecessary scratch files ... > > And yes, we don't have mpd running. > > Environment used is: > > Currently Loaded Modulefiles: > 1) compiler/intel/16.0(default) > 2) mpi/impi/5.1.3-intel-16.0(default) > 3) bio/maker/2.31.8_impi > > Running maker with mpiexec using only 1 node and 8 cores. > > mpiexec -n 8 maker > > :-( > > Any suggestions ? > > -- > Rainer Rutka > University of Konstanz > Communication, Information, Media Centre (KIM) > * High-Performance-Computing (HPC) > * KIM-Support and -Base-Services > Room: V511 > 78457 Konstanz, Germany > +49 7531 88-5413 > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Thu Feb 23 16:40:24 2017 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 23 Feb 2017 16:40:24 -0700 Subject: [maker-devel] genes longer than contig in maker In-Reply-To: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> References: <32A90DA3-1301-4F5D-8798-5EFE2EF3790D@bils.se> Message-ID: 1. You have two contigs with the same ID in your assembly (one longer and one shorter). 2. Your BioPerl index for retrieving the sequence is corrupt somehow. Requires delete of previous output directory before restarting. 3. You are using bad formatted GFF3 as input into maker, and it is somehow not failing right away. The fact you get output means that it was able to translate the sequence into protein with CDS etc. So it was not too short for that. Look at the contig line in the GFF3 to get the length of the contig maker is seeing to compare to the feature positions given. ?Carson > On Feb 23, 2017, at 5:40 AM, lucile.soler at bils.se wrote: > > Hello, > > I am using maker3 to annotate a lizard. > > > The maker part went well but then when I want to do the functional annotation, it seems that for some contigs maker has created genes outside of the contig size. > > For instance : > > > XXX1 maker gene 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8 > > > XXX1 maker mRNA 2 26717 . - . ID=maker- XXX1_pilon-augustus-gene-0.8-mRNA-1;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-1;_AED=0.02;_eAED=0.02;_QI=0|0.8|0.5|1|0.4|0.33|6|5291|105 > > XXX1 maker mRNA 2 26717 . - . ID=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;Parent=maker-XXX1_pilon-augustus-gene-0.8;Name=maker-XXX1_pilon-augustus-gene-0.8-mRNA-2;_AED=0.08;_eAED=0.08;_QI=0|0.5|0.4|1|0.25|0.4|5|4442|170 > > and the length of the contig is 26696 > > > > > Any idea of what could have happened? Have you seen that already? > > > > Let me know what more information you need to answer. > > Thank you very much for your help, > > > > > > Lucile Soler > > PhD in Bioinformatics > Genome Annotation Platform > NBIS (National Bioinformatics Infrastructure Sweden) > mail:lucile.soler at bils.se > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From rainer.rutka at uni-konstanz.de Fri Feb 24 01:43:21 2017 From: rainer.rutka at uni-konstanz.de (Rainer Rutka) Date: Fri, 24 Feb 2017 09:43:21 +0100 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> Message-ID: <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> Hi Carson. First of all THANK YOU FOR YOUR HELP. MUCH APPRECIATED. :-) Am 23.02.2017 um 21:22 schrieb Carson Holt: > It means one of two things. > 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). No, I am starting the right version of mpiexec. This is a list of all our current available MPI-Versions, corresponding to their compiler: UC:[kn at uc1n997 ~]$ module avail mpi ------------------------------------- /opt/bwhpc/common/modulefiles -------------------------------------- mpi/impi/4.1.3-gnu-4.4 mpi/openmpi/1.10-intel-15.0 mpi/impi/4.1.3-gnu-4.7 mpi/openmpi/1.10-intel-16.0 mpi/impi/4.1.3-intel-14.0 mpi/openmpi/1.8-gnu-4.5 mpi/impi/5.0.3-gnu-4.4 mpi/openmpi/1.8-gnu-4.7 mpi/impi/5.0.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.7-m32 mpi/impi/5.0.3-intel-15.0 mpi/openmpi/1.8-gnu-4.8 mpi/impi/5.1.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.9 mpi/impi/5.1.3-gnu-system mpi/openmpi/1.8-intel-14.0(default) mpi/impi/5.1.3-intel-16.0(default) mpi/openmpi/1.8-intel-15.0 mpi/openmpi/1.10-gnu-4.5 mpi/openmpi/2.0-gnu-4.7 mpi/openmpi/1.10-gnu-4.7 mpi/openmpi/2.0-gnu-4.8 mpi/openmpi/1.10-gnu-4.8 mpi/openmpi/2.0-gnu-5.2 mpi/openmpi/1.10-gnu-4.9 mpi/openmpi/2.0-intel-15.0 mpi/openmpi/1.10-gnu-5.2 mpi/openmpi/2.0-intel-16.0 mpi/openmpi/1.10-intel-14.0 Here I load the MPI-Module including all it's dependencies: UC:[kn at uc1n997 ~]$ module load mpi/impi/5.1.3-intel-16.0 Loading module dependency 'compiler/intel/16.0'. Result: UC:[kn at c1n997 ~]$ which mpiexec /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec UC:[kn at uc1n997 ~]$ > 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) Extremely old? [...] Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160601 (build id: 15562) > MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. Yes. And we don't even use MPD at our clusters :-) > So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. I do not have any MPICH available on our cluster(s) now. All of the old versions had been removed some years ago. At a glance --------------- Maker is available as a so-called module on our cluster system. It's been build on a developers' node I can access. But, the MPI-modules are built by other fellows (e.g. at the KIT in Karlsruhe/Germany) on other nodes. Please check the module-file (included in this mail) bio-maker-2.31.8_impi to see how Maker was build including the envoronments set by this module. e.g.: ./Build status verify dependencies: =================================================== STATUS MAKER v2.31.8 =================================================== PERL Dependencies: VERIFIED External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: ENABLED MWAS Web Interface: DISABLED MAKER PACKAGE: CONFIGURATION OK At least, please(!) have a look at our m.moab file (included in this mail). This is the way how we submit a Maker job to our cluster. Maybe something is wrong here? Sorry again for wasting your time. But we imperatively need the Maker software running in parallel mode. :-) -- Rainer Rutka University of Konstanz Communication, Information, Media Centre (KIM) * High-Performance-Computing (HPC) * KIM-Support and -Base-Services Room: V511 78457 Konstanz, Germany +49 7531 88-5413 -------------- next part -------------- #%Module1.0 # # Cluster: bwunicluster # Module: bio/maker/2.31.8 # Revision: knbw01 # TargetSystem: Red-Hat-Enterprise # MainLocation: /opt/bwhpc/common # Status: optional # License: GPL, Artistic License # URL: http://www.yandell-lab.org/software/maker.html # 2ndLevelSupport: bwhpc [at] uni-konstanz.de # ##### (A) Revision history: # # knbw01 20160630 r.rutka Initial revision knbw01 of module version 2.31.8 # knbw02 20161121 r.rutka Initial revision knbw02 of module version 2.31.8 # knbw03 20160222 r.rutka Initial revision knbw02 of module version 2.31.8 with impi # ##### (B) Dependencies: # # conflict: any other maker version # module load compiler/intel/16.0 # module load mpi/impi/5.1.3-intel-16.0 # # # Main environments for build # # VERSION="2.31.8_impi" && echo "Version: ${VERSION}" # MAIN_DIR="/opt/bwhpc/common" && echo ${MAIN_DIR} # SOURCE_DIR=/home/kn/kn_kn/kn_pop235844/src/bio/maker/2.31.8 # TARGET_DIR="${MAIN_DIR}/bio/maker/${VERSION}" && echo ${TARGET_DIR} # ##### (C) How to obtain software? # # You have to register at first before you can download the SW. # # http://yandell.topaz.genetics.utah.edu/cgi-bin/maker_license.cgi # # # Download and unzip the binary distributions # cd ${SOURCE_DIR} && pwd # [ ! -f maker-${VERSION/_impi}.tgz ] && wget http://yandell.topaz.genetics.utah.edu/maker_downloads/CE63/461D/BD41/4510A183DC4571D5EA619E459572/maker-2.31.8.tgz # [[ -d ${TARGET_DIR} ]] && mv ${TARGET_DIR} ${TARGET_DIR}_$(date +%Y.%m.%d:%H.%M) # ##### (D) How to build and install software? # # mkdir -vp ${TARGET_DIR/$VERSION} # cd ${TARGET_DIR/$VERSION} && pwd # tar -xvzf ${SOURCE_DIR}/maker-${VERSION/_impi}.tgz # mv maker ${VERSION} # cd ${VERSION} && pwd # export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so # cd src # # Build required ad-ons. Lot's of required packages will be installed. # # perl Build.PL # | Would you like to configure MAKER for MPI.. ? [N ] Y # | # accept the next default paths and press ENTER # # ignore the error-messages - just continue with the next step # # ./Build installdeps # installs missing PERL dependencies # | ... MAKER .. local installataion: Y # | # Accept the defaults of _all_ further questions by pressing ENTER. # | # ** This is an annoying and long procedure :-( ** # # ./Build installexes # installs missing external programs # | # If are a registered user of RepBase, then MAKER can # | # download and install RepBase for RepeatMasker for you. # | # Register at: http://www.girinst.org/ # | ... download and install RepBase ... : N # We install RepBase later! # # If you type Y, this will not work # # See Install RepBase later in this file # # | # ERROR: Exonerate can't be found. URL Error 404. :-( # | # Locate exonerate entry in locations file and modify path to (line 33): # | # http://ftp.ebi.ac.uk/pub/software/vertebrategenomics/exonerate/exonerate-2.2.0-x86_64.tar.gz # | # restart ./Build installexes # ./Build installexes # # # Install RepBase to prevent "RepBase is not insalled for RepeatMasker" error-message # # http://www.girinst.org/server/RepBase/index.php # p=$(pwd) # cd ${TARGET_DIR}/exe/RepeatMasker && pwd # # http://www.girinst.org/server/RepBase/index.php # # FASTA-Format # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/RepBase22.01.fasta.tar.gz # tar xvzf RepBase22.01.fasta.tar.gz # # # # NEU! # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/repeatmaskerlibraries/RepBaseRepeatMaskerEdition-20170127.tar.gz # # # wget --user RainerRutka --password 13m5c3 http://www.girinst.org/server/RepBase/protected/repeatmaskerlibraries/repeatmaskerlibraries-20160829.tar.gz # tar xvzf RepBaseRepeatMaskerEdition-20170127.tar.gz # # # tar xvzf repeatmaskerlibraries-20160829.tar.gz # # perl ./configure # | Enter path perl ... : ENTER # | Enter path ... RepeatMasker: ENTER # | Enter path TRF programm: /opt/bwhpc/common/bio/maker/2.31.8_impi/exe/RepeatMasker/trf # | Add a search engine: 2 RMBlast # | Enter path: /opt/bwhpc/common/bio/maker/2.31.8_impi/exe/RepeatMasker/rmblast/bin # | search engine ... default: Y or ENTER # | Enter Selection: 5 (Done) # # # rm *.tar.gz # cd $p && pwd # # # Now check if anything's fine until now... # ./Build status # | # veryfy dependencies: # | # =================================================== # | # STATUS MAKER v2.31.8 # | # =================================================== # | # PERL Dependencies: VERIFIED # | # External Programs: VERIFIED # | # External C Libraries: VERIFIED # | # MPI SUPPORT: ENABLED # | # MWAS Web Interface: DISABLED # | # MAKER PACKAGE: CONFIGURATION OK # # ./Build install # # # Test # cd ${TARGET_DIR} # bin/maker -version # 2.31.8 # # # Create module file (this one): # # ${SOURCE_DIR}/modulefiles/bio-maker-2.31.8 # # # Create Moab example files (and read the instructions inside of it): # # ${SOURCE_DIR}/bwhpc-examples/bwunicluster-maker-example.moab # # # Copy Moab/bwHPC, special-scripts, examples and more... # mkdir -vp ${TARGET_DIR}/bwhpc-examples # cp ${SOURCE_DIR}/bwhpc-examples/* ${TARGET_DIR}/bwhpc-examples/. # # # Copy module file (this one): # mkdir -vp ${TARGET_DIR}/modulefiles # cp -v ${SOURCE_DIR}/modulefiles/bio-maker-${VERSION} ${TARGET_DIR}/modulefiles/ # chmod -c 644 ${TARGET_DIR}/modulefiles/* # # # Create module link: # mkdir -vp ${MAIN_DIR}/modulefiles/bio/maker # ln -fs ${TARGET_DIR}/modulefiles/bio-maker-${VERSION} ${MAIN_DIR}/modulefiles/bio/maker/${VERSION} # module avail bio/maker # # # Cleanup build and installation target dir: # # chgrp -R -h -c uc1-adm-sw ${TARGET_DIR} # chmod 777 ${TARGET_DIR}/bwhpc-examples # chmod 666 ${TARGET_DIR}/bwhpc-examples/* # rm -rf ${TARGET_DIR}/src ##### END COMMENTS ### Define procedure "set_envVAR" to work around a module-unload-load bug (optional): # Exported environment variables, which do not change their values, # disappear after an automatic unload and load sequence within a module file. # This is most likely a bug in the modules environment. Unchanged # variables are not re-exported upon load in an automatic unload-load # sequence. The workaround is to "unset" the variable in the right moment # (so the load of the unload-load-chain can set the variable again) # and, in addition, to set the environment variable explicitly. proc set_envVAR {envVAR VARcontent} { # Use global function env: global env # Unset envVAR explicitly: if { [info exists env($envVAR)] } { catch { unset env($envVAR) } } # Call overloaded module command setenv: setenv $envVAR $VARcontent # Set envVAR explicitly: set env([set envVAR]) $VARcontent } ### Define fallback values and source global functions and variables from modulerc file: set first_level_support_email "bwhpc (at) uni-konstanz.de" set global_modulerc_file "/opt/bwhpc/common/etc/modules.conf" if { [ file readable "${global_modulerc_file}" ] } { source "${global_modulerc_file}" } # Override global first_level_support_email email: set first_level_support_email "bwhpc (at) uni-konstanz.de" ### Define version, base_dir and whatis entry: set version "2.31.8_impi" set base_dir "/opt/bwhpc/common/bio/maker/$version" module-whatis "maker $version is a portable and configurable genome annotation pipeline" ### Define convenience environment variables: set maker_version "${version}" set maker_home "${base_dir}" set maker_exa_dir "${base_dir}/bwhpc-examples" set maker_bin_dir "${base_dir}/bin" set maker_bpr_url "http://www.bwhpc-c5.de/wiki/index.php/Maker" set maker_blast_bin "${base_dir}/exe/blast/bin" set maker_exonerate_bin "${base_dir}/exe/exonerate/bin" set maker_snap_bin "${base_dir}/exe/snap" set maker_repeatmasker_bin "${base_dir}/exe/RepeatMasker" set_envVAR MAKER_VERSION "${maker_version}" set_envVAR MAKER_HOME "${maker_home}" set_envVAR MAKER_EXA_DIR "${maker_exa_dir}" set_envVAR MAKER_BIN_DIR "${maker_bin_dir}" set_envVAR MAKER_BPR_URL "http://www.bwhpc-c5.de/wiki/index.php/Maker" set_envVAR MAKER_BLAST_BIN "${maker_blast_bin}" set_envVAR MAKER_EXONERATE_BIN "${maker_exonerate_bin}" set_envVAR MAKER_SNAP_BIN "${maker_snap_bin}" set_envVAR MAKER_REPEATMASKER_BIN "${maker_repeatmasker_bin}" ### Update path environments and define application specific environment vars: prepend-path PATH "${maker_bin_dir}" prepend-path PATH "${maker_blast_bin}" prepend-path PATH "${maker_exonerate_bin}" prepend-path PATH "${maker_snap_bin}" prepend-path PATH "${maker_repeatmasker_bin}" ### Define help text: proc ModulesHelp { } { global maker_home maker_exa_dir first_level_support_email maker_bpr_url maker_blast_bin maker_exonerate_bin maker_snap_bin maker_repeatmasker_bin puts stderr " DESCRIPTION MAKER is a portable and easily configurable genome annotation pipeline. Its purpose is to allow smaller eukaryotic and prokaryotic genome projects to independently annotate their genomes and to create genome databases. MAKER identifies repeats, aligns ESTs and proteins to a genome, produces ab-initio gene predictions and automatically synthesizes these data into gene annotations having evidence-based quality values. EXTERNAL PROGRAMS BLAST Command Line Applications (2.2.28) $maker_blast_bin EXONERATE Generic Sequence Comparison Tool (2.2.0) $maker_exonerate_bin SNAP Semi-HMM-based Nucleic Acid Parser (version 2006-07-28) $maker_snap_bin REPEATMASKER Mask repetitive DNA (open-4.0.5) $maker_repeatmasker_bin DOCUMENTATION * Main MAKER site http://www.yandell-lab.org/software/maker.html * MAKER User Guide, Documents, Wiki-Article http://weatherby.genetics.utah.edu/MAKER/wiki/index.php $maker_home/README * MAKER Download http://yandell.topaz.genetics.utah.edu/maker_downloads/CE63/461D/BD41/4510A183DC4571D5EA619E459572/maker-2.31.8.tgz * bwHPC examples and a moab example script $maker_exa_dir * bwHPC Best Practices Repository $maker_bpr_url CITATION n./a. +-------------------------------------------+ | THIS IS THE IMPI-VERSION BUILT WITH INTEL | +-------------------------------------------+ In case of problems, please contact: $first_level_support_email This module is available for all users. " } module load compiler/intel/16.0 module load mpi/impi/5.1.3-intel-16.0 conflict bio/maker -------------- next part -------------- #!/bin/bash #MSUB -N maker_impi-job #MSUB -j oe #MSUB -o $(JOBNAME).$(JOBID) #MSUB -m ae #MSUB -l nodes=1:ppn=1 #MSUB -l mem=20gb #MSUB -l walltime=10:00:00 # start=$(date +%s) echo " " echo "### Setting up shell environment ..." echo " " # if test -e "/etc/profile"; then source "/etc/profile"; fi; if test -e "$HOME/.bash_profile"; then source "$HOME/.bash_profile"; fi; unset LANG; export LC_ALL="C"; export MKL_NUM_THREADS=1; export OMP_NUM_THREADS=1 export USER=${USER:=`logname`} export MOAB_JOBID=${MOAB_JOBID:=`date +%s`} export MOAB_SUBMITDIR=${MOAB_SUBMITDIR:=`pwd`} export MOAB_JOBNAME=${MOAB_JOBNAME:=`basename "$0"`} export MOAB_JOBNAME=$(echo "${MOAB_JOBNAME}" | sed 's/[^a-zA-Z0-9._-]/_/g') export MOAB_NODECOUNT=${MOAB_NODECOUNT:=1} export MOAB_PROCCOUNT=${MOAB_PROCCOUNT:=1} ulimit -s 200000 echo " " echo "### Printing basic job infos to stdout ..." echo " " echo "START_TIME = `date +'%y-%m-%d %H:%M:%S %s'`" echo "HOSTNAME = ${HOSTNAME}" echo "USER = ${USER}" echo "MOAB_JOBNAME = ${MOAB_JOBNAME}" echo "MOAB_JOBID = ${MOAB_JOBID}" echo "MOAB_SUBMITDIR = ${MOAB_SUBMITDIR}" echo "MOAB_NODECOUNT = ${MOAB_NODECOUNT}" echo "MOAB_PROCCOUNT = ${MOAB_PROCCOUNT}" echo "SLURM_NODELIST = ${SLURM_NODELIST}" echo "PBS_NODEFILE = ${PBS_NODEFILE}" if test -f "${PBS_NODEFILE}"; then echo "PBS_NODEFILE (begin) ---------------------------------" NO_NODES=$(wc -l < ${PBS_NODEFILE}) cat "${PBS_NODEFILE}" sort -u $PBS_NODEFILE > mpd.hosts echo "PBS_NODEFILE (end) -----------------------------------" else NO_NODES=1 fi echo " " echo "### Creating TMP_WORK_DIR directory and changing to it ..." echo " " # Using "$TMPDIR" is strongly recommended for Maker jobs # since these jobs can create a lot of disk IO. # NEVER EVER calculate in your home directory. TMP_BASE_DIR="$TMPDIR" JOB_WORK_DIR="${MOAB_JOBNAME}.${MOAB_JOBID##*.}.$(date +%y%m%d_%H%M%S)" TMP_WORK_DIR="${TMP_BASE_DIR}/${JOB_WORK_DIR}" echo "JOB_WORK_DIR = ${JOB_WORK_DIR}" echo "TMP_BASE_DIR = ${TMP_BASE_DIR}" echo "TMP_WORK_DIR = ${TMP_WORK_DIR}" mkdir -vp "${TMP_WORK_DIR}" cd "${TMP_WORK_DIR}" echo " " echo "### Loading MAKER module:" echo " " module load bio/maker/2.31.8_impi [ "$MAKER_VERSION" ] || { echo "ERROR: Failed to load module 'bio/maker/2.31.8_impi'."; exit 1; } echo "MAKER_VERSION = $MAKER_VERSION" module list echo " " echo "### Copying input examples files for job:" echo " " cp -v ${MAKER_EXA_DIR}/*.{fasta,ctl} . cp -Rv ${MAKER_EXA_DIR}/_Inline . sleep 2 echo " " echo "### Display internal Maker/bwHPC environments..." echo " " echo "MAKER_BIN_DIR = ${MAKER_BIN_DIR}" echo "MAKER_EXA_DIR = ${MAKER_EXA_DIR}" echo "" echo " " echo "### Runing Maker example" echo " " # # Environmental variables to set: # export LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so echo "LD_PRELOAD=${MPI_LIB_DIR}/libmpi.so" export OMPI_MCA_mpi_warn_on_fork=0 echo "OMPI_MCA_mpi_warn_on_fork=0" export I_MPI_CPUINFO="proc" echo "I_MPI_CPUINFO=proc" export I_MPI_PMI_LIBRARY=${MPI_LIB_DIR}/libpmi.so echo "I_MPI_PMI_LIBRARY=${MPI_LIB_DIR}/libpmi.so" export I_MPI_PIN_DOMAIN=node echo "I_MPI_PIN_DOMAIN=node" # otherwise MAKER calls to BLAST and other programs which are parallelized independent # of MPI may not work export I_MPI_FABRICS='shm:tcp' echo "I_MPI_FABRICS=shm:tcp" # avoid potential complication with OpenFabrics libraries (they block system calls because of # how they use registered memory, i.e. MAKER calling BLAST would fail) # set to eth0 if you don?t have an infiniband over # ip inerface (required because of the above I_MPI_FABRICS change) export I_MPI_HYDRA_IFACE=ib0 echo "I_MPI_HYDRA_IFACE=ib0" # The ?c nobrs. option will try and run the BLAST jobs on # nobrs. cpus on the same machine, but will not run via MPI. # -nolocal # wtf? # mpiexec -n $SLURM_NPROCS -env I_MPI_DEBUG 5 ./hello echo "starting mpiexec..." mpiexec -nc 1 maker echo "### Cleaning up files ... removing unnecessary scratch files ..." echo " " # rm -fv sleep 3 # Sleep some time so potential stale nfs handles can disappear. echo " " echo "### Compressing results and copying back result archive ..." echo " " cd "${TMP_BASE_DIR}" mkdir -vp "${MOAB_SUBMITDIR}" # if user has deleted or moved the submit dir echo "Creating result tgz-file '${MOAB_SUBMITDIR}/${JOB_WORK_DIR}.tgz' ..." tar -zcvf "${MOAB_SUBMITDIR}/${JOB_WORK_DIR}.tgz" "${JOB_WORK_DIR}" \ || { echo "ERROR: Failed to create tgz-file. Please cleanup TMP_WORK_DIR '$TMP_WORK_DIR' on host '$HOSTNAME' manually (if not done automatically by queueing system)."; exit 102; } # Remarks: # * The resulting tgz file is copied back to the submit directory. # The name of the tgz file looks similar too # "bwunicluster-maker-example.moab.275.110528_101755.tgz" echo " " echo "### Final cleanup: Remove TMP_WORK_DIR ..." echo " " rm -rvf "${TMP_WORK_DIR}" echo "END_TIME = `date +'%y-%m-%d %H:%M:%S %s'`" end=$(date +%s) echo " " echo "### Calculate duration ..." echo " " diff=$[end-start] if [ $diff -lt 60 ]; then echo "Runtime (approx.): '$diff' secs" elif [ $diff -ge 60 ]; then echo 'Runtime (approx.): '$[$diff / 60] 'min(s) '$[$diff % 60] 'secs' fi -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 5055 bytes Desc: S/MIME Cryptographic Signature URL: From carsonhh at gmail.com Fri Feb 24 10:30:32 2017 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 24 Feb 2017 10:30:32 -0700 Subject: [maker-devel] Maker-Error when started with IMPI In-Reply-To: <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> References: <021ac88b-3574-14cf-ce56-acf9e07f0fab@uni-konstanz.de> <999a411b-9ba3-ec33-e7f7-ab0f8294e777@uni-konstanz.de> <9c57acf0-30ee-3713-65c0-801edac10098@uni-konstanz.de> Message-ID: Specific things. 1. Do not set LD_PRELOAD. That is only for OpenMPI, but it will cause problems with other MPI's. 2. Make sure you recompiled MAKER for Intel MPI (MPI code always has to be compiled for the flavor you are using, so make sure you have a separate installation of MAKER for Intel MPI). Also validate that the mpicc and libmpi.h listed during the MAKER install belong to Intel MPI. Don?t just assume they do because you loaded the module. Manually verify the paths during MAKER?s setup. 3. The error you got previously should not even be possible with the current version of Intel MPI, which is why I say that when you called mpiexec, something else (that was not Intel MPI) was launched. Easy solution is to give the full path of mpiexec in your job, so are not relying on PATH to be unaltered in your job. Do not do ?> mpiexec -nc 1 maker Do this for example ?> /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec -nc maker 4. Build and run on the same node for your test. If you build on one node and run on another, you may be changing your environment in ways you don?t realize that break things. So if you can build and test on the same node and it works, then it fails when you test it elsewhere, then you have to track down how your environment is changing. ?Carson > On Feb 24, 2017, at 1:43 AM, Rainer Rutka wrote: > > Hi Carson. > > First of all THANK YOU FOR YOUR HELP. > MUCH APPRECIATED. :-) > > Am 23.02.2017 um 21:22 schrieb Carson Holt: >> It means one of two things. >> 1. The mpiexec you called is not from Intel MPI (try 'which mpiexec' to verify the location and that it is not some other random mpiexec executable). > > No, I am starting the right version of mpiexec. > > This is a list of all our current available MPI-Versions, corresponding > to their compiler: > > UC:[kn at uc1n997 ~]$ module avail mpi > ------------------------------------- /opt/bwhpc/common/modulefiles -------------------------------------- > mpi/impi/4.1.3-gnu-4.4 mpi/openmpi/1.10-intel-15.0 > mpi/impi/4.1.3-gnu-4.7 mpi/openmpi/1.10-intel-16.0 > mpi/impi/4.1.3-intel-14.0 mpi/openmpi/1.8-gnu-4.5 > mpi/impi/5.0.3-gnu-4.4 mpi/openmpi/1.8-gnu-4.7 > mpi/impi/5.0.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.7-m32 > mpi/impi/5.0.3-intel-15.0 mpi/openmpi/1.8-gnu-4.8 > mpi/impi/5.1.3-gnu-4.7 mpi/openmpi/1.8-gnu-4.9 > mpi/impi/5.1.3-gnu-system mpi/openmpi/1.8-intel-14.0(default) > > mpi/impi/5.1.3-intel-16.0(default) > > mpi/openmpi/1.8-intel-15.0 > mpi/openmpi/1.10-gnu-4.5 mpi/openmpi/2.0-gnu-4.7 > mpi/openmpi/1.10-gnu-4.7 mpi/openmpi/2.0-gnu-4.8 > mpi/openmpi/1.10-gnu-4.8 mpi/openmpi/2.0-gnu-5.2 > mpi/openmpi/1.10-gnu-4.9 mpi/openmpi/2.0-intel-15.0 > mpi/openmpi/1.10-gnu-5.2 mpi/openmpi/2.0-intel-16.0 > mpi/openmpi/1.10-intel-14.0 > > > Here I load the MPI-Module including all it's dependencies: > > UC:[kn at uc1n997 ~]$ module load mpi/impi/5.1.3-intel-16.0 > Loading module dependency 'compiler/intel/16.0'. > > Result: > UC:[kn at c1n997 ~]$ which mpiexec > /opt/bwhpc/common/compiler/intel/compxe.2016.4.258/impi/5.1.3.223/intel64/bin/mpiexec > UC:[kn at uc1n997 ~]$ > >> 2. It is an extremely old version of Intel MPI (in which case the instructions I gave you would not apply) > Extremely old? > > [...] > Intel(R) MPI Library for Linux* OS, Version 5.1.3 Build 20160601 (build id: 15562) > >> MPD is an old launcher used in MPICH1 and early versions of MPICH2. It?s been abandoned since about 2008 when MPICH2 switched to the hydra launcher which is still used in MPICH3. > Yes. And we don't even use MPD at our clusters :-) > >> So either you are pointing to an old version of MPICH or you have a very old version of Intel MPI based off of MPICH. > I do not have any MPICH available on our cluster(s) now. All of the old versions had > been removed some years ago. > > At a glance > --------------- > > Maker is available as a so-called module on our cluster system. It's been build > on a developers' node I can access. But, the MPI-modules are built by other > fellows (e.g. at the KIT in Karlsruhe/Germany) on other nodes. > > Please check the module-file (included in this mail) > > bio-maker-2.31.8_impi > > to see how Maker was build including the envoronments set by this > module. > > e.g.: > ./Build status > verify dependencies: > =================================================== > STATUS MAKER v2.31.8 > =================================================== > PERL Dependencies: VERIFIED > External Programs: VERIFIED > External C Libraries: VERIFIED > MPI SUPPORT: ENABLED > MWAS Web Interface: DISABLED > MAKER PACKAGE: CONFIGURATION OK > > > At least, please(!) have a look at our m.moab file (included in this mail). > This is the way how we submit a Maker job to our cluster. Maybe something > is wrong here? > > Sorry again for wasting your time. But we imperatively need the Maker > software running in parallel mode. > > :-) > > -- > Rainer Rutka > University of Konstanz > Communication, Information, Media Centre (KIM) > * High-Performance-Computing (HPC) > * KIM-Support and -Base-Services > Room: V511 > 78457 Konstanz, Germany > +49 7531 88-5413 > From qlian003 at ucr.edu Sat Feb 25 10:14:04 2017 From: qlian003 at ucr.edu (Qihua Liang) Date: Sat, 25 Feb 2017 09:14:04 -0800 Subject: [maker-devel] SOBA statistics of Maker annotation In-Reply-To: References: <688EB172-FEC8-4995-8AA2-0925AF62201A@ucr.edu> <6551374B-54FF-4047-B7A8-A49327FC0036@gmail.com> <73526BAB-57F8-4A47-AADD-DB6883573EAB@ucr.edu> Message-ID: <2377C5DD-569C-4248-B458-349D7AEA32F5@ucr.edu> Thank you Barry and Carson! I compared the SOBA statistics of RepeatMasker footprint and the report generated by running RepeatMasker alone, I got 2 different parentage of repeats masked. Running RepeatMasker with myTrained.lib, the repeats masked are 42%. But within Maker GFF3, the percentage of repeats masker is only ~18%. What may cause such difference here? Thanks Qihua > On Feb 21, 2017, at 1:34 PM, Carson Holt wrote: > > MAKER merges overlapping RepeatMasker results into a single longer feature. > > ?Carson > > >> On Feb 20, 2017, at 1:34 PM, Qihua Liang wrote: >> >> Hi Carson, >> >> Thanks for your reply! Now I understand the minimal length of SOBA analysis of Maker gene models in GFF3. >> >> I am also using SOBA to calculate the statistics of another sources in the GFF3 file, and I have found another strange thing about RepeatMasker annotation and footprint percentage. >> >> Previously, I ran RepeatMasker outside of Maker once, with my_trained.lib (same as used in Maker), and I had bases masked of ~42% from the output report. >> In running Maker, I provided both ?model_org=all? and ?rmlib=my_trained.lib?. Under these setting, RepeatMasker should be run twice and the merged results of the twice running will be the output of RepeatMasker in GFF3. I am expecting the bases masked by RepeatMasker in the GFF3 will be more than 42%. >> >> But in SOBA calculation, the footprint percentage is only ~18%. Referring to the SOBA paper, footprint is calculated as "non-redundant nucleotide count of all features of a given type?. I assume that when SOBA calculates footprint of RepeatMasker features in GFF3, it should be counting the same as "masked bps" as RepeatMasker itself. >> >> When Maker ?combines? the 2 runs of RepeatMasker, is it a merge or an overlapping of 2 RepeatMasker results? >> Besides, instead of using SOBA, are there any accessory scripts updated in Maker to calculate the statistics of the annotations? >> >> Thanks >> Qihua >> >> >>> On Feb 19, 2017, at 10:05 PM, Carson Holt wrote: >>> >>> IN GFF3 the CDS and UTR lengths are actually the merge of all CDSs or UTR features, but SOBA is reporting each part individually which may be causing your confusion. This is because SOBA reports per feature statistics and not merged feature statistics. >>> >>> CDS?s do not have to take up entire exons. For example start/stop codons may cross splice sites and be split across exons (very common). The result is that each part of the split CDS becomes a separate feature. As a result SOBA will treat each one separately. So a single bp CDS here is not abnormal, since the remaining part of the CDS continues on the next exon as a separate line. The exact same is true for UTR. >>> >>> If you want the merged length of the UTR and CDS, it is bets to pull that info out of the _QI= part of the GFF3 attributes for each mRNA. >>> >>> What about single bp exons? Those cannot occur unless you gave an input GFF3 with predictions that have single bp exons. The predictors like SNAP and Augustus just won?t produce them, with one exception. They can potentially produce them for the first/last exon. This is not because the exon is 1 bp, but rather because the predictor only reports the CDS part of the exon. As a result if the stop/start codon may have only 1 bp overlapping that exon, but one you add UTR the exon will extend from that point and will no longer be 1bp in length. But if the UTR never gets added, then you can be left with a partial initial/terminal exon. >>> >>> However more than likely what you are seeing is just related to how SOBA reports individual feature line stats as opposed to merged stats for CDS and UTR. >>> >>> Thanks, >>> Carson >>> >>>> On Feb 18, 2017, at 9:43 AM, Qihua Liang wrote: >>>> >>>> Dear Maker develop team, >>>> >>>> I used SOBA website to calculate the statistics of Maker annotation, and I found out the length of some features of Maker, like CDS, exon, 5? and 3?UTR, the minimal length of such features can be as short as 1bp. These are confusing, with such features length of 1bp. When Maker combines different gene models and makes such predictions, how will it accept such abnormal exon/CDS length? And is there any parameters in the bopt.ctl or evm.ctl to avoid such abnormal gene models? >>>> >>>> Thanks >>>> Qihua >>>> _______________________________________________ >>>> maker-devel mailing list >>>> maker-devel at box290.bluehost.com >>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org >>> >> > From qwzhang0601 at gmail.com Mon Feb 27 08:25:04 2017 From: qwzhang0601 at gmail.com (Quanwei Zhang) Date: Mon, 27 Feb 2017 10:25:04 -0500 Subject: [maker-devel] PARALLELIZED DE NOVO GENOME ANNOTATION WITHOUT MPI Message-ID: Hello: I am doing genome annotation using Maker on our high performance computational cluster (HPC). Due to some issues of MPI, I submitted the Maker jobs several times under the same directory to HPC. Followed by the example in the protocol (as shown below), when I submit the jobs I make them as background processes by "&" except the first one. Is this necessary when I submit a job to a HPC? I found it costed much much longer time than I expected (according to a testing on a smaller data set). I am not sure whether setting the process as background process lead to this issue? The example in the protocol % maker 2> maker1.error % maker 2> maker2.error & % maker 2> maker3.error & ...... BTW, will the annotation on shorter contig (e.g., 500bp) cost ~ 1/100 of the time that cost for annotation a 50000bp contig? I am using SNAP for an inito and RNA-seq assembly and protein sequences as evidence. I have more than half contigs shorter than 300bp (whose total length is only about 5% of the total length of all contigs), I want to know whether I can save about half (or only about 5%) of the time if I ignore those short contigs. Thanks Best Quanwei -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcg at cau.edu.cn Tue Feb 28 00:43:57 2017 From: dcg at cau.edu.cn (dcg at cau.edu.cn) Date: Tue, 28 Feb 2017 15:43:57 +0800 Subject: [maker-devel] how to deal with Contigs to run maker? Message-ID: <2017022815435664227911@cau.edu.cn> Dear sir: After assemblying, I got many contigs and their order in each chromosome. What I have done is merging these contigs into each chromosomes followed by the order, with 100 Ns inserted betwwen each contigs. So that I got chr1 chr2......Then I ran the repeatmodeler, predictor to annotate it. Could my way reach a high-quality result? Should I use all the contigs to mask repeats and practice predictor? Is there any better way to do genome-wide annotation? I'm looking forward to your reply! Best wishes! Chao Chao 2017.02.28 -------------- next part -------------- An HTML attachment was scrubbed... URL: