[maker-devel] Maker issues
Fields, Christopher J
cjfields at illinois.edu
Mon Nov 26 14:55:04 MST 2012
That makes sense. The change I made should also allow these sequences to 'just work', but this would of course require a BioPerl update.
We could also make that change within BioPerl if needed if we know where it might be; it seems in this case a returned sequence should be 'dna' if the application is exonerate.
chris
On Nov 26, 2012, at 3:37 PM, Carson Holt <carsonhh at gmail.com> wrote:
> The sequence object is being created well down into BioPerl (outside of my
> control), but I think I can explicitly set the alphabet to 'dna' by
> modifying the existing object just before calling the revcomp method to
> get around that.
>
> Thanks,
> Carson
>
>
> On 12-11-26 4:32 PM, "Jason Stajich" <jason.stajich at gmail.com> wrote:
>
>> right - you can explicitly set the alphabet to 'dna' when building a
>> sequence object but I don't know if this down in MAKER code that is
>> tripping up?
>>
>> Jason
>> On Nov 25, 2012, at 9:33 PM, "Fields, Christopher J"
>> <cjfields at illinois.edu> wrote:
>>
>>> This is coming from BioPerl trying to guess the alphabet if one is not
>>> provided. The specific spot:
>>>
>>> if( ($str =~ tr/ATUGCNatugcn//) / $total > 0.7 ) {
>>> if ( $str =~ m/U/i ) {
>>> $alphabet = 'rna';
>>> } else {
>>> $alphabet = 'dna';
>>> }
>>> } else {
>>> $alphabet = 'protein';
>>> }
>>>
>>> Easy enough to fix to allow for additional ambiguous nucleotides (just
>>> committed, in fact). It's probably best to explicitly set this when
>>> possible, though; it is a guess, after all.
>>>
>>> chris
>>>
>>> On Nov 25, 2012, at 10:56 PM, Mark Yandell <myandell at genetics.utah.edu>
>>> wrote:
>>>
>>>> good detective work there Carson!
>>>>
>>>>
>>>> Mark Yandell
>>>> Professor of Human Genetics
>>>> H.A. & Edna Benning Presidential Endowed Chair
>>>> Eccles Institute of Human Genetics
>>>> University of Utah
>>>> 15 North 2030 East, Room 2100
>>>> Salt Lake City, UT 84112-5330
>>>> ph:801-587-7707
>>>>
>>>> ________________________________________
>>>> From: maker-devel-bounces at yandell-lab.org
>>>> [maker-devel-bounces at yandell-lab.org] on behalf of Carson Holt
>>>> [carsonhh at gmail.com]
>>>> Sent: Sunday, November 25, 2012 9:10 PM
>>>> To: Daniel Standage
>>>> Cc: Maker Mailing List
>>>> Subject: Re: [maker-devel] Maker issues
>>>>
>>>> I think the problem is in the sequence of your scaffold. I pulled
>>>> this out of the exonerate alignment -->
>>>> WTGGGGCTATGAAAAAAAAAWTTKMGMMAAAAAWTTWTKRWMRATC
>>>>
>>>> Notice the letters W, K, R, M, etc. While these are technically legal
>>>> nucleotides, many external programs, and in this case BioPerl doesn't
>>>> handle them well.
>>>> That is why you get -->
>>>> ------------- EXCEPTION: Bio::Root::Exception -------------
>>>> MSG: Sequence is a protein. Cannot revcom
>>>>
>>>> You might want to replace them in your input fasta with the letter 'N'
>>>> so they are treated as masked. You will have to delete the mpi_blastdb
>>>> directory to let maker rebuild the fasta indexes and you will probably
>>>> have to set clean_try=1 in the control files so that MAKER deletes old
>>>> result files that contain those characters on the retry. The other
>>>> error may be just a snowball effect from the first error, so you should
>>>> see of it still happens after fixing the input fasta file.
>>>>
>>>> Thanks,
>>>> Carson
>>>>
>>>>
>>>>
>>>> From: Daniel Standage
>>>> <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
>>>> Date: Friday, 23 November, 2012 3:06 PM
>>>> To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
>>>> Cc: Maker Mailing List
>>>> <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
>>>> Subject: Re: Maker issues
>>>>
>>>> Thanks for your reply, and sorry for my delayed response.
>>>>
>>>> I have attached the first file you requested, but the other two do not
>>>> exist. I have attached a listing of the files in that directory. Let me
>>>> know if you need anything else.
>>>>
>>>>
>>>> --
>>>> Daniel S. Standage
>>>> Ph.D. Candidate
>>>> Bioinformatics and Computational Biology Program
>>>> Department of Genetics, Development, and Cell Biology
>>>> Iowa State University
>>>>
>>>>
>>>>
>>>> On Mon, Nov 12, 2012 at 10:02 AM, Carson Holt
>>>> <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
>>>> The first error is an IO error with your system. I've added some more
>>>> detail to the errors in the development version if you do an 'svn
>>>> update'. Then you will know the system specific reason why close or
>>>> opened failed. For the other error, could you send me this file -->
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.3.mason.
>>>> maker.output/maker.pdom.3.mason_datastore/scaffold_7/theVoid.scaffold_7/
>>>> scaffold_7.1869077-1869882.comp59027_c1_seq93.est_exonerate.0
>>>>
>>>> This one -->
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.
>>>> maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_2
>>>> 3/scaffold_23.716125-721460.0.fasta
>>>>
>>>> And this one -->
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.
>>>> maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_2
>>>> 3/comp58983_c0_seq101.for.716125-721460.0.fasta
>>>>
>>>> thanks,
>>>> Carson
>>>>
>>>>
>>>>
>>>>
>>>> From: Daniel Standage
>>>> <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
>>>> Date: Thursday, 8 November, 2012 9:32 AM
>>>>
>>>> To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
>>>> Cc: Maker Mailing List
>>>> <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
>>>> Subject: Re: Maker issues
>>>>
>>>> Scaling up to whole-genome annotation, things seem to be going well.
>>>> However, there are some intermittent issues. I've seen a couple
>>>> occurrences of the following error...
>>>>
>>>> #-------------------------------#
>>>> Calling out to FastaSeq::convert at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.
>>>> running est2genome search.
>>>> #--------- command -------------#
>>>> Widget::exonerate::est2genome:
>>>> /N/hd01/dstandag/Mason/local/bin/exonerate -q
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.
>>>> maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_2
>>>> 3/comp58983_c0_seq101.for.716125-721460.0.fasta -t
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.
>>>> maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_2
>>>> 3/scaffold_23.716125-721460.0.fasta -Q dna -T dna --model est2genome
>>>> --minintron 20 --maxintron 10000 --showcigar --percent 20 >
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.
>>>> maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_2
>>>> 3/scaffold_23.716125-721460.comp58983_c0_seq101.est_exonerate.0
>>>> #-------------------------------#
>>>> Calling out to FastaSeq::convert at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.
>>>> couldn't close
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.
>>>> maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_2
>>>> 3/comp58983_c0_seq37.for.716125-723330.0.fasta at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/FastaFile.pm line
>>>> 60.
>>>> --> rank=NA, hostname=c4
>>>> ERROR: Failed while polishig ESTs
>>>> ERROR: Chunk failed at level:2, tier_type:2
>>>> FAILED CONTIG:scaffold_23
>>>>
>>>> ERROR: Chunk failed at level:5, tier_type:0
>>>> FAILED CONTIG:scaffold_23
>>>>
>>>> examining contents of the fasta file and run log
>>>> Calling Datastore::MD5::mkdir at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>> Calling uri_escape at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>> Calling File::Path::mkpath at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>>
>>>>
>>>> ...as well as one occurrence of this error.
>>>>
>>>> #-------------------------------#
>>>> Calling out to FastaSeq::convert at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.
>>>> running est2genome search.
>>>> #--------- command -------------#
>>>> Widget::exonerate::est2genome:
>>>> /N/hd01/dstandag/Mason/local/bin/exonerate -q
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.3.mason.
>>>> maker.output/maker.pd
>>>>
>>>> om.3.mason_datastore/scaffold_7/theVoid.scaffold_7/comp59027_c1_seq93.fo
>>>> r.1869077-1869882.0.fasta -t /N/dc/scratch/dstandag/PdomGenomic/Anno
>>>>
>>>> tation/output/maker.pdom.3.mason.maker.output/maker.pdom.3.mason_datasto
>>>> re/scaffold_7/theVoid.scaffold_7/scaffold_7.1869077-1869882.0.fasta
>>>> -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000
>>>> --showcigar --percent 20 >
>>>> /N/dc/scratch/dstandag/PdomGenomic/Annotation/
>>>>
>>>> output/maker.pdom.3.mason.maker.output/maker.pdom.3.mason_datastore/scaf
>>>> fold_7/theVoid.scaffold_7/scaffold_7.1869077-1869882.comp59027_c1_se
>>>> q93.est_exonerate.0
>>>> #-------------------------------#
>>>>
>>>> ------------- EXCEPTION: Bio::Root::Exception -------------
>>>> MSG: Sequence is a protein. Cannot revcom
>>>> STACK: Error::throw
>>>> STACK: Bio::Root::Root::throw
>>>> /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/Root/Root.pm:368
>>>> STACK: Bio::PrimarySeqI::revcom
>>>> /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/PrimarySeqI.pm:381
>>>> STACK: Bio::LocatableSeq::revcom
>>>> /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/LocatableSeq.pm:577
>>>> STACK: exonerate::splice_info::needs_to_be_revcomped
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/exonerate/splice_i
>>>> nfo.pm:86<http://splice_info.pm:86>
>>>> STACK: Widget::exonerate::est2genome::assemble
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Widget/exonerate/e
>>>> st2genome.pm:686<http://est2genome.pm:686>
>>>> STACK: Widget::exonerate::est2genome::parse
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Widget/exonerate/e
>>>> st2genome.pm:961<http://est2genome.pm:961>
>>>> STACK: polisher::exonerate::est::e_exonerate
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/polisher/exonerate
>>>> /est.pm:82<http://est.pm:82>
>>>> STACK: polisher::exonerate::est::polish
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/polisher/exonerate
>>>> /est.pm:44<http://est.pm:44>
>>>> STACK: GI::to_polisher
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm:1670
>>>> STACK: GI::polish_exonerate
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm:1517
>>>> STACK: Process::MpiChunk::_go
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m:1663
>>>> STACK: Process::MpiChunk::run
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m:335
>>>> STACK: Process::MpiChunk::run_all
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m:351
>>>> STACK: Process::MpiTiers::run_all
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.p
>>>> m:286
>>>> STACK: Process::MpiTiers::run_all
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.p
>>>> m:286
>>>> STACK: /N/u/dstandag/Mason/local/src/maker-dev/bin/maker:644
>>>> -----------------------------------------------------------
>>>> --> rank=NA, hostname=c4
>>>> ERROR: Failed while polishig ESTs
>>>> ERROR: Chunk failed at level:2, tier_type:2
>>>> FAILED CONTIG:scaffold_7
>>>>
>>>> ERROR: Chunk failed at level:5, tier_type:0
>>>> FAILED CONTIG:scaffold_7
>>>>
>>>> examining contents of the fasta file and run log
>>>> Calling Datastore::MD5::mkdir at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>> Calling uri_escape at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>> Calling File::Path::mkpath at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>>
>>>> I'll let you know if I see anything else.
>>>>
>>>>
>>>> --
>>>> Daniel S. Standage
>>>> Ph.D. Candidate
>>>> Bioinformatics and Computational Biology Program
>>>> Department of Genetics, Development, and Cell Biology
>>>> Iowa State University
>>>>
>>>>
>>>>
>>>> On Wed, Nov 7, 2012 at 11:46 AM, Carson Holt
>>>> <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
>>>> Thanks. Typo now fixed on my end too ;-)
>>>>
>>>> Thanks,
>>>> Carson
>>>>
>>>>
>>>> From: Daniel Standage
>>>> <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
>>>> Date: Wednesday, 7 November, 2012 11:43 AM
>>>>
>>>> To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
>>>> Cc: Maker Mailing List
>>>> <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
>>>> Subject: Re: Maker issues
>>>>
>>>> Looked good for a while, but came across this error.
>>>>
>>>> total clusters:20 now processing 0
>>>> flattening EST clusters
>>>> doing tblastx of alt-ESTs
>>>> Undefined subroutine &GI::loalize_file called at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 2648.
>>>> --> rank=NA, hostname=c4
>>>> ERROR: Failed while doing tblastx of alt-ESTs
>>>> ERROR: Chunk failed at level:4, tier_type:2
>>>> FAILED CONTIG:scaffold_58
>>>>
>>>> ERROR: Chunk failed at level:5, tier_type:0
>>>> FAILED CONTIG:scaffold_58
>>>>
>>>> examining contents of the fasta file and run log
>>>> Calling Datastore::MD5::mkdir at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>> Calling uri_escape at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>> Calling File::Path::mkpath at
>>>> /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.p
>>>> m line 433.
>>>>
>>>>
>>>>
>>>> --Next Contig--
>>>>
>>>> It seems pretty clear that there is a typo in GI.pm. I changed loalize
>>>> to localize and relaunched.
>>>>
>>>>
>>>> --
>>>> Daniel S. Standage
>>>> Ph.D. Candidate
>>>> Bioinformatics and Computational Biology Program
>>>> Department of Genetics, Development, and Cell Biology
>>>> Iowa State University
>>>>
>>>>
>>>>
>>>> On Wed, Nov 7, 2012 at 9:30 AM, Daniel Standage
>>>> <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>> wrote:
>>>> Done.
>>>>
>>>> Test job has successfully cleared the preliminary Fasta indexing steps
>>>> and is repeat masking. I'll let you know if there are any problems.
>>>> Thanks!
>>>>
>>>>
>>>> --
>>>> Daniel S. Standage
>>>> Ph.D. Candidate
>>>> Bioinformatics and Computational Biology Program
>>>> Department of Genetics, Development, and Cell Biology
>>>> Iowa State University
>>>>
>>>>
>>>>
>>>> On Wed, Nov 7, 2012 at 9:00 AM, Carson Holt
>>>> <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
>>>> 1.006902 Bio::Root::Version
>>>> /N/u/dstandag/Mason/local/src/PerlLibs/bioperl-live/Bio/Root/Version.pm
>>>>
>>>> One thing I noticed, in the debug output is that you are using Bioperl
>>>> live (here --> /N/u/dstandag/Mason/local/src/PerlLibs/bioperl-live).
>>>> It's fasta indexer is broken. I have an open bug I am trying to
>>>> resolve with the Bioperl developers, but for now use the CPAN version
>>>> of Bioperl.
>>>>
>>>> Thanks,
>>>> Carson
>>>>
>>>>
>>>>
>>>>
>>>> From: Daniel Standage
>>>> <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
>>>> Date: Monday, 5 November, 2012 10:14 AM
>>>> To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
>>>> Cc: Maker Mailing List
>>>> <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
>>>> Subject: Re: Maker issues
>>>>
>>>> Debug output attached (bzip2 compressed).
>>>>
>>>>
>>>> --
>>>> Daniel S. Standage
>>>> Ph.D. Candidate
>>>> Bioinformatics and Computational Biology Program
>>>> Department of Genetics, Development, and Cell Biology
>>>> Iowa State University
>>>>
>>>>
>>>>
>>>> On Mon, Nov 5, 2012 at 10:08 AM, Carson Holt
>>>> <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
>>>> Thanks. Could you also run with the --debug flag set on the command
>>>> line for a few minutes and send me that.
>>>>
>>>> --Carson
>>>>
>>>>
>>>> From: Daniel Standage
>>>> <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
>>>> Date: Monday, 5 November, 2012 10:05 AM
>>>> To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>, Maker
>>>> Mailing List
>>>> <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
>>>> Subject: Maker issues
>>>>
>>>> Carson,
>>>>
>>>> I updated to the latest development version, made sure the TMP
>>>> directory is on native disk space, and relaunched. I have attached the
>>>> output of the job that failed in <5 minutes. It looks pretty similar to
>>>> the errors I got the last time I used the dev version.
>>>>
>>>> --
>>>> Daniel S. Standage
>>>> Ph.D. Candidate
>>>> Bioinformatics and Computational Biology Program
>>>> Department of Genetics, Development, and Cell Biology
>>>> Iowa State University
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> maker-devel mailing list
>>>> maker-devel at box290.bluehost.com
>>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>>>
>>>
>>> _______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>>
>> Jason Stajich
>> jason.stajich at gmail.com
>> jason at bioperl.org
>>
>>
>> _______________________________________________
>> maker-devel mailing list
>> maker-devel at box290.bluehost.com
>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>
>
More information about the maker-devel
mailing list