[maker-devel] Maker issues

Sun Nov 25 21:56:31 MST 2012

good detective work there Carson!

Mark Yandell
Professor of Human Genetics
H.A. & Edna Benning Presidential Endowed Chair
Eccles Institute of Human Genetics
University of Utah
15 North 2030 East, Room 2100
Salt Lake City, UT 84112-5330
ph:801-587-7707

________________________________________
From: maker-devel-bounces at yandell-lab.org [maker-devel-bounces at yandell-lab.org] on behalf of Carson Holt [carsonhh at gmail.com]
Sent: Sunday, November 25, 2012 9:10 PM
To: Daniel Standage
Cc: Maker Mailing List
Subject: Re: [maker-devel] Maker issues

I think the problem is in the sequence of your scaffold.  I pulled this out of the exonerate alignment -->
WTGGGGCTATGAAAAAAAAAWTTKMGMMAAAAAWTTWTKRWMRATC

Notice the letters W, K, R, M, etc.  While these are technically legal nucleotides, many external programs, and in this case BioPerl doesn't handle them well.
That is why you get -->
------------- EXCEPTION: Bio::Root::Exception -------------
MSG: Sequence is a protein. Cannot revcom

You might want to replace them in your input fasta with the letter 'N' so they are treated as masked.  You will have to delete the mpi_blastdb directory to let maker rebuild the fasta indexes and you will probably have to set clean_try=1 in the control files so that MAKER deletes old result files that contain those characters on the retry.  The other error may be just a snowball effect from the first error, so you should see of it still happens after fixing the input fasta file.

Thanks,
Carson

From: Daniel Standage <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
Date: Friday, 23 November, 2012 3:06 PM
To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
Cc: Maker Mailing List <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
Subject: Re: Maker issues

Thanks for your reply, and sorry for my delayed response.

I have attached the first file you requested, but the other two do not exist. I have attached a listing of the files in that directory. Let me know if you need anything else.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

On Mon, Nov 12, 2012 at 10:02 AM, Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
The first error is an IO error with your system.  I've added some more detail to the errors in the development version if you do an 'svn update'.  Then you will know the system specific reason why close or opened failed.  For the other error, could you send me this file  --> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.3.mason.maker.output/maker.pdom.3.mason_datastore/scaffold_7/theVoid.scaffold_7/scaffold_7.1869077-1869882.comp59027_c1_seq93.est_exonerate.0

This one --> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/scaffold_23.716125-721460.0.fasta

And this one --> /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/comp58983_c0_seq101.for.716125-721460.0.fasta

thanks,
Carson

From: Daniel Standage <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
Date: Thursday, 8 November, 2012 9:32 AM

To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
Cc: Maker Mailing List <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
Subject: Re: Maker issues

Scaling up to whole-genome annotation, things seem to be going well. However, there are some intermittent issues. I've seen a couple occurrences of the following error...

#-------------------------------#
Calling out to FastaSeq::convert at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.
running  est2genome search.
#--------- command -------------#
Widget::exonerate::est2genome:
/N/hd01/dstandag/Mason/local/bin/exonerate  -q /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/comp58983_c0_seq101.for.716125-721460.0.fasta -t /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/scaffold_23.716125-721460.0.fasta -Q dna -T dna --model est2genome  --minintron 20 --maxintron 10000 --showcigar --percent 20 > /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/scaffold_23.716125-721460.comp58983_c0_seq101.est_exonerate.0
#-------------------------------#
Calling out to FastaSeq::convert at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.
couldn't close /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.5.mason.maker.output/maker.pdom.5.mason_datastore/scaffold_23/theVoid.scaffold_23/comp58983_c0_seq37.for.716125-723330.0.fasta at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/FastaFile.pm line 60.
--> rank=NA, hostname=c4
ERROR: Failed while polishig ESTs
ERROR: Chunk failed at level:2, tier_type:2
FAILED CONTIG:scaffold_23

ERROR: Chunk failed at level:5, tier_type:0
FAILED CONTIG:scaffold_23

examining contents of the fasta file and run log
Calling Datastore::MD5::mkdir at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.
Calling uri_escape at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.
Calling File::Path::mkpath at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

...as well as one occurrence of this error.

#-------------------------------#
Calling out to FastaSeq::convert at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 1480.
running  est2genome search.
#--------- command -------------#
Widget::exonerate::est2genome:
/N/hd01/dstandag/Mason/local/bin/exonerate  -q /N/dc/scratch/dstandag/PdomGenomic/Annotation/output/maker.pdom.3.mason.maker.output/maker.pd
om.3.mason_datastore/scaffold_7/theVoid.scaffold_7/comp59027_c1_seq93.for.1869077-1869882.0.fasta -t /N/dc/scratch/dstandag/PdomGenomic/Anno
tation/output/maker.pdom.3.mason.maker.output/maker.pdom.3.mason_datastore/scaffold_7/theVoid.scaffold_7/scaffold_7.1869077-1869882.0.fasta
-Q dna -T dna --model est2genome  --minintron 20 --maxintron 10000 --showcigar --percent 20 > /N/dc/scratch/dstandag/PdomGenomic/Annotation/
output/maker.pdom.3.mason.maker.output/maker.pdom.3.mason_datastore/scaffold_7/theVoid.scaffold_7/scaffold_7.1869077-1869882.comp59027_c1_se
q93.est_exonerate.0
#-------------------------------#

------------- EXCEPTION: Bio::Root::Exception -------------
MSG: Sequence is a protein. Cannot revcom
STACK: Error::throw
STACK: Bio::Root::Root::throw /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/Root/Root.pm:368
STACK: Bio::PrimarySeqI::revcom /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/PrimarySeqI.pm:381
STACK: Bio::LocatableSeq::revcom /N/u/dstandag/Mason/local/src/PerlLibs/lib/perl5/Bio/LocatableSeq.pm:577
STACK: exonerate::splice_info::needs_to_be_revcomped /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/exonerate/splice_info.pm:86<http://splice_info.pm:86>
STACK: Widget::exonerate::est2genome::assemble /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Widget/exonerate/est2genome.pm:686<http://est2genome.pm:686>
STACK: Widget::exonerate::est2genome::parse /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Widget/exonerate/est2genome.pm:961<http://est2genome.pm:961>
STACK: polisher::exonerate::est::e_exonerate /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/polisher/exonerate/est.pm:82<http://est.pm:82>
STACK: polisher::exonerate::est::polish /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/polisher/exonerate/est.pm:44<http://est.pm:44>
STACK: GI::to_polisher /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm:1670
STACK: GI::polish_exonerate /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm:1517
STACK: Process::MpiChunk::_go /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm:1663
STACK: Process::MpiChunk::run /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm:335
STACK: Process::MpiChunk::run_all /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm:351
STACK: Process::MpiTiers::run_all /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm:286
STACK: Process::MpiTiers::run_all /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiTiers.pm:286
STACK: /N/u/dstandag/Mason/local/src/maker-dev/bin/maker:644
-----------------------------------------------------------
--> rank=NA, hostname=c4
ERROR: Failed while polishig ESTs
ERROR: Chunk failed at level:2, tier_type:2
FAILED CONTIG:scaffold_7

ERROR: Chunk failed at level:5, tier_type:0
FAILED CONTIG:scaffold_7

examining contents of the fasta file and run log
Calling Datastore::MD5::mkdir at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.
Calling uri_escape at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.
Calling File::Path::mkpath at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

I'll let you know if I see anything else.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

On Wed, Nov 7, 2012 at 11:46 AM, Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
Thanks.  Typo now fixed on my end too ;-)

Thanks,
Carson

From: Daniel Standage <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
Date: Wednesday, 7 November, 2012 11:43 AM

To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
Cc: Maker Mailing List <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
Subject: Re: Maker issues

Looked good for a while, but came across this error.

total clusters:20 now processing 0
flattening EST clusters
doing tblastx of alt-ESTs
Undefined subroutine &GI::loalize_file called at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/GI.pm line 2648.
--> rank=NA, hostname=c4
ERROR: Failed while doing tblastx of alt-ESTs
ERROR: Chunk failed at level:4, tier_type:2
FAILED CONTIG:scaffold_58

ERROR: Chunk failed at level:5, tier_type:0
FAILED CONTIG:scaffold_58

examining contents of the fasta file and run log
Calling Datastore::MD5::mkdir at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.
Calling uri_escape at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.
Calling File::Path::mkpath at /N/hd01/dstandag/Mason/local/src/maker-dev/bin/../lib/Process/MpiChunk.pm line 433.

--Next Contig--

It seems pretty clear that there is a typo in GI.pm. I changed loalize to localize and relaunched.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

On Wed, Nov 7, 2012 at 9:30 AM, Daniel Standage <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>> wrote:
Done.

Test job has successfully cleared the preliminary Fasta indexing steps and is repeat masking. I'll let you know if there are any problems. Thanks!

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

On Wed, Nov 7, 2012 at 9:00 AM, Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
1.006902        Bio::Root::Version      /N/u/dstandag/Mason/local/src/PerlLibs/bioperl-live/Bio/Root/Version.pm

One thing I noticed, in the debug output is that you are using Bioperl live (here -->  /N/u/dstandag/Mason/local/src/PerlLibs/bioperl-live).  It's fasta indexer is broken.  I have an open bug I am trying to resolve with the Bioperl developers, but for now use the CPAN version of Bioperl.

Thanks,
Carson

From: Daniel Standage <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
Date: Monday, 5 November, 2012 10:14 AM
To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
Cc: Maker Mailing List <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
Subject: Re: Maker issues

Debug output attached (bzip2 compressed).

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University

On Mon, Nov 5, 2012 at 10:08 AM, Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>> wrote:
Thanks. Could you also run with the --debug flag set on the command line for a few minutes and send me that.

--Carson

From: Daniel Standage <daniel.standage at gmail.com<mailto:daniel.standage at gmail.com>>
Date: Monday, 5 November, 2012 10:05 AM
To: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>, Maker Mailing List <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
Subject: Maker issues

Carson,

I updated to the latest development version, made sure the TMP directory is on native disk space, and relaunched. I have attached the output of the job that failed in <5 minutes. It looks pretty similar to the errors I got the last time I used the dev version.

--
Daniel S. Standage
Ph.D. Candidate
Bioinformatics and Computational Biology Program
Department of Genetics, Development, and Cell Biology
Iowa State University