[maker-devel] short scaffolds finish, long scaffolds (almost always) fail

[Carson Holt]

If running under MPI, the reason for a failure may be further back in the STDERR (failures tend snowball other failures, so the initial cause is often way back). If you can capture the STDERR and send it, that would be the most informative. If its memory, you can also set all the blast_depth parameters in maker_botpts.ctl to a value like 20.

—Carson



> On Feb 19, 2020, at 1:54 PM, Devon O'Rourke <devon.orourke at gmail.com> wrote:
> 
> Hello,
> 
> I apologize for not posting directly to the archived forum but it appears that the option to enter new posts is disabled. Perhaps this is by design so emails go directly to this address. I hope this is what you are looking for.
> 
> Thank you for your continued support of Maker and your responses to the forum posts. I have been running Maker (V3.01.02-beta) to annotate a mammalian genome that consists of 22 chromosome-length scaffolds (between ~200-20Mb) and about 10,000 smaller fragments from 1Mb to 10kb in length. In my various tests in running Maker, the vast majority of the smaller fragments are annotated successfully, but nearly all the large scaffolds fail with the same error code when I look at the 'run.log.child.0' file:
> ```
> DIED	RANK	0:6:0:0
> DIED	COUNT	2
> ```
> (the master 'run.log' file just shows "DIED	COUNT	2")
> 
> I struggled to find this exact error code anywhere on the forum and was hoping you might be able to help me determine where I should start troubleshooting. I thought perhaps it was an error concerning memory requirements, so I altered the chunk size from the default to a few larger sequence lengths (I've tried 1e6, 1e7, and 999,999,999 - all produce the same outcome). I've tried running the program with parallel support using either openMPI or mpich. I've tried running on a single node using 24 cpus and 120g of RAM. It always stalls at the same step.
> 
> Interestingly, one of the 22 large scaffolds always finishes and produces the .maker.proteins.fasta, .maker.transcripts.fasta, and .gff files, but the other 21 of 22 large scaffolds fail. This makes me think perhaps it's not a memory issue?
> 
> In the case of both the completed and failed scaffolds, the "theVoid.scaffoldX" subdirectory(ies) containing the .rb.cat.gz, .rb.out, .specific.ori.out, .specific.cat.gz, .specific.out, te_proteins*fasta.repeat runner, the est *fasta.blastn, the altest *fasta.tblastx, and protein *fasta.blastx files are all present (and appear finished from what I can tell).
> However, the particular contents in the parent directory to the "theVoid.scaffold" folder differ. For the failed scaffolds, the contents generally always look something like this (that is, they stall with the same kind of files produced):
> ```
> 0
> evidence_0.gff
> query.fasta
> query.masked.fasta
> query.masked.fasta.index
> query.masked.gff
> run.log.child.0
> scaffold22.0.final.section
> scaffold22.0.pred.raw.section
> scaffold22.0.raw.section
> scaffold22.gff.ann
> scaffold22.gff.def
> scaffold22.gff.seq
> ```
> 
> For the completed scaffold, there are many more files created:
> ```
> 0
> 10
> 100
> 20
> 30
> 40
> 50
> 60
> 70
> 80
> 90
> evidence_0.gff
> evidence_10.gff
> evidence_1.gff
> evidence_2.gff
> evidence_3.gff
> evidence_4.gff
> evidence_5.gff
> evidence_6.gff
> evidence_7.gff
> evidence_8.gff
> evidence_9.gff
> query.fasta
> query.masked.fasta
> query.masked.fasta.index
> query.masked.gff
> run.log.child.0
> run.log.child.1
> run.log.child.10
> run.log.child.2
> run.log.child.3
> run.log.child.4
> run.log.child.5
> run.log.child.6
> run.log.child.7
> run.log.child.8
> run.log.child.9
> scaffold4.0-1.raw.section
> scaffold4.0.final.section
> scaffold4.0.pred.raw.section
> scaffold4.0.raw.section
> scaffold4.10.final.section
> scaffold4.10.pred.raw.section
> scaffold4.10.raw.section
> scaffold4.1-2.raw.section
> scaffold4.1.final.section
> scaffold4.1.pred.raw.section
> scaffold4.1.raw.section
> scaffold4.2-3.raw.section
> scaffold4.2.final.section
> scaffold4.2.pred.raw.section
> scaffold4.2.raw.section
> scaffold4.3-4.raw.section
> scaffold4.3.final.section
> scaffold4.3.pred.raw.section
> scaffold4.3.raw.section
> scaffold4.4-5.raw.section
> scaffold4.4.final.section
> scaffold4.4.pred.raw.section
> scaffold4.4.raw.section
> scaffold4.5-6.raw.section
> scaffold4.5.final.section
> scaffold4.5.pred.raw.section
> scaffold4.5.raw.section
> scaffold4.6-7.raw.section
> scaffold4.6.final.section
> scaffold4.6.pred.raw.section
> scaffold4.6.raw.section
> scaffold4.7-8.raw.section
> scaffold4.7.final.section
> scaffold4.7.pred.raw.section
> scaffold4.7.raw.section
> scaffold4.8-9.raw.section
> scaffold4.8.final.section
> scaffold4.8.pred.raw.section
> scaffold4.8.raw.section
> scaffold4.9-10.raw.section
> scaffold4.9.final.section
> scaffold4.9.pred.raw.section
> scaffold4.9.raw.section
> ```
> 
> Thanks for any troubleshooting tips you can offer. 
> 
> Cheers,
> Devon
> 
> -- 
> Devon O'Rourke
> Postdoctoral researcher, Northern Arizona University
> Lab of Jeffrey T. Foster - https://fozlab.weebly.com/ <https://fozlab.weebly.com/>
> twitter: @thesciencedork
> _______________________________________________
> maker-devel mailing list
> maker-devel at yandell-lab.org
> http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20200226/18185cac/attachment-0004.html>