[maker-devel] failed contigs in master_datastore_index.log but no errors in screen output

[Carson Holt]

Hi Valerie,

Sorry for the slow reply. If you are running in a screen session, instead try redirecting STDERR to a file so you can capture all errors. Example: maker &> log.err

Also the datastore_index.log is a cumulative file. Rather than just just grepping for FAILED. grep for the contig of interest. Example: grep “unclustered_scaffold_3490” Rwill4_master_datastore_index.log

You may get something like this:

unclustered_scaffold_3490	Rwill4_datastore/1D/F9/unclustered_scaffold_3490/	STARTED
unclustered_scaffold_3490	Rwill4_datastore/1D/F9/unclustered_scaffold_3490/	FAILED
unclustered_scaffold_3490	Rwill4_datastore/1D/F9/unclustered_scaffold_3490/	RETRY
unclustered_scaffold_3490	Rwill4_datastore/1D/F9/unclustered_scaffold_3490/	FINISHED

If rather than FINISHED, it shows DIED_SKIPPED_PERMANENT, then increase the maker retry count on the next run (in maker_opts file or command line flag). Then you can see why it fails on the next run by capturing all STDERR to a file.

Thanks,
Carson



> On Feb 12, 2018, at 12:41 PM, Valerie Soza <vsoza at uw.edu> wrote:
> 
> Hi all
> 
> I ran 3 instances of Maker 2.31.9 on a genome assembly using a SNAP training parameters file and my training parameters from running BUSCO on our genome. The job completed on our departmental computing cluster but when I looked at the master_datastore_index.log, 4 scaffolds had FAILED and 2 were indicated as RETRY.
> 
> I previously ran Maker on this same genome just using the SNAP training parameters file and it worked fine so I am perplexed.
> 
> $ grep FAILED Rwill4_master_datastore_index.log 
> LG12_ordered_scaffold_101	Rwill4_datastore/B7/F2/LG12_ordered_scaffold_101/	FAILED 
> unclustered_scaffold_3148	Rwill4_datastore/62/82/unclustered_scaffold_3148/	FAILED 
> unclustered_scaffold_3490	Rwill4_datastore/1D/F9/unclustered_scaffold_3490/	FAILED 
> unclustered_scaffold_7506	Rwill4_datastore/69/B8/unclustered_scaffold_7506/	FAILED
> 
> $ grep RETRY Rwill4_master_datastore_index.log 
> LG12_ordered_scaffold_101	Rwill4_datastore/B7/F2/LG12_ordered_scaffold_101/	RETRY 
> unclustered_scaffold_7506	Rwill4_datastore/69/B8/unclustered_scaffold_7506/	RETRY
> 
> When I looked at the screen output from all 3 instances, there are no errors when I grep for Error, error, or ERROR.
> None of the 3 screen outputs indicated that Maker had finished, so I restarted the job and then immediately got that "Maker is now finished!!!” 
> However when I grep for the 4 failed scaffolds above in the 3 screen outputs, I only get something for 1 of the scaffolds, which also happens to be the last lines in one of the screens' output's:
> 
> #---------------------------------------------------------------------
> Now starting the contig!!
> SeqID: LG12_ordered_scaffold_101
> Length: 89169
> #---------------------------------------------------------------------
> 
> 
> setting up GFF3 output and fasta chunks
> doing repeat masking
> running  repeat masker.
> #--------- command -------------#
> Widget::RepeatMasker:
> cd /tmp/935482.1.ravana.q/maker_ul9sWE; /net/gs/vol3/software/modules-sw/RepeatMasker/4.0.7/Linux/RHEL6/x86_64/RepeatMasker /net/shendure/vol8/projects/R.williamsianum.annotation/Maker_analyses/SNAP_training/Rwill4.maker.output/Rwill4_datastore/B7/F2/LG12_ordered_scaffold_101//theVoid.LG12_ordered_scaffold_101/0/LG12_ordered_scaffold_101.0.all.rb -species all -dir /net/shendure/vol8/projects/R.williamsianum.annotation/Maker_analyses/SNAP_training/Rwill4.maker.output/Rwill4_datastore/B7/F2/LG12_ordered_scaffold_101//theVoid.LG12_ordered_scaffold_101/0 -pa 10
> #———————————————#
> 
> It seems like the run did not finish properly. Am I interpreting this correctly? Does anyone have suggestions on what I should do or how to troubleshoot? 
> 
> Thanks.
> 
> -Valerie
> 	
> Valerie Soza, Ph.D.
> c/o Hall Lab
> Department of Biology
> University of Washington
> Johnson Hall 202A
> Box 351800
> Seattle, WA 98195-1800
> 206-543-6740
> http://staff.washington.edu/vsoza/
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org