[maker-devel] Output question

[Carson Holt]

The FAILED in column 2 means MAKER failed at some point in between getting the next contig ID. You can also see that with contig KE337098.1, it starts, fails, retries, and then finally finishes.  If you are running under MPI, your NFS mount might not be able to handle all of the IO.  This can cause a series of failures that brute force retrying can eventually get through.  If you are using a non-system version of Perl (i.e. a version you installed yourself) sometimes those have a broken Berkley DB and fall back to less efficient indexing Schemes in BioPerl that can be prone to random failures.  Or if you have the temporary directory for MAKER set to an NFS mounted or in memory mounted directory, some dependencies of MAKER can have weird behavior and random failures. We recommend always setting the temporary directory to a locally mounted disk.  Other issues that can cause seemingly random failures are running out of memory or low on disk space in the temporary directory used by MAKER.

Sometime the brute force method of letting MAKER retry multiple times is the fastest solution.  If you will be running MAKER often, then you may want to investigate and system issues affecting performance.

To ensure integrity of the final results, you can also run maker with the -a option.  This will cause MAKER to rebuild final results without rerunning most of the longer running steps.  You can this if the number of transcripts in the final GFF3 is different than the number of transcripts in the fasta files for example.

If you delete the, datastore index, then you can rebuild it with the maker -dbindex option (removing redundant lines).  That will give you a better idea of how many contigs are left to run. You can also set the number of retries in the control file to a higher value or set clean_try if there are stubborn contigs that continuously fail and you want to try getting around them using the brute force method.

—Carson




> On Apr 23, 2015, at 5:16 AM, Muriel Gros-Balthazard <muriel.grosb at gmail.com> wrote:
> 
> Hello !
> 
> After running a genome containing 7,752 scaffolds, I have some differences in outputs.
> For some I have less files in the datastore.
> For instance, for one, I have only :
> KE337098%2E1.gff
> KE337098%2E1.maker.augustus.proteins.fasta
> KE337098%2E1.maker.augustus.transcripts.fasta
> run.log
> theVoid.KE337098.1
> 
> And others have more:
> KE332545%2E1.gff
> KE332545%2E1.maker.augustus_masked.proteins.fasta
> KE332545%2E1.maker.augustus_masked.transcripts.fasta
> KE332545%2E1.maker.augustus.proteins.fasta
> KE332545%2E1.maker.augustus.transcripts.fasta
> KE332545%2E1.maker.non_overlapping_ab_initio.proteins.fasta
> KE332545%2E1.maker.non_overlapping_ab_initio.transcripts.fasta
> KE332545%2E1.maker.proteins.fasta
> KE332545%2E1.maker.transcripts.fasta
> KE332545%2E1.maker.trnascan.transcripts.fasta
> run.log
> theVoid.KE332545.1
> 
> I checked that for these two scaffolds, it is written FINISHED in the output.
> Do we agree that as long as a scaffold is written FINISHED it is OK ?
> And that the process won’t start again if I relaunch Maker in the same directory ?
> I am asking this second question as I had to relaunch several times…
> 
> Also, I have something weird for some scaffolds. It has failed but the FAILED is written in column 2 of the output.
> For instance if I grep a single scaffold:
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1	Pdac_ref2013s_datastore/23/51/KE337098.1/	STARTED
> KE337098.1	Pdac_ref2013s_datastore/23/51/KE337098.1/	FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1		FAILED
> KE337098.1	Pdac_ref2013s_datastore/23/51/KE337098.1/	RETRY
> KE337098.1	Pdac_ref2013s_datastore/23/51/KE337098.1/	FINISHED
> 
> 
> Is this problematic at some point or since it’s also written FINISHED, I should not worry ?
> 
> Thanks a lot,
> 
> Muriel
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20150423/3813d3bc/attachment-0003.html>