[maker-devel] master_datastore_index.log file shrinks.

[Carson Holt]

The file should only be deleted if there are no instances running and a
new one starts.  Then it rebuilds it.  If it is being deleted while other
instances are still active, then yes that is a lock issue.  There are
several other locks that should protect individual contigs while that
particular lock is only protecting the datastore_index.log file.

If any of the contig locks are not working you would start to see failures
of contigs with weird errors that say there are missing files.

Try dialling back on the number of simultaneous instances you start and
instead use MPI or the -cpus option to get the parallelization boost.
Alternatively you can also split up the input file and use the -base
option so everything gets written to the same place (then you never have
to worry about locks affecting individual contigs - as no single instance
has access to all the contigs)

Example:
fasta_tool --chunks 5 maize_assembly.fasta
maker -g maize_assembly_0.fasta -base maize_assembly
maker -g maize_assembly_1.fasta -base maize_assembly

maker -g maize_assembly_2.fasta -base maize_assembly

maker -g maize_assembly_3.fasta -base maize_assembly

maker -g maize_assembly_4.fasta -base maize_assembly

maker -dsindex

Everything then gets written to maize_assembly.maker.output for all
results.  The last call to maker with the -dsindex flag then rebuilds the
datastore_index.log file to match the original maize_assembly.fasta file


Thanks,
Carson





On 13-03-14 12:20 PM, "Michael Nuhn" <mnuhn at ebi.ac.uk> wrote:

>Hello!
>
>I'm trying to keep track of the progress of maker (version 2.27) while
>it is running by looking at the master_datastore_index.log file every
>once in a while.
>
>Sometimes the number of lines in it decreases. Just now it went down
>from more than two hundred to thirty seven.
>
>When I start more instances of maker, the number of lines in it
>increases when they start. But sometimes I check and the number of lines
>has greatly reduced since the last time.
>
>I'm afraid that the newer instances of maker are deleting the file and
>starting it from scratch instead of adding their progress to it.
>
>Is this a file locking issue I should be worried about?
>
>Cheers,
>Michael.
>
>_______________________________________________
>maker-devel mailing list
>maker-devel at box290.bluehost.com
>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org