[maker-devel] master_datastore_index.log file shrinks.

[Michael Nuhn]

Hello Carson!

Thanks for your quick response and your ideas. I'll give them a try.

Cheers,
Michael.

On 03/14/2013 04:21 PM, Carson Holt wrote:
> The file should only be deleted if there are no instances running and a
> new one starts.  Then it rebuilds it.  If it is being deleted while other
> instances are still active, then yes that is a lock issue.  There are
> several other locks that should protect individual contigs while that
> particular lock is only protecting the datastore_index.log file.
>
> If any of the contig locks are not working you would start to see failures
> of contigs with weird errors that say there are missing files.
>
> Try dialling back on the number of simultaneous instances you start and
> instead use MPI or the -cpus option to get the parallelization boost.
> Alternatively you can also split up the input file and use the -base
> option so everything gets written to the same place (then you never have
> to worry about locks affecting individual contigs - as no single instance
> has access to all the contigs)
>
> Example:
> fasta_tool --chunks 5 maize_assembly.fasta
> maker -g maize_assembly_0.fasta -base maize_assembly
> maker -g maize_assembly_1.fasta -base maize_assembly
>
> maker -g maize_assembly_2.fasta -base maize_assembly
>
> maker -g maize_assembly_3.fasta -base maize_assembly
>
> maker -g maize_assembly_4.fasta -base maize_assembly
>
> maker -dsindex
>
> Everything then gets written to maize_assembly.maker.output for all
> results.  The last call to maker with the -dsindex flag then rebuilds the
> datastore_index.log file to match the original maize_assembly.fasta file
>
>
> Thanks,
> Carson
>
>
>
>
>
> On 13-03-14 12:20 PM, "Michael Nuhn" <mnuhn at ebi.ac.uk> wrote:
>
>> Hello!
>>
>> I'm trying to keep track of the progress of maker (version 2.27) while
>> it is running by looking at the master_datastore_index.log file every
>> once in a while.
>>
>> Sometimes the number of lines in it decreases. Just now it went down
>>from more than two hundred to thirty seven.
>>
>> When I start more instances of maker, the number of lines in it
>> increases when they start. But sometimes I check and the number of lines
>> has greatly reduced since the last time.
>>
>> I'm afraid that the newer instances of maker are deleting the file and
>> starting it from scratch instead of adding their progress to it.
>>
>> Is this a file locking issue I should be worried about?
>>
>> Cheers,
>> Michael.
>>
>> _______________________________________________
>> maker-devel mailing list
>> maker-devel at box290.bluehost.com
>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>
>