[maker-devel] maker leaving large numbers of defunct zombies

[Carson Holt]

MAKER doesn't lock the SQL database except on initial creation and
population.  However, SQLite does it's own advisory locking apart from
MAKER (and it may be trying to do this with every transaction).  This
could be problematic on some NFS systems where advisory locking may be
Buggy.

I'm going to try something.  Just update the SVN repository one more time.
 What I've done is to add a check to see if you are running in an NFS
mounted directory.  If so, maker will try and localize the SQLite db file
to /tmp or wherever your TMP variable is set to in the maker_opts.ctl file
(maker will check the NFS status on that as well before attempting the
copy so as not to waste time in just copying to another NFS).  So if you
set TMP or your linux TMPDIR variable is set to an NFS mounted locations,
try changing it to a location that will be local to each machine.  Usually
/tmp (Linux default) or you can use the memory mounted virtual directory
/dev/shm.  Maker will clean up temporary files it creates afterwards.
Automatically.

By using a local rather than NFS location in this step, advisory locks
should work properly for SQLite (if that is the issue you are
experiencing).  I've added checks and optimizations so the file will not
be copied if it has already been localized to a given machine, even if it
was by another node in the MPI job.  So the overall space overhead should
be limited, and you'll probably even get a slight performance boost as
well (local SQLite databases run much faster than NFS located ones).

Thanks,
Carson






On 12-06-28 4:16 PM, "Mikael Brandström Durling" <mikael.durling at slu.se>
wrote:

>Sorry, but I might have been slightly to quick to my conclusions. All the
>initialisation and initial database creation works fine. However, when
>running I start to see locking error from SQLite. Is GFFDB supposed to
>use the NFSLock to lock the database at all accesses? After running for a
>while now, I see locking errors (ie the query or transaction failes) from
>SQLite at these lines in GFFDB.pm: 107/109, 529 and 678.
>
>Mikael