[maker-devel] MAKER and large number of 'ps' processes

[Carson Holt]

FYI.  If you use the -nolock flag, never start MAKER more than once in the
same directory.  The lack of file locks means MAKER won't detect the other
active process and they can end up overwriting each others output.  So do
any parallelization via MPI instead.

Thanks,
Carson


From:  Carson Holt <carsonhh at gmail.com>
Date:  Thursday, August 21, 2014 at 2:17 PM
To:  "Timothy Stitt (TGAC)" <Timothy.Stitt at tgac.ac.uk>,
"maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
Subject:  Re: [maker-devel] MAKER and large number of 'ps' processes

MAKER uses 'ps' every so often to check on certain processes to make sure
they haven't failed or become zombies.  On your system these 'ps' calls may
be hanging which would cause them to build up over time.
You can try and run MAKER with the '-nolock' flag, since it is the NFS file
locking that requires these process checks.

Alternatively you can edit .../maker/lib/Proc/ProcessTable_simple.pm and
change it as follows.

Find the 'new'  subroutine and change it from this -->

sub new {
    if($PS){
        my $self = {};
        my $class = shift;
        bless($self, $class);
        return $self;
    }
    else{
        eval 'require Proc::ProcessTable';
        return Proc::ProcessTable->new(@_);
    }
}

to this -->

sub new {
    eval 'require Proc::ProcessTable';
    return Proc::ProcessTable->new(@_);
}


This will access the process table directly rather than through 'ps', but it
may experience the same hang as 'ps' is experiencing.  Also you will need to
install 'Proc::ProcessTable' via CPAN for it to work, and that particular
module may not install on some Linux systems.

--Carson


From:  "Timothy Stitt (TGAC)" <Timothy.Stitt at tgac.ac.uk>
Date:  Thursday, August 21, 2014 at 2:05 PM
To:  "maker-devel at yandell-lab.org" <maker-devel at yandell-lab.org>
Subject:  [maker-devel] MAKER and large number of 'ps' processes

Dear MAKER developers,

One of my users is running MAKER on our large shared-memory SGI UV2000
system (with over 2000 cores) and the application appears to be generating
large amounts of 'ps' processes that are overwhelming the system and causing
the system to be unusable for other users.

Can you confirm that MAKER would be generating this behaviour and if so, is
there a way to prevent the application from running 'ps' repeatedly?

Thanks in advance,

Tim.
—

Timothy Stitt PhD | Head of Scientific Computing
+44 1603 450378  | timothy.stitt at tgac.ac.uk
The Genome Analysis Centre (TGAC)
Norwich Research Park, Norwich, NR4 7UH, UK | http://www.tgac.ac.uk
<http://www.tgac.ac.uk/>
_______________________________________________ maker-devel mailing list
maker-devel at box290.bluehost.comhttp://box290.bluehost.com/mailman/listinfo/m
aker-devel_yandell-lab.org


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20140821/cc392be3/attachment-0003.html>