[maker-devel] MAKER and large number of 'ps' processes

[Timothy Stitt (TGAC)]

Hi Carson,

I tried the –nolock option and it didn't have much effect. I then installed Proc:ProcessTable (which built successfully via cpan). Running MAKER though I get the following error:

Can't locate auto/Proc/ProcessTable/get_proc_by.al in @INC (@INC contains: /usr/users/TGAC_ga007/stittt/Software/MAKER/UV/maker/bin/../perl/lib /usr/users/TGAC_ga007/stittt/Software/MAKER/UV/maker/bin/../lib /usr/users/TGAC_ga007/stittt/Software/MAKER/UV/maker/bin/../src/inc/perl/lib /usr/users/TGAC_ga007/stittt/Software/Perl/5.18/site/lib /usr/users/TGAC_ga007/stittt/Software/Perl/5.18/lib .) at /usr/users/TGAC_ga007/stittt/Software/MAKER/UV/maker/bin/../lib/Proc/Signal.pm line 143.

I looked within the directories of the ProcessTable build but I don't see the get_proc_by.al file. Should I be using an older version of ProcessTable? The one that was installed is v0.50.

Thanks in advance for any further help with this.

Tim.

---

Timothy Stitt PhD / Head of Scientific Computing

The Genome Analysis Centre (TGAC)

http://www.tgac.ac.uk/


p: +44 1603 450378

e: timothy.stitt at tgac.ac.uk<mailto:timothy.stitt at tgac.ac.uk>

From: Carson Holt <carsonhh at gmail.com<mailto:carsonhh at gmail.com>>
Date: Thursday, 21 August 2014 21:17
To: Timothy Stitt <timothy.stitt at tgac.ac.uk<mailto:timothy.stitt at tgac.ac.uk>>, "maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>" <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
Subject: Re: [maker-devel] MAKER and large number of 'ps' processes

MAKER uses 'ps' every so often to check on certain processes to make sure they haven't failed or become zombies.  On your system these 'ps' calls may be hanging which would cause them to build up over time.
You can try and run MAKER with the '-nolock' flag, since it is the NFS file locking that requires these process checks.

Alternatively you can edit .../maker/lib/Proc/ProcessTable_simple.pm and change it as follows.

Find the 'new'  subroutine and change it from this -->

sub new {
    if($PS){
        my $self = {};
        my $class = shift;
        bless($self, $class);
        return $self;
    }
    else{
        eval 'require Proc::ProcessTable';
        return Proc::ProcessTable->new(@_);
    }
}

to this -->

sub new {
    eval 'require Proc::ProcessTable';
    return Proc::ProcessTable->new(@_);
}


This will access the process table directly rather than through 'ps', but it may experience the same hang as 'ps' is experiencing.  Also you will need to install 'Proc::ProcessTable' via CPAN for it to work, and that particular module may not install on some Linux systems.

--Carson


From: "Timothy Stitt (TGAC)" <Timothy.Stitt at tgac.ac.uk<mailto:Timothy.Stitt at tgac.ac.uk>>
Date: Thursday, August 21, 2014 at 2:05 PM
To: "maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>" <maker-devel at yandell-lab.org<mailto:maker-devel at yandell-lab.org>>
Subject: [maker-devel] MAKER and large number of 'ps' processes

Dear MAKER developers,

One of my users is running MAKER on our large shared-memory SGI UV2000 system (with over 2000 cores) and the application appears to be generating large amounts of 'ps' processes that are overwhelming the system and causing the system to be unusable for other users.

Can you confirm that MAKER would be generating this behaviour and if so, is there a way to prevent the application from running 'ps' repeatedly?

Thanks in advance,

Tim.

—

Timothy Stitt PhD | Head of Scientific Computing
+44 1603 450378  | timothy.stitt at tgac.ac.uk<mailto:timothy.stitt at tgac.ac.uk>

The Genome Analysis Centre (TGAC)
Norwich Research Park, Norwich, NR4 7UH, UK | http://www.tgac.ac.uk<http://www.tgac.ac.uk/>

_______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com<mailto:maker-devel at box290.bluehost.com> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20140904/ee2120ad/attachment-0002.html>