[maker-devel] High memory consumption

[Kyungyong Seong]

Thank you for the tips! How about reducing the time for tblastx? My cluster
has a 3 days run limit. I think what is happening is that MAKER is
terminated because of out-of-memory issues or runtime cap, and when MAKER
is restarted, tblastx needs to start from scratch. Do you think it would be
better not to use MPI and set cpus=30? Or would it be okay to set up mpi =
3 and cpus=10 if I have 30 cores?


On Fri, Dec 17, 2021 at 9:29 AM Carson Holt <carsonhh at gmail.com> wrote:

> 1. Make sure your system is not configured with an in memory /tmp
> directory. If it is, every file written to temporary storage will use RAM.
> 2. If running under MPI, cpu= in maker_opts.ctl must be set to 1.
> 3. max_dna_len= should be 100000 (the default)
> 4. In maker_bopts.ctl, set all the depth_blast= options to something like
> 10 or 20 (there are 3 depth values you will have to set). The default is to
> keep everything, and if you have really deep alignments that can use a lot
> of RAM with out any actual benefit for gene prediction.
>
> —Carson
>
>
>
> > On Dec 16, 2021, at 11:03 AM, Kyungyong Seong <s.kyungyong at berkeley.edu>
> wrote:
> >
> > Hi!
> >
> > MAKER has been running fine on my genome (~1Gb; 800 contigs) but is now
> stuck with ~30 contigs that keep failing because of high memory
> consumption. I am using mpi, running 20-30 contigs for annotation in
> parallel, depending on the machine. I started with 64Gb memory machines but
> have moved up to 1.5 Tb machines as the job kept failing. Unfortunately,
> all memory of this machine is also saturated. It looks like tblastx is
> taking lots of time and resources. The databases I have are about 200 Mb
> for the proteins and 570 Mb for cDNAs. max_dna_len is set as 100000 in
> maker_opt.ctl. Would there be a way to improve this? Decreasing the number
> of jobs for MPI slowed down memory saturation but eventually the same
> happened.
> >
> > Thank you!
> > Kyungyong
> >
> >
> > _______________________________________________
> > maker-devel mailing list
> > maker-devel at yandell-lab.org
> > http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20211219/987d9b40/attachment-0003.html>