From paul at tupac.bio  Mon Mar 11 06:33:06 2019
From: paul at tupac.bio (Paul Sheridan)
Date: Mon, 11 Mar 2019 20:33:06 +0900
Subject: [maker-devel] Why doesn't MAKER use all the cores when I run it on
 an instance?
Message-ID: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>

Dear MAKER Team,

I am running MAKER 2.31.10 on a 32 core instance with 192BG of RAM. I set
cpus=32 in the maker_opts.ctl configuration file. However, the instance is
only using at most 3 cores. I would like to understand if there is a way to
get MAKER to use more cores.

Thanks in Advance,

Paul Sheridan

-- 
CSO at Tupac Bio
Email: paul at tupac.bio
Homepage: www.paulsheridan.net
Mobile: +81 80 7889 0859
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190311/9788e0e8/attachment.html>

From carsonhh at gmail.com  Mon Mar 11 11:55:09 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Mon, 11 Mar 2019 10:55:09 -0600
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
Message-ID: <34141521-B351-448F-BE1C-CA97E025A861@gmail.com>

Sorry for the slow reply. This got lost somehow. The file from the mailing list is 5 years old. Don?t use it. You are actually downgrading, not upgrading. You will break things.

Run maker outside of MPI (if you are using MPI). There might be additional error message, that are further back that are getting covered by extra MPI output.  I don?t actually have a license for fgenesh to test with any more, so debugging it for you may be difficult.  My guess is that a more recent version has slightly different output format which is breaking the parser. Also you need to restore the previous fgenesh.pm file before you can run any tests (may need to reinstall MAKER).  

Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.

Basically after a couple of tests/error messages I may be able to send you something that would at least allow you to run fgenesh outside of MAKER to see what?s going on.

?Carson



> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
> 
> Hi, Carson
> 
> I have been getting this error with fgenesh tool within MAKER. It runs ok with most of the assembly contigs but seems to fail on one contig or part of the contig with the below error
> 
> Widget::fgenesh:
> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh > /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
> #-------------------------------#
>  ...processing 9 of 24
>  ...processing 8 of 28
>  ...processing 10 of 24
>  ...processing 9 of 28
>  ...processing 11 of 24
>  ...processing 10 of 28
>  ...processing 12 of 24
>  ...processing 11 of 28
> deleted:0 genes
> ERROR: FgenesH failed
> --> rank=14, hostname=bnbcompute50
> ERROR: Failed while annotating transcripts
> ERROR: Chunk failed at level:1, tier_type:4
> FAILED CONTIG:Super-Scaffold_14.2_contig2
> 
> I updated the perl module fgenesh.pm <http://fgenesh.pm/> as suggested in the previous threads. Attached are the  maker_opts.ctl and STDERR log file.
> 
> Thanks
> 
> Kapeel
> 
>  
> 
> -- 
> 
> Kapeel Chougule
> Computational Scientist Developer II
> One Bungtown Road Cold Spring Harbor, NY 11724
> http://www.warelab.org/ <http://www.warelab.org/>
> 
> <maker_opts.ctl><stderr.log>_______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190311/ba8e0489/attachment.html>

From kapeelc at gmail.com  Mon Mar 11 12:22:44 2019
From: kapeelc at gmail.com (Kapeel Chougule)
Date: Mon, 11 Mar 2019 13:22:44 -0400
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
	<34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
Message-ID: <CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>

Thanks Carson,

I will try the settings you suggested.

Also, for those failed contigs I ran fgenesh standalone and passed the
results to pred_gff in maker_opts.ctl. The run was successful. The only
issue with this was the name attribute in column predicted by MAKER had a
very different ID compared to other genes predicted.

E.g
Chr10   maker   mRNA    33029   45468   .       -       .
ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;
Name=Chr10mRNA_1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418

Correct Name attribute :
Chr10   maker   mRNA    33029   45468   .       -       .
ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;
Name=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1
;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418

This was also reflected in the fasta headers.

Best

Kapeel



On Mon, Mar 11, 2019 at 12:55 PM Carson Holt <carsonhh at gmail.com> wrote:

> Sorry for the slow reply. This got lost somehow. The file from the mailing
> list is 5 years old. Don?t use it. You are actually downgrading, not
> upgrading. You will break things.
>
> Run maker outside of MPI (if you are using MPI). There might be additional
> error message, that are further back that are getting covered by extra MPI
> output.  I don?t actually have a license for fgenesh to test with any more,
> so debugging it for you may be difficult.  My guess is that a more recent
> version has slightly different output format which is breaking the parser.
> Also you need to restore the previous fgenesh.pm file before you can run
> any tests (may need to reinstall MAKER).
>
> Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.
>
> Basically after a couple of tests/error messages I may be able to send you
> something that would at least allow you to run fgenesh outside of MAKER to
> see what?s going on.
>
> ?Carson
>
>
>
> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
>
> Hi, Carson
>
> I have been getting this error with fgenesh tool within MAKER. It runs ok
> with most of the assembly contigs but seems to fail on one contig or part
> of the contig with the below error
>
> Widget::fgenesh:
> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap
> /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh
> /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new
> /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta
> -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh
> >
> /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
> #-------------------------------#
>  ...processing 9 of 24
>  ...processing 8 of 28
>  ...processing 10 of 24
>  ...processing 9 of 28
>  ...processing 11 of 24
>  ...processing 10 of 28
>  ...processing 12 of 24
>  ...processing 11 of 28
> deleted:0 genes
> ERROR: FgenesH failed
> --> rank=14, hostname=bnbcompute50
> ERROR: Failed while annotating transcripts
> ERROR: Chunk failed at level:1, tier_type:4
> FAILED CONTIG:Super-Scaffold_14.2_contig2
>
> I updated the perl module fgenesh.pm as suggested in the previous
> threads. Attached are the  maker_opts.ctl and STDERR log file.
>
> Thanks
>
> Kapeel
>
>
>
> --
>
>
>
> *Kapeel ChouguleComputational Scientist Developer II*
>
>
> *One Bungtown Road Cold Spring Harbor, NY 11724http://www.warelab.org/
> <http://www.warelab.org/>*
> <maker_opts.ctl><stderr.log>
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>
>
>

-- 



*Kapeel ChouguleComputational Scientist Developer II*


*One Bungtown Road Cold Spring Harbor, NY 11724http://www.warelab.org/
<http://www.warelab.org/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190311/22459c31/attachment.html>

From carsonhh at gmail.com  Mon Mar 11 12:25:19 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Mon, 11 Mar 2019 11:25:19 -0600
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
	<34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
	<CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>
Message-ID: <6729A4F9-E594-4F77-BA11-A36F104E04AA@gmail.com>

If you you pass in a gff3, MAKER uses the name in the gff3.

?Carson

Sent from my iPhone

> On Mar 11, 2019, at 11:22 AM, Kapeel Chougule <kapeelc at gmail.com> wrote:
> 
> Thanks Carson,
> 
> I will try the settings you suggested.
> 
> Also, for those failed contigs I ran fgenesh standalone and passed the results to pred_gff in maker_opts.ctl. The run was successful. The only issue with this was the name attribute in column predicted by MAKER had a very different ID compared to other genes predicted.
> 
> E.g
> Chr10   maker   mRNA    33029   45468   .       -       .       ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;Name=Chr10mRNA_1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418
> 
> Correct Name attribute :
> Chr10   maker   mRNA    33029   45468   .       -       .       ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;Name=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418
> 
> This was also reflected in the fasta headers. 
> 
> Best
> 
> Kapeel
> 
> 
> 
>> On Mon, Mar 11, 2019 at 12:55 PM Carson Holt <carsonhh at gmail.com> wrote:
>> Sorry for the slow reply. This got lost somehow. The file from the mailing list is 5 years old. Don?t use it. You are actually downgrading, not upgrading. You will break things.
>> 
>> Run maker outside of MPI (if you are using MPI). There might be additional error message, that are further back that are getting covered by extra MPI output.  I don?t actually have a license for fgenesh to test with any more, so debugging it for you may be difficult.  My guess is that a more recent version has slightly different output format which is breaking the parser. Also you need to restore the previous fgenesh.pm file before you can run any tests (may need to reinstall MAKER).  
>> 
>> Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.
>> 
>> Basically after a couple of tests/error messages I may be able to send you something that would at least allow you to run fgenesh outside of MAKER to see what?s going on.
>> 
>> ?Carson
>> 
>> 
>> 
>>> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
>>> 
>>> Hi, Carson
>>> 
>>> I have been getting this error with fgenesh tool within MAKER. It runs ok with most of the assembly contigs but seems to fail on one contig or part of the contig with the below error
>>> 
>>> Widget::fgenesh:
>>> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh > /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
>>> #-------------------------------#
>>>  ...processing 9 of 24
>>>  ...processing 8 of 28
>>>  ...processing 10 of 24
>>>  ...processing 9 of 28
>>>  ...processing 11 of 24
>>>  ...processing 10 of 28
>>>  ...processing 12 of 24
>>>  ...processing 11 of 28
>>> deleted:0 genes
>>> ERROR: FgenesH failed
>>> --> rank=14, hostname=bnbcompute50
>>> ERROR: Failed while annotating transcripts
>>> ERROR: Chunk failed at level:1, tier_type:4
>>> FAILED CONTIG:Super-Scaffold_14.2_contig2
>>> 
>>> I updated the perl module fgenesh.pm as suggested in the previous threads. Attached are the  maker_opts.ctl and STDERR log file.
>>> 
>>> Thanks
>>> 
>>> Kapeel
>>> 
>>>  
>>> 
>>> -- 
>>> 
>>> Kapeel Chougule
>>> Computational Scientist Developer II
>>> One Bungtown Road Cold Spring Harbor, NY 11724
>>> http://www.warelab.org/
>>> 
>>> <maker_opts.ctl><stderr.log>_______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>> 
> 
> 
> -- 
> 
> Kapeel Chougule
> Computational Scientist Developer II
> One Bungtown Road Cold Spring Harbor, NY 11724
> http://www.warelab.org/
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190311/bcbebead/attachment.html>

From khs3z at virginia.edu  Tue Mar 12 08:39:39 2019
From: khs3z at virginia.edu (Siller, Karsten H (khs3z))
Date: Tue, 12 Mar 2019 13:39:39 +0000
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
Message-ID: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:	VERIFIED
External Programs:	VERIFIED
External C Libraries:	VERIFIED
MPI SUPPORT:		ENABLED
MWAS Web Interface:	DISABLED
MAKER PACKAGE:		CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten

?

Karsten Siller
University of Virginia


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190312/ec672589/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6097 bytes
Desc: not available
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190312/ec672589/attachment.p7s>

From carsonhh at gmail.com  Tue Mar 12 10:07:22 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 12 Mar 2019 08:07:22 -0700
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
In-Reply-To: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>
References: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>
Message-ID: <C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>

Launch with OpenMPI?s mpiexec command. The slurm stun command you are using won?t launch it as expected.

?Carson

> On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <khs3z at virginia.edu> wrote:
> 
> Hi,
> 
> We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status
> 
> ==============================================================================
> STATUS MAKER v2.31.10
> ==============================================================================
> PERL Dependencies:	VERIFIED
> External Programs:	VERIFIED
> External C Libraries:	VERIFIED
> MPI SUPPORT:		ENABLED
> MWAS Web Interface:	DISABLED
> MAKER PACKAGE:		CONFIGURATION OK
> 
> This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):
> 
> srun maker > maker_small_PARALLEL
> 
> This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.
> 
> Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 
> 
> Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?
> 
> Thanks,
> Karsten
> 
> ?
> 
> Karsten Siller
> University of Virginia
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190312/add51654/attachment.html>

From carsonhh at gmail.com  Tue Mar 12 18:57:27 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 12 Mar 2019 16:57:27 -0700
Subject: [maker-devel] Why doesn't MAKER use all the cores when I run it
 on an instance?
In-Reply-To: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>
References: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>
Message-ID: <91F12D37-2A8D-4867-84EA-0245846A76B8@gmail.com>

The -cpu option is only for BLAST. However, if you run MAKER under MPI (even on a single machine), you can parallelize all steps. Set cpu=1 when running under MPI, otherwise BLAST will multiply the -cpu value with the MPI value, and oversubscribe the CPU.

?Carson


> On Mar 11, 2019, at 4:33 AM, Paul Sheridan <paul at tupac.bio> wrote:
> 
> Dear MAKER Team,
> 
> I am running MAKER 2.31.10 on a 32 core instance with 192BG of RAM. I set cpus=32 in the maker_opts.ctl configuration file. However, the instance is only using at most 3 cores. I would like to understand if there is a way to get MAKER to use more cores.
> 
> Thanks in Advance,
> 
> Paul Sheridan
> 
> -- 
> CSO at Tupac Bio
> Email: paul at tupac.bio
> Homepage: www.paulsheridan.net <http://www.paulsheridan.net/>
> Mobile: +81 80 7889 0859
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190312/3afbcd7e/attachment.html>

From khs3z at virginia.edu  Wed Mar 13 17:26:40 2019
From: khs3z at virginia.edu (Siller, Karsten H (khs3z))
Date: Wed, 13 Mar 2019 22:26:40 +0000
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
In-Reply-To: <C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>
References: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>,
	<C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>
Message-ID: <DM6PR13MB324172C635C6773225EBC2929C4A0@DM6PR13MB3241.namprd13.prod.outlook.com>

Ok, we?ll try it with mpiexec. Thanks for your help.

Karsten

________________________________
From: Carson Holt <carsonhh at gmail.com>
Sent: Tuesday, March 12, 2019 11:07 AM
To: Siller, Karsten H (khs3z)
Cc: maker-devel at yandell-lab.org
Subject: Re: [maker-devel] Running MAKER with MPI support and SLURM scheduler

Launch with OpenMPI?s mpiexec command. The slurm stun command you are using won?t launch it as expected.

?Carson

On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <khs3z at virginia.edu<mailto:khs3z at virginia.edu>> wrote:

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:VERIFIED
External Programs:VERIFIED
External C Libraries:VERIFIED
MPI SUPPORT:ENABLED
MWAS Web Interface:DISABLED
MAKER PACKAGE:CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation.

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten

?

Karsten Siller
University of Virginia


_______________________________________________
maker-devel mailing list
maker-devel at box290.bluehost.com<mailto:maker-devel at box290.bluehost.com>
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190313/cfe389de/attachment.html>

From isa.schon at uhasselt.be  Thu Mar 14 03:51:27 2019
From: isa.schon at uhasselt.be (Isa SCHON)
Date: Thu, 14 Mar 2019 09:51:27 +0100
Subject: [maker-devel] problem with MWAS
Message-ID: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>

Dear colleague,

I am trying to run MWAS as a first trial using MAKER for annotations. I had first trouble uploading my data and since a few days, I cannot log in anymore at all with my account. I get the error message that the website is insecure.
Could you please check what is going on?

Many thanks in advance and kind regards,

Isa Sch?n


From carsonhh at gmail.com  Thu Mar 28 13:05:25 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 28 Mar 2019 12:05:25 -0600
Subject: [maker-devel] problem with MWAS
In-Reply-To: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>
References: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>
Message-ID: <4947A0AB-456C-4583-9618-EEB1F376F598@gmail.com>

?Insecure? means your browser is using http rather than https. You can add the s manually to an address if you want. It?s a campaign from browser makers to try and get people to always use https instead. Chrome started doing this in 2018, so now anytime you use http in an address or link, it will display ?insecure".

Why your login doesn?t work would be a different issue. The server has been restarted. See if that helps.

?Carson


> On Mar 14, 2019, at 2:51 AM, Isa SCHON <isa.schon at uhasselt.be> wrote:
> 
> Dear colleague,
> 
> I am trying to run MWAS as a first trial using MAKER for annotations. I had first trouble uploading my data and since a few days, I cannot log in anymore at all with my account. I get the error message that the website is insecure.
> Could you please check what is going on?
> 
> Many thanks in advance and kind regards,
> 
> Isa Sch?n
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From sj2641 at nyu.edu  Thu Mar 14 17:25:02 2019
From: sj2641 at nyu.edu (Shaowen Jiang)
Date: Thu, 14 Mar 2019 22:25:02 -0000
Subject: [maker-devel] Installation Failure with pg_config?
Message-ID: <CADA28Zhf5q1ND8U=aT_njcGPGapEmbPTcFowBpM6CwT2jMv32Q@mail.gmail.com>

Dear MAKER2 admins:

Hi, I have read some tutorials for annotating a newly assembly genome and
MAKER2 seems to be a very good and functional pipeline to me. So I am
trying to use it to annotate a new assembly mammalian genome that our lab
just generated.
But I was stuck while I was trying to install MAKER2 to our slurm HPC
server.
I think the pipeline is trying to install several perl packages locally,
but one of them called DBD::Pg requires the path of pg_config?
screenshot as below

But I think our server doesn't have this path and I don't have root to
install some other stuff, like libpq-dev or PostgreSQL.
Is that any other methods that can circle around that?

Any help or advice would be appreciated!

Thanks,
Shaowen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://box290.bluehost.com/pipermail/maker-devel_yandell-lab.org/attachments/20190314/0b5599e2/attachment.html>

From paul at tupac.bio  Mon Mar 11 05:33:06 2019
From: paul at tupac.bio (Paul Sheridan)
Date: Mon, 11 Mar 2019 20:33:06 +0900
Subject: [maker-devel] Why doesn't MAKER use all the cores when I run it on
 an instance?
Message-ID: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>

Dear MAKER Team,

I am running MAKER 2.31.10 on a 32 core instance with 192BG of RAM. I set
cpus=32 in the maker_opts.ctl configuration file. However, the instance is
only using at most 3 cores. I would like to understand if there is a way to
get MAKER to use more cores.

Thanks in Advance,

Paul Sheridan

-- 
CSO at Tupac Bio
Email: paul at tupac.bio
Homepage: www.paulsheridan.net
Mobile: +81 80 7889 0859
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/9788e0e8/attachment-0001.html>

From carsonhh at gmail.com  Mon Mar 11 10:55:09 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Mon, 11 Mar 2019 10:55:09 -0600
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
Message-ID: <34141521-B351-448F-BE1C-CA97E025A861@gmail.com>

Sorry for the slow reply. This got lost somehow. The file from the mailing list is 5 years old. Don?t use it. You are actually downgrading, not upgrading. You will break things.

Run maker outside of MPI (if you are using MPI). There might be additional error message, that are further back that are getting covered by extra MPI output.  I don?t actually have a license for fgenesh to test with any more, so debugging it for you may be difficult.  My guess is that a more recent version has slightly different output format which is breaking the parser. Also you need to restore the previous fgenesh.pm file before you can run any tests (may need to reinstall MAKER).  

Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.

Basically after a couple of tests/error messages I may be able to send you something that would at least allow you to run fgenesh outside of MAKER to see what?s going on.

?Carson



> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
> 
> Hi, Carson
> 
> I have been getting this error with fgenesh tool within MAKER. It runs ok with most of the assembly contigs but seems to fail on one contig or part of the contig with the below error
> 
> Widget::fgenesh:
> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh > /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
> #-------------------------------#
>  ...processing 9 of 24
>  ...processing 8 of 28
>  ...processing 10 of 24
>  ...processing 9 of 28
>  ...processing 11 of 24
>  ...processing 10 of 28
>  ...processing 12 of 24
>  ...processing 11 of 28
> deleted:0 genes
> ERROR: FgenesH failed
> --> rank=14, hostname=bnbcompute50
> ERROR: Failed while annotating transcripts
> ERROR: Chunk failed at level:1, tier_type:4
> FAILED CONTIG:Super-Scaffold_14.2_contig2
> 
> I updated the perl module fgenesh.pm <http://fgenesh.pm/> as suggested in the previous threads. Attached are the  maker_opts.ctl and STDERR log file.
> 
> Thanks
> 
> Kapeel
> 
>  
> 
> -- 
> 
> Kapeel Chougule
> Computational Scientist Developer II
> One Bungtown Road Cold Spring Harbor, NY 11724
> http://www.warelab.org/ <http://www.warelab.org/>
> 
> <maker_opts.ctl><stderr.log>_______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/ba8e0489/attachment-0001.html>

From kapeelc at gmail.com  Mon Mar 11 11:22:44 2019
From: kapeelc at gmail.com (Kapeel Chougule)
Date: Mon, 11 Mar 2019 13:22:44 -0400
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
	<34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
Message-ID: <CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>

Thanks Carson,

I will try the settings you suggested.

Also, for those failed contigs I ran fgenesh standalone and passed the
results to pred_gff in maker_opts.ctl. The run was successful. The only
issue with this was the name attribute in column predicted by MAKER had a
very different ID compared to other genes predicted.

E.g
Chr10   maker   mRNA    33029   45468   .       -       .
ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;
Name=Chr10mRNA_1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418

Correct Name attribute :
Chr10   maker   mRNA    33029   45468   .       -       .
ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;
Name=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1
;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418

This was also reflected in the fasta headers.

Best

Kapeel



On Mon, Mar 11, 2019 at 12:55 PM Carson Holt <carsonhh at gmail.com> wrote:

> Sorry for the slow reply. This got lost somehow. The file from the mailing
> list is 5 years old. Don?t use it. You are actually downgrading, not
> upgrading. You will break things.
>
> Run maker outside of MPI (if you are using MPI). There might be additional
> error message, that are further back that are getting covered by extra MPI
> output.  I don?t actually have a license for fgenesh to test with any more,
> so debugging it for you may be difficult.  My guess is that a more recent
> version has slightly different output format which is breaking the parser.
> Also you need to restore the previous fgenesh.pm file before you can run
> any tests (may need to reinstall MAKER).
>
> Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.
>
> Basically after a couple of tests/error messages I may be able to send you
> something that would at least allow you to run fgenesh outside of MAKER to
> see what?s going on.
>
> ?Carson
>
>
>
> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
>
> Hi, Carson
>
> I have been getting this error with fgenesh tool within MAKER. It runs ok
> with most of the assembly contigs but seems to fail on one contig or part
> of the contig with the below error
>
> Widget::fgenesh:
> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap
> /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh
> /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new
> /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta
> -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh
> >
> /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
> #-------------------------------#
>  ...processing 9 of 24
>  ...processing 8 of 28
>  ...processing 10 of 24
>  ...processing 9 of 28
>  ...processing 11 of 24
>  ...processing 10 of 28
>  ...processing 12 of 24
>  ...processing 11 of 28
> deleted:0 genes
> ERROR: FgenesH failed
> --> rank=14, hostname=bnbcompute50
> ERROR: Failed while annotating transcripts
> ERROR: Chunk failed at level:1, tier_type:4
> FAILED CONTIG:Super-Scaffold_14.2_contig2
>
> I updated the perl module fgenesh.pm as suggested in the previous
> threads. Attached are the  maker_opts.ctl and STDERR log file.
>
> Thanks
>
> Kapeel
>
>
>
> --
>
>
>
> *Kapeel ChouguleComputational Scientist Developer II*
>
>
> *One Bungtown Road Cold Spring Harbor, NY 11724http://www.warelab.org/
> <http://www.warelab.org/>*
> <maker_opts.ctl><stderr.log>
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>
>
>

-- 



*Kapeel ChouguleComputational Scientist Developer II*


*One Bungtown Road Cold Spring Harbor, NY 11724http://www.warelab.org/
<http://www.warelab.org/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/22459c31/attachment-0001.html>

From carsonhh at gmail.com  Mon Mar 11 11:25:19 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Mon, 11 Mar 2019 11:25:19 -0600
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
	<34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
	<CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>
Message-ID: <6729A4F9-E594-4F77-BA11-A36F104E04AA@gmail.com>

If you you pass in a gff3, MAKER uses the name in the gff3.

?Carson

Sent from my iPhone

> On Mar 11, 2019, at 11:22 AM, Kapeel Chougule <kapeelc at gmail.com> wrote:
> 
> Thanks Carson,
> 
> I will try the settings you suggested.
> 
> Also, for those failed contigs I ran fgenesh standalone and passed the results to pred_gff in maker_opts.ctl. The run was successful. The only issue with this was the name attribute in column predicted by MAKER had a very different ID compared to other genes predicted.
> 
> E.g
> Chr10   maker   mRNA    33029   45468   .       -       .       ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;Name=Chr10mRNA_1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418
> 
> Correct Name attribute :
> Chr10   maker   mRNA    33029   45468   .       -       .       ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;Name=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418
> 
> This was also reflected in the fasta headers. 
> 
> Best
> 
> Kapeel
> 
> 
> 
>> On Mon, Mar 11, 2019 at 12:55 PM Carson Holt <carsonhh at gmail.com> wrote:
>> Sorry for the slow reply. This got lost somehow. The file from the mailing list is 5 years old. Don?t use it. You are actually downgrading, not upgrading. You will break things.
>> 
>> Run maker outside of MPI (if you are using MPI). There might be additional error message, that are further back that are getting covered by extra MPI output.  I don?t actually have a license for fgenesh to test with any more, so debugging it for you may be difficult.  My guess is that a more recent version has slightly different output format which is breaking the parser. Also you need to restore the previous fgenesh.pm file before you can run any tests (may need to reinstall MAKER).  
>> 
>> Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.
>> 
>> Basically after a couple of tests/error messages I may be able to send you something that would at least allow you to run fgenesh outside of MAKER to see what?s going on.
>> 
>> ?Carson
>> 
>> 
>> 
>>> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
>>> 
>>> Hi, Carson
>>> 
>>> I have been getting this error with fgenesh tool within MAKER. It runs ok with most of the assembly contigs but seems to fail on one contig or part of the contig with the below error
>>> 
>>> Widget::fgenesh:
>>> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh > /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
>>> #-------------------------------#
>>>  ...processing 9 of 24
>>>  ...processing 8 of 28
>>>  ...processing 10 of 24
>>>  ...processing 9 of 28
>>>  ...processing 11 of 24
>>>  ...processing 10 of 28
>>>  ...processing 12 of 24
>>>  ...processing 11 of 28
>>> deleted:0 genes
>>> ERROR: FgenesH failed
>>> --> rank=14, hostname=bnbcompute50
>>> ERROR: Failed while annotating transcripts
>>> ERROR: Chunk failed at level:1, tier_type:4
>>> FAILED CONTIG:Super-Scaffold_14.2_contig2
>>> 
>>> I updated the perl module fgenesh.pm as suggested in the previous threads. Attached are the  maker_opts.ctl and STDERR log file.
>>> 
>>> Thanks
>>> 
>>> Kapeel
>>> 
>>>  
>>> 
>>> -- 
>>> 
>>> Kapeel Chougule
>>> Computational Scientist Developer II
>>> One Bungtown Road Cold Spring Harbor, NY 11724
>>> http://www.warelab.org/
>>> 
>>> <maker_opts.ctl><stderr.log>_______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>> 
> 
> 
> -- 
> 
> Kapeel Chougule
> Computational Scientist Developer II
> One Bungtown Road Cold Spring Harbor, NY 11724
> http://www.warelab.org/
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/bcbebead/attachment-0001.html>

From khs3z at virginia.edu  Tue Mar 12 07:39:39 2019
From: khs3z at virginia.edu (Siller, Karsten H (khs3z))
Date: Tue, 12 Mar 2019 13:39:39 +0000
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
Message-ID: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:	VERIFIED
External Programs:	VERIFIED
External C Libraries:	VERIFIED
MPI SUPPORT:		ENABLED
MWAS Web Interface:	DISABLED
MAKER PACKAGE:		CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten

?

Karsten Siller
University of Virginia


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/ec672589/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6097 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/ec672589/attachment-0001.p7s>

From carsonhh at gmail.com  Tue Mar 12 09:07:22 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 12 Mar 2019 08:07:22 -0700
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
In-Reply-To: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>
References: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>
Message-ID: <C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>

Launch with OpenMPI?s mpiexec command. The slurm stun command you are using won?t launch it as expected.

?Carson

> On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <khs3z at virginia.edu> wrote:
> 
> Hi,
> 
> We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status
> 
> ==============================================================================
> STATUS MAKER v2.31.10
> ==============================================================================
> PERL Dependencies:	VERIFIED
> External Programs:	VERIFIED
> External C Libraries:	VERIFIED
> MPI SUPPORT:		ENABLED
> MWAS Web Interface:	DISABLED
> MAKER PACKAGE:		CONFIGURATION OK
> 
> This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):
> 
> srun maker > maker_small_PARALLEL
> 
> This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.
> 
> Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 
> 
> Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?
> 
> Thanks,
> Karsten
> 
> ?
> 
> Karsten Siller
> University of Virginia
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/add51654/attachment-0001.html>

From carsonhh at gmail.com  Tue Mar 12 17:57:27 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 12 Mar 2019 16:57:27 -0700
Subject: [maker-devel] Why doesn't MAKER use all the cores when I run it
 on an instance?
In-Reply-To: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>
References: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>
Message-ID: <91F12D37-2A8D-4867-84EA-0245846A76B8@gmail.com>

The -cpu option is only for BLAST. However, if you run MAKER under MPI (even on a single machine), you can parallelize all steps. Set cpu=1 when running under MPI, otherwise BLAST will multiply the -cpu value with the MPI value, and oversubscribe the CPU.

?Carson


> On Mar 11, 2019, at 4:33 AM, Paul Sheridan <paul at tupac.bio> wrote:
> 
> Dear MAKER Team,
> 
> I am running MAKER 2.31.10 on a 32 core instance with 192BG of RAM. I set cpus=32 in the maker_opts.ctl configuration file. However, the instance is only using at most 3 cores. I would like to understand if there is a way to get MAKER to use more cores.
> 
> Thanks in Advance,
> 
> Paul Sheridan
> 
> -- 
> CSO at Tupac Bio
> Email: paul at tupac.bio
> Homepage: www.paulsheridan.net <http://www.paulsheridan.net/>
> Mobile: +81 80 7889 0859
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/3afbcd7e/attachment-0001.html>

From khs3z at virginia.edu  Wed Mar 13 16:26:40 2019
From: khs3z at virginia.edu (Siller, Karsten H (khs3z))
Date: Wed, 13 Mar 2019 22:26:40 +0000
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
In-Reply-To: <C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>
References: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>,
	<C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>
Message-ID: <DM6PR13MB324172C635C6773225EBC2929C4A0@DM6PR13MB3241.namprd13.prod.outlook.com>

Ok, we?ll try it with mpiexec. Thanks for your help.

Karsten

________________________________
From: Carson Holt <carsonhh at gmail.com>
Sent: Tuesday, March 12, 2019 11:07 AM
To: Siller, Karsten H (khs3z)
Cc: maker-devel at yandell-lab.org
Subject: Re: [maker-devel] Running MAKER with MPI support and SLURM scheduler

Launch with OpenMPI?s mpiexec command. The slurm stun command you are using won?t launch it as expected.

?Carson

On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <khs3z at virginia.edu<mailto:khs3z at virginia.edu>> wrote:

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:VERIFIED
External Programs:VERIFIED
External C Libraries:VERIFIED
MPI SUPPORT:ENABLED
MWAS Web Interface:DISABLED
MAKER PACKAGE:CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation.

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten

?

Karsten Siller
University of Virginia


_______________________________________________
maker-devel mailing list
maker-devel at box290.bluehost.com<mailto:maker-devel at box290.bluehost.com>
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190313/cfe389de/attachment-0001.html>

From isa.schon at uhasselt.be  Thu Mar 14 02:51:27 2019
From: isa.schon at uhasselt.be (Isa SCHON)
Date: Thu, 14 Mar 2019 09:51:27 +0100
Subject: [maker-devel] problem with MWAS
Message-ID: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>

Dear colleague,

I am trying to run MWAS as a first trial using MAKER for annotations. I had first trouble uploading my data and since a few days, I cannot log in anymore at all with my account. I get the error message that the website is insecure.
Could you please check what is going on?

Many thanks in advance and kind regards,

Isa Sch?n


From carsonhh at gmail.com  Thu Mar 28 12:05:25 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 28 Mar 2019 12:05:25 -0600
Subject: [maker-devel] problem with MWAS
In-Reply-To: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>
References: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>
Message-ID: <4947A0AB-456C-4583-9618-EEB1F376F598@gmail.com>

?Insecure? means your browser is using http rather than https. You can add the s manually to an address if you want. It?s a campaign from browser makers to try and get people to always use https instead. Chrome started doing this in 2018, so now anytime you use http in an address or link, it will display ?insecure".

Why your login doesn?t work would be a different issue. The server has been restarted. See if that helps.

?Carson


> On Mar 14, 2019, at 2:51 AM, Isa SCHON <isa.schon at uhasselt.be> wrote:
> 
> Dear colleague,
> 
> I am trying to run MWAS as a first trial using MAKER for annotations. I had first trouble uploading my data and since a few days, I cannot log in anymore at all with my account. I get the error message that the website is insecure.
> Could you please check what is going on?
> 
> Many thanks in advance and kind regards,
> 
> Isa Sch?n
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From sj2641 at nyu.edu  Thu Mar 14 16:25:02 2019
From: sj2641 at nyu.edu (Shaowen Jiang)
Date: Thu, 14 Mar 2019 22:25:02 -0000
Subject: [maker-devel] Installation Failure with pg_config?
Message-ID: <CADA28Zhf5q1ND8U=aT_njcGPGapEmbPTcFowBpM6CwT2jMv32Q@mail.gmail.com>

Dear MAKER2 admins:

Hi, I have read some tutorials for annotating a newly assembly genome and
MAKER2 seems to be a very good and functional pipeline to me. So I am
trying to use it to annotate a new assembly mammalian genome that our lab
just generated.
But I was stuck while I was trying to install MAKER2 to our slurm HPC
server.
I think the pipeline is trying to install several perl packages locally,
but one of them called DBD::Pg requires the path of pg_config?
screenshot as below

But I think our server doesn't have this path and I don't have root to
install some other stuff, like libpq-dev or PostgreSQL.
Is that any other methods that can circle around that?

Any help or advice would be appreciated!

Thanks,
Shaowen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190314/0b5599e2/attachment-0001.html>

From paul at tupac.bio  Mon Mar 11 05:33:06 2019
From: paul at tupac.bio (Paul Sheridan)
Date: Mon, 11 Mar 2019 20:33:06 +0900
Subject: [maker-devel] Why doesn't MAKER use all the cores when I run it on
 an instance?
Message-ID: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>

Dear MAKER Team,

I am running MAKER 2.31.10 on a 32 core instance with 192BG of RAM. I set
cpus=32 in the maker_opts.ctl configuration file. However, the instance is
only using at most 3 cores. I would like to understand if there is a way to
get MAKER to use more cores.

Thanks in Advance,

Paul Sheridan

-- 
CSO at Tupac Bio
Email: paul at tupac.bio
Homepage: www.paulsheridan.net
Mobile: +81 80 7889 0859
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/9788e0e8/attachment-0002.html>

From carsonhh at gmail.com  Mon Mar 11 10:55:09 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Mon, 11 Mar 2019 10:55:09 -0600
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
Message-ID: <34141521-B351-448F-BE1C-CA97E025A861@gmail.com>

Sorry for the slow reply. This got lost somehow. The file from the mailing list is 5 years old. Don?t use it. You are actually downgrading, not upgrading. You will break things.

Run maker outside of MPI (if you are using MPI). There might be additional error message, that are further back that are getting covered by extra MPI output.  I don?t actually have a license for fgenesh to test with any more, so debugging it for you may be difficult.  My guess is that a more recent version has slightly different output format which is breaking the parser. Also you need to restore the previous fgenesh.pm file before you can run any tests (may need to reinstall MAKER).  

Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.

Basically after a couple of tests/error messages I may be able to send you something that would at least allow you to run fgenesh outside of MAKER to see what?s going on.

?Carson



> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
> 
> Hi, Carson
> 
> I have been getting this error with fgenesh tool within MAKER. It runs ok with most of the assembly contigs but seems to fail on one contig or part of the contig with the below error
> 
> Widget::fgenesh:
> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh > /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
> #-------------------------------#
>  ...processing 9 of 24
>  ...processing 8 of 28
>  ...processing 10 of 24
>  ...processing 9 of 28
>  ...processing 11 of 24
>  ...processing 10 of 28
>  ...processing 12 of 24
>  ...processing 11 of 28
> deleted:0 genes
> ERROR: FgenesH failed
> --> rank=14, hostname=bnbcompute50
> ERROR: Failed while annotating transcripts
> ERROR: Chunk failed at level:1, tier_type:4
> FAILED CONTIG:Super-Scaffold_14.2_contig2
> 
> I updated the perl module fgenesh.pm <http://fgenesh.pm/> as suggested in the previous threads. Attached are the  maker_opts.ctl and STDERR log file.
> 
> Thanks
> 
> Kapeel
> 
>  
> 
> -- 
> 
> Kapeel Chougule
> Computational Scientist Developer II
> One Bungtown Road Cold Spring Harbor, NY 11724
> http://www.warelab.org/ <http://www.warelab.org/>
> 
> <maker_opts.ctl><stderr.log>_______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/ba8e0489/attachment-0002.html>

From kapeelc at gmail.com  Mon Mar 11 11:22:44 2019
From: kapeelc at gmail.com (Kapeel Chougule)
Date: Mon, 11 Mar 2019 13:22:44 -0400
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
	<34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
Message-ID: <CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>

Thanks Carson,

I will try the settings you suggested.

Also, for those failed contigs I ran fgenesh standalone and passed the
results to pred_gff in maker_opts.ctl. The run was successful. The only
issue with this was the name attribute in column predicted by MAKER had a
very different ID compared to other genes predicted.

E.g
Chr10   maker   mRNA    33029   45468   .       -       .
ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;
Name=Chr10mRNA_1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418

Correct Name attribute :
Chr10   maker   mRNA    33029   45468   .       -       .
ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;
Name=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1
;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418

This was also reflected in the fasta headers.

Best

Kapeel



On Mon, Mar 11, 2019 at 12:55 PM Carson Holt <carsonhh at gmail.com> wrote:

> Sorry for the slow reply. This got lost somehow. The file from the mailing
> list is 5 years old. Don?t use it. You are actually downgrading, not
> upgrading. You will break things.
>
> Run maker outside of MPI (if you are using MPI). There might be additional
> error message, that are further back that are getting covered by extra MPI
> output.  I don?t actually have a license for fgenesh to test with any more,
> so debugging it for you may be difficult.  My guess is that a more recent
> version has slightly different output format which is breaking the parser.
> Also you need to restore the previous fgenesh.pm file before you can run
> any tests (may need to reinstall MAKER).
>
> Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.
>
> Basically after a couple of tests/error messages I may be able to send you
> something that would at least allow you to run fgenesh outside of MAKER to
> see what?s going on.
>
> ?Carson
>
>
>
> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
>
> Hi, Carson
>
> I have been getting this error with fgenesh tool within MAKER. It runs ok
> with most of the assembly contigs but seems to fail on one contig or part
> of the contig with the below error
>
> Widget::fgenesh:
> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap
> /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh
> /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new
> /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta
> -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh
> >
> /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
> #-------------------------------#
>  ...processing 9 of 24
>  ...processing 8 of 28
>  ...processing 10 of 24
>  ...processing 9 of 28
>  ...processing 11 of 24
>  ...processing 10 of 28
>  ...processing 12 of 24
>  ...processing 11 of 28
> deleted:0 genes
> ERROR: FgenesH failed
> --> rank=14, hostname=bnbcompute50
> ERROR: Failed while annotating transcripts
> ERROR: Chunk failed at level:1, tier_type:4
> FAILED CONTIG:Super-Scaffold_14.2_contig2
>
> I updated the perl module fgenesh.pm as suggested in the previous
> threads. Attached are the  maker_opts.ctl and STDERR log file.
>
> Thanks
>
> Kapeel
>
>
>
> --
>
>
>
> *Kapeel ChouguleComputational Scientist Developer II*
>
>
> *One Bungtown Road Cold Spring Harbor, NY 11724http://www.warelab.org/
> <http://www.warelab.org/>*
> <maker_opts.ctl><stderr.log>
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>
>
>

-- 



*Kapeel ChouguleComputational Scientist Developer II*


*One Bungtown Road Cold Spring Harbor, NY 11724http://www.warelab.org/
<http://www.warelab.org/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/22459c31/attachment-0002.html>

From carsonhh at gmail.com  Mon Mar 11 11:25:19 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Mon, 11 Mar 2019 11:25:19 -0600
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
	<34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
	<CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>
Message-ID: <6729A4F9-E594-4F77-BA11-A36F104E04AA@gmail.com>

If you you pass in a gff3, MAKER uses the name in the gff3.

?Carson

Sent from my iPhone

> On Mar 11, 2019, at 11:22 AM, Kapeel Chougule <kapeelc at gmail.com> wrote:
> 
> Thanks Carson,
> 
> I will try the settings you suggested.
> 
> Also, for those failed contigs I ran fgenesh standalone and passed the results to pred_gff in maker_opts.ctl. The run was successful. The only issue with this was the name attribute in column predicted by MAKER had a very different ID compared to other genes predicted.
> 
> E.g
> Chr10   maker   mRNA    33029   45468   .       -       .       ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;Name=Chr10mRNA_1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418
> 
> Correct Name attribute :
> Chr10   maker   mRNA    33029   45468   .       -       .       ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;Name=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418
> 
> This was also reflected in the fasta headers. 
> 
> Best
> 
> Kapeel
> 
> 
> 
>> On Mon, Mar 11, 2019 at 12:55 PM Carson Holt <carsonhh at gmail.com> wrote:
>> Sorry for the slow reply. This got lost somehow. The file from the mailing list is 5 years old. Don?t use it. You are actually downgrading, not upgrading. You will break things.
>> 
>> Run maker outside of MPI (if you are using MPI). There might be additional error message, that are further back that are getting covered by extra MPI output.  I don?t actually have a license for fgenesh to test with any more, so debugging it for you may be difficult.  My guess is that a more recent version has slightly different output format which is breaking the parser. Also you need to restore the previous fgenesh.pm file before you can run any tests (may need to reinstall MAKER).  
>> 
>> Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.
>> 
>> Basically after a couple of tests/error messages I may be able to send you something that would at least allow you to run fgenesh outside of MAKER to see what?s going on.
>> 
>> ?Carson
>> 
>> 
>> 
>>> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
>>> 
>>> Hi, Carson
>>> 
>>> I have been getting this error with fgenesh tool within MAKER. It runs ok with most of the assembly contigs but seems to fail on one contig or part of the contig with the below error
>>> 
>>> Widget::fgenesh:
>>> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh > /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
>>> #-------------------------------#
>>>  ...processing 9 of 24
>>>  ...processing 8 of 28
>>>  ...processing 10 of 24
>>>  ...processing 9 of 28
>>>  ...processing 11 of 24
>>>  ...processing 10 of 28
>>>  ...processing 12 of 24
>>>  ...processing 11 of 28
>>> deleted:0 genes
>>> ERROR: FgenesH failed
>>> --> rank=14, hostname=bnbcompute50
>>> ERROR: Failed while annotating transcripts
>>> ERROR: Chunk failed at level:1, tier_type:4
>>> FAILED CONTIG:Super-Scaffold_14.2_contig2
>>> 
>>> I updated the perl module fgenesh.pm as suggested in the previous threads. Attached are the  maker_opts.ctl and STDERR log file.
>>> 
>>> Thanks
>>> 
>>> Kapeel
>>> 
>>>  
>>> 
>>> -- 
>>> 
>>> Kapeel Chougule
>>> Computational Scientist Developer II
>>> One Bungtown Road Cold Spring Harbor, NY 11724
>>> http://www.warelab.org/
>>> 
>>> <maker_opts.ctl><stderr.log>_______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>> 
> 
> 
> -- 
> 
> Kapeel Chougule
> Computational Scientist Developer II
> One Bungtown Road Cold Spring Harbor, NY 11724
> http://www.warelab.org/
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/bcbebead/attachment-0002.html>

From khs3z at virginia.edu  Tue Mar 12 07:39:39 2019
From: khs3z at virginia.edu (Siller, Karsten H (khs3z))
Date: Tue, 12 Mar 2019 13:39:39 +0000
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
Message-ID: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:	VERIFIED
External Programs:	VERIFIED
External C Libraries:	VERIFIED
MPI SUPPORT:		ENABLED
MWAS Web Interface:	DISABLED
MAKER PACKAGE:		CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten

?

Karsten Siller
University of Virginia


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/ec672589/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6097 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/ec672589/attachment-0002.p7s>

From carsonhh at gmail.com  Tue Mar 12 09:07:22 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 12 Mar 2019 08:07:22 -0700
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
In-Reply-To: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>
References: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>
Message-ID: <C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>

Launch with OpenMPI?s mpiexec command. The slurm stun command you are using won?t launch it as expected.

?Carson

> On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <khs3z at virginia.edu> wrote:
> 
> Hi,
> 
> We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status
> 
> ==============================================================================
> STATUS MAKER v2.31.10
> ==============================================================================
> PERL Dependencies:	VERIFIED
> External Programs:	VERIFIED
> External C Libraries:	VERIFIED
> MPI SUPPORT:		ENABLED
> MWAS Web Interface:	DISABLED
> MAKER PACKAGE:		CONFIGURATION OK
> 
> This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):
> 
> srun maker > maker_small_PARALLEL
> 
> This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.
> 
> Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 
> 
> Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?
> 
> Thanks,
> Karsten
> 
> ?
> 
> Karsten Siller
> University of Virginia
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/add51654/attachment-0002.html>

From carsonhh at gmail.com  Tue Mar 12 17:57:27 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 12 Mar 2019 16:57:27 -0700
Subject: [maker-devel] Why doesn't MAKER use all the cores when I run it
 on an instance?
In-Reply-To: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>
References: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>
Message-ID: <91F12D37-2A8D-4867-84EA-0245846A76B8@gmail.com>

The -cpu option is only for BLAST. However, if you run MAKER under MPI (even on a single machine), you can parallelize all steps. Set cpu=1 when running under MPI, otherwise BLAST will multiply the -cpu value with the MPI value, and oversubscribe the CPU.

?Carson


> On Mar 11, 2019, at 4:33 AM, Paul Sheridan <paul at tupac.bio> wrote:
> 
> Dear MAKER Team,
> 
> I am running MAKER 2.31.10 on a 32 core instance with 192BG of RAM. I set cpus=32 in the maker_opts.ctl configuration file. However, the instance is only using at most 3 cores. I would like to understand if there is a way to get MAKER to use more cores.
> 
> Thanks in Advance,
> 
> Paul Sheridan
> 
> -- 
> CSO at Tupac Bio
> Email: paul at tupac.bio
> Homepage: www.paulsheridan.net <http://www.paulsheridan.net/>
> Mobile: +81 80 7889 0859
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/3afbcd7e/attachment-0002.html>

From khs3z at virginia.edu  Wed Mar 13 16:26:40 2019
From: khs3z at virginia.edu (Siller, Karsten H (khs3z))
Date: Wed, 13 Mar 2019 22:26:40 +0000
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
In-Reply-To: <C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>
References: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>,
	<C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>
Message-ID: <DM6PR13MB324172C635C6773225EBC2929C4A0@DM6PR13MB3241.namprd13.prod.outlook.com>

Ok, we?ll try it with mpiexec. Thanks for your help.

Karsten

________________________________
From: Carson Holt <carsonhh at gmail.com>
Sent: Tuesday, March 12, 2019 11:07 AM
To: Siller, Karsten H (khs3z)
Cc: maker-devel at yandell-lab.org
Subject: Re: [maker-devel] Running MAKER with MPI support and SLURM scheduler

Launch with OpenMPI?s mpiexec command. The slurm stun command you are using won?t launch it as expected.

?Carson

On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <khs3z at virginia.edu<mailto:khs3z at virginia.edu>> wrote:

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:VERIFIED
External Programs:VERIFIED
External C Libraries:VERIFIED
MPI SUPPORT:ENABLED
MWAS Web Interface:DISABLED
MAKER PACKAGE:CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation.

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten

?

Karsten Siller
University of Virginia


_______________________________________________
maker-devel mailing list
maker-devel at box290.bluehost.com<mailto:maker-devel at box290.bluehost.com>
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190313/cfe389de/attachment-0002.html>

From isa.schon at uhasselt.be  Thu Mar 14 02:51:27 2019
From: isa.schon at uhasselt.be (Isa SCHON)
Date: Thu, 14 Mar 2019 09:51:27 +0100
Subject: [maker-devel] problem with MWAS
Message-ID: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>

Dear colleague,

I am trying to run MWAS as a first trial using MAKER for annotations. I had first trouble uploading my data and since a few days, I cannot log in anymore at all with my account. I get the error message that the website is insecure.
Could you please check what is going on?

Many thanks in advance and kind regards,

Isa Sch?n


From carsonhh at gmail.com  Thu Mar 28 12:05:25 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 28 Mar 2019 12:05:25 -0600
Subject: [maker-devel] problem with MWAS
In-Reply-To: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>
References: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>
Message-ID: <4947A0AB-456C-4583-9618-EEB1F376F598@gmail.com>

?Insecure? means your browser is using http rather than https. You can add the s manually to an address if you want. It?s a campaign from browser makers to try and get people to always use https instead. Chrome started doing this in 2018, so now anytime you use http in an address or link, it will display ?insecure".

Why your login doesn?t work would be a different issue. The server has been restarted. See if that helps.

?Carson


> On Mar 14, 2019, at 2:51 AM, Isa SCHON <isa.schon at uhasselt.be> wrote:
> 
> Dear colleague,
> 
> I am trying to run MWAS as a first trial using MAKER for annotations. I had first trouble uploading my data and since a few days, I cannot log in anymore at all with my account. I get the error message that the website is insecure.
> Could you please check what is going on?
> 
> Many thanks in advance and kind regards,
> 
> Isa Sch?n
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From sj2641 at nyu.edu  Thu Mar 14 16:25:02 2019
From: sj2641 at nyu.edu (Shaowen Jiang)
Date: Thu, 14 Mar 2019 22:25:02 -0000
Subject: [maker-devel] Installation Failure with pg_config?
Message-ID: <CADA28Zhf5q1ND8U=aT_njcGPGapEmbPTcFowBpM6CwT2jMv32Q@mail.gmail.com>

Dear MAKER2 admins:

Hi, I have read some tutorials for annotating a newly assembly genome and
MAKER2 seems to be a very good and functional pipeline to me. So I am
trying to use it to annotate a new assembly mammalian genome that our lab
just generated.
But I was stuck while I was trying to install MAKER2 to our slurm HPC
server.
I think the pipeline is trying to install several perl packages locally,
but one of them called DBD::Pg requires the path of pg_config?
screenshot as below

But I think our server doesn't have this path and I don't have root to
install some other stuff, like libpq-dev or PostgreSQL.
Is that any other methods that can circle around that?

Any help or advice would be appreciated!

Thanks,
Shaowen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190314/0b5599e2/attachment-0002.html>

From paul at tupac.bio  Mon Mar 11 05:33:06 2019
From: paul at tupac.bio (Paul Sheridan)
Date: Mon, 11 Mar 2019 20:33:06 +0900
Subject: [maker-devel] Why doesn't MAKER use all the cores when I run it on
 an instance?
Message-ID: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>

Dear MAKER Team,

I am running MAKER 2.31.10 on a 32 core instance with 192BG of RAM. I set
cpus=32 in the maker_opts.ctl configuration file. However, the instance is
only using at most 3 cores. I would like to understand if there is a way to
get MAKER to use more cores.

Thanks in Advance,

Paul Sheridan

-- 
CSO at Tupac Bio
Email: paul at tupac.bio
Homepage: www.paulsheridan.net
Mobile: +81 80 7889 0859
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/9788e0e8/attachment-0003.html>

From carsonhh at gmail.com  Mon Mar 11 10:55:09 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Mon, 11 Mar 2019 10:55:09 -0600
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
Message-ID: <34141521-B351-448F-BE1C-CA97E025A861@gmail.com>

Sorry for the slow reply. This got lost somehow. The file from the mailing list is 5 years old. Don?t use it. You are actually downgrading, not upgrading. You will break things.

Run maker outside of MPI (if you are using MPI). There might be additional error message, that are further back that are getting covered by extra MPI output.  I don?t actually have a license for fgenesh to test with any more, so debugging it for you may be difficult.  My guess is that a more recent version has slightly different output format which is breaking the parser. Also you need to restore the previous fgenesh.pm file before you can run any tests (may need to reinstall MAKER).  

Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.

Basically after a couple of tests/error messages I may be able to send you something that would at least allow you to run fgenesh outside of MAKER to see what?s going on.

?Carson



> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
> 
> Hi, Carson
> 
> I have been getting this error with fgenesh tool within MAKER. It runs ok with most of the assembly contigs but seems to fail on one contig or part of the contig with the below error
> 
> Widget::fgenesh:
> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh > /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
> #-------------------------------#
>  ...processing 9 of 24
>  ...processing 8 of 28
>  ...processing 10 of 24
>  ...processing 9 of 28
>  ...processing 11 of 24
>  ...processing 10 of 28
>  ...processing 12 of 24
>  ...processing 11 of 28
> deleted:0 genes
> ERROR: FgenesH failed
> --> rank=14, hostname=bnbcompute50
> ERROR: Failed while annotating transcripts
> ERROR: Chunk failed at level:1, tier_type:4
> FAILED CONTIG:Super-Scaffold_14.2_contig2
> 
> I updated the perl module fgenesh.pm <http://fgenesh.pm/> as suggested in the previous threads. Attached are the  maker_opts.ctl and STDERR log file.
> 
> Thanks
> 
> Kapeel
> 
>  
> 
> -- 
> 
> Kapeel Chougule
> Computational Scientist Developer II
> One Bungtown Road Cold Spring Harbor, NY 11724
> http://www.warelab.org/ <http://www.warelab.org/>
> 
> <maker_opts.ctl><stderr.log>_______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/ba8e0489/attachment-0003.html>

From kapeelc at gmail.com  Mon Mar 11 11:22:44 2019
From: kapeelc at gmail.com (Kapeel Chougule)
Date: Mon, 11 Mar 2019 13:22:44 -0400
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
	<34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
Message-ID: <CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>

Thanks Carson,

I will try the settings you suggested.

Also, for those failed contigs I ran fgenesh standalone and passed the
results to pred_gff in maker_opts.ctl. The run was successful. The only
issue with this was the name attribute in column predicted by MAKER had a
very different ID compared to other genes predicted.

E.g
Chr10   maker   mRNA    33029   45468   .       -       .
ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;
Name=Chr10mRNA_1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418

Correct Name attribute :
Chr10   maker   mRNA    33029   45468   .       -       .
ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;
Name=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1
;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418

This was also reflected in the fasta headers.

Best

Kapeel



On Mon, Mar 11, 2019 at 12:55 PM Carson Holt <carsonhh at gmail.com> wrote:

> Sorry for the slow reply. This got lost somehow. The file from the mailing
> list is 5 years old. Don?t use it. You are actually downgrading, not
> upgrading. You will break things.
>
> Run maker outside of MPI (if you are using MPI). There might be additional
> error message, that are further back that are getting covered by extra MPI
> output.  I don?t actually have a license for fgenesh to test with any more,
> so debugging it for you may be difficult.  My guess is that a more recent
> version has slightly different output format which is breaking the parser.
> Also you need to restore the previous fgenesh.pm file before you can run
> any tests (may need to reinstall MAKER).
>
> Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.
>
> Basically after a couple of tests/error messages I may be able to send you
> something that would at least allow you to run fgenesh outside of MAKER to
> see what?s going on.
>
> ?Carson
>
>
>
> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
>
> Hi, Carson
>
> I have been getting this error with fgenesh tool within MAKER. It runs ok
> with most of the assembly contigs but seems to fail on one contig or part
> of the contig with the below error
>
> Widget::fgenesh:
> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap
> /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh
> /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new
> /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta
> -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh
> >
> /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
> #-------------------------------#
>  ...processing 9 of 24
>  ...processing 8 of 28
>  ...processing 10 of 24
>  ...processing 9 of 28
>  ...processing 11 of 24
>  ...processing 10 of 28
>  ...processing 12 of 24
>  ...processing 11 of 28
> deleted:0 genes
> ERROR: FgenesH failed
> --> rank=14, hostname=bnbcompute50
> ERROR: Failed while annotating transcripts
> ERROR: Chunk failed at level:1, tier_type:4
> FAILED CONTIG:Super-Scaffold_14.2_contig2
>
> I updated the perl module fgenesh.pm as suggested in the previous
> threads. Attached are the  maker_opts.ctl and STDERR log file.
>
> Thanks
>
> Kapeel
>
>
>
> --
>
>
>
> *Kapeel ChouguleComputational Scientist Developer II*
>
>
> *One Bungtown Road Cold Spring Harbor, NY 11724http://www.warelab.org/
> <http://www.warelab.org/>*
> <maker_opts.ctl><stderr.log>
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>
>
>

-- 



*Kapeel ChouguleComputational Scientist Developer II*


*One Bungtown Road Cold Spring Harbor, NY 11724http://www.warelab.org/
<http://www.warelab.org/>*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/22459c31/attachment-0003.html>

From carsonhh at gmail.com  Mon Mar 11 11:25:19 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Mon, 11 Mar 2019 11:25:19 -0600
Subject: [maker-devel] MAKER v3 Fgenesh ERROR
In-Reply-To: <CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>
References: <CA+DOtefuUEc5_fFh7j2ykb4yBKmtEp1vgt0Pea-RF+7GCqr9ig@mail.gmail.com>
	<34141521-B351-448F-BE1C-CA97E025A861@gmail.com>
	<CA+DOtedJyndo4OEadgRv9Em_ncH02aMZLAZtKRQww-+0rj=iWA@mail.gmail.com>
Message-ID: <6729A4F9-E594-4F77-BA11-A36F104E04AA@gmail.com>

If you you pass in a gff3, MAKER uses the name in the gff3.

?Carson

Sent from my iPhone

> On Mar 11, 2019, at 11:22 AM, Kapeel Chougule <kapeelc at gmail.com> wrote:
> 
> Thanks Carson,
> 
> I will try the settings you suggested.
> 
> Also, for those failed contigs I ran fgenesh standalone and passed the results to pred_gff in maker_opts.ctl. The run was successful. The only issue with this was the name attribute in column predicted by MAKER had a very different ID compared to other genes predicted.
> 
> E.g
> Chr10   maker   mRNA    33029   45468   .       -       .       ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;Name=Chr10mRNA_1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418
> 
> Correct Name attribute :
> Chr10   maker   mRNA    33029   45468   .       -       .       ID=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;Parent=maker-Chr10-pred_gff_Fgenesh-gene-0.5;Name=maker-Chr10-pred_gff_Fgenesh-gene-0.5-mRNA-1;_AED=0.21;_eAED=0.21;_QI=76|0.88|1|1|1|1|10|315|418
> 
> This was also reflected in the fasta headers. 
> 
> Best
> 
> Kapeel
> 
> 
> 
>> On Mon, Mar 11, 2019 at 12:55 PM Carson Holt <carsonhh at gmail.com> wrote:
>> Sorry for the slow reply. This got lost somehow. The file from the mailing list is 5 years old. Don?t use it. You are actually downgrading, not upgrading. You will break things.
>> 
>> Run maker outside of MPI (if you are using MPI). There might be additional error message, that are further back that are getting covered by extra MPI output.  I don?t actually have a license for fgenesh to test with any more, so debugging it for you may be difficult.  My guess is that a more recent version has slightly different output format which is breaking the parser. Also you need to restore the previous fgenesh.pm file before you can run any tests (may need to reinstall MAKER).  
>> 
>> Also try MAKER3 vs MAKER2. They both run fgenesh slightly differently.
>> 
>> Basically after a couple of tests/error messages I may be able to send you something that would at least allow you to run fgenesh outside of MAKER to see what?s going on.
>> 
>> ?Carson
>> 
>> 
>> 
>>> On Feb 7, 2019, at 12:43 PM, Kapeel Chougule <kapeelc at gmail.com> wrote:
>>> 
>>> Hi, Carson
>>> 
>>> I have been getting this error with fgenesh tool within MAKER. It runs ok with most of the assembly contigs but seems to fail on one contig or part of the contig with the below error
>>> 
>>> Widget::fgenesh:
>>> /mnt/grid/ware/hpc/home/data/mcampbel/applications/maker/bin/../lib/Widget/fgenesh/fgenesh_wrap /mnt/grid/ware/hpc_norepl/data/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/fgenesh /sonas-hs/ware/hpc_norepl/data/programs/fgenesh_v8/fgenesh_suite_v8.0.0a/Zeamays.mpar.dat.new /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh.fasta -exon_table:/tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.xdef.fgenesh > /tmp/uge/53139300.1.primary.q/maker_j3ttxX/6/6_1.600610-613023.Zeamays.mpar.dat.new.auto_annotator.fgenesh
>>> #-------------------------------#
>>>  ...processing 9 of 24
>>>  ...processing 8 of 28
>>>  ...processing 10 of 24
>>>  ...processing 9 of 28
>>>  ...processing 11 of 24
>>>  ...processing 10 of 28
>>>  ...processing 12 of 24
>>>  ...processing 11 of 28
>>> deleted:0 genes
>>> ERROR: FgenesH failed
>>> --> rank=14, hostname=bnbcompute50
>>> ERROR: Failed while annotating transcripts
>>> ERROR: Chunk failed at level:1, tier_type:4
>>> FAILED CONTIG:Super-Scaffold_14.2_contig2
>>> 
>>> I updated the perl module fgenesh.pm as suggested in the previous threads. Attached are the  maker_opts.ctl and STDERR log file.
>>> 
>>> Thanks
>>> 
>>> Kapeel
>>> 
>>>  
>>> 
>>> -- 
>>> 
>>> Kapeel Chougule
>>> Computational Scientist Developer II
>>> One Bungtown Road Cold Spring Harbor, NY 11724
>>> http://www.warelab.org/
>>> 
>>> <maker_opts.ctl><stderr.log>_______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>> 
> 
> 
> -- 
> 
> Kapeel Chougule
> Computational Scientist Developer II
> One Bungtown Road Cold Spring Harbor, NY 11724
> http://www.warelab.org/
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190311/bcbebead/attachment-0003.html>

From khs3z at virginia.edu  Tue Mar 12 07:39:39 2019
From: khs3z at virginia.edu (Siller, Karsten H (khs3z))
Date: Tue, 12 Mar 2019 13:39:39 +0000
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
Message-ID: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:	VERIFIED
External Programs:	VERIFIED
External C Libraries:	VERIFIED
MPI SUPPORT:		ENABLED
MWAS Web Interface:	DISABLED
MAKER PACKAGE:		CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten

?

Karsten Siller
University of Virginia


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/ec672589/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6097 bytes
Desc: not available
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/ec672589/attachment-0003.p7s>

From carsonhh at gmail.com  Tue Mar 12 09:07:22 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 12 Mar 2019 08:07:22 -0700
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
In-Reply-To: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>
References: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>
Message-ID: <C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>

Launch with OpenMPI?s mpiexec command. The slurm stun command you are using won?t launch it as expected.

?Carson

> On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <khs3z at virginia.edu> wrote:
> 
> Hi,
> 
> We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status
> 
> ==============================================================================
> STATUS MAKER v2.31.10
> ==============================================================================
> PERL Dependencies:	VERIFIED
> External Programs:	VERIFIED
> External C Libraries:	VERIFIED
> MPI SUPPORT:		ENABLED
> MWAS Web Interface:	DISABLED
> MAKER PACKAGE:		CONFIGURATION OK
> 
> This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):
> 
> srun maker > maker_small_PARALLEL
> 
> This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.
> 
> Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation. 
> 
> Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?
> 
> Thanks,
> Karsten
> 
> ?
> 
> Karsten Siller
> University of Virginia
> 
> 
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/add51654/attachment-0003.html>

From carsonhh at gmail.com  Tue Mar 12 17:57:27 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Tue, 12 Mar 2019 16:57:27 -0700
Subject: [maker-devel] Why doesn't MAKER use all the cores when I run it
 on an instance?
In-Reply-To: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>
References: <CALbXf6u8urgyALJHCbTXiwmRxHbdchQFZWp+0qCwL0q2pqfQNQ@mail.gmail.com>
Message-ID: <91F12D37-2A8D-4867-84EA-0245846A76B8@gmail.com>

The -cpu option is only for BLAST. However, if you run MAKER under MPI (even on a single machine), you can parallelize all steps. Set cpu=1 when running under MPI, otherwise BLAST will multiply the -cpu value with the MPI value, and oversubscribe the CPU.

?Carson


> On Mar 11, 2019, at 4:33 AM, Paul Sheridan <paul at tupac.bio> wrote:
> 
> Dear MAKER Team,
> 
> I am running MAKER 2.31.10 on a 32 core instance with 192BG of RAM. I set cpus=32 in the maker_opts.ctl configuration file. However, the instance is only using at most 3 cores. I would like to understand if there is a way to get MAKER to use more cores.
> 
> Thanks in Advance,
> 
> Paul Sheridan
> 
> -- 
> CSO at Tupac Bio
> Email: paul at tupac.bio
> Homepage: www.paulsheridan.net <http://www.paulsheridan.net/>
> Mobile: +81 80 7889 0859
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190312/3afbcd7e/attachment-0003.html>

From khs3z at virginia.edu  Wed Mar 13 16:26:40 2019
From: khs3z at virginia.edu (Siller, Karsten H (khs3z))
Date: Wed, 13 Mar 2019 22:26:40 +0000
Subject: [maker-devel] Running MAKER with MPI support and SLURM scheduler
In-Reply-To: <C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>
References: <DE21DC01-A557-4377-B283-1A9B7C4AF325@virginia.edu>,
	<C7AA6B46-8CAF-4868-8A39-821790A01FFF@gmail.com>
Message-ID: <DM6PR13MB324172C635C6773225EBC2929C4A0@DM6PR13MB3241.namprd13.prod.outlook.com>

Ok, we?ll try it with mpiexec. Thanks for your help.

Karsten

________________________________
From: Carson Holt <carsonhh at gmail.com>
Sent: Tuesday, March 12, 2019 11:07 AM
To: Siller, Karsten H (khs3z)
Cc: maker-devel at yandell-lab.org
Subject: Re: [maker-devel] Running MAKER with MPI support and SLURM scheduler

Launch with OpenMPI?s mpiexec command. The slurm stun command you are using won?t launch it as expected.

?Carson

On Mar 12, 2019, at 6:39 AM, Siller, Karsten H (khs3z) <khs3z at virginia.edu<mailto:khs3z at virginia.edu>> wrote:

Hi,

We have configured and installed MAKER 2.31 with OpenMPI 2.1.5 on our HPC cluster.  During build/installation, the MPI libs and mpicc wrapper are found.  This is the output of ./Build status

==============================================================================
STATUS MAKER v2.31.10
==============================================================================
PERL Dependencies:VERIFIED
External Programs:VERIFIED
External C Libraries:VERIFIED
MPI SUPPORT:ENABLED
MWAS Web Interface:DISABLED
MAKER PACKAGE:CONFIGURATION OK

This is how we run MAKER (the number of nodes and tasks are set via #SBATCH directives):

srun maker > maker_small_PARALLEL

This launches the correct number of MAKER processes across nodes but we?re seeing messages in the output "WARNING: Multiple MAKER processes have been started in the same directory.?  The processes appear to run ok, but we want to make sure there?s nothing fundamentally wrong with how we run MAKER.

Browsing the mailing list, I came across similar posts and it was suggested that there?s an issue either with the MAKER MPI configuration or with the scheduler.  We?re positive that the scheduler is configured properly since many other MPI applications run without problems on the cluster with the same scheduler and MPI implementation.

Is it possible that launching MAKER processes via srun is not supported?  Or is the warning harmless and can be ignored?

Thanks,
Karsten

?

Karsten Siller
University of Virginia


_______________________________________________
maker-devel mailing list
maker-devel at box290.bluehost.com<mailto:maker-devel at box290.bluehost.com>
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190313/cfe389de/attachment-0003.html>

From isa.schon at uhasselt.be  Thu Mar 14 02:51:27 2019
From: isa.schon at uhasselt.be (Isa SCHON)
Date: Thu, 14 Mar 2019 09:51:27 +0100
Subject: [maker-devel] problem with MWAS
Message-ID: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>

Dear colleague,

I am trying to run MWAS as a first trial using MAKER for annotations. I had first trouble uploading my data and since a few days, I cannot log in anymore at all with my account. I get the error message that the website is insecure.
Could you please check what is going on?

Many thanks in advance and kind regards,

Isa Sch?n


From carsonhh at gmail.com  Thu Mar 28 12:05:25 2019
From: carsonhh at gmail.com (Carson Holt)
Date: Thu, 28 Mar 2019 12:05:25 -0600
Subject: [maker-devel] problem with MWAS
In-Reply-To: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>
References: <1E02FAB5-35BE-437A-9159-C4D4D55EAE27@uhasselt.be>
Message-ID: <4947A0AB-456C-4583-9618-EEB1F376F598@gmail.com>

?Insecure? means your browser is using http rather than https. You can add the s manually to an address if you want. It?s a campaign from browser makers to try and get people to always use https instead. Chrome started doing this in 2018, so now anytime you use http in an address or link, it will display ?insecure".

Why your login doesn?t work would be a different issue. The server has been restarted. See if that helps.

?Carson


> On Mar 14, 2019, at 2:51 AM, Isa SCHON <isa.schon at uhasselt.be> wrote:
> 
> Dear colleague,
> 
> I am trying to run MWAS as a first trial using MAKER for annotations. I had first trouble uploading my data and since a few days, I cannot log in anymore at all with my account. I get the error message that the website is insecure.
> Could you please check what is going on?
> 
> Many thanks in advance and kind regards,
> 
> Isa Sch?n
> _______________________________________________
> maker-devel mailing list
> maker-devel at box290.bluehost.com
> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org




From sj2641 at nyu.edu  Thu Mar 14 16:25:02 2019
From: sj2641 at nyu.edu (Shaowen Jiang)
Date: Thu, 14 Mar 2019 22:25:02 -0000
Subject: [maker-devel] Installation Failure with pg_config?
Message-ID: <CADA28Zhf5q1ND8U=aT_njcGPGapEmbPTcFowBpM6CwT2jMv32Q@mail.gmail.com>

Dear MAKER2 admins:

Hi, I have read some tutorials for annotating a newly assembly genome and
MAKER2 seems to be a very good and functional pipeline to me. So I am
trying to use it to annotate a new assembly mammalian genome that our lab
just generated.
But I was stuck while I was trying to install MAKER2 to our slurm HPC
server.
I think the pipeline is trying to install several perl packages locally,
but one of them called DBD::Pg requires the path of pg_config?
screenshot as below

But I think our server doesn't have this path and I don't have root to
install some other stuff, like libpq-dev or PostgreSQL.
Is that any other methods that can circle around that?

Any help or advice would be appreciated!

Thanks,
Shaowen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20190314/0b5599e2/attachment-0003.html>