From carsonhh at gmail.com Tue Aug 2 09:08:41 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:08:41 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> If you move it, then the executables won?t be able to locate dependencies in the ?/maker/data, ?/maker/lib, and ?/maker/perl directories. You should really either add the location of ?/maker/bin to you PATH environmental variable or at most soft link the executables somewhere else using the ?ln -s? command. ?Carson > On Aug 2, 2016, at 5:57 AM, Sebastien Moretti wrote: > > OK, just need to move the maker/bin folder. > > Thanks > >> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >> >> ?Carson >> >> >> >>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I try to install maker in a custom directory following regular Perl >>> commands with Build.PL. >>> >>> Unfortunately if I use >>> perl Build.PL --installdirs= >>> then >>> ./Build install --destdir= >>> build install fails with the error: >>> Can't use an undefined value as a HASH reference at >>> .../Module/Build/Base.pm line 3038. >>> >>> It does not happen if I don't use --installdirs >>> >>> >>> Is there an "official" way to install maker in a custom directory? >>> >>> >>> Also I found typos in the description of maker at >>> http://www.yandell-lab.org/software/maker.html >>> ouputs should be outputs >>> seusequent should be ??? >>> >>> >>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>> >>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 06:57:32 2016 From: sebastien.moretti at unil.ch (Sebastien Moretti) Date: Tue, 2 Aug 2016 13:57:32 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: OK, just need to move the maker/bin folder. Thanks > MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. > > ?Carson > > > >> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >> >> Hi >> >> I try to install maker in a custom directory following regular Perl >> commands with Build.PL. >> >> Unfortunately if I use >> perl Build.PL --installdirs= >> then >> ./Build install --destdir= >> build install fails with the error: >> Can't use an undefined value as a HASH reference at >> .../Module/Build/Base.pm line 3038. >> >> It does not happen if I don't use --installdirs >> >> >> Is there an "official" way to install maker in a custom directory? >> >> >> Also I found typos in the description of maker at >> http://www.yandell-lab.org/software/maker.html >> ouputs should be outputs >> seusequent should be ??? >> >> >> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >> >> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 07:00:53 2016 From: sebastien.moretti at unil.ch (Sebastien Moretti) Date: Tue, 2 Aug 2016 14:00:53 +0200 Subject: [maker-devel] MPI test Message-ID: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Hi I compiled maker with MPI support and would like to test it. So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm is missing. What does mpi_evaluator do? How to test properly MPI in maker? maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 openmpi 1.8.1 Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 09:14:23 2016 From: sebastien.moretti at unil.ch (sebastien.moretti at unil.ch) Date: Tue, 2 Aug 2016 16:14:23 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> Message-ID: <196799cd-d370-a343-f87c-1292e58593a4@unil.ch> I moved maker/bin/, maker/perl/, maker/data/, maker/lib/ + LICENSE README RELEASE and it looks to be fine. Thanks > If you move it, then the executables won?t be able to locate dependencies in the ?/maker/data, ?/maker/lib, and ?/maker/perl directories. You should really either add the location of ?/maker/bin to you PATH environmental variable or at most soft link the executables somewhere else using the ?ln -s? command. > > ?Carson > > >> On Aug 2, 2016, at 5:57 AM, Sebastien Moretti wrote: >> >> OK, just need to move the maker/bin folder. >> >> Thanks >> >>> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >>> >>> ?Carson >>> >>> >>> >>>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>>> >>>> Hi >>>> >>>> I try to install maker in a custom directory following regular Perl >>>> commands with Build.PL. >>>> >>>> Unfortunately if I use >>>> perl Build.PL --installdirs= >>>> then >>>> ./Build install --destdir= >>>> build install fails with the error: >>>> Can't use an undefined value as a HASH reference at >>>> .../Module/Build/Base.pm line 3038. >>>> >>>> It does not happen if I don't use --installdirs >>>> >>>> >>>> Is there an "official" way to install maker in a custom directory? >>>> >>>> >>>> Also I found typos in the description of maker at >>>> http://www.yandell-lab.org/software/maker.html >>>> ouputs should be outputs >>>> seusequent should be ??? >>>> >>>> >>>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>>> >>>> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From carsonhh at gmail.com Tue Aug 2 09:19:35 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:19:35 -0600 Subject: [maker-devel] MPI test In-Reply-To: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: <97587487-EAB0-461E-8DB1-2E23FA3E3C4C@gmail.com> mpi_evaluator is a maker development related accessory script. You should ignore it. ?Carson > On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: > > Hi > > I compiled maker with MPI support and would like to test it. > So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm > is missing. > > What does mpi_evaluator do? > How to test properly MPI in maker? > > maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 > openmpi 1.8.1 > > Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Aug 2 09:59:40 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:59:40 -0600 Subject: [maker-devel] MPI test In-Reply-To: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. ?Carson > On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: > > Hi > > I compiled maker with MPI support and would like to test it. > So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm > is missing. > > What does mpi_evaluator do? > How to test properly MPI in maker? > > maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 > openmpi 1.8.1 > > Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From mmokrejs at gmail.com Tue Aug 2 10:29:49 2016 From: mmokrejs at gmail.com (=?UTF-8?Q?Martin_MOKREJ=c5=a0?=) Date: Tue, 2 Aug 2016 17:29:49 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: Hi Sebastien, I had similar concerns in the past as well. I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". Martin Sebastien Moretti wrote: > OK, just need to move the maker/bin folder. > > Thanks > >> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >> >> ?Carson >> >> >> >>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I try to install maker in a custom directory following regular Perl >>> commands with Build.PL. >>> >>> Unfortunately if I use >>> perl Build.PL --installdirs= >>> then >>> ./Build install --destdir= >>> build install fails with the error: >>> Can't use an undefined value as a HASH reference at >>> .../Module/Build/Base.pm line 3038. >>> >>> It does not happen if I don't use --installdirs >>> >>> >>> Is there an "official" way to install maker in a custom directory? From carsonhh at gmail.com Tue Aug 2 10:51:25 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 09:51:25 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. ?Carson > On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: > > Hi Sebastien, > I had similar concerns in the past as well. > > I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. > > I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse > > From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. > The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. > > Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". > > Martin > > > Sebastien Moretti wrote: >> OK, just need to move the maker/bin folder. >> >> Thanks >> >>> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >>> >>> ?Carson >>> >>> >>> >>>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>>> >>>> Hi >>>> >>>> I try to install maker in a custom directory following regular Perl >>>> commands with Build.PL. >>>> >>>> Unfortunately if I use >>>> perl Build.PL --installdirs= >>>> then >>>> ./Build install --destdir= >>>> build install fails with the error: >>>> Can't use an undefined value as a HASH reference at >>>> .../Module/Build/Base.pm line 3038. >>>> >>>> It does not happen if I don't use --installdirs >>>> >>>> >>>> Is there an "official" way to install maker in a custom directory? From mmokrejs at gmail.com Tue Aug 2 11:04:05 2016 From: mmokrejs at gmail.com (=?UTF-8?Q?Martin_MOKREJ=c5=a0?=) Date: Tue, 2 Aug 2016 18:04:05 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> Hi Carson, Carson Holt wrote: > You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. Maybe I was fooled by the install step. > > ?Carson > > >> On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: >> >> Hi Sebastien, >> I had similar concerns in the past as well. >> >> I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. >> >> I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse >> >> From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. >> The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. >> >> Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". >>> Emerging (1 of 1) sci-biology/maker-2.31.8::science * maker-2.31.8.tgz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>> Unpacking source... >>> Unpacking maker-2.31.8.tgz to /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>> Source unpacked in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>> Preparing source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>> Source prepared. >>> Configuring source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>> Source configured. >>> Compiling source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... MAKER supports distributed parallelization via MPI. Would you like to configure MAKER for MPI (This requires that you have an MPI client installed)? [N ]N Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'MAKER' version 'v2.31.8' The file 'Build' has been created for you to finish installing MAKER. ============================================================================== STATUS MAKER v2.31.8 ============================================================================== PERL Dependencies: VERIFIED External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: DISABLED MWAS Web Interface: DISABLED MAKER PACKAGE: CONFIGURATION OK Important Commands: ./Build installdeps #installs missing PERL dependencies ./Build installexes #installs all missing external programs ./Build install #installs MAKER ./Build status #Shows this status menu Other Commands: ./Build repeatmasker #installs RepeatMasker (asks for RepBase) ./Build blast #installs BLAST (NCBI BLAST+) ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) ./Build snap #installs SNAP ./Build augustus #installs Augustus ./Build apollo #installs Apollo ./Build gbrowse #installs GBrowse (must be root) ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) ./Build webapollo #installs WebApollo (use maker2wap to create DBs) ./Build mpich2 #installs MPICH2 (but manual install recommended) Building MAKER Installing MAKER... Building MAKER Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/MAKER/ConfigData.pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/Parallel/Application/MPI.pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/man/MAKER::ConfigData.3pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_fasta_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cufflinks2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2eval_gtf Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/genemark_gtf2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2wap Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_map_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/ipr_update_gff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2jbrowse Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2chado Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/compare Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_tool Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan_wrap Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/evaluator Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2zff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cegma2zff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_merge Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_data_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_fasta Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/chado2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/gff3_merge Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/tophat2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map2assembly Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_gff_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_gff >>> Source compiled. >>> Test phase [not enabled]: sci-biology/maker-2.31.8 >>> Install maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ category sci-biology >>> Completed installing maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ * Final size of build directory: 40312 KiB * Final size of installed tree: 12656 KiB ecompressdir: bzip2 -9 /usr/share/man ecompressdir: bzip2 -9 /usr/share/doc >>> Installing (1 of 1) sci-biology/maker-2.31.8::science >>> Auto-cleaning packages... That gives me these files installed on my system: # equery files maker * Searching for maker ... * Contents of sci-biology/maker-2.31.8: /usr /usr/bin /usr/bin/cegma2zff /usr/bin/chado2gff3 /usr/bin/compare_gff3_to_chado /usr/bin/cufflinks2gff3 /usr/bin/evaluator /usr/bin/fasta_merge /usr/bin/genemark_gtf2gff3 /usr/bin/gff3_merge /usr/bin/ipr_update_gff /usr/bin/iprscan2gff3 /usr/bin/iprscan_wrap /usr/bin/maker /usr/bin/maker2chado /usr/bin/maker2eval_gtf /usr/bin/maker2jbrowse /usr/bin/maker2wap /usr/bin/maker2zff /usr/bin/maker_functional /usr/bin/maker_functional_fasta /usr/bin/maker_functional_gff /usr/bin/maker_map_ids /usr/bin/map2assembly /usr/bin/map_data_ids /usr/bin/map_fasta_ids /usr/bin/map_gff_ids /usr/bin/tophat2gff3 /usr/lib64 /usr/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/vendor_perl/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/MAKER /usr/lib64/perl5/vendor_perl/5.20.2/MAKER/ConfigData.pm /usr/lib64/perl5/vendor_perl/5.20.2/Parallel /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application/MPI.pm /usr/share /usr/share/doc /usr/share/doc/maker-2.31.8 /usr/share/doc/maker-2.31.8/INSTALL.bz2 /usr/share/doc/maker-2.31.8/README.bz2 /usr/share/maker /usr/share/maker/GMOD /usr/share/maker/GMOD/Apollo /usr/share/maker/GMOD/Apollo/gff3.tiers /usr/share/maker/GMOD/JBrowse /usr/share/maker/GMOD/JBrowse/maker.css /usr/share/maker/data /usr/share/maker/data/dpp_contig.fasta /usr/share/maker/data/dpp_est.fasta /usr/share/maker/data/dpp_protein.fasta /usr/share/maker/data/hsap_contig.fasta /usr/share/maker/data/hsap_est.fasta /usr/share/maker/data/hsap_protein.fasta /usr/share/maker/data/te_proteins.fasta /usr/share/man /usr/share/man/man3 /usr/share/man/man3/MAKER::ConfigData.3pm.bz2 $ /usr/bin/maker Argument "2.49_01" isn't numeric in numeric ge (>=) at /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi/forks.pm line 1570. Can't locate GI.pm in @INC (you may need to install the GI module) (@INC contains: /home/mmokrejs/proj/sci/sci-biology/maker/_Inline/lib /usr/bin/../perl/lib /usr/bin/../lib /usr/bin/../src/inc/perl/lib /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/5.20.2 .) at /usr/bin/maker line 63. BEGIN failed--compilation aborted at /usr/bin/maker line 63. $ Of course, because as I said, I need to install properly all the dependencies from maker/lib/ subdirectory. BTW, did perl Build.PL introduce the _Inline/lib/ directory? It somehow escaped from the sanbox I think. :( But thank you for you kind answer anyway, Martin From carsonhh at gmail.com Tue Aug 2 11:09:12 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 10:09:12 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> Message-ID: <57C25EAE-FF0C-4CE6-9000-5B67A7260C28@gmail.com> The install step is installing the C/Perl shared object bindings for MPI to their final location. I could call that internally to ./Build, but since it can fail, and I also since it is configured to fail on missing Perl dependencies, it is better to run it as the final step. ?Carson > On Aug 2, 2016, at 10:04 AM, Martin MOKREJ? wrote: > > Hi Carson, > > Carson Holt wrote: >> You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. > > Maybe I was fooled by the install step. > >> >> ?Carson >> >> >>> On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: >>> >>> Hi Sebastien, >>> I had similar concerns in the past as well. >>> >>> I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. >>> >>> I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse >>> >>> From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. >>> The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. >>> >>> Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". > > >>>> Emerging (1 of 1) sci-biology/maker-2.31.8::science > * maker-2.31.8.tgz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>>> Unpacking source... >>>> Unpacking maker-2.31.8.tgz to /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>>> Source unpacked in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>>> Preparing source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>>> Source prepared. >>>> Configuring source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>>> Source configured. >>>> Compiling source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... > > MAKER supports distributed parallelization via MPI. > Would you like to configure MAKER for MPI (This > requires that you have an MPI client installed)? [N ]N > Created MYMETA.yml and MYMETA.json > Creating new 'Build' script for 'MAKER' version 'v2.31.8' > > > The file 'Build' has been created for you to finish installing MAKER. > > > ============================================================================== > STATUS MAKER v2.31.8 > ============================================================================== > PERL Dependencies: VERIFIED > External Programs: VERIFIED > External C Libraries: VERIFIED > MPI SUPPORT: DISABLED > MWAS Web Interface: DISABLED > MAKER PACKAGE: CONFIGURATION OK > > > Important Commands: > ./Build installdeps #installs missing PERL dependencies > ./Build installexes #installs all missing external programs > ./Build install #installs MAKER > ./Build status #Shows this status menu > > Other Commands: > ./Build repeatmasker #installs RepeatMasker (asks for RepBase) > ./Build blast #installs BLAST (NCBI BLAST+) > ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) > ./Build snap #installs SNAP > ./Build augustus #installs Augustus > ./Build apollo #installs Apollo > ./Build gbrowse #installs GBrowse (must be root) > ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) > ./Build webapollo #installs WebApollo (use maker2wap to create DBs) > ./Build mpich2 #installs MPICH2 (but manual install recommended) > Building MAKER > Installing MAKER... > Building MAKER > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/MAKER/ConfigData.pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/Parallel/Application/MPI.pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/man/MAKER::ConfigData.3pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_fasta_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cufflinks2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2eval_gtf > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/genemark_gtf2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2wap > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_map_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/ipr_update_gff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2jbrowse > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2chado > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/compare > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_tool > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan_wrap > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/evaluator > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2zff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cegma2zff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_merge > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_data_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_fasta > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/chado2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/gff3_merge > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/tophat2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map2assembly > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_gff_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_gff >>>> Source compiled. >>>> Test phase [not enabled]: sci-biology/maker-2.31.8 > >>>> Install maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ category sci-biology >>>> Completed installing maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ > > * Final size of build directory: 40312 KiB > * Final size of installed tree: 12656 KiB > > ecompressdir: bzip2 -9 /usr/share/man > ecompressdir: bzip2 -9 /usr/share/doc > >>>> Installing (1 of 1) sci-biology/maker-2.31.8::science >>>> Auto-cleaning packages... > > > > > That gives me these files installed on my system: > > # equery files maker > * Searching for maker ... > * Contents of sci-biology/maker-2.31.8: > /usr > /usr/bin > /usr/bin/cegma2zff > /usr/bin/chado2gff3 > /usr/bin/compare_gff3_to_chado > /usr/bin/cufflinks2gff3 > /usr/bin/evaluator > /usr/bin/fasta_merge > /usr/bin/genemark_gtf2gff3 > /usr/bin/gff3_merge > /usr/bin/ipr_update_gff > /usr/bin/iprscan2gff3 > /usr/bin/iprscan_wrap > /usr/bin/maker > /usr/bin/maker2chado > /usr/bin/maker2eval_gtf > /usr/bin/maker2jbrowse > /usr/bin/maker2wap > /usr/bin/maker2zff > /usr/bin/maker_functional > /usr/bin/maker_functional_fasta > /usr/bin/maker_functional_gff > /usr/bin/maker_map_ids > /usr/bin/map2assembly > /usr/bin/map_data_ids > /usr/bin/map_fasta_ids > /usr/bin/map_gff_ids > /usr/bin/tophat2gff3 > /usr/lib64 > /usr/lib64/perl5 > /usr/lib64/perl5/vendor_perl > /usr/lib64/perl5/vendor_perl/5.20.2 > /usr/lib64/perl5/vendor_perl/5.20.2/MAKER > /usr/lib64/perl5/vendor_perl/5.20.2/MAKER/ConfigData.pm > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application/MPI.pm > /usr/share > /usr/share/doc > /usr/share/doc/maker-2.31.8 > /usr/share/doc/maker-2.31.8/INSTALL.bz2 > /usr/share/doc/maker-2.31.8/README.bz2 > /usr/share/maker > /usr/share/maker/GMOD > /usr/share/maker/GMOD/Apollo > /usr/share/maker/GMOD/Apollo/gff3.tiers > /usr/share/maker/GMOD/JBrowse > /usr/share/maker/GMOD/JBrowse/maker.css > /usr/share/maker/data > /usr/share/maker/data/dpp_contig.fasta > /usr/share/maker/data/dpp_est.fasta > /usr/share/maker/data/dpp_protein.fasta > /usr/share/maker/data/hsap_contig.fasta > /usr/share/maker/data/hsap_est.fasta > /usr/share/maker/data/hsap_protein.fasta > /usr/share/maker/data/te_proteins.fasta > /usr/share/man > /usr/share/man/man3 > /usr/share/man/man3/MAKER::ConfigData.3pm.bz2 > > > > $ /usr/bin/maker > Argument "2.49_01" isn't numeric in numeric ge (>=) at /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi/forks.pm line 1570. > Can't locate GI.pm in @INC (you may need to install the GI module) (@INC contains: /home/mmokrejs/proj/sci/sci-biology/maker/_Inline/lib /usr/bin/../perl/lib /usr/bin/../lib /usr/bin/../src/inc/perl/lib /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/5.20.2 .) at /usr/bin/maker line 63. > BEGIN failed--compilation aborted at /usr/bin/maker line 63. > $ > > Of course, because as I said, I need to install properly all the dependencies from maker/lib/ subdirectory. > > > BTW, did perl Build.PL introduce the _Inline/lib/ directory? It somehow escaped from the sanbox I think. :( > > But thank you for you kind answer anyway, > Martin From sebastien.moretti at unil.ch Wed Aug 3 04:25:56 2016 From: sebastien.moretti at unil.ch (sebastien.moretti at unil.ch) Date: Wed, 3 Aug 2016 11:25:56 +0200 Subject: [maker-devel] MPI test In-Reply-To: References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: OK, I removed execution rights for mpi_evaluator. I tried mpiexec -n 10 maker MPI tries to write in the maker/perl/ directory. This is an issue for me because maker is installed in a read-only file system. Is there an option or something to tell MPI to write elsewhere? S?bastien > To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. > > Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker > > Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. > > ?Carson > > > >> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: >> >> Hi >> >> I compiled maker with MPI support and would like to test it. >> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >> is missing. >> >> What does mpi_evaluator do? >> How to test properly MPI in maker? >> >> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >> openmpi 1.8.1 >> >> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From carsonhh at gmail.com Wed Aug 3 11:16:57 2016 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 3 Aug 2016 10:16:57 -0600 Subject: [maker-devel] MPI test In-Reply-To: References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: <72BDD9C3-9E02-4394-AE1B-521411F9B86D@gmail.com> The issue is that you moved the ?/maker/perl directory after initial setup. So when maker runs it can?t find the files, and tries to rebuild the shared objects for C/Perl MPI binding that are missing (compiles on demand). Solutions (one of the following two options) 1. Install maker as indicated in the INTALL file and then append the path for ?/maker/bin/ to the PATH environmental variable in /etc/profile (i.e. export PATH=/usr/local/maker/bin:$PATH). 2. Install maker as indicated in the INTALL file and then soft link the executables in ?/maker/bin/ to the location of your choice (i.e. /usr/bin/ or /usr/local/bin/ - this is what homebrew does for example). Example: MacBook-Pro:~ cholt$ ls -al /usr/local/bin/maker lrwxr-xr-x 1 cholt admin 32 Oct 25 2015 /usr/local/bin/maker -> ../Cellar/maker/2.31.8/bin/maker Notice that the softink in /usr/local/bin/ is pointing to the actual location where homebrew placed the installation. Doing something similar would be better than you trying to break up the installation structure. Basically they move the contents of ?/maker/* to /usr/local/Cellar/maker/2.31.8/, then they performed all installation steps there, and then softlinked the executables elsewhere using ?ln -s' When you install maker, you have to run the setup steps in ?/maker/src/. After install, ?/maker/bin/ will contain executables and ?/maker/perl/ and ?/maker/lib/ will contain libraries and shared objects used by those executables. Remember the executables are not binaries, they are scripts, so they need access to the libraries and shared objects used by the scripts. The executables know to look for libraries/files in the installation folder, but if you move them, the @INC search list used by perl will not find them. You can try and manually declare new locations to search in the @INC list using the PERL5LIB environmental variable (but using PERL5LIB is beyond the scope of the maker-devel list and can be investigated using Perl?s own docs). Also as explained in the homebrew example above, the base directory does not have to be named ?maker?. You can name it whatever you want. It is just the installation structure within the containing directory that has to be consistent. Alternatively if you are doing something more complex like a ?module? setup using lua based modules just use the prepend_path command to add the .../maker/bin foldert to the environment on demand ?> Example: -- -*- lua -*- help( [[ This module loads the MAKER genome annotation pipeline ]]) whatis("Name: MAKER") whatis("Version: 2.31.8") whatis("Category: Gene Annotation") whatis("Keywords: Gene Annotation") whatis("URL: http://www.yandell-lab.org/software/maker.html") whatis("Description: MAKER is a highly parallelized genome annotation/re-annotation pipeline") -- Load required modules load("openmpi/1.8.4.i", "genemark-ES", "augustus", "snap", "exonerate", "ncbi-blast+", "RepeatMasker") -- Prepare needed values local APPS = "/ucgd/apps/" local id = "maker" local version = "2.31.8" local base = pathJoin(APPS, id, version) local bin = pathJoin(base, "bin") -- Set PATH and other environmental variables prepend_path("PATH", bin) prepend_path("LD_PRELOAD", "/ucgd/apps/openmpi/1.8.4i/lib/libmpi.so") setenv("OMPI_MCA_mpi_warn_on_fork", 0) ?Carson > On Aug 3, 2016, at 3:25 AM, sebastien.moretti at unil.ch wrote: > > OK, I removed execution rights for mpi_evaluator. > > I tried mpiexec -n 10 maker > MPI tries to write in the maker/perl/ directory. > This is an issue for me because maker is installed in a read-only file > system. > > Is there an option or something to tell MPI to write elsewhere? > > S?bastien > >> To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. >> >> Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker >> >> Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. >> >> ?Carson >> >> >> >>> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I compiled maker with MPI support and would like to test it. >>> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >>> is missing. >>> >>> What does mpi_evaluator do? >>> How to test properly MPI in maker? >>> >>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>> openmpi 1.8.1 >>> >>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Aug 4 15:21:36 2016 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 4 Aug 2016 14:21:36 -0600 Subject: [maker-devel] MPI test In-Reply-To: <9e01f987-d284-f73b-63c0-e2d84a87cb27@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> <72BDD9C3-9E02-4394-AE1B-521411F9B86D@gmail.com> <9e01f987-d284-f73b-63c0-e2d84a87cb27@unil.ch> Message-ID: Hi Sebastien, As long as you are not following the ?/maker/INSTALL instructions, are instead moving files around manually, and trying to modify the default behavior of ./Build I can not provide support. I?ve tried to best explain how you should be doing things in the previous e-mails as well as the information below. If you are installing for a module based setup, then the proper way to do this would be to move the maker tarball to the desired module location, then follow the INSTALL instructions. Example (bent for a module based setup since you said that is what you are doing): #load required tools module load snap module load openmpi module load blast module load exonerate module load RepeatMasker #set up environment for OpenMPI install (explained in ?/maker/INSTALL instructions) export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so #download and untar maker into desired location mkdir $APPS/maker cd $APPS/maker wget http://maker_url/maker.tgz tar -zxvf maker.tgz mv maker 2.31.8 #go to src directory cd $APPS/maker/2.31.8/src #install according to ?/maker/INSTALL instructions perl Build.PL ./Build installdeps ./Build installexes ./Build ./Build install #test normal installation $APPS/maker/2.31.8/bin/maker --help #test MPI installation (will print only one help message - something is wrong if it prints 5) mpiexec -mca btl ^openib -n 5 $APPS/maker/2.31.8/bin/maker --help #then setup a module file for maker that includes: 1. steps for loading prerequisite tools and modules (such as openmpi) 2. setting OMPI_MCA_mpi_warn_on_fork=0 3. setting LD_PRELOAD=/location/of/openmpi/lib/libmpi.so 4. prepend path for $APPS/maker/2.31.8/bin to PATH environmental variable Thanks, Carson > On Aug 4, 2016, at 8:28 AM, Sebastien Moretti wrote: > > I already use a module to load maker and external tools ;-) > As well as to set OMPI_MCA_mpi_warn_on_fork = 0 > > > In my case perl/config-x86_64-linux-thread-multi-5.018002 was created > properly but not copied. > I use ./Build install --destdir= and the destdir value is not > reported at the end of the install process when > blib/config-x86_64-linux-thread-multi-5.018002 is copied to ../perl/ > > I manually copy blib/config-x86_64-linux-thread-multi-5.018002 to > my_path/../perl/ and MPI does not try to write it now. > > > > Now I have issues when MPI tries to send messages. > Don't know if you can help me with this. > Here is part of the error message I got: > > [devfrt01.XXX:10256] mca: base: component_find: unable to open > /software/lib64/openmpi/lib/openmpi/mca_shmem_mmap: perhaps a missing > symbol, or compiled for a different version of Open MPI? (ignored) > ... > > It looks like opal_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during opal_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_shmem_base_select failed > --> Returned value -1 instead of OPAL_SUCCESS > ... > It looks like orte_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during orte_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_init failed > --> Returned value Error (-1) instead of ORTE_SUCCESS > ... > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10252] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10256] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10253] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10257] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10251] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > ------------------------------------------------------- > Primary job terminated normally, but 1 process returned > a non-zero exit code.. Per user-direction, the job has been aborted. > ------------------------------------------------------- > -------------------------------------------------------------------------- > mpiexec detected that one or more processes exited with non-zero status, > thus causing > the job to be terminated. The first process to do so was: > > Process name: [[21120,1],1] > Exit code: 1 > > >> The issue is that you moved the ?/maker/perl directory after initial >> setup. So when maker runs it can?t find the files, and tries to rebuild >> the shared objects for C/Perl MPI binding that are missing (compiles on >> demand). >> >> Solutions (one of the following two options) >> 1. Install maker as indicated in the INTALL file and then append the >> path for ?/maker/bin/ to the PATH environmental variable in /etc/profile >> (i.e. export PATH=/usr/local/maker/bin:$PATH). >> 2. Install maker as indicated in the INTALL file and then soft link the >> executables in ?/maker/bin/ to the location of your choice (i.e. >> /usr/bin/ or /usr/local/bin/ - this is what homebrew does for example). >> >> Example: >> MacBook-Pro:~ cholt$ ls -al /usr/local/bin/maker >> lrwxr-xr-x 1 cholt admin 32 Oct 25 2015 /usr/local/bin/maker -> >> ../Cellar/maker/2.31.8/bin/maker >> >> Notice that the softink in /usr/local/bin/ is pointing to the actual >> location where homebrew placed the installation. Doing something similar >> would be better than you trying to break up the installation structure. >> Basically they move the contents of ?/maker/* to >> /usr/local/Cellar/maker/2.31.8/, then they performed all installation >> steps there, and then softlinked the executables elsewhere using ?ln -s' >> >> >> When you install maker, you have to run the setup steps in ?/maker/src/. >> After install, ?/maker/bin/ will contain executables and ?/maker/perl/ >> and ?/maker/lib/ will contain libraries and shared objects used by those >> executables. Remember the executables are not binaries, they are >> scripts, so they need access to the libraries and shared objects used by >> the scripts. The executables know to look for libraries/files in the >> installation folder, but if you move them, the @INC search list used by >> perl will not find them. You can try and manually declare new locations >> to search in the @INC list using the PERL5LIB environmental variable >> (but using PERL5LIB is beyond the scope of the maker-devel list and can >> be investigated using Perl?s own docs). >> >> Also as explained in the homebrew example above, the base directory does >> not have to be named ?maker?. You can name it whatever you want. It is >> just the installation structure within the containing directory that has >> to be consistent. >> >> Alternatively if you are doing something more complex like a ?module? >> setup using lua based modules just use the prepend_path command to add >> the .../maker/bin foldert to the environment on demand ?> >> >> Example: >> >> -- -*- lua -*- >> help( >> [[ >> This module loads the MAKER genome annotation pipeline >> ]]) >> >> whatis("Name: MAKER") >> whatis("Version: 2.31.8") >> whatis("Category: Gene Annotation") >> whatis("Keywords: Gene Annotation") >> whatis("URL: http://www.yandell-lab.org/software/maker.html") >> whatis("Description: MAKER is a highly parallelized genome >> annotation/re-annotation pipeline") >> >> -- Load required modules >> load("openmpi/1.8.4.i", "genemark-ES", "augustus", "snap", "exonerate", >> "ncbi-blast+", "RepeatMasker") >> >> -- Prepare needed values >> local APPS = "/ucgd/apps/" >> local id = "maker" >> local version = "2.31.8" >> local base = pathJoin(APPS, id, version) >> local bin = pathJoin(base, "bin") >> >> -- Set PATH and other environmental variables >> prepend_path("PATH", bin) >> prepend_path("LD_PRELOAD", "/ucgd/apps/openmpi/1.8.4i/lib/libmpi.so") >> setenv("OMPI_MCA_mpi_warn_on_fork", 0) >> >> >> ?Carson >> >> >>> On Aug 3, 2016, at 3:25 AM, sebastien.moretti at unil.ch >>> wrote: >>> >>> OK, I removed execution rights for mpi_evaluator. >>> >>> I tried mpiexec -n 10 maker >>> MPI tries to write in the maker/perl/ directory. >>> This is an issue for me because maker is installed in a read-only file >>> system. >>> >>> Is there an option or something to tell MPI to write elsewhere? >>> >>> S?bastien >>> >>>> To run MAKER with MPI, you just call maker with mpiexec. Examples can >>>> be found on the MAKER wiki, in the devel list archives, and in the >>>> README/INSTALL guide. >>>> >>>> Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker >>>> >>>> Any other MPI options will depend on the flavor of MPI you use, and >>>> your cluster settings. >>>> >>>> ?Carson >>>> >>>> >>>> >>>>> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti >>>>> > wrote: >>>>> >>>>> Hi >>>>> >>>>> I compiled maker with MPI support and would like to test it. >>>>> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >>>>> is missing. >>>>> >>>>> What does mpi_evaluator do? >>>>> How to test properly MPI in maker? >>>>> >>>>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>>>> openmpi 1.8.1 >>>>> >>>>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From lassance at fas.harvard.edu Tue Aug 30 08:19:13 2016 From: lassance at fas.harvard.edu (Lassance, Jean-Marc) Date: Tue, 30 Aug 2016 13:19:13 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats Message-ID: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Tue Aug 30 11:16:37 2016 From: dence at genetics.utah.edu (Daniel Ence) Date: Tue, 30 Aug 2016 16:16:37 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Message-ID: Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? Thanks, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From me.mark at gmail.com Tue Aug 30 11:10:16 2016 From: me.mark at gmail.com (Mark Ebbert) Date: Tue, 30 Aug 2016 09:10:16 -0700 Subject: [maker-devel] (no subject) Message-ID: <57c5af23a842850000919b43@polymail.io> Good day everyone! I?m getting the error stating: ?WARNING: Multiple MAKER processes have been started in the same directory.? Everything I?ve seen mentions version issues with MPICH. The difference in my situation is that my initial run ran just fine, but died because of the cluster time constraints. We?re only allowed 3 days.? There are a bunch of .NFSLock files in the output directory. I?m guessing Maker wasn?t able to clear the locks when the jobs died? Can I safely delete those lock files? What?s the best way to handle this going forward since I can only run jobs for 3 days at a time? Thanks! Mark T. W. Ebbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 30 11:54:39 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 30 Aug 2016 10:54:39 -0600 Subject: [maker-devel] (no subject) In-Reply-To: <57c5af23a842850000919b43@polymail.io> References: <57c5af23a842850000919b43@polymail.io> Message-ID: Run 'maker -help? with mpiexec. Example: mpiexec -n 10 maker -help If the MPI communication ring is working correctly, then it will print the help message only once (from the root process). If it is not working, it will print the help message 10 time because each of the 10 MPI processes will think they are the root process. It is a simple test that can identify if it is an MPI issue or not. If it is not an MPI issue, you can just search for the NFSLock files using find and delete them,. ?Carson > On Aug 30, 2016, at 10:10 AM, Mark Ebbert wrote: > > > Good day everyone! > > I?m getting the error stating: ?WARNING: Multiple MAKER processes have been started in the same directory.? Everything I?ve seen mentions version issues with MPICH. The difference in my situation is that my initial run ran just fine, but died because of the cluster time constraints. We?re only allowed 3 days. > > There are a bunch of .NFSLock files in the output directory. I?m guessing Maker wasn?t able to clear the locks when the jobs died? Can I safely delete those lock files? What?s the best way to handle this going forward since I can only run jobs for 3 days at a time? > > Thanks! > > Mark T. W. Ebbert > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 30 15:12:44 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 30 Aug 2016 14:12:44 -0600 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Message-ID: <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> Also make sure you pass the data in using rm_gff and not maker_gff if the repeats were not MAKER generated. ?Carson > On Aug 30, 2016, at 10:16 AM, Daniel Ence wrote: > > Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? > > And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? > > Thanks, > Daniel > > > Daniel Ence > Graduate Student > Eccles Institute of Human Genetics > University of Utah > 15 North 2030 East, Room 2100 > Salt Lake City, UT 84112-5330 > >> On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: >> >> Hi. >> >> I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). >> >> Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: >> >> rm_gff=myanimal.repeatmasker.out.gff3 >> >> #-----Re-annotation Using MAKER Derived GFF3 >> maker_gff=myanimal.all.maker.pass1.gff >> rm_pass=1 >> >> Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: >> >> Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. >> --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu >> ERROR: Failed while processing all repeats >> ERROR: Chunk failed at level:3, tier_type:1 >> FAILED CONTIG:scaffold00013 >> >> I wonder if you have an idea of what could be wrong here. >> >> Thanks for your help, >> >> >> Jean-Marc >> >> ?????????????????? >> Jean-Marc Lassance, PhD >> >> Harvard University >> Department of Organismic and Evolutionary Biology >> Department of Molecular and Cellular Biology >> Museum of Comparative Zoology >> >> 26, Oxford Street >> Cambridge MA 02138 >> USA >> >> email: lassance at fas.harvard.edu >> twitter: @lassancejm >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lassance at fas.harvard.edu Tue Aug 30 15:31:44 2016 From: lassance at fas.harvard.edu (Lassance, Jean-Marc) Date: Tue, 30 Aug 2016 20:31:44 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> Message-ID: <1F3B92BC-1717-4CB5-A26D-6F2126667E53@fas.harvard.edu> Let me clarify one thing: the first pass was performed with Maker running repeatMasker internally, which is why I decided to use them in the second pass, as well as the data from the independent run of RepeatMasker. From reading earlier posts, I gathered that Maker would use first the evidence from the rm_gff, and then from maker_gff if rm_pass=1 is activated, but that having both would not hurt. Correct? JM On Aug 30, 2016, at 4:12 PM, Carson Holt > wrote: Also make sure you pass the data in using rm_gff and not maker_gff if the repeats were not MAKER generated. ?Carson On Aug 30, 2016, at 10:16 AM, Daniel Ence > wrote: Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? Thanks, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm -------------- next part -------------- An HTML attachment was scrubbed... URL: From psh65 at cornell.edu Wed Aug 31 09:35:28 2016 From: psh65 at cornell.edu (Prashant S Hosmani) Date: Wed, 31 Aug 2016 14:35:28 +0000 Subject: [maker-devel] mapping cDNA to updated genome Message-ID: Hi All, I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. Thank you for your help in advance. Prashant Prashant Hosmani Sol Genomics Network Boyce Thompson Institute, Ithaca, NY, USA -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Wed Aug 31 11:10:22 2016 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Wed, 31 Aug 2016 12:10:22 -0400 Subject: [maker-devel] mapping cDNA to updated genome In-Reply-To: References: Message-ID: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> Hi Prashant, I?m almost positive that the additional genes are coming from multiply aligning cDNAs. Did you repeat mask your genome before mapping things forward? Another thought, what kind of whole genome duplications has your plant been through. it may be that the multiple alignments are to pseudogenes is some stage of decay. If that is the case it would probably be safe to keep the the gene from longest/best aligned cDNA. Thanks, Mike > On Aug 31, 2016, at 10:35 AM, Prashant S Hosmani wrote: > > Hi All, > > I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. > > I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. > > Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. > > Thank you for your help in advance. > Prashant > > > Prashant Hosmani > Sol Genomics Network > Boyce Thompson Institute, Ithaca, NY, USA > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Aug 31 11:12:53 2016 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 31 Aug 2016 10:12:53 -0600 Subject: [maker-devel] mapping cDNA to updated genome In-Reply-To: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> References: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> Message-ID: <9FBCB1C4-C319-4933-8741-53DAFCB82458@gmail.com> Also if you have multiple alignments of the same cDNA, you can use the score column of the mRNA feature to see which aligns best. If they have the same score, you will have to disambiguate manually or just remove all copies. ?Carson > On Aug 31, 2016, at 10:10 AM, Michael Campbell wrote: > > Hi Prashant, > > I?m almost positive that the additional genes are coming from multiply aligning cDNAs. Did you repeat mask your genome before mapping things forward? > > Another thought, what kind of whole genome duplications has your plant been through. it may be that the multiple alignments are to pseudogenes is some stage of decay. If that is the case it would probably be safe to keep the the gene from longest/best aligned cDNA. > > Thanks, > Mike >> On Aug 31, 2016, at 10:35 AM, Prashant S Hosmani > wrote: >> >> Hi All, >> >> I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. >> >> I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. >> >> Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. >> >> Thank you for your help in advance. >> Prashant >> >> >> Prashant Hosmani >> Sol Genomics Network >> Boyce Thompson Institute, Ithaca, NY, USA >> >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 2 08:08:41 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:08:41 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> If you move it, then the executables won?t be able to locate dependencies in the ?/maker/data, ?/maker/lib, and ?/maker/perl directories. You should really either add the location of ?/maker/bin to you PATH environmental variable or at most soft link the executables somewhere else using the ?ln -s? command. ?Carson > On Aug 2, 2016, at 5:57 AM, Sebastien Moretti wrote: > > OK, just need to move the maker/bin folder. > > Thanks > >> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >> >> ?Carson >> >> >> >>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I try to install maker in a custom directory following regular Perl >>> commands with Build.PL. >>> >>> Unfortunately if I use >>> perl Build.PL --installdirs= >>> then >>> ./Build install --destdir= >>> build install fails with the error: >>> Can't use an undefined value as a HASH reference at >>> .../Module/Build/Base.pm line 3038. >>> >>> It does not happen if I don't use --installdirs >>> >>> >>> Is there an "official" way to install maker in a custom directory? >>> >>> >>> Also I found typos in the description of maker at >>> http://www.yandell-lab.org/software/maker.html >>> ouputs should be outputs >>> seusequent should be ??? >>> >>> >>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>> >>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 05:57:32 2016 From: sebastien.moretti at unil.ch (Sebastien Moretti) Date: Tue, 2 Aug 2016 13:57:32 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: OK, just need to move the maker/bin folder. Thanks > MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. > > ?Carson > > > >> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >> >> Hi >> >> I try to install maker in a custom directory following regular Perl >> commands with Build.PL. >> >> Unfortunately if I use >> perl Build.PL --installdirs= >> then >> ./Build install --destdir= >> build install fails with the error: >> Can't use an undefined value as a HASH reference at >> .../Module/Build/Base.pm line 3038. >> >> It does not happen if I don't use --installdirs >> >> >> Is there an "official" way to install maker in a custom directory? >> >> >> Also I found typos in the description of maker at >> http://www.yandell-lab.org/software/maker.html >> ouputs should be outputs >> seusequent should be ??? >> >> >> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >> >> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 06:00:53 2016 From: sebastien.moretti at unil.ch (Sebastien Moretti) Date: Tue, 2 Aug 2016 14:00:53 +0200 Subject: [maker-devel] MPI test Message-ID: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Hi I compiled maker with MPI support and would like to test it. So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm is missing. What does mpi_evaluator do? How to test properly MPI in maker? maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 openmpi 1.8.1 Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 08:14:23 2016 From: sebastien.moretti at unil.ch (sebastien.moretti at unil.ch) Date: Tue, 2 Aug 2016 16:14:23 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> Message-ID: <196799cd-d370-a343-f87c-1292e58593a4@unil.ch> I moved maker/bin/, maker/perl/, maker/data/, maker/lib/ + LICENSE README RELEASE and it looks to be fine. Thanks > If you move it, then the executables won?t be able to locate dependencies in the ?/maker/data, ?/maker/lib, and ?/maker/perl directories. You should really either add the location of ?/maker/bin to you PATH environmental variable or at most soft link the executables somewhere else using the ?ln -s? command. > > ?Carson > > >> On Aug 2, 2016, at 5:57 AM, Sebastien Moretti wrote: >> >> OK, just need to move the maker/bin folder. >> >> Thanks >> >>> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >>> >>> ?Carson >>> >>> >>> >>>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>>> >>>> Hi >>>> >>>> I try to install maker in a custom directory following regular Perl >>>> commands with Build.PL. >>>> >>>> Unfortunately if I use >>>> perl Build.PL --installdirs= >>>> then >>>> ./Build install --destdir= >>>> build install fails with the error: >>>> Can't use an undefined value as a HASH reference at >>>> .../Module/Build/Base.pm line 3038. >>>> >>>> It does not happen if I don't use --installdirs >>>> >>>> >>>> Is there an "official" way to install maker in a custom directory? >>>> >>>> >>>> Also I found typos in the description of maker at >>>> http://www.yandell-lab.org/software/maker.html >>>> ouputs should be outputs >>>> seusequent should be ??? >>>> >>>> >>>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>>> >>>> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From carsonhh at gmail.com Tue Aug 2 08:19:35 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:19:35 -0600 Subject: [maker-devel] MPI test In-Reply-To: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: <97587487-EAB0-461E-8DB1-2E23FA3E3C4C@gmail.com> mpi_evaluator is a maker development related accessory script. You should ignore it. ?Carson > On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: > > Hi > > I compiled maker with MPI support and would like to test it. > So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm > is missing. > > What does mpi_evaluator do? > How to test properly MPI in maker? > > maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 > openmpi 1.8.1 > > Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Aug 2 08:59:40 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:59:40 -0600 Subject: [maker-devel] MPI test In-Reply-To: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. ?Carson > On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: > > Hi > > I compiled maker with MPI support and would like to test it. > So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm > is missing. > > What does mpi_evaluator do? > How to test properly MPI in maker? > > maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 > openmpi 1.8.1 > > Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From mmokrejs at gmail.com Tue Aug 2 09:29:49 2016 From: mmokrejs at gmail.com (=?UTF-8?Q?Martin_MOKREJ=c5=a0?=) Date: Tue, 2 Aug 2016 17:29:49 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: Hi Sebastien, I had similar concerns in the past as well. I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". Martin Sebastien Moretti wrote: > OK, just need to move the maker/bin folder. > > Thanks > >> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >> >> ?Carson >> >> >> >>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I try to install maker in a custom directory following regular Perl >>> commands with Build.PL. >>> >>> Unfortunately if I use >>> perl Build.PL --installdirs= >>> then >>> ./Build install --destdir= >>> build install fails with the error: >>> Can't use an undefined value as a HASH reference at >>> .../Module/Build/Base.pm line 3038. >>> >>> It does not happen if I don't use --installdirs >>> >>> >>> Is there an "official" way to install maker in a custom directory? From carsonhh at gmail.com Tue Aug 2 09:51:25 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 09:51:25 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. ?Carson > On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: > > Hi Sebastien, > I had similar concerns in the past as well. > > I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. > > I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse > > From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. > The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. > > Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". > > Martin > > > Sebastien Moretti wrote: >> OK, just need to move the maker/bin folder. >> >> Thanks >> >>> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >>> >>> ?Carson >>> >>> >>> >>>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>>> >>>> Hi >>>> >>>> I try to install maker in a custom directory following regular Perl >>>> commands with Build.PL. >>>> >>>> Unfortunately if I use >>>> perl Build.PL --installdirs= >>>> then >>>> ./Build install --destdir= >>>> build install fails with the error: >>>> Can't use an undefined value as a HASH reference at >>>> .../Module/Build/Base.pm line 3038. >>>> >>>> It does not happen if I don't use --installdirs >>>> >>>> >>>> Is there an "official" way to install maker in a custom directory? From mmokrejs at gmail.com Tue Aug 2 10:04:05 2016 From: mmokrejs at gmail.com (=?UTF-8?Q?Martin_MOKREJ=c5=a0?=) Date: Tue, 2 Aug 2016 18:04:05 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> Hi Carson, Carson Holt wrote: > You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. Maybe I was fooled by the install step. > > ?Carson > > >> On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: >> >> Hi Sebastien, >> I had similar concerns in the past as well. >> >> I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. >> >> I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse >> >> From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. >> The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. >> >> Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". >>> Emerging (1 of 1) sci-biology/maker-2.31.8::science * maker-2.31.8.tgz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>> Unpacking source... >>> Unpacking maker-2.31.8.tgz to /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>> Source unpacked in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>> Preparing source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>> Source prepared. >>> Configuring source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>> Source configured. >>> Compiling source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... MAKER supports distributed parallelization via MPI. Would you like to configure MAKER for MPI (This requires that you have an MPI client installed)? [N ]N Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'MAKER' version 'v2.31.8' The file 'Build' has been created for you to finish installing MAKER. ============================================================================== STATUS MAKER v2.31.8 ============================================================================== PERL Dependencies: VERIFIED External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: DISABLED MWAS Web Interface: DISABLED MAKER PACKAGE: CONFIGURATION OK Important Commands: ./Build installdeps #installs missing PERL dependencies ./Build installexes #installs all missing external programs ./Build install #installs MAKER ./Build status #Shows this status menu Other Commands: ./Build repeatmasker #installs RepeatMasker (asks for RepBase) ./Build blast #installs BLAST (NCBI BLAST+) ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) ./Build snap #installs SNAP ./Build augustus #installs Augustus ./Build apollo #installs Apollo ./Build gbrowse #installs GBrowse (must be root) ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) ./Build webapollo #installs WebApollo (use maker2wap to create DBs) ./Build mpich2 #installs MPICH2 (but manual install recommended) Building MAKER Installing MAKER... Building MAKER Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/MAKER/ConfigData.pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/Parallel/Application/MPI.pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/man/MAKER::ConfigData.3pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_fasta_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cufflinks2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2eval_gtf Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/genemark_gtf2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2wap Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_map_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/ipr_update_gff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2jbrowse Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2chado Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/compare Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_tool Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan_wrap Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/evaluator Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2zff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cegma2zff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_merge Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_data_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_fasta Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/chado2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/gff3_merge Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/tophat2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map2assembly Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_gff_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_gff >>> Source compiled. >>> Test phase [not enabled]: sci-biology/maker-2.31.8 >>> Install maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ category sci-biology >>> Completed installing maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ * Final size of build directory: 40312 KiB * Final size of installed tree: 12656 KiB ecompressdir: bzip2 -9 /usr/share/man ecompressdir: bzip2 -9 /usr/share/doc >>> Installing (1 of 1) sci-biology/maker-2.31.8::science >>> Auto-cleaning packages... That gives me these files installed on my system: # equery files maker * Searching for maker ... * Contents of sci-biology/maker-2.31.8: /usr /usr/bin /usr/bin/cegma2zff /usr/bin/chado2gff3 /usr/bin/compare_gff3_to_chado /usr/bin/cufflinks2gff3 /usr/bin/evaluator /usr/bin/fasta_merge /usr/bin/genemark_gtf2gff3 /usr/bin/gff3_merge /usr/bin/ipr_update_gff /usr/bin/iprscan2gff3 /usr/bin/iprscan_wrap /usr/bin/maker /usr/bin/maker2chado /usr/bin/maker2eval_gtf /usr/bin/maker2jbrowse /usr/bin/maker2wap /usr/bin/maker2zff /usr/bin/maker_functional /usr/bin/maker_functional_fasta /usr/bin/maker_functional_gff /usr/bin/maker_map_ids /usr/bin/map2assembly /usr/bin/map_data_ids /usr/bin/map_fasta_ids /usr/bin/map_gff_ids /usr/bin/tophat2gff3 /usr/lib64 /usr/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/vendor_perl/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/MAKER /usr/lib64/perl5/vendor_perl/5.20.2/MAKER/ConfigData.pm /usr/lib64/perl5/vendor_perl/5.20.2/Parallel /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application/MPI.pm /usr/share /usr/share/doc /usr/share/doc/maker-2.31.8 /usr/share/doc/maker-2.31.8/INSTALL.bz2 /usr/share/doc/maker-2.31.8/README.bz2 /usr/share/maker /usr/share/maker/GMOD /usr/share/maker/GMOD/Apollo /usr/share/maker/GMOD/Apollo/gff3.tiers /usr/share/maker/GMOD/JBrowse /usr/share/maker/GMOD/JBrowse/maker.css /usr/share/maker/data /usr/share/maker/data/dpp_contig.fasta /usr/share/maker/data/dpp_est.fasta /usr/share/maker/data/dpp_protein.fasta /usr/share/maker/data/hsap_contig.fasta /usr/share/maker/data/hsap_est.fasta /usr/share/maker/data/hsap_protein.fasta /usr/share/maker/data/te_proteins.fasta /usr/share/man /usr/share/man/man3 /usr/share/man/man3/MAKER::ConfigData.3pm.bz2 $ /usr/bin/maker Argument "2.49_01" isn't numeric in numeric ge (>=) at /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi/forks.pm line 1570. Can't locate GI.pm in @INC (you may need to install the GI module) (@INC contains: /home/mmokrejs/proj/sci/sci-biology/maker/_Inline/lib /usr/bin/../perl/lib /usr/bin/../lib /usr/bin/../src/inc/perl/lib /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/5.20.2 .) at /usr/bin/maker line 63. BEGIN failed--compilation aborted at /usr/bin/maker line 63. $ Of course, because as I said, I need to install properly all the dependencies from maker/lib/ subdirectory. BTW, did perl Build.PL introduce the _Inline/lib/ directory? It somehow escaped from the sanbox I think. :( But thank you for you kind answer anyway, Martin From carsonhh at gmail.com Tue Aug 2 10:09:12 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 10:09:12 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> Message-ID: <57C25EAE-FF0C-4CE6-9000-5B67A7260C28@gmail.com> The install step is installing the C/Perl shared object bindings for MPI to their final location. I could call that internally to ./Build, but since it can fail, and I also since it is configured to fail on missing Perl dependencies, it is better to run it as the final step. ?Carson > On Aug 2, 2016, at 10:04 AM, Martin MOKREJ? wrote: > > Hi Carson, > > Carson Holt wrote: >> You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. > > Maybe I was fooled by the install step. > >> >> ?Carson >> >> >>> On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: >>> >>> Hi Sebastien, >>> I had similar concerns in the past as well. >>> >>> I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. >>> >>> I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse >>> >>> From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. >>> The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. >>> >>> Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". > > >>>> Emerging (1 of 1) sci-biology/maker-2.31.8::science > * maker-2.31.8.tgz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>>> Unpacking source... >>>> Unpacking maker-2.31.8.tgz to /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>>> Source unpacked in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>>> Preparing source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>>> Source prepared. >>>> Configuring source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>>> Source configured. >>>> Compiling source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... > > MAKER supports distributed parallelization via MPI. > Would you like to configure MAKER for MPI (This > requires that you have an MPI client installed)? [N ]N > Created MYMETA.yml and MYMETA.json > Creating new 'Build' script for 'MAKER' version 'v2.31.8' > > > The file 'Build' has been created for you to finish installing MAKER. > > > ============================================================================== > STATUS MAKER v2.31.8 > ============================================================================== > PERL Dependencies: VERIFIED > External Programs: VERIFIED > External C Libraries: VERIFIED > MPI SUPPORT: DISABLED > MWAS Web Interface: DISABLED > MAKER PACKAGE: CONFIGURATION OK > > > Important Commands: > ./Build installdeps #installs missing PERL dependencies > ./Build installexes #installs all missing external programs > ./Build install #installs MAKER > ./Build status #Shows this status menu > > Other Commands: > ./Build repeatmasker #installs RepeatMasker (asks for RepBase) > ./Build blast #installs BLAST (NCBI BLAST+) > ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) > ./Build snap #installs SNAP > ./Build augustus #installs Augustus > ./Build apollo #installs Apollo > ./Build gbrowse #installs GBrowse (must be root) > ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) > ./Build webapollo #installs WebApollo (use maker2wap to create DBs) > ./Build mpich2 #installs MPICH2 (but manual install recommended) > Building MAKER > Installing MAKER... > Building MAKER > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/MAKER/ConfigData.pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/Parallel/Application/MPI.pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/man/MAKER::ConfigData.3pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_fasta_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cufflinks2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2eval_gtf > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/genemark_gtf2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2wap > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_map_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/ipr_update_gff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2jbrowse > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2chado > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/compare > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_tool > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan_wrap > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/evaluator > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2zff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cegma2zff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_merge > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_data_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_fasta > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/chado2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/gff3_merge > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/tophat2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map2assembly > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_gff_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_gff >>>> Source compiled. >>>> Test phase [not enabled]: sci-biology/maker-2.31.8 > >>>> Install maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ category sci-biology >>>> Completed installing maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ > > * Final size of build directory: 40312 KiB > * Final size of installed tree: 12656 KiB > > ecompressdir: bzip2 -9 /usr/share/man > ecompressdir: bzip2 -9 /usr/share/doc > >>>> Installing (1 of 1) sci-biology/maker-2.31.8::science >>>> Auto-cleaning packages... > > > > > That gives me these files installed on my system: > > # equery files maker > * Searching for maker ... > * Contents of sci-biology/maker-2.31.8: > /usr > /usr/bin > /usr/bin/cegma2zff > /usr/bin/chado2gff3 > /usr/bin/compare_gff3_to_chado > /usr/bin/cufflinks2gff3 > /usr/bin/evaluator > /usr/bin/fasta_merge > /usr/bin/genemark_gtf2gff3 > /usr/bin/gff3_merge > /usr/bin/ipr_update_gff > /usr/bin/iprscan2gff3 > /usr/bin/iprscan_wrap > /usr/bin/maker > /usr/bin/maker2chado > /usr/bin/maker2eval_gtf > /usr/bin/maker2jbrowse > /usr/bin/maker2wap > /usr/bin/maker2zff > /usr/bin/maker_functional > /usr/bin/maker_functional_fasta > /usr/bin/maker_functional_gff > /usr/bin/maker_map_ids > /usr/bin/map2assembly > /usr/bin/map_data_ids > /usr/bin/map_fasta_ids > /usr/bin/map_gff_ids > /usr/bin/tophat2gff3 > /usr/lib64 > /usr/lib64/perl5 > /usr/lib64/perl5/vendor_perl > /usr/lib64/perl5/vendor_perl/5.20.2 > /usr/lib64/perl5/vendor_perl/5.20.2/MAKER > /usr/lib64/perl5/vendor_perl/5.20.2/MAKER/ConfigData.pm > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application/MPI.pm > /usr/share > /usr/share/doc > /usr/share/doc/maker-2.31.8 > /usr/share/doc/maker-2.31.8/INSTALL.bz2 > /usr/share/doc/maker-2.31.8/README.bz2 > /usr/share/maker > /usr/share/maker/GMOD > /usr/share/maker/GMOD/Apollo > /usr/share/maker/GMOD/Apollo/gff3.tiers > /usr/share/maker/GMOD/JBrowse > /usr/share/maker/GMOD/JBrowse/maker.css > /usr/share/maker/data > /usr/share/maker/data/dpp_contig.fasta > /usr/share/maker/data/dpp_est.fasta > /usr/share/maker/data/dpp_protein.fasta > /usr/share/maker/data/hsap_contig.fasta > /usr/share/maker/data/hsap_est.fasta > /usr/share/maker/data/hsap_protein.fasta > /usr/share/maker/data/te_proteins.fasta > /usr/share/man > /usr/share/man/man3 > /usr/share/man/man3/MAKER::ConfigData.3pm.bz2 > > > > $ /usr/bin/maker > Argument "2.49_01" isn't numeric in numeric ge (>=) at /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi/forks.pm line 1570. > Can't locate GI.pm in @INC (you may need to install the GI module) (@INC contains: /home/mmokrejs/proj/sci/sci-biology/maker/_Inline/lib /usr/bin/../perl/lib /usr/bin/../lib /usr/bin/../src/inc/perl/lib /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/5.20.2 .) at /usr/bin/maker line 63. > BEGIN failed--compilation aborted at /usr/bin/maker line 63. > $ > > Of course, because as I said, I need to install properly all the dependencies from maker/lib/ subdirectory. > > > BTW, did perl Build.PL introduce the _Inline/lib/ directory? It somehow escaped from the sanbox I think. :( > > But thank you for you kind answer anyway, > Martin From sebastien.moretti at unil.ch Wed Aug 3 03:25:56 2016 From: sebastien.moretti at unil.ch (sebastien.moretti at unil.ch) Date: Wed, 3 Aug 2016 11:25:56 +0200 Subject: [maker-devel] MPI test In-Reply-To: References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: OK, I removed execution rights for mpi_evaluator. I tried mpiexec -n 10 maker MPI tries to write in the maker/perl/ directory. This is an issue for me because maker is installed in a read-only file system. Is there an option or something to tell MPI to write elsewhere? S?bastien > To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. > > Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker > > Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. > > ?Carson > > > >> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: >> >> Hi >> >> I compiled maker with MPI support and would like to test it. >> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >> is missing. >> >> What does mpi_evaluator do? >> How to test properly MPI in maker? >> >> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >> openmpi 1.8.1 >> >> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From carsonhh at gmail.com Wed Aug 3 10:16:57 2016 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 3 Aug 2016 10:16:57 -0600 Subject: [maker-devel] MPI test In-Reply-To: References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: <72BDD9C3-9E02-4394-AE1B-521411F9B86D@gmail.com> The issue is that you moved the ?/maker/perl directory after initial setup. So when maker runs it can?t find the files, and tries to rebuild the shared objects for C/Perl MPI binding that are missing (compiles on demand). Solutions (one of the following two options) 1. Install maker as indicated in the INTALL file and then append the path for ?/maker/bin/ to the PATH environmental variable in /etc/profile (i.e. export PATH=/usr/local/maker/bin:$PATH). 2. Install maker as indicated in the INTALL file and then soft link the executables in ?/maker/bin/ to the location of your choice (i.e. /usr/bin/ or /usr/local/bin/ - this is what homebrew does for example). Example: MacBook-Pro:~ cholt$ ls -al /usr/local/bin/maker lrwxr-xr-x 1 cholt admin 32 Oct 25 2015 /usr/local/bin/maker -> ../Cellar/maker/2.31.8/bin/maker Notice that the softink in /usr/local/bin/ is pointing to the actual location where homebrew placed the installation. Doing something similar would be better than you trying to break up the installation structure. Basically they move the contents of ?/maker/* to /usr/local/Cellar/maker/2.31.8/, then they performed all installation steps there, and then softlinked the executables elsewhere using ?ln -s' When you install maker, you have to run the setup steps in ?/maker/src/. After install, ?/maker/bin/ will contain executables and ?/maker/perl/ and ?/maker/lib/ will contain libraries and shared objects used by those executables. Remember the executables are not binaries, they are scripts, so they need access to the libraries and shared objects used by the scripts. The executables know to look for libraries/files in the installation folder, but if you move them, the @INC search list used by perl will not find them. You can try and manually declare new locations to search in the @INC list using the PERL5LIB environmental variable (but using PERL5LIB is beyond the scope of the maker-devel list and can be investigated using Perl?s own docs). Also as explained in the homebrew example above, the base directory does not have to be named ?maker?. You can name it whatever you want. It is just the installation structure within the containing directory that has to be consistent. Alternatively if you are doing something more complex like a ?module? setup using lua based modules just use the prepend_path command to add the .../maker/bin foldert to the environment on demand ?> Example: -- -*- lua -*- help( [[ This module loads the MAKER genome annotation pipeline ]]) whatis("Name: MAKER") whatis("Version: 2.31.8") whatis("Category: Gene Annotation") whatis("Keywords: Gene Annotation") whatis("URL: http://www.yandell-lab.org/software/maker.html") whatis("Description: MAKER is a highly parallelized genome annotation/re-annotation pipeline") -- Load required modules load("openmpi/1.8.4.i", "genemark-ES", "augustus", "snap", "exonerate", "ncbi-blast+", "RepeatMasker") -- Prepare needed values local APPS = "/ucgd/apps/" local id = "maker" local version = "2.31.8" local base = pathJoin(APPS, id, version) local bin = pathJoin(base, "bin") -- Set PATH and other environmental variables prepend_path("PATH", bin) prepend_path("LD_PRELOAD", "/ucgd/apps/openmpi/1.8.4i/lib/libmpi.so") setenv("OMPI_MCA_mpi_warn_on_fork", 0) ?Carson > On Aug 3, 2016, at 3:25 AM, sebastien.moretti at unil.ch wrote: > > OK, I removed execution rights for mpi_evaluator. > > I tried mpiexec -n 10 maker > MPI tries to write in the maker/perl/ directory. > This is an issue for me because maker is installed in a read-only file > system. > > Is there an option or something to tell MPI to write elsewhere? > > S?bastien > >> To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. >> >> Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker >> >> Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. >> >> ?Carson >> >> >> >>> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I compiled maker with MPI support and would like to test it. >>> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >>> is missing. >>> >>> What does mpi_evaluator do? >>> How to test properly MPI in maker? >>> >>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>> openmpi 1.8.1 >>> >>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Aug 4 14:21:36 2016 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 4 Aug 2016 14:21:36 -0600 Subject: [maker-devel] MPI test In-Reply-To: <9e01f987-d284-f73b-63c0-e2d84a87cb27@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> <72BDD9C3-9E02-4394-AE1B-521411F9B86D@gmail.com> <9e01f987-d284-f73b-63c0-e2d84a87cb27@unil.ch> Message-ID: Hi Sebastien, As long as you are not following the ?/maker/INSTALL instructions, are instead moving files around manually, and trying to modify the default behavior of ./Build I can not provide support. I?ve tried to best explain how you should be doing things in the previous e-mails as well as the information below. If you are installing for a module based setup, then the proper way to do this would be to move the maker tarball to the desired module location, then follow the INSTALL instructions. Example (bent for a module based setup since you said that is what you are doing): #load required tools module load snap module load openmpi module load blast module load exonerate module load RepeatMasker #set up environment for OpenMPI install (explained in ?/maker/INSTALL instructions) export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so #download and untar maker into desired location mkdir $APPS/maker cd $APPS/maker wget http://maker_url/maker.tgz tar -zxvf maker.tgz mv maker 2.31.8 #go to src directory cd $APPS/maker/2.31.8/src #install according to ?/maker/INSTALL instructions perl Build.PL ./Build installdeps ./Build installexes ./Build ./Build install #test normal installation $APPS/maker/2.31.8/bin/maker --help #test MPI installation (will print only one help message - something is wrong if it prints 5) mpiexec -mca btl ^openib -n 5 $APPS/maker/2.31.8/bin/maker --help #then setup a module file for maker that includes: 1. steps for loading prerequisite tools and modules (such as openmpi) 2. setting OMPI_MCA_mpi_warn_on_fork=0 3. setting LD_PRELOAD=/location/of/openmpi/lib/libmpi.so 4. prepend path for $APPS/maker/2.31.8/bin to PATH environmental variable Thanks, Carson > On Aug 4, 2016, at 8:28 AM, Sebastien Moretti wrote: > > I already use a module to load maker and external tools ;-) > As well as to set OMPI_MCA_mpi_warn_on_fork = 0 > > > In my case perl/config-x86_64-linux-thread-multi-5.018002 was created > properly but not copied. > I use ./Build install --destdir= and the destdir value is not > reported at the end of the install process when > blib/config-x86_64-linux-thread-multi-5.018002 is copied to ../perl/ > > I manually copy blib/config-x86_64-linux-thread-multi-5.018002 to > my_path/../perl/ and MPI does not try to write it now. > > > > Now I have issues when MPI tries to send messages. > Don't know if you can help me with this. > Here is part of the error message I got: > > [devfrt01.XXX:10256] mca: base: component_find: unable to open > /software/lib64/openmpi/lib/openmpi/mca_shmem_mmap: perhaps a missing > symbol, or compiled for a different version of Open MPI? (ignored) > ... > > It looks like opal_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during opal_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_shmem_base_select failed > --> Returned value -1 instead of OPAL_SUCCESS > ... > It looks like orte_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during orte_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_init failed > --> Returned value Error (-1) instead of ORTE_SUCCESS > ... > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10252] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10256] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10253] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10257] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10251] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > ------------------------------------------------------- > Primary job terminated normally, but 1 process returned > a non-zero exit code.. Per user-direction, the job has been aborted. > ------------------------------------------------------- > -------------------------------------------------------------------------- > mpiexec detected that one or more processes exited with non-zero status, > thus causing > the job to be terminated. The first process to do so was: > > Process name: [[21120,1],1] > Exit code: 1 > > >> The issue is that you moved the ?/maker/perl directory after initial >> setup. So when maker runs it can?t find the files, and tries to rebuild >> the shared objects for C/Perl MPI binding that are missing (compiles on >> demand). >> >> Solutions (one of the following two options) >> 1. Install maker as indicated in the INTALL file and then append the >> path for ?/maker/bin/ to the PATH environmental variable in /etc/profile >> (i.e. export PATH=/usr/local/maker/bin:$PATH). >> 2. Install maker as indicated in the INTALL file and then soft link the >> executables in ?/maker/bin/ to the location of your choice (i.e. >> /usr/bin/ or /usr/local/bin/ - this is what homebrew does for example). >> >> Example: >> MacBook-Pro:~ cholt$ ls -al /usr/local/bin/maker >> lrwxr-xr-x 1 cholt admin 32 Oct 25 2015 /usr/local/bin/maker -> >> ../Cellar/maker/2.31.8/bin/maker >> >> Notice that the softink in /usr/local/bin/ is pointing to the actual >> location where homebrew placed the installation. Doing something similar >> would be better than you trying to break up the installation structure. >> Basically they move the contents of ?/maker/* to >> /usr/local/Cellar/maker/2.31.8/, then they performed all installation >> steps there, and then softlinked the executables elsewhere using ?ln -s' >> >> >> When you install maker, you have to run the setup steps in ?/maker/src/. >> After install, ?/maker/bin/ will contain executables and ?/maker/perl/ >> and ?/maker/lib/ will contain libraries and shared objects used by those >> executables. Remember the executables are not binaries, they are >> scripts, so they need access to the libraries and shared objects used by >> the scripts. The executables know to look for libraries/files in the >> installation folder, but if you move them, the @INC search list used by >> perl will not find them. You can try and manually declare new locations >> to search in the @INC list using the PERL5LIB environmental variable >> (but using PERL5LIB is beyond the scope of the maker-devel list and can >> be investigated using Perl?s own docs). >> >> Also as explained in the homebrew example above, the base directory does >> not have to be named ?maker?. You can name it whatever you want. It is >> just the installation structure within the containing directory that has >> to be consistent. >> >> Alternatively if you are doing something more complex like a ?module? >> setup using lua based modules just use the prepend_path command to add >> the .../maker/bin foldert to the environment on demand ?> >> >> Example: >> >> -- -*- lua -*- >> help( >> [[ >> This module loads the MAKER genome annotation pipeline >> ]]) >> >> whatis("Name: MAKER") >> whatis("Version: 2.31.8") >> whatis("Category: Gene Annotation") >> whatis("Keywords: Gene Annotation") >> whatis("URL: http://www.yandell-lab.org/software/maker.html") >> whatis("Description: MAKER is a highly parallelized genome >> annotation/re-annotation pipeline") >> >> -- Load required modules >> load("openmpi/1.8.4.i", "genemark-ES", "augustus", "snap", "exonerate", >> "ncbi-blast+", "RepeatMasker") >> >> -- Prepare needed values >> local APPS = "/ucgd/apps/" >> local id = "maker" >> local version = "2.31.8" >> local base = pathJoin(APPS, id, version) >> local bin = pathJoin(base, "bin") >> >> -- Set PATH and other environmental variables >> prepend_path("PATH", bin) >> prepend_path("LD_PRELOAD", "/ucgd/apps/openmpi/1.8.4i/lib/libmpi.so") >> setenv("OMPI_MCA_mpi_warn_on_fork", 0) >> >> >> ?Carson >> >> >>> On Aug 3, 2016, at 3:25 AM, sebastien.moretti at unil.ch >>> wrote: >>> >>> OK, I removed execution rights for mpi_evaluator. >>> >>> I tried mpiexec -n 10 maker >>> MPI tries to write in the maker/perl/ directory. >>> This is an issue for me because maker is installed in a read-only file >>> system. >>> >>> Is there an option or something to tell MPI to write elsewhere? >>> >>> S?bastien >>> >>>> To run MAKER with MPI, you just call maker with mpiexec. Examples can >>>> be found on the MAKER wiki, in the devel list archives, and in the >>>> README/INSTALL guide. >>>> >>>> Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker >>>> >>>> Any other MPI options will depend on the flavor of MPI you use, and >>>> your cluster settings. >>>> >>>> ?Carson >>>> >>>> >>>> >>>>> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti >>>>> > wrote: >>>>> >>>>> Hi >>>>> >>>>> I compiled maker with MPI support and would like to test it. >>>>> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >>>>> is missing. >>>>> >>>>> What does mpi_evaluator do? >>>>> How to test properly MPI in maker? >>>>> >>>>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>>>> openmpi 1.8.1 >>>>> >>>>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From lassance at fas.harvard.edu Tue Aug 30 07:19:13 2016 From: lassance at fas.harvard.edu (Lassance, Jean-Marc) Date: Tue, 30 Aug 2016 13:19:13 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats Message-ID: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Tue Aug 30 10:16:37 2016 From: dence at genetics.utah.edu (Daniel Ence) Date: Tue, 30 Aug 2016 16:16:37 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Message-ID: Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? Thanks, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From me.mark at gmail.com Tue Aug 30 10:10:16 2016 From: me.mark at gmail.com (Mark Ebbert) Date: Tue, 30 Aug 2016 09:10:16 -0700 Subject: [maker-devel] (no subject) Message-ID: <57c5af23a842850000919b43@polymail.io> Good day everyone! I?m getting the error stating: ?WARNING: Multiple MAKER processes have been started in the same directory.? Everything I?ve seen mentions version issues with MPICH. The difference in my situation is that my initial run ran just fine, but died because of the cluster time constraints. We?re only allowed 3 days.? There are a bunch of .NFSLock files in the output directory. I?m guessing Maker wasn?t able to clear the locks when the jobs died? Can I safely delete those lock files? What?s the best way to handle this going forward since I can only run jobs for 3 days at a time? Thanks! Mark T. W. Ebbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 30 10:54:39 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 30 Aug 2016 10:54:39 -0600 Subject: [maker-devel] (no subject) In-Reply-To: <57c5af23a842850000919b43@polymail.io> References: <57c5af23a842850000919b43@polymail.io> Message-ID: Run 'maker -help? with mpiexec. Example: mpiexec -n 10 maker -help If the MPI communication ring is working correctly, then it will print the help message only once (from the root process). If it is not working, it will print the help message 10 time because each of the 10 MPI processes will think they are the root process. It is a simple test that can identify if it is an MPI issue or not. If it is not an MPI issue, you can just search for the NFSLock files using find and delete them,. ?Carson > On Aug 30, 2016, at 10:10 AM, Mark Ebbert wrote: > > > Good day everyone! > > I?m getting the error stating: ?WARNING: Multiple MAKER processes have been started in the same directory.? Everything I?ve seen mentions version issues with MPICH. The difference in my situation is that my initial run ran just fine, but died because of the cluster time constraints. We?re only allowed 3 days. > > There are a bunch of .NFSLock files in the output directory. I?m guessing Maker wasn?t able to clear the locks when the jobs died? Can I safely delete those lock files? What?s the best way to handle this going forward since I can only run jobs for 3 days at a time? > > Thanks! > > Mark T. W. Ebbert > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 30 14:12:44 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 30 Aug 2016 14:12:44 -0600 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Message-ID: <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> Also make sure you pass the data in using rm_gff and not maker_gff if the repeats were not MAKER generated. ?Carson > On Aug 30, 2016, at 10:16 AM, Daniel Ence wrote: > > Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? > > And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? > > Thanks, > Daniel > > > Daniel Ence > Graduate Student > Eccles Institute of Human Genetics > University of Utah > 15 North 2030 East, Room 2100 > Salt Lake City, UT 84112-5330 > >> On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: >> >> Hi. >> >> I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). >> >> Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: >> >> rm_gff=myanimal.repeatmasker.out.gff3 >> >> #-----Re-annotation Using MAKER Derived GFF3 >> maker_gff=myanimal.all.maker.pass1.gff >> rm_pass=1 >> >> Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: >> >> Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. >> --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu >> ERROR: Failed while processing all repeats >> ERROR: Chunk failed at level:3, tier_type:1 >> FAILED CONTIG:scaffold00013 >> >> I wonder if you have an idea of what could be wrong here. >> >> Thanks for your help, >> >> >> Jean-Marc >> >> ?????????????????? >> Jean-Marc Lassance, PhD >> >> Harvard University >> Department of Organismic and Evolutionary Biology >> Department of Molecular and Cellular Biology >> Museum of Comparative Zoology >> >> 26, Oxford Street >> Cambridge MA 02138 >> USA >> >> email: lassance at fas.harvard.edu >> twitter: @lassancejm >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lassance at fas.harvard.edu Tue Aug 30 14:31:44 2016 From: lassance at fas.harvard.edu (Lassance, Jean-Marc) Date: Tue, 30 Aug 2016 20:31:44 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> Message-ID: <1F3B92BC-1717-4CB5-A26D-6F2126667E53@fas.harvard.edu> Let me clarify one thing: the first pass was performed with Maker running repeatMasker internally, which is why I decided to use them in the second pass, as well as the data from the independent run of RepeatMasker. From reading earlier posts, I gathered that Maker would use first the evidence from the rm_gff, and then from maker_gff if rm_pass=1 is activated, but that having both would not hurt. Correct? JM On Aug 30, 2016, at 4:12 PM, Carson Holt > wrote: Also make sure you pass the data in using rm_gff and not maker_gff if the repeats were not MAKER generated. ?Carson On Aug 30, 2016, at 10:16 AM, Daniel Ence > wrote: Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? Thanks, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm -------------- next part -------------- An HTML attachment was scrubbed... URL: From psh65 at cornell.edu Wed Aug 31 08:35:28 2016 From: psh65 at cornell.edu (Prashant S Hosmani) Date: Wed, 31 Aug 2016 14:35:28 +0000 Subject: [maker-devel] mapping cDNA to updated genome Message-ID: Hi All, I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. Thank you for your help in advance. Prashant Prashant Hosmani Sol Genomics Network Boyce Thompson Institute, Ithaca, NY, USA -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Wed Aug 31 10:10:22 2016 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Wed, 31 Aug 2016 12:10:22 -0400 Subject: [maker-devel] mapping cDNA to updated genome In-Reply-To: References: Message-ID: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> Hi Prashant, I?m almost positive that the additional genes are coming from multiply aligning cDNAs. Did you repeat mask your genome before mapping things forward? Another thought, what kind of whole genome duplications has your plant been through. it may be that the multiple alignments are to pseudogenes is some stage of decay. If that is the case it would probably be safe to keep the the gene from longest/best aligned cDNA. Thanks, Mike > On Aug 31, 2016, at 10:35 AM, Prashant S Hosmani wrote: > > Hi All, > > I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. > > I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. > > Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. > > Thank you for your help in advance. > Prashant > > > Prashant Hosmani > Sol Genomics Network > Boyce Thompson Institute, Ithaca, NY, USA > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Aug 31 10:12:53 2016 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 31 Aug 2016 10:12:53 -0600 Subject: [maker-devel] mapping cDNA to updated genome In-Reply-To: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> References: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> Message-ID: <9FBCB1C4-C319-4933-8741-53DAFCB82458@gmail.com> Also if you have multiple alignments of the same cDNA, you can use the score column of the mRNA feature to see which aligns best. If they have the same score, you will have to disambiguate manually or just remove all copies. ?Carson > On Aug 31, 2016, at 10:10 AM, Michael Campbell wrote: > > Hi Prashant, > > I?m almost positive that the additional genes are coming from multiply aligning cDNAs. Did you repeat mask your genome before mapping things forward? > > Another thought, what kind of whole genome duplications has your plant been through. it may be that the multiple alignments are to pseudogenes is some stage of decay. If that is the case it would probably be safe to keep the the gene from longest/best aligned cDNA. > > Thanks, > Mike >> On Aug 31, 2016, at 10:35 AM, Prashant S Hosmani > wrote: >> >> Hi All, >> >> I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. >> >> I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. >> >> Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. >> >> Thank you for your help in advance. >> Prashant >> >> >> Prashant Hosmani >> Sol Genomics Network >> Boyce Thompson Institute, Ithaca, NY, USA >> >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 2 08:08:41 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:08:41 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> If you move it, then the executables won?t be able to locate dependencies in the ?/maker/data, ?/maker/lib, and ?/maker/perl directories. You should really either add the location of ?/maker/bin to you PATH environmental variable or at most soft link the executables somewhere else using the ?ln -s? command. ?Carson > On Aug 2, 2016, at 5:57 AM, Sebastien Moretti wrote: > > OK, just need to move the maker/bin folder. > > Thanks > >> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >> >> ?Carson >> >> >> >>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I try to install maker in a custom directory following regular Perl >>> commands with Build.PL. >>> >>> Unfortunately if I use >>> perl Build.PL --installdirs= >>> then >>> ./Build install --destdir= >>> build install fails with the error: >>> Can't use an undefined value as a HASH reference at >>> .../Module/Build/Base.pm line 3038. >>> >>> It does not happen if I don't use --installdirs >>> >>> >>> Is there an "official" way to install maker in a custom directory? >>> >>> >>> Also I found typos in the description of maker at >>> http://www.yandell-lab.org/software/maker.html >>> ouputs should be outputs >>> seusequent should be ??? >>> >>> >>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>> >>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 05:57:32 2016 From: sebastien.moretti at unil.ch (Sebastien Moretti) Date: Tue, 2 Aug 2016 13:57:32 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: OK, just need to move the maker/bin folder. Thanks > MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. > > ?Carson > > > >> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >> >> Hi >> >> I try to install maker in a custom directory following regular Perl >> commands with Build.PL. >> >> Unfortunately if I use >> perl Build.PL --installdirs= >> then >> ./Build install --destdir= >> build install fails with the error: >> Can't use an undefined value as a HASH reference at >> .../Module/Build/Base.pm line 3038. >> >> It does not happen if I don't use --installdirs >> >> >> Is there an "official" way to install maker in a custom directory? >> >> >> Also I found typos in the description of maker at >> http://www.yandell-lab.org/software/maker.html >> ouputs should be outputs >> seusequent should be ??? >> >> >> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >> >> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 06:00:53 2016 From: sebastien.moretti at unil.ch (Sebastien Moretti) Date: Tue, 2 Aug 2016 14:00:53 +0200 Subject: [maker-devel] MPI test Message-ID: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Hi I compiled maker with MPI support and would like to test it. So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm is missing. What does mpi_evaluator do? How to test properly MPI in maker? maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 openmpi 1.8.1 Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 08:14:23 2016 From: sebastien.moretti at unil.ch (sebastien.moretti at unil.ch) Date: Tue, 2 Aug 2016 16:14:23 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> Message-ID: <196799cd-d370-a343-f87c-1292e58593a4@unil.ch> I moved maker/bin/, maker/perl/, maker/data/, maker/lib/ + LICENSE README RELEASE and it looks to be fine. Thanks > If you move it, then the executables won?t be able to locate dependencies in the ?/maker/data, ?/maker/lib, and ?/maker/perl directories. You should really either add the location of ?/maker/bin to you PATH environmental variable or at most soft link the executables somewhere else using the ?ln -s? command. > > ?Carson > > >> On Aug 2, 2016, at 5:57 AM, Sebastien Moretti wrote: >> >> OK, just need to move the maker/bin folder. >> >> Thanks >> >>> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >>> >>> ?Carson >>> >>> >>> >>>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>>> >>>> Hi >>>> >>>> I try to install maker in a custom directory following regular Perl >>>> commands with Build.PL. >>>> >>>> Unfortunately if I use >>>> perl Build.PL --installdirs= >>>> then >>>> ./Build install --destdir= >>>> build install fails with the error: >>>> Can't use an undefined value as a HASH reference at >>>> .../Module/Build/Base.pm line 3038. >>>> >>>> It does not happen if I don't use --installdirs >>>> >>>> >>>> Is there an "official" way to install maker in a custom directory? >>>> >>>> >>>> Also I found typos in the description of maker at >>>> http://www.yandell-lab.org/software/maker.html >>>> ouputs should be outputs >>>> seusequent should be ??? >>>> >>>> >>>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>>> >>>> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From carsonhh at gmail.com Tue Aug 2 08:19:35 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:19:35 -0600 Subject: [maker-devel] MPI test In-Reply-To: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: <97587487-EAB0-461E-8DB1-2E23FA3E3C4C@gmail.com> mpi_evaluator is a maker development related accessory script. You should ignore it. ?Carson > On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: > > Hi > > I compiled maker with MPI support and would like to test it. > So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm > is missing. > > What does mpi_evaluator do? > How to test properly MPI in maker? > > maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 > openmpi 1.8.1 > > Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Aug 2 08:59:40 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:59:40 -0600 Subject: [maker-devel] MPI test In-Reply-To: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. ?Carson > On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: > > Hi > > I compiled maker with MPI support and would like to test it. > So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm > is missing. > > What does mpi_evaluator do? > How to test properly MPI in maker? > > maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 > openmpi 1.8.1 > > Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From mmokrejs at gmail.com Tue Aug 2 09:29:49 2016 From: mmokrejs at gmail.com (=?UTF-8?Q?Martin_MOKREJ=c5=a0?=) Date: Tue, 2 Aug 2016 17:29:49 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: Hi Sebastien, I had similar concerns in the past as well. I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". Martin Sebastien Moretti wrote: > OK, just need to move the maker/bin folder. > > Thanks > >> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >> >> ?Carson >> >> >> >>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I try to install maker in a custom directory following regular Perl >>> commands with Build.PL. >>> >>> Unfortunately if I use >>> perl Build.PL --installdirs= >>> then >>> ./Build install --destdir= >>> build install fails with the error: >>> Can't use an undefined value as a HASH reference at >>> .../Module/Build/Base.pm line 3038. >>> >>> It does not happen if I don't use --installdirs >>> >>> >>> Is there an "official" way to install maker in a custom directory? From carsonhh at gmail.com Tue Aug 2 09:51:25 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 09:51:25 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. ?Carson > On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: > > Hi Sebastien, > I had similar concerns in the past as well. > > I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. > > I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse > > From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. > The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. > > Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". > > Martin > > > Sebastien Moretti wrote: >> OK, just need to move the maker/bin folder. >> >> Thanks >> >>> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >>> >>> ?Carson >>> >>> >>> >>>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>>> >>>> Hi >>>> >>>> I try to install maker in a custom directory following regular Perl >>>> commands with Build.PL. >>>> >>>> Unfortunately if I use >>>> perl Build.PL --installdirs= >>>> then >>>> ./Build install --destdir= >>>> build install fails with the error: >>>> Can't use an undefined value as a HASH reference at >>>> .../Module/Build/Base.pm line 3038. >>>> >>>> It does not happen if I don't use --installdirs >>>> >>>> >>>> Is there an "official" way to install maker in a custom directory? From mmokrejs at gmail.com Tue Aug 2 10:04:05 2016 From: mmokrejs at gmail.com (=?UTF-8?Q?Martin_MOKREJ=c5=a0?=) Date: Tue, 2 Aug 2016 18:04:05 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> Hi Carson, Carson Holt wrote: > You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. Maybe I was fooled by the install step. > > ?Carson > > >> On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: >> >> Hi Sebastien, >> I had similar concerns in the past as well. >> >> I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. >> >> I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse >> >> From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. >> The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. >> >> Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". >>> Emerging (1 of 1) sci-biology/maker-2.31.8::science * maker-2.31.8.tgz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>> Unpacking source... >>> Unpacking maker-2.31.8.tgz to /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>> Source unpacked in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>> Preparing source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>> Source prepared. >>> Configuring source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>> Source configured. >>> Compiling source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... MAKER supports distributed parallelization via MPI. Would you like to configure MAKER for MPI (This requires that you have an MPI client installed)? [N ]N Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'MAKER' version 'v2.31.8' The file 'Build' has been created for you to finish installing MAKER. ============================================================================== STATUS MAKER v2.31.8 ============================================================================== PERL Dependencies: VERIFIED External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: DISABLED MWAS Web Interface: DISABLED MAKER PACKAGE: CONFIGURATION OK Important Commands: ./Build installdeps #installs missing PERL dependencies ./Build installexes #installs all missing external programs ./Build install #installs MAKER ./Build status #Shows this status menu Other Commands: ./Build repeatmasker #installs RepeatMasker (asks for RepBase) ./Build blast #installs BLAST (NCBI BLAST+) ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) ./Build snap #installs SNAP ./Build augustus #installs Augustus ./Build apollo #installs Apollo ./Build gbrowse #installs GBrowse (must be root) ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) ./Build webapollo #installs WebApollo (use maker2wap to create DBs) ./Build mpich2 #installs MPICH2 (but manual install recommended) Building MAKER Installing MAKER... Building MAKER Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/MAKER/ConfigData.pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/Parallel/Application/MPI.pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/man/MAKER::ConfigData.3pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_fasta_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cufflinks2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2eval_gtf Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/genemark_gtf2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2wap Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_map_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/ipr_update_gff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2jbrowse Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2chado Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/compare Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_tool Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan_wrap Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/evaluator Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2zff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cegma2zff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_merge Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_data_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_fasta Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/chado2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/gff3_merge Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/tophat2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map2assembly Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_gff_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_gff >>> Source compiled. >>> Test phase [not enabled]: sci-biology/maker-2.31.8 >>> Install maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ category sci-biology >>> Completed installing maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ * Final size of build directory: 40312 KiB * Final size of installed tree: 12656 KiB ecompressdir: bzip2 -9 /usr/share/man ecompressdir: bzip2 -9 /usr/share/doc >>> Installing (1 of 1) sci-biology/maker-2.31.8::science >>> Auto-cleaning packages... That gives me these files installed on my system: # equery files maker * Searching for maker ... * Contents of sci-biology/maker-2.31.8: /usr /usr/bin /usr/bin/cegma2zff /usr/bin/chado2gff3 /usr/bin/compare_gff3_to_chado /usr/bin/cufflinks2gff3 /usr/bin/evaluator /usr/bin/fasta_merge /usr/bin/genemark_gtf2gff3 /usr/bin/gff3_merge /usr/bin/ipr_update_gff /usr/bin/iprscan2gff3 /usr/bin/iprscan_wrap /usr/bin/maker /usr/bin/maker2chado /usr/bin/maker2eval_gtf /usr/bin/maker2jbrowse /usr/bin/maker2wap /usr/bin/maker2zff /usr/bin/maker_functional /usr/bin/maker_functional_fasta /usr/bin/maker_functional_gff /usr/bin/maker_map_ids /usr/bin/map2assembly /usr/bin/map_data_ids /usr/bin/map_fasta_ids /usr/bin/map_gff_ids /usr/bin/tophat2gff3 /usr/lib64 /usr/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/vendor_perl/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/MAKER /usr/lib64/perl5/vendor_perl/5.20.2/MAKER/ConfigData.pm /usr/lib64/perl5/vendor_perl/5.20.2/Parallel /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application/MPI.pm /usr/share /usr/share/doc /usr/share/doc/maker-2.31.8 /usr/share/doc/maker-2.31.8/INSTALL.bz2 /usr/share/doc/maker-2.31.8/README.bz2 /usr/share/maker /usr/share/maker/GMOD /usr/share/maker/GMOD/Apollo /usr/share/maker/GMOD/Apollo/gff3.tiers /usr/share/maker/GMOD/JBrowse /usr/share/maker/GMOD/JBrowse/maker.css /usr/share/maker/data /usr/share/maker/data/dpp_contig.fasta /usr/share/maker/data/dpp_est.fasta /usr/share/maker/data/dpp_protein.fasta /usr/share/maker/data/hsap_contig.fasta /usr/share/maker/data/hsap_est.fasta /usr/share/maker/data/hsap_protein.fasta /usr/share/maker/data/te_proteins.fasta /usr/share/man /usr/share/man/man3 /usr/share/man/man3/MAKER::ConfigData.3pm.bz2 $ /usr/bin/maker Argument "2.49_01" isn't numeric in numeric ge (>=) at /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi/forks.pm line 1570. Can't locate GI.pm in @INC (you may need to install the GI module) (@INC contains: /home/mmokrejs/proj/sci/sci-biology/maker/_Inline/lib /usr/bin/../perl/lib /usr/bin/../lib /usr/bin/../src/inc/perl/lib /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/5.20.2 .) at /usr/bin/maker line 63. BEGIN failed--compilation aborted at /usr/bin/maker line 63. $ Of course, because as I said, I need to install properly all the dependencies from maker/lib/ subdirectory. BTW, did perl Build.PL introduce the _Inline/lib/ directory? It somehow escaped from the sanbox I think. :( But thank you for you kind answer anyway, Martin From carsonhh at gmail.com Tue Aug 2 10:09:12 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 10:09:12 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> Message-ID: <57C25EAE-FF0C-4CE6-9000-5B67A7260C28@gmail.com> The install step is installing the C/Perl shared object bindings for MPI to their final location. I could call that internally to ./Build, but since it can fail, and I also since it is configured to fail on missing Perl dependencies, it is better to run it as the final step. ?Carson > On Aug 2, 2016, at 10:04 AM, Martin MOKREJ? wrote: > > Hi Carson, > > Carson Holt wrote: >> You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. > > Maybe I was fooled by the install step. > >> >> ?Carson >> >> >>> On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: >>> >>> Hi Sebastien, >>> I had similar concerns in the past as well. >>> >>> I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. >>> >>> I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse >>> >>> From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. >>> The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. >>> >>> Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". > > >>>> Emerging (1 of 1) sci-biology/maker-2.31.8::science > * maker-2.31.8.tgz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>>> Unpacking source... >>>> Unpacking maker-2.31.8.tgz to /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>>> Source unpacked in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>>> Preparing source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>>> Source prepared. >>>> Configuring source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>>> Source configured. >>>> Compiling source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... > > MAKER supports distributed parallelization via MPI. > Would you like to configure MAKER for MPI (This > requires that you have an MPI client installed)? [N ]N > Created MYMETA.yml and MYMETA.json > Creating new 'Build' script for 'MAKER' version 'v2.31.8' > > > The file 'Build' has been created for you to finish installing MAKER. > > > ============================================================================== > STATUS MAKER v2.31.8 > ============================================================================== > PERL Dependencies: VERIFIED > External Programs: VERIFIED > External C Libraries: VERIFIED > MPI SUPPORT: DISABLED > MWAS Web Interface: DISABLED > MAKER PACKAGE: CONFIGURATION OK > > > Important Commands: > ./Build installdeps #installs missing PERL dependencies > ./Build installexes #installs all missing external programs > ./Build install #installs MAKER > ./Build status #Shows this status menu > > Other Commands: > ./Build repeatmasker #installs RepeatMasker (asks for RepBase) > ./Build blast #installs BLAST (NCBI BLAST+) > ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) > ./Build snap #installs SNAP > ./Build augustus #installs Augustus > ./Build apollo #installs Apollo > ./Build gbrowse #installs GBrowse (must be root) > ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) > ./Build webapollo #installs WebApollo (use maker2wap to create DBs) > ./Build mpich2 #installs MPICH2 (but manual install recommended) > Building MAKER > Installing MAKER... > Building MAKER > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/MAKER/ConfigData.pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/Parallel/Application/MPI.pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/man/MAKER::ConfigData.3pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_fasta_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cufflinks2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2eval_gtf > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/genemark_gtf2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2wap > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_map_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/ipr_update_gff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2jbrowse > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2chado > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/compare > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_tool > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan_wrap > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/evaluator > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2zff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cegma2zff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_merge > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_data_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_fasta > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/chado2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/gff3_merge > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/tophat2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map2assembly > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_gff_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_gff >>>> Source compiled. >>>> Test phase [not enabled]: sci-biology/maker-2.31.8 > >>>> Install maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ category sci-biology >>>> Completed installing maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ > > * Final size of build directory: 40312 KiB > * Final size of installed tree: 12656 KiB > > ecompressdir: bzip2 -9 /usr/share/man > ecompressdir: bzip2 -9 /usr/share/doc > >>>> Installing (1 of 1) sci-biology/maker-2.31.8::science >>>> Auto-cleaning packages... > > > > > That gives me these files installed on my system: > > # equery files maker > * Searching for maker ... > * Contents of sci-biology/maker-2.31.8: > /usr > /usr/bin > /usr/bin/cegma2zff > /usr/bin/chado2gff3 > /usr/bin/compare_gff3_to_chado > /usr/bin/cufflinks2gff3 > /usr/bin/evaluator > /usr/bin/fasta_merge > /usr/bin/genemark_gtf2gff3 > /usr/bin/gff3_merge > /usr/bin/ipr_update_gff > /usr/bin/iprscan2gff3 > /usr/bin/iprscan_wrap > /usr/bin/maker > /usr/bin/maker2chado > /usr/bin/maker2eval_gtf > /usr/bin/maker2jbrowse > /usr/bin/maker2wap > /usr/bin/maker2zff > /usr/bin/maker_functional > /usr/bin/maker_functional_fasta > /usr/bin/maker_functional_gff > /usr/bin/maker_map_ids > /usr/bin/map2assembly > /usr/bin/map_data_ids > /usr/bin/map_fasta_ids > /usr/bin/map_gff_ids > /usr/bin/tophat2gff3 > /usr/lib64 > /usr/lib64/perl5 > /usr/lib64/perl5/vendor_perl > /usr/lib64/perl5/vendor_perl/5.20.2 > /usr/lib64/perl5/vendor_perl/5.20.2/MAKER > /usr/lib64/perl5/vendor_perl/5.20.2/MAKER/ConfigData.pm > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application/MPI.pm > /usr/share > /usr/share/doc > /usr/share/doc/maker-2.31.8 > /usr/share/doc/maker-2.31.8/INSTALL.bz2 > /usr/share/doc/maker-2.31.8/README.bz2 > /usr/share/maker > /usr/share/maker/GMOD > /usr/share/maker/GMOD/Apollo > /usr/share/maker/GMOD/Apollo/gff3.tiers > /usr/share/maker/GMOD/JBrowse > /usr/share/maker/GMOD/JBrowse/maker.css > /usr/share/maker/data > /usr/share/maker/data/dpp_contig.fasta > /usr/share/maker/data/dpp_est.fasta > /usr/share/maker/data/dpp_protein.fasta > /usr/share/maker/data/hsap_contig.fasta > /usr/share/maker/data/hsap_est.fasta > /usr/share/maker/data/hsap_protein.fasta > /usr/share/maker/data/te_proteins.fasta > /usr/share/man > /usr/share/man/man3 > /usr/share/man/man3/MAKER::ConfigData.3pm.bz2 > > > > $ /usr/bin/maker > Argument "2.49_01" isn't numeric in numeric ge (>=) at /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi/forks.pm line 1570. > Can't locate GI.pm in @INC (you may need to install the GI module) (@INC contains: /home/mmokrejs/proj/sci/sci-biology/maker/_Inline/lib /usr/bin/../perl/lib /usr/bin/../lib /usr/bin/../src/inc/perl/lib /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/5.20.2 .) at /usr/bin/maker line 63. > BEGIN failed--compilation aborted at /usr/bin/maker line 63. > $ > > Of course, because as I said, I need to install properly all the dependencies from maker/lib/ subdirectory. > > > BTW, did perl Build.PL introduce the _Inline/lib/ directory? It somehow escaped from the sanbox I think. :( > > But thank you for you kind answer anyway, > Martin From sebastien.moretti at unil.ch Wed Aug 3 03:25:56 2016 From: sebastien.moretti at unil.ch (sebastien.moretti at unil.ch) Date: Wed, 3 Aug 2016 11:25:56 +0200 Subject: [maker-devel] MPI test In-Reply-To: References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: OK, I removed execution rights for mpi_evaluator. I tried mpiexec -n 10 maker MPI tries to write in the maker/perl/ directory. This is an issue for me because maker is installed in a read-only file system. Is there an option or something to tell MPI to write elsewhere? S?bastien > To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. > > Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker > > Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. > > ?Carson > > > >> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: >> >> Hi >> >> I compiled maker with MPI support and would like to test it. >> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >> is missing. >> >> What does mpi_evaluator do? >> How to test properly MPI in maker? >> >> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >> openmpi 1.8.1 >> >> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From carsonhh at gmail.com Wed Aug 3 10:16:57 2016 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 3 Aug 2016 10:16:57 -0600 Subject: [maker-devel] MPI test In-Reply-To: References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: <72BDD9C3-9E02-4394-AE1B-521411F9B86D@gmail.com> The issue is that you moved the ?/maker/perl directory after initial setup. So when maker runs it can?t find the files, and tries to rebuild the shared objects for C/Perl MPI binding that are missing (compiles on demand). Solutions (one of the following two options) 1. Install maker as indicated in the INTALL file and then append the path for ?/maker/bin/ to the PATH environmental variable in /etc/profile (i.e. export PATH=/usr/local/maker/bin:$PATH). 2. Install maker as indicated in the INTALL file and then soft link the executables in ?/maker/bin/ to the location of your choice (i.e. /usr/bin/ or /usr/local/bin/ - this is what homebrew does for example). Example: MacBook-Pro:~ cholt$ ls -al /usr/local/bin/maker lrwxr-xr-x 1 cholt admin 32 Oct 25 2015 /usr/local/bin/maker -> ../Cellar/maker/2.31.8/bin/maker Notice that the softink in /usr/local/bin/ is pointing to the actual location where homebrew placed the installation. Doing something similar would be better than you trying to break up the installation structure. Basically they move the contents of ?/maker/* to /usr/local/Cellar/maker/2.31.8/, then they performed all installation steps there, and then softlinked the executables elsewhere using ?ln -s' When you install maker, you have to run the setup steps in ?/maker/src/. After install, ?/maker/bin/ will contain executables and ?/maker/perl/ and ?/maker/lib/ will contain libraries and shared objects used by those executables. Remember the executables are not binaries, they are scripts, so they need access to the libraries and shared objects used by the scripts. The executables know to look for libraries/files in the installation folder, but if you move them, the @INC search list used by perl will not find them. You can try and manually declare new locations to search in the @INC list using the PERL5LIB environmental variable (but using PERL5LIB is beyond the scope of the maker-devel list and can be investigated using Perl?s own docs). Also as explained in the homebrew example above, the base directory does not have to be named ?maker?. You can name it whatever you want. It is just the installation structure within the containing directory that has to be consistent. Alternatively if you are doing something more complex like a ?module? setup using lua based modules just use the prepend_path command to add the .../maker/bin foldert to the environment on demand ?> Example: -- -*- lua -*- help( [[ This module loads the MAKER genome annotation pipeline ]]) whatis("Name: MAKER") whatis("Version: 2.31.8") whatis("Category: Gene Annotation") whatis("Keywords: Gene Annotation") whatis("URL: http://www.yandell-lab.org/software/maker.html") whatis("Description: MAKER is a highly parallelized genome annotation/re-annotation pipeline") -- Load required modules load("openmpi/1.8.4.i", "genemark-ES", "augustus", "snap", "exonerate", "ncbi-blast+", "RepeatMasker") -- Prepare needed values local APPS = "/ucgd/apps/" local id = "maker" local version = "2.31.8" local base = pathJoin(APPS, id, version) local bin = pathJoin(base, "bin") -- Set PATH and other environmental variables prepend_path("PATH", bin) prepend_path("LD_PRELOAD", "/ucgd/apps/openmpi/1.8.4i/lib/libmpi.so") setenv("OMPI_MCA_mpi_warn_on_fork", 0) ?Carson > On Aug 3, 2016, at 3:25 AM, sebastien.moretti at unil.ch wrote: > > OK, I removed execution rights for mpi_evaluator. > > I tried mpiexec -n 10 maker > MPI tries to write in the maker/perl/ directory. > This is an issue for me because maker is installed in a read-only file > system. > > Is there an option or something to tell MPI to write elsewhere? > > S?bastien > >> To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. >> >> Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker >> >> Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. >> >> ?Carson >> >> >> >>> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I compiled maker with MPI support and would like to test it. >>> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >>> is missing. >>> >>> What does mpi_evaluator do? >>> How to test properly MPI in maker? >>> >>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>> openmpi 1.8.1 >>> >>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Aug 4 14:21:36 2016 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 4 Aug 2016 14:21:36 -0600 Subject: [maker-devel] MPI test In-Reply-To: <9e01f987-d284-f73b-63c0-e2d84a87cb27@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> <72BDD9C3-9E02-4394-AE1B-521411F9B86D@gmail.com> <9e01f987-d284-f73b-63c0-e2d84a87cb27@unil.ch> Message-ID: Hi Sebastien, As long as you are not following the ?/maker/INSTALL instructions, are instead moving files around manually, and trying to modify the default behavior of ./Build I can not provide support. I?ve tried to best explain how you should be doing things in the previous e-mails as well as the information below. If you are installing for a module based setup, then the proper way to do this would be to move the maker tarball to the desired module location, then follow the INSTALL instructions. Example (bent for a module based setup since you said that is what you are doing): #load required tools module load snap module load openmpi module load blast module load exonerate module load RepeatMasker #set up environment for OpenMPI install (explained in ?/maker/INSTALL instructions) export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so #download and untar maker into desired location mkdir $APPS/maker cd $APPS/maker wget http://maker_url/maker.tgz tar -zxvf maker.tgz mv maker 2.31.8 #go to src directory cd $APPS/maker/2.31.8/src #install according to ?/maker/INSTALL instructions perl Build.PL ./Build installdeps ./Build installexes ./Build ./Build install #test normal installation $APPS/maker/2.31.8/bin/maker --help #test MPI installation (will print only one help message - something is wrong if it prints 5) mpiexec -mca btl ^openib -n 5 $APPS/maker/2.31.8/bin/maker --help #then setup a module file for maker that includes: 1. steps for loading prerequisite tools and modules (such as openmpi) 2. setting OMPI_MCA_mpi_warn_on_fork=0 3. setting LD_PRELOAD=/location/of/openmpi/lib/libmpi.so 4. prepend path for $APPS/maker/2.31.8/bin to PATH environmental variable Thanks, Carson > On Aug 4, 2016, at 8:28 AM, Sebastien Moretti wrote: > > I already use a module to load maker and external tools ;-) > As well as to set OMPI_MCA_mpi_warn_on_fork = 0 > > > In my case perl/config-x86_64-linux-thread-multi-5.018002 was created > properly but not copied. > I use ./Build install --destdir= and the destdir value is not > reported at the end of the install process when > blib/config-x86_64-linux-thread-multi-5.018002 is copied to ../perl/ > > I manually copy blib/config-x86_64-linux-thread-multi-5.018002 to > my_path/../perl/ and MPI does not try to write it now. > > > > Now I have issues when MPI tries to send messages. > Don't know if you can help me with this. > Here is part of the error message I got: > > [devfrt01.XXX:10256] mca: base: component_find: unable to open > /software/lib64/openmpi/lib/openmpi/mca_shmem_mmap: perhaps a missing > symbol, or compiled for a different version of Open MPI? (ignored) > ... > > It looks like opal_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during opal_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_shmem_base_select failed > --> Returned value -1 instead of OPAL_SUCCESS > ... > It looks like orte_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during orte_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_init failed > --> Returned value Error (-1) instead of ORTE_SUCCESS > ... > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10252] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10256] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10253] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10257] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10251] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > ------------------------------------------------------- > Primary job terminated normally, but 1 process returned > a non-zero exit code.. Per user-direction, the job has been aborted. > ------------------------------------------------------- > -------------------------------------------------------------------------- > mpiexec detected that one or more processes exited with non-zero status, > thus causing > the job to be terminated. The first process to do so was: > > Process name: [[21120,1],1] > Exit code: 1 > > >> The issue is that you moved the ?/maker/perl directory after initial >> setup. So when maker runs it can?t find the files, and tries to rebuild >> the shared objects for C/Perl MPI binding that are missing (compiles on >> demand). >> >> Solutions (one of the following two options) >> 1. Install maker as indicated in the INTALL file and then append the >> path for ?/maker/bin/ to the PATH environmental variable in /etc/profile >> (i.e. export PATH=/usr/local/maker/bin:$PATH). >> 2. Install maker as indicated in the INTALL file and then soft link the >> executables in ?/maker/bin/ to the location of your choice (i.e. >> /usr/bin/ or /usr/local/bin/ - this is what homebrew does for example). >> >> Example: >> MacBook-Pro:~ cholt$ ls -al /usr/local/bin/maker >> lrwxr-xr-x 1 cholt admin 32 Oct 25 2015 /usr/local/bin/maker -> >> ../Cellar/maker/2.31.8/bin/maker >> >> Notice that the softink in /usr/local/bin/ is pointing to the actual >> location where homebrew placed the installation. Doing something similar >> would be better than you trying to break up the installation structure. >> Basically they move the contents of ?/maker/* to >> /usr/local/Cellar/maker/2.31.8/, then they performed all installation >> steps there, and then softlinked the executables elsewhere using ?ln -s' >> >> >> When you install maker, you have to run the setup steps in ?/maker/src/. >> After install, ?/maker/bin/ will contain executables and ?/maker/perl/ >> and ?/maker/lib/ will contain libraries and shared objects used by those >> executables. Remember the executables are not binaries, they are >> scripts, so they need access to the libraries and shared objects used by >> the scripts. The executables know to look for libraries/files in the >> installation folder, but if you move them, the @INC search list used by >> perl will not find them. You can try and manually declare new locations >> to search in the @INC list using the PERL5LIB environmental variable >> (but using PERL5LIB is beyond the scope of the maker-devel list and can >> be investigated using Perl?s own docs). >> >> Also as explained in the homebrew example above, the base directory does >> not have to be named ?maker?. You can name it whatever you want. It is >> just the installation structure within the containing directory that has >> to be consistent. >> >> Alternatively if you are doing something more complex like a ?module? >> setup using lua based modules just use the prepend_path command to add >> the .../maker/bin foldert to the environment on demand ?> >> >> Example: >> >> -- -*- lua -*- >> help( >> [[ >> This module loads the MAKER genome annotation pipeline >> ]]) >> >> whatis("Name: MAKER") >> whatis("Version: 2.31.8") >> whatis("Category: Gene Annotation") >> whatis("Keywords: Gene Annotation") >> whatis("URL: http://www.yandell-lab.org/software/maker.html") >> whatis("Description: MAKER is a highly parallelized genome >> annotation/re-annotation pipeline") >> >> -- Load required modules >> load("openmpi/1.8.4.i", "genemark-ES", "augustus", "snap", "exonerate", >> "ncbi-blast+", "RepeatMasker") >> >> -- Prepare needed values >> local APPS = "/ucgd/apps/" >> local id = "maker" >> local version = "2.31.8" >> local base = pathJoin(APPS, id, version) >> local bin = pathJoin(base, "bin") >> >> -- Set PATH and other environmental variables >> prepend_path("PATH", bin) >> prepend_path("LD_PRELOAD", "/ucgd/apps/openmpi/1.8.4i/lib/libmpi.so") >> setenv("OMPI_MCA_mpi_warn_on_fork", 0) >> >> >> ?Carson >> >> >>> On Aug 3, 2016, at 3:25 AM, sebastien.moretti at unil.ch >>> wrote: >>> >>> OK, I removed execution rights for mpi_evaluator. >>> >>> I tried mpiexec -n 10 maker >>> MPI tries to write in the maker/perl/ directory. >>> This is an issue for me because maker is installed in a read-only file >>> system. >>> >>> Is there an option or something to tell MPI to write elsewhere? >>> >>> S?bastien >>> >>>> To run MAKER with MPI, you just call maker with mpiexec. Examples can >>>> be found on the MAKER wiki, in the devel list archives, and in the >>>> README/INSTALL guide. >>>> >>>> Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker >>>> >>>> Any other MPI options will depend on the flavor of MPI you use, and >>>> your cluster settings. >>>> >>>> ?Carson >>>> >>>> >>>> >>>>> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti >>>>> > wrote: >>>>> >>>>> Hi >>>>> >>>>> I compiled maker with MPI support and would like to test it. >>>>> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >>>>> is missing. >>>>> >>>>> What does mpi_evaluator do? >>>>> How to test properly MPI in maker? >>>>> >>>>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>>>> openmpi 1.8.1 >>>>> >>>>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From lassance at fas.harvard.edu Tue Aug 30 07:19:13 2016 From: lassance at fas.harvard.edu (Lassance, Jean-Marc) Date: Tue, 30 Aug 2016 13:19:13 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats Message-ID: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Tue Aug 30 10:16:37 2016 From: dence at genetics.utah.edu (Daniel Ence) Date: Tue, 30 Aug 2016 16:16:37 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Message-ID: Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? Thanks, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From me.mark at gmail.com Tue Aug 30 10:10:16 2016 From: me.mark at gmail.com (Mark Ebbert) Date: Tue, 30 Aug 2016 09:10:16 -0700 Subject: [maker-devel] (no subject) Message-ID: <57c5af23a842850000919b43@polymail.io> Good day everyone! I?m getting the error stating: ?WARNING: Multiple MAKER processes have been started in the same directory.? Everything I?ve seen mentions version issues with MPICH. The difference in my situation is that my initial run ran just fine, but died because of the cluster time constraints. We?re only allowed 3 days.? There are a bunch of .NFSLock files in the output directory. I?m guessing Maker wasn?t able to clear the locks when the jobs died? Can I safely delete those lock files? What?s the best way to handle this going forward since I can only run jobs for 3 days at a time? Thanks! Mark T. W. Ebbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 30 10:54:39 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 30 Aug 2016 10:54:39 -0600 Subject: [maker-devel] (no subject) In-Reply-To: <57c5af23a842850000919b43@polymail.io> References: <57c5af23a842850000919b43@polymail.io> Message-ID: Run 'maker -help? with mpiexec. Example: mpiexec -n 10 maker -help If the MPI communication ring is working correctly, then it will print the help message only once (from the root process). If it is not working, it will print the help message 10 time because each of the 10 MPI processes will think they are the root process. It is a simple test that can identify if it is an MPI issue or not. If it is not an MPI issue, you can just search for the NFSLock files using find and delete them,. ?Carson > On Aug 30, 2016, at 10:10 AM, Mark Ebbert wrote: > > > Good day everyone! > > I?m getting the error stating: ?WARNING: Multiple MAKER processes have been started in the same directory.? Everything I?ve seen mentions version issues with MPICH. The difference in my situation is that my initial run ran just fine, but died because of the cluster time constraints. We?re only allowed 3 days. > > There are a bunch of .NFSLock files in the output directory. I?m guessing Maker wasn?t able to clear the locks when the jobs died? Can I safely delete those lock files? What?s the best way to handle this going forward since I can only run jobs for 3 days at a time? > > Thanks! > > Mark T. W. Ebbert > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 30 14:12:44 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 30 Aug 2016 14:12:44 -0600 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Message-ID: <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> Also make sure you pass the data in using rm_gff and not maker_gff if the repeats were not MAKER generated. ?Carson > On Aug 30, 2016, at 10:16 AM, Daniel Ence wrote: > > Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? > > And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? > > Thanks, > Daniel > > > Daniel Ence > Graduate Student > Eccles Institute of Human Genetics > University of Utah > 15 North 2030 East, Room 2100 > Salt Lake City, UT 84112-5330 > >> On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: >> >> Hi. >> >> I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). >> >> Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: >> >> rm_gff=myanimal.repeatmasker.out.gff3 >> >> #-----Re-annotation Using MAKER Derived GFF3 >> maker_gff=myanimal.all.maker.pass1.gff >> rm_pass=1 >> >> Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: >> >> Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. >> --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu >> ERROR: Failed while processing all repeats >> ERROR: Chunk failed at level:3, tier_type:1 >> FAILED CONTIG:scaffold00013 >> >> I wonder if you have an idea of what could be wrong here. >> >> Thanks for your help, >> >> >> Jean-Marc >> >> ?????????????????? >> Jean-Marc Lassance, PhD >> >> Harvard University >> Department of Organismic and Evolutionary Biology >> Department of Molecular and Cellular Biology >> Museum of Comparative Zoology >> >> 26, Oxford Street >> Cambridge MA 02138 >> USA >> >> email: lassance at fas.harvard.edu >> twitter: @lassancejm >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lassance at fas.harvard.edu Tue Aug 30 14:31:44 2016 From: lassance at fas.harvard.edu (Lassance, Jean-Marc) Date: Tue, 30 Aug 2016 20:31:44 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> Message-ID: <1F3B92BC-1717-4CB5-A26D-6F2126667E53@fas.harvard.edu> Let me clarify one thing: the first pass was performed with Maker running repeatMasker internally, which is why I decided to use them in the second pass, as well as the data from the independent run of RepeatMasker. From reading earlier posts, I gathered that Maker would use first the evidence from the rm_gff, and then from maker_gff if rm_pass=1 is activated, but that having both would not hurt. Correct? JM On Aug 30, 2016, at 4:12 PM, Carson Holt > wrote: Also make sure you pass the data in using rm_gff and not maker_gff if the repeats were not MAKER generated. ?Carson On Aug 30, 2016, at 10:16 AM, Daniel Ence > wrote: Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? Thanks, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm -------------- next part -------------- An HTML attachment was scrubbed... URL: From psh65 at cornell.edu Wed Aug 31 08:35:28 2016 From: psh65 at cornell.edu (Prashant S Hosmani) Date: Wed, 31 Aug 2016 14:35:28 +0000 Subject: [maker-devel] mapping cDNA to updated genome Message-ID: Hi All, I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. Thank you for your help in advance. Prashant Prashant Hosmani Sol Genomics Network Boyce Thompson Institute, Ithaca, NY, USA -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Wed Aug 31 10:10:22 2016 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Wed, 31 Aug 2016 12:10:22 -0400 Subject: [maker-devel] mapping cDNA to updated genome In-Reply-To: References: Message-ID: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> Hi Prashant, I?m almost positive that the additional genes are coming from multiply aligning cDNAs. Did you repeat mask your genome before mapping things forward? Another thought, what kind of whole genome duplications has your plant been through. it may be that the multiple alignments are to pseudogenes is some stage of decay. If that is the case it would probably be safe to keep the the gene from longest/best aligned cDNA. Thanks, Mike > On Aug 31, 2016, at 10:35 AM, Prashant S Hosmani wrote: > > Hi All, > > I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. > > I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. > > Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. > > Thank you for your help in advance. > Prashant > > > Prashant Hosmani > Sol Genomics Network > Boyce Thompson Institute, Ithaca, NY, USA > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Aug 31 10:12:53 2016 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 31 Aug 2016 10:12:53 -0600 Subject: [maker-devel] mapping cDNA to updated genome In-Reply-To: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> References: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> Message-ID: <9FBCB1C4-C319-4933-8741-53DAFCB82458@gmail.com> Also if you have multiple alignments of the same cDNA, you can use the score column of the mRNA feature to see which aligns best. If they have the same score, you will have to disambiguate manually or just remove all copies. ?Carson > On Aug 31, 2016, at 10:10 AM, Michael Campbell wrote: > > Hi Prashant, > > I?m almost positive that the additional genes are coming from multiply aligning cDNAs. Did you repeat mask your genome before mapping things forward? > > Another thought, what kind of whole genome duplications has your plant been through. it may be that the multiple alignments are to pseudogenes is some stage of decay. If that is the case it would probably be safe to keep the the gene from longest/best aligned cDNA. > > Thanks, > Mike >> On Aug 31, 2016, at 10:35 AM, Prashant S Hosmani > wrote: >> >> Hi All, >> >> I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. >> >> I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. >> >> Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. >> >> Thank you for your help in advance. >> Prashant >> >> >> Prashant Hosmani >> Sol Genomics Network >> Boyce Thompson Institute, Ithaca, NY, USA >> >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 2 08:08:41 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:08:41 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> If you move it, then the executables won?t be able to locate dependencies in the ?/maker/data, ?/maker/lib, and ?/maker/perl directories. You should really either add the location of ?/maker/bin to you PATH environmental variable or at most soft link the executables somewhere else using the ?ln -s? command. ?Carson > On Aug 2, 2016, at 5:57 AM, Sebastien Moretti wrote: > > OK, just need to move the maker/bin folder. > > Thanks > >> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >> >> ?Carson >> >> >> >>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I try to install maker in a custom directory following regular Perl >>> commands with Build.PL. >>> >>> Unfortunately if I use >>> perl Build.PL --installdirs= >>> then >>> ./Build install --destdir= >>> build install fails with the error: >>> Can't use an undefined value as a HASH reference at >>> .../Module/Build/Base.pm line 3038. >>> >>> It does not happen if I don't use --installdirs >>> >>> >>> Is there an "official" way to install maker in a custom directory? >>> >>> >>> Also I found typos in the description of maker at >>> http://www.yandell-lab.org/software/maker.html >>> ouputs should be outputs >>> seusequent should be ??? >>> >>> >>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>> >>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 05:57:32 2016 From: sebastien.moretti at unil.ch (Sebastien Moretti) Date: Tue, 2 Aug 2016 13:57:32 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: OK, just need to move the maker/bin folder. Thanks > MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. > > ?Carson > > > >> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >> >> Hi >> >> I try to install maker in a custom directory following regular Perl >> commands with Build.PL. >> >> Unfortunately if I use >> perl Build.PL --installdirs= >> then >> ./Build install --destdir= >> build install fails with the error: >> Can't use an undefined value as a HASH reference at >> .../Module/Build/Base.pm line 3038. >> >> It does not happen if I don't use --installdirs >> >> >> Is there an "official" way to install maker in a custom directory? >> >> >> Also I found typos in the description of maker at >> http://www.yandell-lab.org/software/maker.html >> ouputs should be outputs >> seusequent should be ??? >> >> >> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >> >> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 06:00:53 2016 From: sebastien.moretti at unil.ch (Sebastien Moretti) Date: Tue, 2 Aug 2016 14:00:53 +0200 Subject: [maker-devel] MPI test Message-ID: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Hi I compiled maker with MPI support and would like to test it. So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm is missing. What does mpi_evaluator do? How to test properly MPI in maker? maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 openmpi 1.8.1 Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From sebastien.moretti at unil.ch Tue Aug 2 08:14:23 2016 From: sebastien.moretti at unil.ch (sebastien.moretti at unil.ch) Date: Tue, 2 Aug 2016 16:14:23 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> <66EE1C24-A796-4F36-BE56-D77443EE70D8@gmail.com> Message-ID: <196799cd-d370-a343-f87c-1292e58593a4@unil.ch> I moved maker/bin/, maker/perl/, maker/data/, maker/lib/ + LICENSE README RELEASE and it looks to be fine. Thanks > If you move it, then the executables won?t be able to locate dependencies in the ?/maker/data, ?/maker/lib, and ?/maker/perl directories. You should really either add the location of ?/maker/bin to you PATH environmental variable or at most soft link the executables somewhere else using the ?ln -s? command. > > ?Carson > > >> On Aug 2, 2016, at 5:57 AM, Sebastien Moretti wrote: >> >> OK, just need to move the maker/bin folder. >> >> Thanks >> >>> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >>> >>> ?Carson >>> >>> >>> >>>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>>> >>>> Hi >>>> >>>> I try to install maker in a custom directory following regular Perl >>>> commands with Build.PL. >>>> >>>> Unfortunately if I use >>>> perl Build.PL --installdirs= >>>> then >>>> ./Build install --destdir= >>>> build install fails with the error: >>>> Can't use an undefined value as a HASH reference at >>>> .../Module/Build/Base.pm line 3038. >>>> >>>> It does not happen if I don't use --installdirs >>>> >>>> >>>> Is there an "official" way to install maker in a custom directory? >>>> >>>> >>>> Also I found typos in the description of maker at >>>> http://www.yandell-lab.org/software/maker.html >>>> ouputs should be outputs >>>> seusequent should be ??? >>>> >>>> >>>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>>> >>>> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From carsonhh at gmail.com Tue Aug 2 08:19:35 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:19:35 -0600 Subject: [maker-devel] MPI test In-Reply-To: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: <97587487-EAB0-461E-8DB1-2E23FA3E3C4C@gmail.com> mpi_evaluator is a maker development related accessory script. You should ignore it. ?Carson > On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: > > Hi > > I compiled maker with MPI support and would like to test it. > So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm > is missing. > > What does mpi_evaluator do? > How to test properly MPI in maker? > > maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 > openmpi 1.8.1 > > Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From carsonhh at gmail.com Tue Aug 2 08:59:40 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 08:59:40 -0600 Subject: [maker-devel] MPI test In-Reply-To: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. ?Carson > On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: > > Hi > > I compiled maker with MPI support and would like to test it. > So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm > is missing. > > What does mpi_evaluator do? > How to test properly MPI in maker? > > maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 > openmpi 1.8.1 > > Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org From mmokrejs at gmail.com Tue Aug 2 09:29:49 2016 From: mmokrejs at gmail.com (=?UTF-8?Q?Martin_MOKREJ=c5=a0?=) Date: Tue, 2 Aug 2016 17:29:49 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: Hi Sebastien, I had similar concerns in the past as well. I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". Martin Sebastien Moretti wrote: > OK, just need to move the maker/bin folder. > > Thanks > >> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >> >> ?Carson >> >> >> >>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I try to install maker in a custom directory following regular Perl >>> commands with Build.PL. >>> >>> Unfortunately if I use >>> perl Build.PL --installdirs= >>> then >>> ./Build install --destdir= >>> build install fails with the error: >>> Can't use an undefined value as a HASH reference at >>> .../Module/Build/Base.pm line 3038. >>> >>> It does not happen if I don't use --installdirs >>> >>> >>> Is there an "official" way to install maker in a custom directory? From carsonhh at gmail.com Tue Aug 2 09:51:25 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 09:51:25 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. ?Carson > On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: > > Hi Sebastien, > I had similar concerns in the past as well. > > I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. > > I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse > > From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. > The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. > > Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". > > Martin > > > Sebastien Moretti wrote: >> OK, just need to move the maker/bin folder. >> >> Thanks >> >>> MAKER will by default put executables in ?/maker/bin. You should put ?/maker/bin into your PATH. The maker Build.PL is configured to install things in a set directory configuration to properly locate perl libraries, C shared objects, and data files. >>> >>> ?Carson >>> >>> >>> >>>> On Jul 12, 2016, at 2:35 AM, Sebastien Moretti wrote: >>>> >>>> Hi >>>> >>>> I try to install maker in a custom directory following regular Perl >>>> commands with Build.PL. >>>> >>>> Unfortunately if I use >>>> perl Build.PL --installdirs= >>>> then >>>> ./Build install --destdir= >>>> build install fails with the error: >>>> Can't use an undefined value as a HASH reference at >>>> .../Module/Build/Base.pm line 3038. >>>> >>>> It does not happen if I don't use --installdirs >>>> >>>> >>>> Is there an "official" way to install maker in a custom directory? From mmokrejs at gmail.com Tue Aug 2 10:04:05 2016 From: mmokrejs at gmail.com (=?UTF-8?Q?Martin_MOKREJ=c5=a0?=) Date: Tue, 2 Aug 2016 18:04:05 +0200 Subject: [maker-devel] Installation in a custom directory In-Reply-To: References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> Message-ID: <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> Hi Carson, Carson Holt wrote: > You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. Maybe I was fooled by the install step. > > ?Carson > > >> On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: >> >> Hi Sebastien, >> I had similar concerns in the past as well. >> >> I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. >> >> I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse >> >> From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. >> The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. >> >> Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". >>> Emerging (1 of 1) sci-biology/maker-2.31.8::science * maker-2.31.8.tgz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>> Unpacking source... >>> Unpacking maker-2.31.8.tgz to /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>> Source unpacked in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>> Preparing source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>> Source prepared. >>> Configuring source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>> Source configured. >>> Compiling source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... MAKER supports distributed parallelization via MPI. Would you like to configure MAKER for MPI (This requires that you have an MPI client installed)? [N ]N Created MYMETA.yml and MYMETA.json Creating new 'Build' script for 'MAKER' version 'v2.31.8' The file 'Build' has been created for you to finish installing MAKER. ============================================================================== STATUS MAKER v2.31.8 ============================================================================== PERL Dependencies: VERIFIED External Programs: VERIFIED External C Libraries: VERIFIED MPI SUPPORT: DISABLED MWAS Web Interface: DISABLED MAKER PACKAGE: CONFIGURATION OK Important Commands: ./Build installdeps #installs missing PERL dependencies ./Build installexes #installs all missing external programs ./Build install #installs MAKER ./Build status #Shows this status menu Other Commands: ./Build repeatmasker #installs RepeatMasker (asks for RepBase) ./Build blast #installs BLAST (NCBI BLAST+) ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) ./Build snap #installs SNAP ./Build augustus #installs Augustus ./Build apollo #installs Apollo ./Build gbrowse #installs GBrowse (must be root) ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) ./Build webapollo #installs WebApollo (use maker2wap to create DBs) ./Build mpich2 #installs MPICH2 (but manual install recommended) Building MAKER Installing MAKER... Building MAKER Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/MAKER/ConfigData.pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/Parallel/Application/MPI.pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/man/MAKER::ConfigData.3pm Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_fasta_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cufflinks2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2eval_gtf Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/genemark_gtf2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2wap Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_map_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/ipr_update_gff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2jbrowse Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2chado Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/compare Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_tool Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan_wrap Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/evaluator Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2zff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cegma2zff Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_merge Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_data_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_fasta Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/chado2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/gff3_merge Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/tophat2gff3 Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map2assembly Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_gff_ids Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_gff >>> Source compiled. >>> Test phase [not enabled]: sci-biology/maker-2.31.8 >>> Install maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ category sci-biology >>> Completed installing maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ * Final size of build directory: 40312 KiB * Final size of installed tree: 12656 KiB ecompressdir: bzip2 -9 /usr/share/man ecompressdir: bzip2 -9 /usr/share/doc >>> Installing (1 of 1) sci-biology/maker-2.31.8::science >>> Auto-cleaning packages... That gives me these files installed on my system: # equery files maker * Searching for maker ... * Contents of sci-biology/maker-2.31.8: /usr /usr/bin /usr/bin/cegma2zff /usr/bin/chado2gff3 /usr/bin/compare_gff3_to_chado /usr/bin/cufflinks2gff3 /usr/bin/evaluator /usr/bin/fasta_merge /usr/bin/genemark_gtf2gff3 /usr/bin/gff3_merge /usr/bin/ipr_update_gff /usr/bin/iprscan2gff3 /usr/bin/iprscan_wrap /usr/bin/maker /usr/bin/maker2chado /usr/bin/maker2eval_gtf /usr/bin/maker2jbrowse /usr/bin/maker2wap /usr/bin/maker2zff /usr/bin/maker_functional /usr/bin/maker_functional_fasta /usr/bin/maker_functional_gff /usr/bin/maker_map_ids /usr/bin/map2assembly /usr/bin/map_data_ids /usr/bin/map_fasta_ids /usr/bin/map_gff_ids /usr/bin/tophat2gff3 /usr/lib64 /usr/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/vendor_perl/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/MAKER /usr/lib64/perl5/vendor_perl/5.20.2/MAKER/ConfigData.pm /usr/lib64/perl5/vendor_perl/5.20.2/Parallel /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application/MPI.pm /usr/share /usr/share/doc /usr/share/doc/maker-2.31.8 /usr/share/doc/maker-2.31.8/INSTALL.bz2 /usr/share/doc/maker-2.31.8/README.bz2 /usr/share/maker /usr/share/maker/GMOD /usr/share/maker/GMOD/Apollo /usr/share/maker/GMOD/Apollo/gff3.tiers /usr/share/maker/GMOD/JBrowse /usr/share/maker/GMOD/JBrowse/maker.css /usr/share/maker/data /usr/share/maker/data/dpp_contig.fasta /usr/share/maker/data/dpp_est.fasta /usr/share/maker/data/dpp_protein.fasta /usr/share/maker/data/hsap_contig.fasta /usr/share/maker/data/hsap_est.fasta /usr/share/maker/data/hsap_protein.fasta /usr/share/maker/data/te_proteins.fasta /usr/share/man /usr/share/man/man3 /usr/share/man/man3/MAKER::ConfigData.3pm.bz2 $ /usr/bin/maker Argument "2.49_01" isn't numeric in numeric ge (>=) at /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi/forks.pm line 1570. Can't locate GI.pm in @INC (you may need to install the GI module) (@INC contains: /home/mmokrejs/proj/sci/sci-biology/maker/_Inline/lib /usr/bin/../perl/lib /usr/bin/../lib /usr/bin/../src/inc/perl/lib /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/5.20.2 .) at /usr/bin/maker line 63. BEGIN failed--compilation aborted at /usr/bin/maker line 63. $ Of course, because as I said, I need to install properly all the dependencies from maker/lib/ subdirectory. BTW, did perl Build.PL introduce the _Inline/lib/ directory? It somehow escaped from the sanbox I think. :( But thank you for you kind answer anyway, Martin From carsonhh at gmail.com Tue Aug 2 10:09:12 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 2 Aug 2016 10:09:12 -0600 Subject: [maker-devel] Installation in a custom directory In-Reply-To: <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> References: <9A2E906F-F0B9-4CB3-9C18-CA57F4213D55@gmail.com> <47d657b8-4ae4-5e50-e374-67030156ec19@gmail.com> Message-ID: <57C25EAE-FF0C-4CE6-9000-5B67A7260C28@gmail.com> The install step is installing the C/Perl shared object bindings for MPI to their final location. I could call that internally to ./Build, but since it can fail, and I also since it is configured to fail on missing Perl dependencies, it is better to run it as the final step. ?Carson > On Aug 2, 2016, at 10:04 AM, Martin MOKREJ? wrote: > > Hi Carson, > > Carson Holt wrote: >> You are probably thinking of times you have used Build.PL for installing modules (primarily from CPAN). MAKER is not a module. You should really think of Build.PL as being akin to a Perl version of ?make?. Just like ?make', Build.PL can be configured to do the necessary configuration/setup for nearly anything. In this case, it?s not installing a module, rather Build.PL is just configuring external dependencies for a package of scripts. So the options you are trying to configure for have no meaning given how Build.PL is set up in this case. > > Maybe I was fooled by the install step. > >> >> ?Carson >> >> >>> On Aug 2, 2016, at 9:29 AM, Martin MOKREJ? wrote: >>> >>> Hi Sebastien, >>> I had similar concerns in the past as well. >>> >>> I also would like to install it elsewhere, but usually Build.PL works with other packages. This is not the case of maker. >>> >>> I would also would need two or three commandline switches: --destdir=${DESTDIR} --prefix=${EPREFIX}/usr --perl-inc-path=${EPREFIX}/${VENDOR_LIB} --perl-man-path=$somewhereelse >>> >>> From the above, the ${EPREFIX}/usr could be compiled into binaries, config files, etc. >>> The ${DESTDIR} should contain subdirectories /usr, /usr/lib, /usr/share, etc. so that a script could do: cd /my/temp/path; tar cf - . | maker.tgz and later unpack it over my / (root directory) of the filesystem. >>> >>> Then I could finish my abandoned attempt to make a package for maker in Gentoo Linux ( https://cgit.gentoo.org/proj/sci.git/diff/sci-biology/maker/maker-2.31.8.ebuild ), exactly because of these issues. ;-) As you may see, I am especially stuck with what to do with maker/lib/ contents, seems like pre-installed perl apps. I need to create a package for each of them separately before proceeding with "rm -rf maker/lib". > > >>>> Emerging (1 of 1) sci-biology/maker-2.31.8::science > * maker-2.31.8.tgz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>>> Unpacking source... >>>> Unpacking maker-2.31.8.tgz to /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>>> Source unpacked in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work >>>> Preparing source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>>> Source prepared. >>>> Configuring source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... >>>> Source configured. >>>> Compiling source in /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src ... > > MAKER supports distributed parallelization via MPI. > Would you like to configure MAKER for MPI (This > requires that you have an MPI client installed)? [N ]N > Created MYMETA.yml and MYMETA.json > Creating new 'Build' script for 'MAKER' version 'v2.31.8' > > > The file 'Build' has been created for you to finish installing MAKER. > > > ============================================================================== > STATUS MAKER v2.31.8 > ============================================================================== > PERL Dependencies: VERIFIED > External Programs: VERIFIED > External C Libraries: VERIFIED > MPI SUPPORT: DISABLED > MWAS Web Interface: DISABLED > MAKER PACKAGE: CONFIGURATION OK > > > Important Commands: > ./Build installdeps #installs missing PERL dependencies > ./Build installexes #installs all missing external programs > ./Build install #installs MAKER > ./Build status #Shows this status menu > > Other Commands: > ./Build repeatmasker #installs RepeatMasker (asks for RepBase) > ./Build blast #installs BLAST (NCBI BLAST+) > ./Build exonerate #installs Exonerate (v2 on UNIX / v1 on Mac OSX) > ./Build snap #installs SNAP > ./Build augustus #installs Augustus > ./Build apollo #installs Apollo > ./Build gbrowse #installs GBrowse (must be root) > ./Build jbrowse #installs JBrowse (MAKER copy, not web accecible) > ./Build webapollo #installs WebApollo (use maker2wap to create DBs) > ./Build mpich2 #installs MPICH2 (but manual install recommended) > Building MAKER > Installing MAKER... > Building MAKER > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/MAKER/ConfigData.pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/lib/Parallel/Application/MPI.pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../perl/man/MAKER::ConfigData.3pm > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_fasta_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cufflinks2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2eval_gtf > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/genemark_gtf2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2wap > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_map_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/ipr_update_gff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2jbrowse > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2chado > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/compare > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_tool > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/iprscan_wrap > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/evaluator > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker2zff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/cegma2zff > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/fasta_merge > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_data_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_fasta > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/chado2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/gff3_merge > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/tophat2gff3 > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map2assembly > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/map_gff_ids > Installing /scratch/var/tmp/portage/sci-biology/maker-2.31.8/work/maker/src/../bin/maker_functional_gff >>>> Source compiled. >>>> Test phase [not enabled]: sci-biology/maker-2.31.8 > >>>> Install maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ category sci-biology >>>> Completed installing maker-2.31.8 into /scratch/var/tmp/portage/sci-biology/maker-2.31.8/image/ > > * Final size of build directory: 40312 KiB > * Final size of installed tree: 12656 KiB > > ecompressdir: bzip2 -9 /usr/share/man > ecompressdir: bzip2 -9 /usr/share/doc > >>>> Installing (1 of 1) sci-biology/maker-2.31.8::science >>>> Auto-cleaning packages... > > > > > That gives me these files installed on my system: > > # equery files maker > * Searching for maker ... > * Contents of sci-biology/maker-2.31.8: > /usr > /usr/bin > /usr/bin/cegma2zff > /usr/bin/chado2gff3 > /usr/bin/compare_gff3_to_chado > /usr/bin/cufflinks2gff3 > /usr/bin/evaluator > /usr/bin/fasta_merge > /usr/bin/genemark_gtf2gff3 > /usr/bin/gff3_merge > /usr/bin/ipr_update_gff > /usr/bin/iprscan2gff3 > /usr/bin/iprscan_wrap > /usr/bin/maker > /usr/bin/maker2chado > /usr/bin/maker2eval_gtf > /usr/bin/maker2jbrowse > /usr/bin/maker2wap > /usr/bin/maker2zff > /usr/bin/maker_functional > /usr/bin/maker_functional_fasta > /usr/bin/maker_functional_gff > /usr/bin/maker_map_ids > /usr/bin/map2assembly > /usr/bin/map_data_ids > /usr/bin/map_fasta_ids > /usr/bin/map_gff_ids > /usr/bin/tophat2gff3 > /usr/lib64 > /usr/lib64/perl5 > /usr/lib64/perl5/vendor_perl > /usr/lib64/perl5/vendor_perl/5.20.2 > /usr/lib64/perl5/vendor_perl/5.20.2/MAKER > /usr/lib64/perl5/vendor_perl/5.20.2/MAKER/ConfigData.pm > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application > /usr/lib64/perl5/vendor_perl/5.20.2/Parallel/Application/MPI.pm > /usr/share > /usr/share/doc > /usr/share/doc/maker-2.31.8 > /usr/share/doc/maker-2.31.8/INSTALL.bz2 > /usr/share/doc/maker-2.31.8/README.bz2 > /usr/share/maker > /usr/share/maker/GMOD > /usr/share/maker/GMOD/Apollo > /usr/share/maker/GMOD/Apollo/gff3.tiers > /usr/share/maker/GMOD/JBrowse > /usr/share/maker/GMOD/JBrowse/maker.css > /usr/share/maker/data > /usr/share/maker/data/dpp_contig.fasta > /usr/share/maker/data/dpp_est.fasta > /usr/share/maker/data/dpp_protein.fasta > /usr/share/maker/data/hsap_contig.fasta > /usr/share/maker/data/hsap_est.fasta > /usr/share/maker/data/hsap_protein.fasta > /usr/share/maker/data/te_proteins.fasta > /usr/share/man > /usr/share/man/man3 > /usr/share/man/man3/MAKER::ConfigData.3pm.bz2 > > > > $ /usr/bin/maker > Argument "2.49_01" isn't numeric in numeric ge (>=) at /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi/forks.pm line 1570. > Can't locate GI.pm in @INC (you may need to install the GI module) (@INC contains: /home/mmokrejs/proj/sci/sci-biology/maker/_Inline/lib /usr/bin/../perl/lib /usr/bin/../lib /usr/bin/../src/inc/perl/lib /etc/perl /usr/local/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.20.2 /usr/lib64/perl5/vendor_perl/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.20.2 /usr/local/lib64/perl5 /usr/lib64/perl5/vendor_perl /usr/lib64/perl5/5.20.2/x86_64-linux-thread-multi /usr/lib64/perl5/5.20.2 .) at /usr/bin/maker line 63. > BEGIN failed--compilation aborted at /usr/bin/maker line 63. > $ > > Of course, because as I said, I need to install properly all the dependencies from maker/lib/ subdirectory. > > > BTW, did perl Build.PL introduce the _Inline/lib/ directory? It somehow escaped from the sanbox I think. :( > > But thank you for you kind answer anyway, > Martin From sebastien.moretti at unil.ch Wed Aug 3 03:25:56 2016 From: sebastien.moretti at unil.ch (sebastien.moretti at unil.ch) Date: Wed, 3 Aug 2016 11:25:56 +0200 Subject: [maker-devel] MPI test In-Reply-To: References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: OK, I removed execution rights for mpi_evaluator. I tried mpiexec -n 10 maker MPI tries to write in the maker/perl/ directory. This is an issue for me because maker is installed in a read-only file system. Is there an option or something to tell MPI to write elsewhere? S?bastien > To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. > > Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker > > Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. > > ?Carson > > > >> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: >> >> Hi >> >> I compiled maker with MPI support and would like to test it. >> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >> is missing. >> >> What does mpi_evaluator do? >> How to test properly MPI in maker? >> >> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >> openmpi 1.8.1 >> >> Regards -- S?bastien Moretti Staff Scientist SIB Vital-IT EMBnet, Quartier Sorge - Genopode CH-1015 Lausanne, Switzerland Tel.: +41 (21) 692 4079/4221 http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From carsonhh at gmail.com Wed Aug 3 10:16:57 2016 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 3 Aug 2016 10:16:57 -0600 Subject: [maker-devel] MPI test In-Reply-To: References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> Message-ID: <72BDD9C3-9E02-4394-AE1B-521411F9B86D@gmail.com> The issue is that you moved the ?/maker/perl directory after initial setup. So when maker runs it can?t find the files, and tries to rebuild the shared objects for C/Perl MPI binding that are missing (compiles on demand). Solutions (one of the following two options) 1. Install maker as indicated in the INTALL file and then append the path for ?/maker/bin/ to the PATH environmental variable in /etc/profile (i.e. export PATH=/usr/local/maker/bin:$PATH). 2. Install maker as indicated in the INTALL file and then soft link the executables in ?/maker/bin/ to the location of your choice (i.e. /usr/bin/ or /usr/local/bin/ - this is what homebrew does for example). Example: MacBook-Pro:~ cholt$ ls -al /usr/local/bin/maker lrwxr-xr-x 1 cholt admin 32 Oct 25 2015 /usr/local/bin/maker -> ../Cellar/maker/2.31.8/bin/maker Notice that the softink in /usr/local/bin/ is pointing to the actual location where homebrew placed the installation. Doing something similar would be better than you trying to break up the installation structure. Basically they move the contents of ?/maker/* to /usr/local/Cellar/maker/2.31.8/, then they performed all installation steps there, and then softlinked the executables elsewhere using ?ln -s' When you install maker, you have to run the setup steps in ?/maker/src/. After install, ?/maker/bin/ will contain executables and ?/maker/perl/ and ?/maker/lib/ will contain libraries and shared objects used by those executables. Remember the executables are not binaries, they are scripts, so they need access to the libraries and shared objects used by the scripts. The executables know to look for libraries/files in the installation folder, but if you move them, the @INC search list used by perl will not find them. You can try and manually declare new locations to search in the @INC list using the PERL5LIB environmental variable (but using PERL5LIB is beyond the scope of the maker-devel list and can be investigated using Perl?s own docs). Also as explained in the homebrew example above, the base directory does not have to be named ?maker?. You can name it whatever you want. It is just the installation structure within the containing directory that has to be consistent. Alternatively if you are doing something more complex like a ?module? setup using lua based modules just use the prepend_path command to add the .../maker/bin foldert to the environment on demand ?> Example: -- -*- lua -*- help( [[ This module loads the MAKER genome annotation pipeline ]]) whatis("Name: MAKER") whatis("Version: 2.31.8") whatis("Category: Gene Annotation") whatis("Keywords: Gene Annotation") whatis("URL: http://www.yandell-lab.org/software/maker.html") whatis("Description: MAKER is a highly parallelized genome annotation/re-annotation pipeline") -- Load required modules load("openmpi/1.8.4.i", "genemark-ES", "augustus", "snap", "exonerate", "ncbi-blast+", "RepeatMasker") -- Prepare needed values local APPS = "/ucgd/apps/" local id = "maker" local version = "2.31.8" local base = pathJoin(APPS, id, version) local bin = pathJoin(base, "bin") -- Set PATH and other environmental variables prepend_path("PATH", bin) prepend_path("LD_PRELOAD", "/ucgd/apps/openmpi/1.8.4i/lib/libmpi.so") setenv("OMPI_MCA_mpi_warn_on_fork", 0) ?Carson > On Aug 3, 2016, at 3:25 AM, sebastien.moretti at unil.ch wrote: > > OK, I removed execution rights for mpi_evaluator. > > I tried mpiexec -n 10 maker > MPI tries to write in the maker/perl/ directory. > This is an issue for me because maker is installed in a read-only file > system. > > Is there an option or something to tell MPI to write elsewhere? > > S?bastien > >> To run MAKER with MPI, you just call maker with mpiexec. Examples can be found on the MAKER wiki, in the devel list archives, and in the README/INSTALL guide. >> >> Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker >> >> Any other MPI options will depend on the flavor of MPI you use, and your cluster settings. >> >> ?Carson >> >> >> >>> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti wrote: >>> >>> Hi >>> >>> I compiled maker with MPI support and would like to test it. >>> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >>> is missing. >>> >>> What does mpi_evaluator do? >>> How to test properly MPI in maker? >>> >>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>> openmpi 1.8.1 >>> >>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Thu Aug 4 14:21:36 2016 From: carsonhh at gmail.com (Carson Holt) Date: Thu, 4 Aug 2016 14:21:36 -0600 Subject: [maker-devel] MPI test In-Reply-To: <9e01f987-d284-f73b-63c0-e2d84a87cb27@unil.ch> References: <172fda57-c9e3-3c3d-ef90-04c88b21e7d9@unil.ch> <72BDD9C3-9E02-4394-AE1B-521411F9B86D@gmail.com> <9e01f987-d284-f73b-63c0-e2d84a87cb27@unil.ch> Message-ID: Hi Sebastien, As long as you are not following the ?/maker/INSTALL instructions, are instead moving files around manually, and trying to modify the default behavior of ./Build I can not provide support. I?ve tried to best explain how you should be doing things in the previous e-mails as well as the information below. If you are installing for a module based setup, then the proper way to do this would be to move the maker tarball to the desired module location, then follow the INSTALL instructions. Example (bent for a module based setup since you said that is what you are doing): #load required tools module load snap module load openmpi module load blast module load exonerate module load RepeatMasker #set up environment for OpenMPI install (explained in ?/maker/INSTALL instructions) export LD_PRELOAD=/location/of/openmpi/lib/libmpi.so #download and untar maker into desired location mkdir $APPS/maker cd $APPS/maker wget http://maker_url/maker.tgz tar -zxvf maker.tgz mv maker 2.31.8 #go to src directory cd $APPS/maker/2.31.8/src #install according to ?/maker/INSTALL instructions perl Build.PL ./Build installdeps ./Build installexes ./Build ./Build install #test normal installation $APPS/maker/2.31.8/bin/maker --help #test MPI installation (will print only one help message - something is wrong if it prints 5) mpiexec -mca btl ^openib -n 5 $APPS/maker/2.31.8/bin/maker --help #then setup a module file for maker that includes: 1. steps for loading prerequisite tools and modules (such as openmpi) 2. setting OMPI_MCA_mpi_warn_on_fork=0 3. setting LD_PRELOAD=/location/of/openmpi/lib/libmpi.so 4. prepend path for $APPS/maker/2.31.8/bin to PATH environmental variable Thanks, Carson > On Aug 4, 2016, at 8:28 AM, Sebastien Moretti wrote: > > I already use a module to load maker and external tools ;-) > As well as to set OMPI_MCA_mpi_warn_on_fork = 0 > > > In my case perl/config-x86_64-linux-thread-multi-5.018002 was created > properly but not copied. > I use ./Build install --destdir= and the destdir value is not > reported at the end of the install process when > blib/config-x86_64-linux-thread-multi-5.018002 is copied to ../perl/ > > I manually copy blib/config-x86_64-linux-thread-multi-5.018002 to > my_path/../perl/ and MPI does not try to write it now. > > > > Now I have issues when MPI tries to send messages. > Don't know if you can help me with this. > Here is part of the error message I got: > > [devfrt01.XXX:10256] mca: base: component_find: unable to open > /software/lib64/openmpi/lib/openmpi/mca_shmem_mmap: perhaps a missing > symbol, or compiled for a different version of Open MPI? (ignored) > ... > > It looks like opal_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during opal_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_shmem_base_select failed > --> Returned value -1 instead of OPAL_SUCCESS > ... > It looks like orte_init failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during orte_init; some of which are due to configuration or > environment problems. This failure appears to be an internal failure; > here's some additional information (which may only be relevant to an > Open MPI developer): > > opal_init failed > --> Returned value Error (-1) instead of ORTE_SUCCESS > ... > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10252] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10256] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10253] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10257] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > -------------------------------------------------------------------------- > It looks like MPI_INIT failed for some reason; your parallel process is > likely to abort. There are many reasons that a parallel process can > fail during MPI_INIT; some of which are due to configuration or environment > problems. This failure appears to be an internal failure; here's some > additional information (which may only be relevant to an Open MPI > developer): > > ompi_mpi_init: ompi_rte_init failed > --> Returned "Error" (-1) instead of "Success" (0) > -------------------------------------------------------------------------- > *** An error occurred in MPI_Init > *** on a NULL communicator > *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort, > *** and potentially your MPI job) > [devfrt01.vital-it.ch:10251] Local abort before MPI_INIT completed > successfully; not able to aggregate error messages, and not able to > guarantee that all other processes were killed! > ------------------------------------------------------- > Primary job terminated normally, but 1 process returned > a non-zero exit code.. Per user-direction, the job has been aborted. > ------------------------------------------------------- > -------------------------------------------------------------------------- > mpiexec detected that one or more processes exited with non-zero status, > thus causing > the job to be terminated. The first process to do so was: > > Process name: [[21120,1],1] > Exit code: 1 > > >> The issue is that you moved the ?/maker/perl directory after initial >> setup. So when maker runs it can?t find the files, and tries to rebuild >> the shared objects for C/Perl MPI binding that are missing (compiles on >> demand). >> >> Solutions (one of the following two options) >> 1. Install maker as indicated in the INTALL file and then append the >> path for ?/maker/bin/ to the PATH environmental variable in /etc/profile >> (i.e. export PATH=/usr/local/maker/bin:$PATH). >> 2. Install maker as indicated in the INTALL file and then soft link the >> executables in ?/maker/bin/ to the location of your choice (i.e. >> /usr/bin/ or /usr/local/bin/ - this is what homebrew does for example). >> >> Example: >> MacBook-Pro:~ cholt$ ls -al /usr/local/bin/maker >> lrwxr-xr-x 1 cholt admin 32 Oct 25 2015 /usr/local/bin/maker -> >> ../Cellar/maker/2.31.8/bin/maker >> >> Notice that the softink in /usr/local/bin/ is pointing to the actual >> location where homebrew placed the installation. Doing something similar >> would be better than you trying to break up the installation structure. >> Basically they move the contents of ?/maker/* to >> /usr/local/Cellar/maker/2.31.8/, then they performed all installation >> steps there, and then softlinked the executables elsewhere using ?ln -s' >> >> >> When you install maker, you have to run the setup steps in ?/maker/src/. >> After install, ?/maker/bin/ will contain executables and ?/maker/perl/ >> and ?/maker/lib/ will contain libraries and shared objects used by those >> executables. Remember the executables are not binaries, they are >> scripts, so they need access to the libraries and shared objects used by >> the scripts. The executables know to look for libraries/files in the >> installation folder, but if you move them, the @INC search list used by >> perl will not find them. You can try and manually declare new locations >> to search in the @INC list using the PERL5LIB environmental variable >> (but using PERL5LIB is beyond the scope of the maker-devel list and can >> be investigated using Perl?s own docs). >> >> Also as explained in the homebrew example above, the base directory does >> not have to be named ?maker?. You can name it whatever you want. It is >> just the installation structure within the containing directory that has >> to be consistent. >> >> Alternatively if you are doing something more complex like a ?module? >> setup using lua based modules just use the prepend_path command to add >> the .../maker/bin foldert to the environment on demand ?> >> >> Example: >> >> -- -*- lua -*- >> help( >> [[ >> This module loads the MAKER genome annotation pipeline >> ]]) >> >> whatis("Name: MAKER") >> whatis("Version: 2.31.8") >> whatis("Category: Gene Annotation") >> whatis("Keywords: Gene Annotation") >> whatis("URL: http://www.yandell-lab.org/software/maker.html") >> whatis("Description: MAKER is a highly parallelized genome >> annotation/re-annotation pipeline") >> >> -- Load required modules >> load("openmpi/1.8.4.i", "genemark-ES", "augustus", "snap", "exonerate", >> "ncbi-blast+", "RepeatMasker") >> >> -- Prepare needed values >> local APPS = "/ucgd/apps/" >> local id = "maker" >> local version = "2.31.8" >> local base = pathJoin(APPS, id, version) >> local bin = pathJoin(base, "bin") >> >> -- Set PATH and other environmental variables >> prepend_path("PATH", bin) >> prepend_path("LD_PRELOAD", "/ucgd/apps/openmpi/1.8.4i/lib/libmpi.so") >> setenv("OMPI_MCA_mpi_warn_on_fork", 0) >> >> >> ?Carson >> >> >>> On Aug 3, 2016, at 3:25 AM, sebastien.moretti at unil.ch >>> wrote: >>> >>> OK, I removed execution rights for mpi_evaluator. >>> >>> I tried mpiexec -n 10 maker >>> MPI tries to write in the maker/perl/ directory. >>> This is an issue for me because maker is installed in a read-only file >>> system. >>> >>> Is there an option or something to tell MPI to write elsewhere? >>> >>> S?bastien >>> >>>> To run MAKER with MPI, you just call maker with mpiexec. Examples can >>>> be found on the MAKER wiki, in the devel list archives, and in the >>>> README/INSTALL guide. >>>> >>>> Basically something like this to run on 10 CPUs ?> mpiexec -n 10 maker >>>> >>>> Any other MPI options will depend on the flavor of MPI you use, and >>>> your cluster settings. >>>> >>>> ?Carson >>>> >>>> >>>> >>>>> On Aug 2, 2016, at 6:00 AM, Sebastien Moretti >>>>> > wrote: >>>>> >>>>> Hi >>>>> >>>>> I compiled maker with MPI support and would like to test it. >>>>> So I tried to run mpi_evaluator but it fails because Parallel/MPIcar.pm >>>>> is missing. >>>>> >>>>> What does mpi_evaluator do? >>>>> How to test properly MPI in maker? >>>>> >>>>> maker 2.31.8, Linux x86_64 2.6.32, Perl 5.18.2 >>>>> openmpi 1.8.1 >>>>> >>>>> Regards > > -- > S?bastien Moretti > Staff Scientist > SIB Vital-IT EMBnet, Quartier Sorge - Genopode > CH-1015 Lausanne, Switzerland > Tel.: +41 (21) 692 4079/4221 > http://www.vital-it.ch/ http://myhits.vital-it.ch/ http://MetaNetX.org/ From lassance at fas.harvard.edu Tue Aug 30 07:19:13 2016 From: lassance at fas.harvard.edu (Lassance, Jean-Marc) Date: Tue, 30 Aug 2016 13:19:13 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats Message-ID: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm -------------- next part -------------- An HTML attachment was scrubbed... URL: From dence at genetics.utah.edu Tue Aug 30 10:16:37 2016 From: dence at genetics.utah.edu (Daniel Ence) Date: Tue, 30 Aug 2016 16:16:37 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Message-ID: Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? Thanks, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From me.mark at gmail.com Tue Aug 30 10:10:16 2016 From: me.mark at gmail.com (Mark Ebbert) Date: Tue, 30 Aug 2016 09:10:16 -0700 Subject: [maker-devel] (no subject) Message-ID: <57c5af23a842850000919b43@polymail.io> Good day everyone! I?m getting the error stating: ?WARNING: Multiple MAKER processes have been started in the same directory.? Everything I?ve seen mentions version issues with MPICH. The difference in my situation is that my initial run ran just fine, but died because of the cluster time constraints. We?re only allowed 3 days.? There are a bunch of .NFSLock files in the output directory. I?m guessing Maker wasn?t able to clear the locks when the jobs died? Can I safely delete those lock files? What?s the best way to handle this going forward since I can only run jobs for 3 days at a time? Thanks! Mark T. W. Ebbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 30 10:54:39 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 30 Aug 2016 10:54:39 -0600 Subject: [maker-devel] (no subject) In-Reply-To: <57c5af23a842850000919b43@polymail.io> References: <57c5af23a842850000919b43@polymail.io> Message-ID: Run 'maker -help? with mpiexec. Example: mpiexec -n 10 maker -help If the MPI communication ring is working correctly, then it will print the help message only once (from the root process). If it is not working, it will print the help message 10 time because each of the 10 MPI processes will think they are the root process. It is a simple test that can identify if it is an MPI issue or not. If it is not an MPI issue, you can just search for the NFSLock files using find and delete them,. ?Carson > On Aug 30, 2016, at 10:10 AM, Mark Ebbert wrote: > > > Good day everyone! > > I?m getting the error stating: ?WARNING: Multiple MAKER processes have been started in the same directory.? Everything I?ve seen mentions version issues with MPICH. The difference in my situation is that my initial run ran just fine, but died because of the cluster time constraints. We?re only allowed 3 days. > > There are a bunch of .NFSLock files in the output directory. I?m guessing Maker wasn?t able to clear the locks when the jobs died? Can I safely delete those lock files? What?s the best way to handle this going forward since I can only run jobs for 3 days at a time? > > Thanks! > > Mark T. W. Ebbert > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Tue Aug 30 14:12:44 2016 From: carsonhh at gmail.com (Carson Holt) Date: Tue, 30 Aug 2016 14:12:44 -0600 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> Message-ID: <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> Also make sure you pass the data in using rm_gff and not maker_gff if the repeats were not MAKER generated. ?Carson > On Aug 30, 2016, at 10:16 AM, Daniel Ence wrote: > > Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? > > And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? > > Thanks, > Daniel > > > Daniel Ence > Graduate Student > Eccles Institute of Human Genetics > University of Utah > 15 North 2030 East, Room 2100 > Salt Lake City, UT 84112-5330 > >> On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: >> >> Hi. >> >> I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). >> >> Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: >> >> rm_gff=myanimal.repeatmasker.out.gff3 >> >> #-----Re-annotation Using MAKER Derived GFF3 >> maker_gff=myanimal.all.maker.pass1.gff >> rm_pass=1 >> >> Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: >> >> Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. >> --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu >> ERROR: Failed while processing all repeats >> ERROR: Chunk failed at level:3, tier_type:1 >> FAILED CONTIG:scaffold00013 >> >> I wonder if you have an idea of what could be wrong here. >> >> Thanks for your help, >> >> >> Jean-Marc >> >> ?????????????????? >> Jean-Marc Lassance, PhD >> >> Harvard University >> Department of Organismic and Evolutionary Biology >> Department of Molecular and Cellular Biology >> Museum of Comparative Zoology >> >> 26, Oxford Street >> Cambridge MA 02138 >> USA >> >> email: lassance at fas.harvard.edu >> twitter: @lassancejm >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From lassance at fas.harvard.edu Tue Aug 30 14:31:44 2016 From: lassance at fas.harvard.edu (Lassance, Jean-Marc) Date: Tue, 30 Aug 2016 20:31:44 +0000 Subject: [maker-devel] maker2.31.8 _ failure when processing repeats In-Reply-To: <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> References: <76B77664-2EA9-45AA-A1C1-1B5124DC0025@fas.harvard.edu> <01FEE9E8-69C7-4E42-9E8C-07E029BB01A5@gmail.com> Message-ID: <1F3B92BC-1717-4CB5-A26D-6F2126667E53@fas.harvard.edu> Let me clarify one thing: the first pass was performed with Maker running repeatMasker internally, which is why I decided to use them in the second pass, as well as the data from the independent run of RepeatMasker. From reading earlier posts, I gathered that Maker would use first the evidence from the rm_gff, and then from maker_gff if rm_pass=1 is activated, but that having both would not hurt. Correct? JM On Aug 30, 2016, at 4:12 PM, Carson Holt > wrote: Also make sure you pass the data in using rm_gff and not maker_gff if the repeats were not MAKER generated. ?Carson On Aug 30, 2016, at 10:16 AM, Daniel Ence > wrote: Hi Jean-Marc, so the first question I have is whether maker is still annotating repeats, even though you?re providing the rm_gff file. Are you providing a file or parameter for repeat masker in the maker_opts.ctl file? And secondly, what about the scaffold that is failing? How long is it, what is the percent N?s in the sequence there, and how much of it was masked in the rm_gff file? Thanks, Daniel Daniel Ence Graduate Student Eccles Institute of Human Genetics University of Utah 15 North 2030 East, Room 2100 Salt Lake City, UT 84112-5330 On Aug 30, 2016, at 7:19 AM, Lassance, Jean-Marc > wrote: Hi. I am using Maker2.31.8 to annotate a mammalian genome (with OpenMPI, Linux server). Basically, after running Maker a first time to generate a training set for SNAP, I am running it a second time with SNAP and Augustus enabled. Because we ran RepeatMasker independently, I am providing the gff3 like so: rm_gff=myanimal.repeatmasker.out.gff3 #-----Re-annotation Using MAKER Derived GFF3 maker_gff=myanimal.all.maker.pass1.gff rm_pass=1 Things seem to progress nicely (the vast majority of the scaffolds ?finish?), but one of the scaffolds keeps failing (I have attempted to restart after erasing the entire content of the output folder). This is the message that I could associated with this error: Died at /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/maker/2.31.8-fasrc01/bin/../perl/lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=26, hostname=holy2a11102.rc.fas.harvard.edu ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 FAILED CONTIG:scaffold00013 I wonder if you have an idea of what could be wrong here. Thanks for your help, Jean-Marc ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org _______________________________________________ maker-devel mailing list maker-devel at box290.bluehost.com http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org ?????????????????? Jean-Marc Lassance, PhD Harvard University Department of Organismic and Evolutionary Biology Department of Molecular and Cellular Biology Museum of Comparative Zoology 26, Oxford Street Cambridge MA 02138 USA email: lassance at fas.harvard.edu twitter: @lassancejm -------------- next part -------------- An HTML attachment was scrubbed... URL: From psh65 at cornell.edu Wed Aug 31 08:35:28 2016 From: psh65 at cornell.edu (Prashant S Hosmani) Date: Wed, 31 Aug 2016 14:35:28 +0000 Subject: [maker-devel] mapping cDNA to updated genome Message-ID: Hi All, I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. Thank you for your help in advance. Prashant Prashant Hosmani Sol Genomics Network Boyce Thompson Institute, Ithaca, NY, USA -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael.s.campbell1 at gmail.com Wed Aug 31 10:10:22 2016 From: michael.s.campbell1 at gmail.com (Michael Campbell) Date: Wed, 31 Aug 2016 12:10:22 -0400 Subject: [maker-devel] mapping cDNA to updated genome In-Reply-To: References: Message-ID: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> Hi Prashant, I?m almost positive that the additional genes are coming from multiply aligning cDNAs. Did you repeat mask your genome before mapping things forward? Another thought, what kind of whole genome duplications has your plant been through. it may be that the multiple alignments are to pseudogenes is some stage of decay. If that is the case it would probably be safe to keep the the gene from longest/best aligned cDNA. Thanks, Mike > On Aug 31, 2016, at 10:35 AM, Prashant S Hosmani wrote: > > Hi All, > > I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. > > I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. > > Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. > > Thank you for your help in advance. > Prashant > > > Prashant Hosmani > Sol Genomics Network > Boyce Thompson Institute, Ithaca, NY, USA > > > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Aug 31 10:12:53 2016 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 31 Aug 2016 10:12:53 -0600 Subject: [maker-devel] mapping cDNA to updated genome In-Reply-To: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> References: <646B795A-1B04-4300-94C7-BEBEF0B37323@gmail.com> Message-ID: <9FBCB1C4-C319-4933-8741-53DAFCB82458@gmail.com> Also if you have multiple alignments of the same cDNA, you can use the score column of the mRNA feature to see which aligns best. If they have the same score, you will have to disambiguate manually or just remove all copies. ?Carson > On Aug 31, 2016, at 10:10 AM, Michael Campbell wrote: > > Hi Prashant, > > I?m almost positive that the additional genes are coming from multiply aligning cDNAs. Did you repeat mask your genome before mapping things forward? > > Another thought, what kind of whole genome duplications has your plant been through. it may be that the multiple alignments are to pseudogenes is some stage of decay. If that is the case it would probably be safe to keep the the gene from longest/best aligned cDNA. > > Thanks, > Mike >> On Aug 31, 2016, at 10:35 AM, Prashant S Hosmani > wrote: >> >> Hi All, >> >> I am working on updating a plant genome annotation. I would like to map genes from previous annotation to a new genome build. There is a protocol about this in Campbell et al 2014, current protocols in bioinformatics (basic protocol 4 - Mapping annotations to a new assembly). I followed that protocol exactly with setting est_forward=1. But in output I?m getting large number of genes. My input cDNA fasta contains ~35K genes and after mapping there are ~58K genes. >> >> I?m using maker version 3.0. There are few changes in the genome and I?m not expecting many changes in the mapping previous genes. >> >> Please let me know if there are any other parameters to control mapping of EST?s. I was hoping to get similar number of genes mapped on to new assembly with very few changes. >> >> Thank you for your help in advance. >> Prashant >> >> >> Prashant Hosmani >> Sol Genomics Network >> Boyce Thompson Institute, Ithaca, NY, USA >> >> >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at box290.bluehost.com >> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at box290.bluehost.com > http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: