From carsonhh at gmail.com Fri Mar 5 17:41:43 2021 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 5 Mar 2021 17:41:43 -0700 Subject: [maker-devel] MAKER Commercial User -- Question In-Reply-To: <08318e5bf6d847f2a1c4b6423e09dc13@utah.edu> References: <08318e5bf6d847f2a1c4b6423e09dc13@utah.edu> Message-ID: <41C60EAE-6685-4259-A48A-1EFF7E185D2E@gmail.com> Hi Hayley, Which flavor of MPI are you using? OpenMPI, MVAPICH2, MPICH, or Intel MPI? There is a known issue with infiniband network cards that causes them to freeze on certain system calls, and I bet that is what you are seeing. It is not a MAKER issue, but a limitation in order to allow performance benefits elsewhere with MPI. You need to add extra command line flags to tell MPI to communicate a bit differently to get around it. You can find multiple threads with more info in the maker-devel archives here ?> https://groups.google.com/g/maker-devel For OpenMPI you may need to add these flags to the mpiexec command line ?> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 --mca btl_openib_want_fork_support 1 --mca mpi_warn_on_fork 0 For intel MPI set the environmental variable I_MPI_FABRICS=shm:tcp before running mpiexec For MPICH, no modifications should be needed (it doesn?t experience the issue). For MVAPICH2, there is no work around and you just need to use one of the other MPI flavors. For the second failure, if you can regenerate it, send me the STDERR and I will take a look. Thanks, Carson > > From: Hayley Mangelson > > Sent: Thursday, March 4, 2021 12:03 PM > To: AARON A DUFFY > > Cc: Mary Wood > > Subject: MAKER Commercial User -- Question > > Hi Aaron, > > Mary and I are bioinformaticians at Phase Genomics, and we are using the MAKER software (for which Phase purchased a licence) to annotate a couple of genomes. We are using the same workflow that I used for a previous project, but are having the same issues while working on separate projects. I am hoping you could help us out or direct us to someone who can help! > > Here is some background. I am trying to get it to run using mpiexec for parallelization. I let it run for about a week and all processes remained ?active?, but no new output seemed to be generated. It was hung up on ?prepare section files? for nearly the whole 1-week period. I canceled the run, then tried running again without parallelization (in the same folder as the failed run). I got an error about each contig failing, and something about Augustus. Unfortunately, I didn?t save the error message. I tried to regenerate the error, but it looks like I would probably need to completely rerun? this is the error message now (repeated for each contig), if I run in the same folder again. > > The contig failed 2 times!! > Maker will not try again!! > The contig will be stored in a fasta file that you can use for debugging. > SeqID: PGA_scaffold4__8_contigs__length_31799528 > Length: 31799528 > FASTA: /home/ubuntu/PGScratch/chunk1/chunk1.maker.output/chunk1_datastore/10/91/PGA_scaffold4__8_contigs__length_31799528//PGA_scaffold4__8_contigs__length_31799528.died.fasta > > Any thoughts about why mpiexec seems unable to finish? I tried a run on the test data using parallelization, and it completed in under a minute. > > Thanks! > > Hayley > -- > Hayley Mangelson > Bioinformatics Scientist > Phase Genomics, Inc -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1376 bytes Desc: not available URL: From jwliao25 at gmail.com Mon Mar 8 03:25:30 2021 From: jwliao25 at gmail.com (=?UTF-8?B?5buW5a6257ev?=) Date: Mon, 8 Mar 2021 18:25:30 +0800 Subject: [maker-devel] Maker annotation result contain 10% of gene with incorrect start or stop codon Message-ID: Hi maker-devel group, I used the maker with SNAP and Augustus for annotating a green algae genome. I always set the 'always_complete=1' from the first round of annotation. After Augustus training, I still get around 1111 and 208 genes that don't have the correct start and stop codon. (Total annotated gene number is 12696) I provided the close species proteome and the green algae its own RNA-seq data for EST hint. Does that make sense? Does there have any way to improve the result or fix the incorrect start and stop codon for those gene? best, Chai-Wei Liao -------------------------------------------------- Chia-Wei Liao ??? Research Assistant Institute of Molecular Biology, Academia Sinica, Taipei City, Taiwan Phone: 886-2-2789-9216 (Lab) -------------- next part -------------- An HTML attachment was scrubbed... URL: From MARCO.SOLLITTO at phd.units.it Mon Mar 8 11:27:14 2021 From: MARCO.SOLLITTO at phd.units.it (SOLLITTO MARCO [PHD0100064]) Date: Mon, 8 Mar 2021 18:27:14 +0000 Subject: [maker-devel] Maker- Issue with repbase Message-ID: Dear all, I am using MAKER for the annotation of my de novo genome assembly. I was able to install its conda version, but I have some problems when I run the following command: mpiexec -n 64 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl | tee maker.log It gives me the following output: STATUS: Parsing control files... ERROR: Could not open the control file "maker_opts.ctl". --> rank=NA, hostname=atlas mpiexec: Forwarding signal 20 to job How could I solve it? Thanks for your support and availability! Best regards, Marco -------------- next part -------------- An HTML attachment was scrubbed... URL: From hayley at phasegenomics.com Tue Mar 9 12:17:10 2021 From: hayley at phasegenomics.com (Hayley Mangelson) Date: Tue, 9 Mar 2021 14:17:10 -0500 Subject: [maker-devel] MAKER Commercial User -- Question In-Reply-To: <41C60EAE-6685-4259-A48A-1EFF7E185D2E@gmail.com> References: <08318e5bf6d847f2a1c4b6423e09dc13@utah.edu> <41C60EAE-6685-4259-A48A-1EFF7E185D2E@gmail.com> Message-ID: Hi Carson, Unfortunately, it looks like I am running MPICH version 3.0.4, so that doesn't explain the problem. I regenerated the error for you: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /home/ubuntu/PGScratch/scaff1/scaff0.maker.output/scaff0_datastore To access files for individual sequences use the datastore index: /home/ubuntu/PGScratch/scaff1/scaff0.maker.output/scaff0_master_datastore_index.log STATUS: Now running MAKER... examining contents of the fasta file and run log --Next Contig-- Processing run.log file... MAKER WARNING: The file scaff0.maker.output/scaff0_datastore/0B/6F/PGA_scaffold0__8_contigs__length_34808196//theVoid.PGA_scaffold0__8_contigs__length_34808196/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.1.BUSCO_hemistictus_3028031324.augustus did not finish on the last run and must be erased MAKER WARNING: The file scaff0.maker.output/scaff0_datastore/0B/6F/PGA_scaffold0__8_contigs__length_34808196//theVoid.PGA_scaffold0__8_contigs__length_34808196/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.0.BUSCO_hemistictus_3028031324.augustus did not finish on the last run and must be erased MAKER WARNING: The file scaff0.maker.output/scaff0_datastore/0B/6F/PGA_scaffold0__8_contigs__length_34808196//theVoid.PGA_scaffold0__8_contigs__length_34808196/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.2.BUSCO_hemistictus_3028031324.augustus did not finish on the last run and must be erased #--------------------------------------------------------------------- Now retrying the contig!! SeqID: PGA_scaffold0__8_contigs__length_34808196 Length: 34808196 Tries: 2!! #--------------------------------------------------------------------- setting up GFF3 output and fasta chunks preparing ab-inits running augustus. #--------- command -------------# Widget::augustus: /home/ubuntu/PGTools/Augustus/bin/augustus --species=BUSCO_hemistictus_3028031324 --UTR=off /tmp/maker_pzUjBq/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.0 > /tmp/maker_pzUjBq/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.0.BUSCO_hemistictus_3028031324.augustus #-------------------------------# /home/ubuntu/PGTools/Augustus/bin/augustus: ERROR Invalid nucleotide '' encountered. /home/ubuntu/PGTools/Augustus/bin/augustus: ERROR Invalid nucleotide '?' encountered. ERROR: Augustus failed --> rank=NA, hostname=ip-172-31-14-148 ERROR: Failed while preparing ab-inits ERROR: Chunk failed at level:0, tier_type:2 FAILED CONTIG:PGA_scaffold0__8_contigs__length_34808196 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:PGA_scaffold0__8_contigs__length_34808196 examining contents of the fasta file and run log --Next Contig-- Processing run.log file... MAKER WARNING: The file scaff0.maker.output/scaff0_datastore/0B/6F/PGA_scaffold0__8_contigs__length_34808196//theVoid.PGA_scaffold0__8_contigs__length_34808196/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.0.BUSCO_hemistictus_3028031324.augustus did not finish on the last run and must be erased Maker is now finished!!! Start_time: 1615317386 End_time: 1615317388 Elapsed: 2 On Fri, Mar 5, 2021 at 7:41 PM Carson Holt wrote: > Hi Hayley, > > Which flavor of MPI are you using? OpenMPI, MVAPICH2, MPICH, or Intel > MPI? There is a known issue with infiniband network cards that causes them > to freeze on certain system calls, and I bet that is what you are seeing. > It is not a MAKER issue, but a limitation in order to allow performance > benefits elsewhere with MPI. You need to add extra command line flags to > tell MPI to communicate a bit differently to get around it. You can find > multiple threads with more info in the maker-devel archives here ?> > https://groups.google.com/g/maker-devel > > > For OpenMPI you may need to add these flags to the mpiexec command line > ?> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 --mca > btl_openib_want_fork_support 1 --mca mpi_warn_on_fork 0 > > For intel MPI set the environmental variable I_MPI_FABRICS=shm:tcp before > running mpiexec > > For MPICH, no modifications should be needed (it doesn?t experience the > issue). > > For MVAPICH2, there is no work around and you just need to use one of the > other MPI flavors. > > > For the second failure, if you can regenerate it, send me the STDERR and I > will take a look. > > Thanks, > Carson > > > > > *From:* Hayley Mangelson > *Sent:* Thursday, March 4, 2021 12:03 PM > *To:* AARON A DUFFY > *Cc:* Mary Wood > *Subject:* MAKER Commercial User -- Question > > Hi Aaron, > > Mary and I are bioinformaticians at Phase Genomics, and we are using the > MAKER software (for which Phase purchased a licence) to annotate a couple > of genomes. We are using the same workflow that I used for a previous > project, but are having the same issues while working on separate projects. > I am hoping you could help us out or direct us to someone who can help! > > Here is some background. I am trying to get it to run using *mpiexec* for > parallelization. I let it run for about a week and all processes remained > ?active?, but no new output seemed to be generated. It was hung up on > ?prepare section files? for nearly the whole 1-week period. I canceled the > run, then tried running again without parallelization (in the same folder > as the failed run). I got an error about each contig failing, and something > about Augustus. Unfortunately, I didn?t save the error message. I tried to > regenerate the error, but it looks like I would probably need to completely > rerun? this is the error message now (repeated for each contig), if I run > in the same folder again. > > > > > > > *The contig failed 2 times!!Maker will not try again!!The contig will be > stored in a fasta file that you can use for debugging.SeqID: > PGA_scaffold4__8_contigs__length_31799528Length: 31799528FASTA: > /home/ubuntu/PGScratch/chunk1/chunk1.maker.output/chunk1_datastore/10/91/PGA_scaffold4__8_contigs__length_31799528//PGA_scaffold4__8_contigs__length_31799528.died.fasta* > > Any thoughts about why *mpiexec *seems unable to finish? I tried a run on > the test data using parallelization, and it completed in under a minute. > > Thanks! > > Hayley > -- > Hayley Mangelson > Bioinformatics Scientist > Phase Genomics, Inc > > > -- Hayley Mangelson Bioinformatics Scientist Phase Genomics, Inc -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Mar 10 13:44:12 2021 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 10 Mar 2021 13:44:12 -0700 Subject: [maker-devel] Find evidence for specific gene annotation (without any genome browser!) In-Reply-To: <6ba7a71ce9970fb0dbf072a61df0f18f@uni-duesseldorf.de> References: <6ba7a71ce9970fb0dbf072a61df0f18f@uni-duesseldorf.de> Message-ID: <0C551067-369B-4341-9F78-5D959BB61798@gmail.com> It?s based on overlap. If you want to know the specific evidence you take everything that overlap the gene coordinates and cluster that together. Bedtools might be the easiest way to get everything for a given set of coordinates. For naming based off of protein2genome alignment, one option is to set est_forward=1 in the control files during the maker run (must manually add it). It is a hack for fast protein mapping of previous annotations onto new assemblies. But you can use it to force naming of any protein source. There are several more examples of using this option in the maker-devel archives ?> https://groups.google.com/g/maker-devel/search?q=est_forward ?Carson > On Jan 22, 2021, at 5:42 AM, Ricardo Nuno Ferreira Martins Guerreiro wrote: > > > Hello, > > Simple question: How do I know which evidence generated one specific gene? Or even, which evidences are used for it's AED calculation? > > How do I do this if the gene is predicted directly by a prot2genome alignment? And how when it's an Augustus annotation? > > I want to do this without Jbrowser, which is a huge waste of time. I want the original name in my input protein set, not the maker name. > > > Cheers, > Ricardo > > _______________________________________________ > maker-devel mailing list > maker-devel at yandell-lab.org > http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1376 bytes Desc: not available URL: From carsonhh at gmail.com Wed Mar 10 14:26:25 2021 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 10 Mar 2021 14:26:25 -0700 Subject: [maker-devel] Issues with locking in MPI mode In-Reply-To: References: Message-ID: Look for hidden files with .NFSLock in the name, delete them, and see if they come back. find | grep .NFSLock | xargs rm find | grep .NFSLock If the files come back after deleting them, it can mean another MAKER job is still running and updating the lock. Can happen in weird situations like when process managers like slurm OOM kill a job but only wipe out some of the processes and not all. ?Carson > On Jan 21, 2021, at 6:30 AM, Michele Vidotto wrote: > > Dear all, > > as reported in the subject I'm having issues with locking mechanism of MAKER when it is runs in parallel-mode through mpi. > I'm using maker version 3.01.03 but the same happens in my system when I build and install version 2.31.11. > All prerequisites were installed in a conda environment. Perl was installed from anaconda channel in version 5.26.2. Hard-coded paths to the compilers were fixed. Necessary perl modules were installed via cpanm: > > "DBD::SQLite", > "DBI", > "Error", > "Error::Simple", > "File::NFSLock", > "File::Which", > "forks", > "forks::shared", > "Inline", > "Inline::C", > "IO::All", > "IO::Prompt", > "LWP::Simple" > "Perl::Unsafe::Signals", > "PerlIO::gzip", > "Proc::Simple", > "URI::Escape", > "DBD::Pg" > > additional libraries and components were installed via conda > > - gcc_linux-64=7.3.0 > - gxx_linux-64=7.3.0 > - openmpi=4.1.0 > - zlib=1.2.11 > - libdb=6.1.26 > - expat=2.2.9 > - libxml2=2.9.10 > - exonerate=2.4.0 > - snoscan=1.0 > - rapsearch=2.24 > > other components were installed manually. MAKER compile and install with no errors, but when I execute the program via MPI with: > > # to devoid OPEN MPI segmentation fault > export THREADS_DAEMON_MODEL=1 > > mpiexec -mca btl ^openib -n 1 \ > maker \ > -force \ > -cpus 8 \ > --fix_nucleotides \ > maker_opts.ctl \ > maker_bopts.ctl \ > maker_exe.ctl > > It always ends up with following error: > > > STATUS: Parsing control files... > ERROR: The directory is locked. Perhaps by an instance of MAKER. > > --> rank=NA, hostname=april.corp.igatechnology.com > -------------------------------------------------------------------------- > Primary job terminated normally, but 1 process returned > a non-zero exit code. Per user-direction, the job has been aborted. > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > mpiexec detected that one or more processes exited with non-zero status, thus causing > the job to be terminated. The first process to do so was: > > Process name: [[19321,1],0] > Exit code: 10 > -------------------------------------------------------------------------- > > if I look inside *.maker.output directory a lock file remains: > > .NFSLock.gi_lock.NFSLock > > If instead I run maker with the -nolock flag. MAKER runs with no problems at all. > > My filesystem is oneFS from ISILON, exported to a virtual server through nfs4 protocol. > By looking at the code MAKER uses File::NFSLock Perl module for locking. This module fails some tests when installed on my system with cipanm: > > # Failed test at t/300_bl_sh.t line 115. > Shared locks not running simultaneously at t/300_bl_sh.t line 116, <$rd3> line 18. > # Looks like your test exited with 4 just after 27. > t/300_bl_sh.t ..... Dubious, test returned 4 (wstat 1024, 0x400) > Failed 47/73 subtests > t/400_kill.t ...... ok > t/410_die.t ....... ok > t/420_crash.t ..... ok > t/430_taint.t ..... ok > > Test Summary Report > ------------------- > t/300_bl_sh.t (Wstat: 1024 Tests: 27 Failed: 1) > Failed test: 27 > Non-zero exit status: 4 > Parse errors: Bad plan. You planned 73 tests but ran 27. > > > > But anyway I was able to install it with --notest flag. > Do you have any idea on how I can overcome my problem and have MAKER run in parallel with MPI? > > Thanks in advance, > > > > > --- > Michele Vidotto > mailto: michele.vidotto at gmail.com _______________________________________________ > maker-devel mailing list > maker-devel at yandell-lab.org > http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1376 bytes Desc: not available URL: From bioinf49 at gmail.com Thu Mar 18 18:58:25 2021 From: bioinf49 at gmail.com (Inf Bio) Date: Fri, 19 Mar 2021 08:58:25 +0800 Subject: [maker-devel] ERROR: Failed while processing all repeats Message-ID: Dear Developers, I've researched all available forums and tried to resolve this recurring error in maker: processing all repeats in cluster::shadow_cluster... Died at /home/user/bin/maker/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=19, hostname=m30g4 ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 So far, I have tried: reformatting my gff file including purify script from Carson, Daren Card's script, and agat; uninstalled and reinstalled ncbi-blast+ and updated path; reinstated RMBlast path in repeatmasker; changed max_dna_length to 300000; set clean_try=1; replaced RepeatMasker.pm. However, none of the above helped get rid of the issue. I cannot see what is wrong with my gff. Can you please help? thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jacques.dainat at nbis.se Thu Mar 25 15:03:49 2021 From: jacques.dainat at nbis.se (Jacques Dainat) Date: Thu, 25 Mar 2021 22:03:49 +0100 Subject: [maker-devel] Maker annotation result contain 10% of gene with incorrect start or stop codon In-Reply-To: References: Message-ID: Hi, I met this problem in some projects where the ORFs were not well defined. In the mRNA it was not the longest ORF chosen, which is not necessarily wrong but here it was obviously not the correct one chosen. Probably due to bad training of my abinitio tools. I ended up to develop a script to fix the predictions and use the longest ORF as CDS. The script is called agat_sp_fix_longest_ORF.pl It is available within AGAT (https://github.com/NBISweden/AGAT) Hoping it could help, Best regards, Jacques Dainat, Ph.D. > On 8 Mar 2021, at 11:25, ??? wrote: > > Hi maker-devel group, > > I used the maker with SNAP and Augustus for annotating a green algae genome. I always set the 'always_complete=1' from the first round of annotation. > > After Augustus training, I still get around 1111 and 208 genes that don't have the correct start and stop codon. (Total annotated gene number is 12696) > > I provided the close species proteome and the green algae its own RNA-seq data for EST hint. > > Does that make sense? Does there have any way to improve the result or fix the incorrect start and stop codon for those gene? > > best, > Chai-Wei Liao > > -------------------------------------------------- > > Chia-Wei Liao ??? > > Research Assistant > > Institute of Molecular Biology, > > Academia Sinica, Taipei City, Taiwan > > Phone: 886-2-2789-9216 (Lab) > > _______________________________________________ > maker-devel mailing list > maker-devel at yandell-lab.org > http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Mar 26 09:59:55 2021 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Mar 2021 09:59:55 -0600 Subject: [maker-devel] Maker annotation result contain 10% of gene with incorrect start or stop codon In-Reply-To: References:

Message-ID: Thanks for this. Just some info on how MAKER sets the start/stop codon. MAKER will maintain the reading frame and codons of the gene predictor, so if a longer ORF exists in another reading frame, MAKER will not switch to it (this is because the gene predictor is saying the other reading frame is less probable). MAKER can extend the ORF in the same reading frame, but only if the initial prediction is partial, mRNA alignment suggests an extension, and the ORF extends to a canonical start in the same frame as the prediction. MAKER can also truncate the ORF if the gene prediction is partial to begin with and there is mRNA evidence of UTR (this can indicate an assembly error in the gene that artificially splits the ORF - or also false merge of neighboring genes through bad mRNA-seq assembly). The always_complete options adds one extra step that is not necessarily biologically correct, but does help make genes more canonical. When set, maker will walk off the edge of the CDS without (mRNA evidence) in both directions and extend to the first canonical start or canonical stop that it encounters. It?s beneficial when using protein2genome alignments for homology based annotation since the alignments can be fuzzy near the edges. Thanks, Carson > On Mar 25, 2021, at 3:03 PM, Jacques Dainat wrote: > > Hi, > > I met this problem in some projects where the ORFs were not well defined. In the mRNA it was not the longest ORF chosen, which is not necessarily wrong but here it was obviously not the correct one chosen. Probably due to bad training of my abinitio tools. > I ended up to develop a script to fix the predictions and use the longest ORF as CDS. The script is called agat_sp_fix_longest_ORF.pl > It is available within AGAT (https://github.com/NBISweden/AGAT ) > > Hoping it could help, > > Best regards, > > Jacques Dainat, Ph.D. > > >> On 8 Mar 2021, at 11:25, ??? > wrote: >> >> Hi maker-devel group, >> >> I used the maker with SNAP and Augustus for annotating a green algae genome. I always set the 'always_complete=1' from the first round of annotation. >> >> After Augustus training, I still get around 1111 and 208 genes that don't have the correct start and stop codon. (Total annotated gene number is 12696) >> >> I provided the close species proteome and the green algae its own RNA-seq data for EST hint. >> >> Does that make sense? Does there have any way to improve the result or fix the incorrect start and stop codon for those gene? >> >> best, >> Chai-Wei Liao >> >> -------------------------------------------------- >> >> Chia-Wei Liao ??? >> >> Research Assistant >> >> Institute of Molecular Biology, >> >> Academia Sinica, Taipei City, Taiwan >> >> Phone: 886-2-2789-9216 (Lab) >> >> _______________________________________________ >> maker-devel mailing list >> maker-devel at yandell-lab.org >> http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org > > _______________________________________________ > maker-devel mailing list > maker-devel at yandell-lab.org > http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1376 bytes Desc: not available URL: From carson.holt at genetics.utah.edu Wed Mar 31 12:00:51 2021 From: carson.holt at genetics.utah.edu (Carson Hinton Holt) Date: Wed, 31 Mar 2021 18:00:51 +0000 Subject: [maker-devel] Question about MAKER In-Reply-To: References: Message-ID: Hello, The google group is an archive only and cannot be posted to. You can however send questions to the email list (CCed). The behavior you see is normal. Exonerate gets rerun because it is a relatively fast step that can produce a lot of output and saving the results for later use can be very IO intensive when running under MPI. Some users were actually killing the NFS servers at their institutions. Since the performance gain from archiving exonerate results is small, but the consequences for IO were large, we don?t archive those results. ?Carson Sent from my iPhone > On Mar 31, 2021, at 11:51 AM, Kyungyong Seong wrote: > > ? > Dear Carson, > > I wanted to leave a question on the google groups, but I didn't have permission to do so. I am having a small problem with MAKER and wanted to get some advice from you. > > I ran the initial run with the following figuration successfully. > > #-----Genome (these are always required) > genome=/global/scratch/skyungyong/S.habrochaites/1.Final_Assembly/5.Final_data/SH1353.primary.scaffolds.noPlasmid.fa #genome sequence (fasta file or fasta embeded in GFF3 file) > organism_type=eukaryotic #eukaryotic or prokaryotic. Default is eukaryotic > > #-----Re-annotation Using MAKER Derived GFF3 > maker_gff= #MAKER derived GFF3 file > est_pass=0 #use ESTs in maker_gff: 1 = yes, 0 = no > altest_pass=0 #use alternate organism ESTs in maker_gff: 1 = yes, 0 = no > protein_pass=0 #use protein alignments in maker_gff: 1 = yes, 0 = no > rm_pass=0 #use repeats in maker_gff: 1 = yes, 0 = no > model_pass=0 #use gene models in maker_gff: 1 = yes, 0 = no > pred_pass=0 #use ab-initio predictions in maker_gff: 1 = yes, 0 = no > other_pass=0 #passthrough anyything else in maker_gff: 1 = yes, 0 = no > > #-----EST Evidence (for best results provide a file for at least one) > est=/global/scratch/skyungyong/S.habrochaites/2.Annotation/2.Maker/DB/Transcriptome/SH_ALL.superscripts.fa #set of ESTs or assembled mRNA-seq in fasta format > altest=/global/scratch/skyungyong/S.habrochaites/2.Annotation/2.Maker/DB/ITAG4.1_cDNA.fasta #EST/cDNA sequence file in fasta format from an alternate organism > est_gff= #aligned ESTs or mRNA-seq from an external GFF3 file > altest_gff= #aligned ESTs from a closly relate species in GFF3 format > > #-----Protein Homology Evidence (for best results provide a file for at least one) > protein=/global/scratch/skyungyong/S.habrochaites/2.Annotation/2.Maker/DB/Prot.evidence.fa #protein sequence file in fasta format (i.e. from mutiple organisms) > protein_gff= #aligned protein homology evidence from an external GFF3 file > > #-----Repeat Masking (leave values blank to skip repeat masking) > model_org=Solanum #select a model organism for RepBase masking in RepeatMasker > rmlib=/global/scratch/skyungyong/S.habrochaites/2.Annotation/1.RepeatModeler/SH1353-families.fa #provide an organism specific repeat library in fasta format for RepeatMasker > repeat_protein=/global/scratch/skyungyong/Software/maker/data/te_proteins.fasta #provide a fasta file of transposable element proteins for RepeatRunner > rm_gff= #pre-identified repeat elements from an external GFF3 file > prok_rm=0 #forces MAKER to repeatmask prokaryotes (no reason to change this), 1 = yes, 0 = no > softmask=1 #use soft-masking rather than hard-masking in BLAST (i.e. seg and dust filtering) > > #-----Gene Prediction > snaphmm=0 #SNAP HMM file > gmhmm= #GeneMark HMM file > augustus_species=0 #Augustus gene prediction species model > fgenesh_par_file= #FGENESH parameter file > pred_gff= #ab-initio predictions from an external GFF3 file > model_gff= #annotated gene models from an external GFF3 file (annotation pass-through) > run_evm=0 #run EvidenceModeler, 1 = yes, 0 = no > est2genome=1 #infer gene predictions directly from ESTs, 1 = yes, 0 = no > protein2genome=1 #infer predictions from protein homology, 1 = yes, 0 = no > trna=0 #find tRNAs with tRNAscan, 1 = yes, 0 = no > snoscan_rrna= #rRNA file to have Snoscan find snoRNAs > snoscan_meth= #-O-methylation site fileto have Snoscan find snoRNAs > unmask=0 #also run ab-initio prediction programs on unmasked sequence, 1 = yes, 0 = no > allow_overlap= #allowed gene overlap fraction (value from 0 to 1, blank for default) > > #-----Other Annotation Feature Types (features MAKER doesn't recognize) > other_gff= #extra features to pass-through to final MAKER generated GFF3 file > > #-----External Application Behavior Options > alt_peptide=C #amino acid used to replace non-standard amino acids in BLAST databases > cpus=1 #max number of cpus to use in BLAST and RepeatMasker (not for MPI, leave 1 when using MPI) > > #-----MAKER Behavior Options > max_dna_len=100000 #length for dividing up contigs into chunks (increases/decreases memory usage) > min_contig=1 #skip genome contigs below this length (under 10kb are often useless) > > pred_flank=200 #flank for extending evidence clusters sent to gene predictors > pred_stats=0 #report AED and QI statistics for all predictions as well as models > AED_threshold=1 #Maximum Annotation Edit Distance allowed (bound by 0 and 1) > min_protein=0 #require at least this many amino acids in predicted proteins > alt_splice=0 #Take extra steps to try and find alternative splicing, 1 = yes, 0 = no > always_complete=0 #extra steps to force start and stop codons, 1 = yes, 0 = no > map_forward=0 #map names and attributes forward from old GFF3 genes, 1 = yes, 0 = no > keep_preds=0 #Concordance threshold to add unsupported gene prediction (bound by 0 and 1) > > split_hit=10000 #length for the splitting of hits (expected max intron size for evidence alignments) > min_intron=20 #minimum intron length (used for alignment polishing) > single_exon=0 #consider single exon EST evidence when generating annotations, 1 = yes, 0 = no > single_length=250 #min length required for single exon ESTs if 'single_exon is enabled' > correct_est_fusion=0 #limits use of ESTs in annotation to avoid fusion genes > > tries=3 #number of times to try a contig if there is a failure for some reason > clean_try=0 #remove all data from previous run before retrying, 1 = yes, 0 = no > clean_up=0 #removes theVoid directory with individual analysis files, 1 = yes, 0 = no > TMP= > > For the second run, I trained SNAP and Augustus and provided the path to the HMMs in maker_opts.ctl. With est2genome=0 and prot2genome=0, I ran MAKER in the same directory, hoping MAKER to reuse the info from the previous run. > > The run starts with warning > MAKER WARNING: Changes in control files make re-use of hint based predictions impossible > Old hint based prediction files will be erased before continuing > > And it seems to run exonerate anew: > running est2genome search. > #--------- command -------------# > Widget::exonerate::est2genome: > /global/scratch/skyungyong/Software/maker/exe/exonerate/bin/exonerate -q /tmp/maker_gdlFj8/53/LA2119%2ETRINITY_DN24321_c0_g1.for.475-1651.53.fasta -t /tmp/maker_gdlFj8/53/scaffold53.475-1651.53.fasta -Q dna -T dna --model est2genome --minintron 20 --maxintron 10000 --showcigar --percent 20 > /tmp/maker_gdlFj8/53/scaffold53.475-1651.LA2119%2ETRINITY_DN24321_c0_g1.e.exonerate > > Is this normal? > Thank you for your help! > Kyungyong > > > > > > From yvan.papa at vuw.ac.nz Sun Mar 28 17:59:48 2021 From: yvan.papa at vuw.ac.nz (Yvan Papa) Date: Sun, 28 Mar 2021 23:59:48 +0000 Subject: [maker-devel] Is the MAKER wiki website down? Message-ID: Hi, I have been using the tutorials from the MAKER wiki (http://weatherby.genetics.utah.edu/MAKER/wiki/index.php) to assist in my usage of the software. Especially the detailed "WGS_Assembly_and_Annotation_Winter_School_2018" (http://weatherby.genetics.utah.edu/MAKER/wiki/index.php/MAKER_Tutorial_for_WGS_Assembly_and_Annotation_Winter_School_2018) was particularly useful to me to follow the steps along. Unfortunately it looks like the wiki is down, giving a 404 not found error, and I can't access the tutorials anymore. Is this a global issue? Is it (hopefully) going to be repaired? Thank you in advance for your help, and for providing us with all these great resources for genome annotation. Best wishes Yvan -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Mar 5 17:41:43 2021 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 5 Mar 2021 17:41:43 -0700 Subject: [maker-devel] MAKER Commercial User -- Question In-Reply-To: <08318e5bf6d847f2a1c4b6423e09dc13@utah.edu> References: <08318e5bf6d847f2a1c4b6423e09dc13@utah.edu> Message-ID: <41C60EAE-6685-4259-A48A-1EFF7E185D2E@gmail.com> Hi Hayley, Which flavor of MPI are you using? OpenMPI, MVAPICH2, MPICH, or Intel MPI? There is a known issue with infiniband network cards that causes them to freeze on certain system calls, and I bet that is what you are seeing. It is not a MAKER issue, but a limitation in order to allow performance benefits elsewhere with MPI. You need to add extra command line flags to tell MPI to communicate a bit differently to get around it. You can find multiple threads with more info in the maker-devel archives here ?> https://groups.google.com/g/maker-devel For OpenMPI you may need to add these flags to the mpiexec command line ?> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 --mca btl_openib_want_fork_support 1 --mca mpi_warn_on_fork 0 For intel MPI set the environmental variable I_MPI_FABRICS=shm:tcp before running mpiexec For MPICH, no modifications should be needed (it doesn?t experience the issue). For MVAPICH2, there is no work around and you just need to use one of the other MPI flavors. For the second failure, if you can regenerate it, send me the STDERR and I will take a look. Thanks, Carson > > From: Hayley Mangelson > > Sent: Thursday, March 4, 2021 12:03 PM > To: AARON A DUFFY > > Cc: Mary Wood > > Subject: MAKER Commercial User -- Question > > Hi Aaron, > > Mary and I are bioinformaticians at Phase Genomics, and we are using the MAKER software (for which Phase purchased a licence) to annotate a couple of genomes. We are using the same workflow that I used for a previous project, but are having the same issues while working on separate projects. I am hoping you could help us out or direct us to someone who can help! > > Here is some background. I am trying to get it to run using mpiexec for parallelization. I let it run for about a week and all processes remained ?active?, but no new output seemed to be generated. It was hung up on ?prepare section files? for nearly the whole 1-week period. I canceled the run, then tried running again without parallelization (in the same folder as the failed run). I got an error about each contig failing, and something about Augustus. Unfortunately, I didn?t save the error message. I tried to regenerate the error, but it looks like I would probably need to completely rerun? this is the error message now (repeated for each contig), if I run in the same folder again. > > The contig failed 2 times!! > Maker will not try again!! > The contig will be stored in a fasta file that you can use for debugging. > SeqID: PGA_scaffold4__8_contigs__length_31799528 > Length: 31799528 > FASTA: /home/ubuntu/PGScratch/chunk1/chunk1.maker.output/chunk1_datastore/10/91/PGA_scaffold4__8_contigs__length_31799528//PGA_scaffold4__8_contigs__length_31799528.died.fasta > > Any thoughts about why mpiexec seems unable to finish? I tried a run on the test data using parallelization, and it completed in under a minute. > > Thanks! > > Hayley > -- > Hayley Mangelson > Bioinformatics Scientist > Phase Genomics, Inc -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1376 bytes Desc: not available URL: From jwliao25 at gmail.com Mon Mar 8 03:25:30 2021 From: jwliao25 at gmail.com (=?UTF-8?B?5buW5a6257ev?=) Date: Mon, 8 Mar 2021 18:25:30 +0800 Subject: [maker-devel] Maker annotation result contain 10% of gene with incorrect start or stop codon Message-ID: Hi maker-devel group, I used the maker with SNAP and Augustus for annotating a green algae genome. I always set the 'always_complete=1' from the first round of annotation. After Augustus training, I still get around 1111 and 208 genes that don't have the correct start and stop codon. (Total annotated gene number is 12696) I provided the close species proteome and the green algae its own RNA-seq data for EST hint. Does that make sense? Does there have any way to improve the result or fix the incorrect start and stop codon for those gene? best, Chai-Wei Liao -------------------------------------------------- Chia-Wei Liao ??? Research Assistant Institute of Molecular Biology, Academia Sinica, Taipei City, Taiwan Phone: 886-2-2789-9216 (Lab) -------------- next part -------------- An HTML attachment was scrubbed... URL: From MARCO.SOLLITTO at phd.units.it Mon Mar 8 11:27:14 2021 From: MARCO.SOLLITTO at phd.units.it (SOLLITTO MARCO [PHD0100064]) Date: Mon, 8 Mar 2021 18:27:14 +0000 Subject: [maker-devel] Maker- Issue with repbase Message-ID: Dear all, I am using MAKER for the annotation of my de novo genome assembly. I was able to install its conda version, but I have some problems when I run the following command: mpiexec -n 64 maker maker_opts.ctl maker_bopts.ctl maker_exe.ctl | tee maker.log It gives me the following output: STATUS: Parsing control files... ERROR: Could not open the control file "maker_opts.ctl". --> rank=NA, hostname=atlas mpiexec: Forwarding signal 20 to job How could I solve it? Thanks for your support and availability! Best regards, Marco -------------- next part -------------- An HTML attachment was scrubbed... URL: From hayley at phasegenomics.com Tue Mar 9 12:17:10 2021 From: hayley at phasegenomics.com (Hayley Mangelson) Date: Tue, 9 Mar 2021 14:17:10 -0500 Subject: [maker-devel] MAKER Commercial User -- Question In-Reply-To: <41C60EAE-6685-4259-A48A-1EFF7E185D2E@gmail.com> References: <08318e5bf6d847f2a1c4b6423e09dc13@utah.edu> <41C60EAE-6685-4259-A48A-1EFF7E185D2E@gmail.com> Message-ID: Hi Carson, Unfortunately, it looks like I am running MPICH version 3.0.4, so that doesn't explain the problem. I regenerated the error for you: STATUS: Parsing control files... STATUS: Processing and indexing input FASTA files... STATUS: Setting up database for any GFF3 input... A data structure will be created for you at: /home/ubuntu/PGScratch/scaff1/scaff0.maker.output/scaff0_datastore To access files for individual sequences use the datastore index: /home/ubuntu/PGScratch/scaff1/scaff0.maker.output/scaff0_master_datastore_index.log STATUS: Now running MAKER... examining contents of the fasta file and run log --Next Contig-- Processing run.log file... MAKER WARNING: The file scaff0.maker.output/scaff0_datastore/0B/6F/PGA_scaffold0__8_contigs__length_34808196//theVoid.PGA_scaffold0__8_contigs__length_34808196/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.1.BUSCO_hemistictus_3028031324.augustus did not finish on the last run and must be erased MAKER WARNING: The file scaff0.maker.output/scaff0_datastore/0B/6F/PGA_scaffold0__8_contigs__length_34808196//theVoid.PGA_scaffold0__8_contigs__length_34808196/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.0.BUSCO_hemistictus_3028031324.augustus did not finish on the last run and must be erased MAKER WARNING: The file scaff0.maker.output/scaff0_datastore/0B/6F/PGA_scaffold0__8_contigs__length_34808196//theVoid.PGA_scaffold0__8_contigs__length_34808196/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.2.BUSCO_hemistictus_3028031324.augustus did not finish on the last run and must be erased #--------------------------------------------------------------------- Now retrying the contig!! SeqID: PGA_scaffold0__8_contigs__length_34808196 Length: 34808196 Tries: 2!! #--------------------------------------------------------------------- setting up GFF3 output and fasta chunks preparing ab-inits running augustus. #--------- command -------------# Widget::augustus: /home/ubuntu/PGTools/Augustus/bin/augustus --species=BUSCO_hemistictus_3028031324 --UTR=off /tmp/maker_pzUjBq/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.0 > /tmp/maker_pzUjBq/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.0.BUSCO_hemistictus_3028031324.augustus #-------------------------------# /home/ubuntu/PGTools/Augustus/bin/augustus: ERROR Invalid nucleotide '' encountered. /home/ubuntu/PGTools/Augustus/bin/augustus: ERROR Invalid nucleotide '?' encountered. ERROR: Augustus failed --> rank=NA, hostname=ip-172-31-14-148 ERROR: Failed while preparing ab-inits ERROR: Chunk failed at level:0, tier_type:2 FAILED CONTIG:PGA_scaffold0__8_contigs__length_34808196 ERROR: Chunk failed at level:4, tier_type:0 FAILED CONTIG:PGA_scaffold0__8_contigs__length_34808196 examining contents of the fasta file and run log --Next Contig-- Processing run.log file... MAKER WARNING: The file scaff0.maker.output/scaff0_datastore/0B/6F/PGA_scaffold0__8_contigs__length_34808196//theVoid.PGA_scaffold0__8_contigs__length_34808196/PGA_scaffold0__8_contigs__length_34808196.abinit_masked.0.BUSCO_hemistictus_3028031324.augustus did not finish on the last run and must be erased Maker is now finished!!! Start_time: 1615317386 End_time: 1615317388 Elapsed: 2 On Fri, Mar 5, 2021 at 7:41 PM Carson Holt wrote: > Hi Hayley, > > Which flavor of MPI are you using? OpenMPI, MVAPICH2, MPICH, or Intel > MPI? There is a known issue with infiniband network cards that causes them > to freeze on certain system calls, and I bet that is what you are seeing. > It is not a MAKER issue, but a limitation in order to allow performance > benefits elsewhere with MPI. You need to add extra command line flags to > tell MPI to communicate a bit differently to get around it. You can find > multiple threads with more info in the maker-devel archives here ?> > https://groups.google.com/g/maker-devel > > > For OpenMPI you may need to add these flags to the mpiexec command line > ?> --mca btl vader,tcp,self --mca btl_tcp_if_include ib0 --mca > btl_openib_want_fork_support 1 --mca mpi_warn_on_fork 0 > > For intel MPI set the environmental variable I_MPI_FABRICS=shm:tcp before > running mpiexec > > For MPICH, no modifications should be needed (it doesn?t experience the > issue). > > For MVAPICH2, there is no work around and you just need to use one of the > other MPI flavors. > > > For the second failure, if you can regenerate it, send me the STDERR and I > will take a look. > > Thanks, > Carson > > > > > *From:* Hayley Mangelson > *Sent:* Thursday, March 4, 2021 12:03 PM > *To:* AARON A DUFFY > *Cc:* Mary Wood > *Subject:* MAKER Commercial User -- Question > > Hi Aaron, > > Mary and I are bioinformaticians at Phase Genomics, and we are using the > MAKER software (for which Phase purchased a licence) to annotate a couple > of genomes. We are using the same workflow that I used for a previous > project, but are having the same issues while working on separate projects. > I am hoping you could help us out or direct us to someone who can help! > > Here is some background. I am trying to get it to run using *mpiexec* for > parallelization. I let it run for about a week and all processes remained > ?active?, but no new output seemed to be generated. It was hung up on > ?prepare section files? for nearly the whole 1-week period. I canceled the > run, then tried running again without parallelization (in the same folder > as the failed run). I got an error about each contig failing, and something > about Augustus. Unfortunately, I didn?t save the error message. I tried to > regenerate the error, but it looks like I would probably need to completely > rerun? this is the error message now (repeated for each contig), if I run > in the same folder again. > > > > > > > *The contig failed 2 times!!Maker will not try again!!The contig will be > stored in a fasta file that you can use for debugging.SeqID: > PGA_scaffold4__8_contigs__length_31799528Length: 31799528FASTA: > /home/ubuntu/PGScratch/chunk1/chunk1.maker.output/chunk1_datastore/10/91/PGA_scaffold4__8_contigs__length_31799528//PGA_scaffold4__8_contigs__length_31799528.died.fasta* > > Any thoughts about why *mpiexec *seems unable to finish? I tried a run on > the test data using parallelization, and it completed in under a minute. > > Thanks! > > Hayley > -- > Hayley Mangelson > Bioinformatics Scientist > Phase Genomics, Inc > > > -- Hayley Mangelson Bioinformatics Scientist Phase Genomics, Inc -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Wed Mar 10 13:44:12 2021 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 10 Mar 2021 13:44:12 -0700 Subject: [maker-devel] Find evidence for specific gene annotation (without any genome browser!) In-Reply-To: <6ba7a71ce9970fb0dbf072a61df0f18f@uni-duesseldorf.de> References: <6ba7a71ce9970fb0dbf072a61df0f18f@uni-duesseldorf.de> Message-ID: <0C551067-369B-4341-9F78-5D959BB61798@gmail.com> It?s based on overlap. If you want to know the specific evidence you take everything that overlap the gene coordinates and cluster that together. Bedtools might be the easiest way to get everything for a given set of coordinates. For naming based off of protein2genome alignment, one option is to set est_forward=1 in the control files during the maker run (must manually add it). It is a hack for fast protein mapping of previous annotations onto new assemblies. But you can use it to force naming of any protein source. There are several more examples of using this option in the maker-devel archives ?> https://groups.google.com/g/maker-devel/search?q=est_forward ?Carson > On Jan 22, 2021, at 5:42 AM, Ricardo Nuno Ferreira Martins Guerreiro wrote: > > > Hello, > > Simple question: How do I know which evidence generated one specific gene? Or even, which evidences are used for it's AED calculation? > > How do I do this if the gene is predicted directly by a prot2genome alignment? And how when it's an Augustus annotation? > > I want to do this without Jbrowser, which is a huge waste of time. I want the original name in my input protein set, not the maker name. > > > Cheers, > Ricardo > > _______________________________________________ > maker-devel mailing list > maker-devel at yandell-lab.org > http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1376 bytes Desc: not available URL: From carsonhh at gmail.com Wed Mar 10 14:26:25 2021 From: carsonhh at gmail.com (Carson Holt) Date: Wed, 10 Mar 2021 14:26:25 -0700 Subject: [maker-devel] Issues with locking in MPI mode In-Reply-To: References: Message-ID: Look for hidden files with .NFSLock in the name, delete them, and see if they come back. find | grep .NFSLock | xargs rm find | grep .NFSLock If the files come back after deleting them, it can mean another MAKER job is still running and updating the lock. Can happen in weird situations like when process managers like slurm OOM kill a job but only wipe out some of the processes and not all. ?Carson > On Jan 21, 2021, at 6:30 AM, Michele Vidotto wrote: > > Dear all, > > as reported in the subject I'm having issues with locking mechanism of MAKER when it is runs in parallel-mode through mpi. > I'm using maker version 3.01.03 but the same happens in my system when I build and install version 2.31.11. > All prerequisites were installed in a conda environment. Perl was installed from anaconda channel in version 5.26.2. Hard-coded paths to the compilers were fixed. Necessary perl modules were installed via cpanm: > > "DBD::SQLite", > "DBI", > "Error", > "Error::Simple", > "File::NFSLock", > "File::Which", > "forks", > "forks::shared", > "Inline", > "Inline::C", > "IO::All", > "IO::Prompt", > "LWP::Simple" > "Perl::Unsafe::Signals", > "PerlIO::gzip", > "Proc::Simple", > "URI::Escape", > "DBD::Pg" > > additional libraries and components were installed via conda > > - gcc_linux-64=7.3.0 > - gxx_linux-64=7.3.0 > - openmpi=4.1.0 > - zlib=1.2.11 > - libdb=6.1.26 > - expat=2.2.9 > - libxml2=2.9.10 > - exonerate=2.4.0 > - snoscan=1.0 > - rapsearch=2.24 > > other components were installed manually. MAKER compile and install with no errors, but when I execute the program via MPI with: > > # to devoid OPEN MPI segmentation fault > export THREADS_DAEMON_MODEL=1 > > mpiexec -mca btl ^openib -n 1 \ > maker \ > -force \ > -cpus 8 \ > --fix_nucleotides \ > maker_opts.ctl \ > maker_bopts.ctl \ > maker_exe.ctl > > It always ends up with following error: > > > STATUS: Parsing control files... > ERROR: The directory is locked. Perhaps by an instance of MAKER. > > --> rank=NA, hostname=april.corp.igatechnology.com > -------------------------------------------------------------------------- > Primary job terminated normally, but 1 process returned > a non-zero exit code. Per user-direction, the job has been aborted. > -------------------------------------------------------------------------- > -------------------------------------------------------------------------- > mpiexec detected that one or more processes exited with non-zero status, thus causing > the job to be terminated. The first process to do so was: > > Process name: [[19321,1],0] > Exit code: 10 > -------------------------------------------------------------------------- > > if I look inside *.maker.output directory a lock file remains: > > .NFSLock.gi_lock.NFSLock > > If instead I run maker with the -nolock flag. MAKER runs with no problems at all. > > My filesystem is oneFS from ISILON, exported to a virtual server through nfs4 protocol. > By looking at the code MAKER uses File::NFSLock Perl module for locking. This module fails some tests when installed on my system with cipanm: > > # Failed test at t/300_bl_sh.t line 115. > Shared locks not running simultaneously at t/300_bl_sh.t line 116, <$rd3> line 18. > # Looks like your test exited with 4 just after 27. > t/300_bl_sh.t ..... Dubious, test returned 4 (wstat 1024, 0x400) > Failed 47/73 subtests > t/400_kill.t ...... ok > t/410_die.t ....... ok > t/420_crash.t ..... ok > t/430_taint.t ..... ok > > Test Summary Report > ------------------- > t/300_bl_sh.t (Wstat: 1024 Tests: 27 Failed: 1) > Failed test: 27 > Non-zero exit status: 4 > Parse errors: Bad plan. You planned 73 tests but ran 27. > > > > But anyway I was able to install it with --notest flag. > Do you have any idea on how I can overcome my problem and have MAKER run in parallel with MPI? > > Thanks in advance, > > > > > --- > Michele Vidotto > mailto: michele.vidotto at gmail.com _______________________________________________ > maker-devel mailing list > maker-devel at yandell-lab.org > http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 1376 bytes Desc: not available URL: From bioinf49 at gmail.com Thu Mar 18 18:58:25 2021 From: bioinf49 at gmail.com (Inf Bio) Date: Fri, 19 Mar 2021 08:58:25 +0800 Subject: [maker-devel] ERROR: Failed while processing all repeats Message-ID: Dear Developers, I've researched all available forums and tried to resolve this recurring error in maker: processing all repeats in cluster::shadow_cluster... Died at /home/user/bin/maker/bin/../lib/Bio/Search/Hit/PhatHit/Base.pm line 188. --> rank=19, hostname=m30g4 ERROR: Failed while processing all repeats ERROR: Chunk failed at level:3, tier_type:1 So far, I have tried: reformatting my gff file including purify script from Carson, Daren Card's script, and agat; uninstalled and reinstalled ncbi-blast+ and updated path; reinstated RMBlast path in repeatmasker; changed max_dna_length to 300000; set clean_try=1; replaced RepeatMasker.pm. However, none of the above helped get rid of the issue. I cannot see what is wrong with my gff. Can you please help? thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jacques.dainat at nbis.se Thu Mar 25 15:03:49 2021 From: jacques.dainat at nbis.se (Jacques Dainat) Date: Thu, 25 Mar 2021 22:03:49 +0100 Subject: [maker-devel] Maker annotation result contain 10% of gene with incorrect start or stop codon In-Reply-To: References: Message-ID: Hi, I met this problem in some projects where the ORFs were not well defined. In the mRNA it was not the longest ORF chosen, which is not necessarily wrong but here it was obviously not the correct one chosen. Probably due to bad training of my abinitio tools. I ended up to develop a script to fix the predictions and use the longest ORF as CDS. The script is called agat_sp_fix_longest_ORF.pl It is available within AGAT (https://github.com/NBISweden/AGAT) Hoping it could help, Best regards, Jacques Dainat, Ph.D. > On 8 Mar 2021, at 11:25, ??? wrote: > > Hi maker-devel group, > > I used the maker with SNAP and Augustus for annotating a green algae genome. I always set the 'always_complete=1' from the first round of annotation. > > After Augustus training, I still get around 1111 and 208 genes that don't have the correct start and stop codon. (Total annotated gene number is 12696) > > I provided the close species proteome and the green algae its own RNA-seq data for EST hint. > > Does that make sense? Does there have any way to improve the result or fix the incorrect start and stop codon for those gene? > > best, > Chai-Wei Liao > > -------------------------------------------------- > > Chia-Wei Liao ??? > > Research Assistant > > Institute of Molecular Biology, > > Academia Sinica, Taipei City, Taiwan > > Phone: 886-2-2789-9216 (Lab) > > _______________________________________________ > maker-devel mailing list > maker-devel at yandell-lab.org > http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From carsonhh at gmail.com Fri Mar 26 09:59:55 2021 From: carsonhh at gmail.com (Carson Holt) Date: Fri, 26 Mar 2021 09:59:55 -0600 Subject: [maker-devel] Maker annotation result contain 10% of gene with incorrect start or stop codon In-Reply-To: References: