<div dir="ltr">Thank you very much Carson for your timely response,<div>Yes I think so. The Maker MPI should support the multi-processing of an ultra-long single sequence. But I cannot run it successfully for a single sequence.</div><div>First I make sure the openMPI with maker has been installed properly. It works well for multiple DNA sequences in a parallel way.</div><div>When I submit a maker job for an ultra-long single sequence (mpiexec -mca btl ^openib -n 40 maker -g scaffold1.fasta -fix_nucleotides, The max_dna_len is set to 100000). It always left only one maker thread run. The other maker threads are disappeared and show finished in the output information. See the output information below. Please help me to check it. Thanks for your kind help and your time.</div><div><br></div><div>#-----MAKER Behavior Options<br>max_dna_len=100000 #length for dividing up contigs into chunks (increases/decreases memory usage)<br>min_contig=1 #skip genome contigs below this length (under 10kb are often useless)<br></div><div><br></div><div>Best regards,</div><div>Taosheng</div><div><br></div><div><br></div><div><br></div><div>OUTPUT Information<br>STATUS: Processing and indexing input FASTA files...<br>STATUS: Setting up database for any GFF3 input...<br>examining contents of the fasta file and run log<br>A data structure will be created for you at:<br>/data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore<br><br>To access files for individual sequences use the datastore index:<br>/data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_master_datastore_index.log<br><br>STATUS: Now running MAKER...<br><br><br><br>--Next Contig--<br><br>Processing run.log file...<br>examining contents of the fasta file and run log<br><br><br><br>--Next Contig--<br><br>#---------------------------------------------------------------------<br>Another instance of maker is processing this contig!!<br>SeqID: scaffold#1<br>Length: 73580997<br>#---------------------------------------------------------------------<br><br><br>examining contents of the fasta file and run log<br><br><br><br>--Next Contig--<br><br>#---------------------------------------------------------------------<br>Another instance of maker is processing this contig!!<br>SeqID: scaffold#1<br>Length: 73580997<br>#---------------------------------------------------------------------<br><br><br><br><br><font color="#ff0000">Maker is now finished!!!</font><br><br><br><br>Start_time: 1603000030<br>End_time: 1603000033<br>Elapsed: 3<br>A data structure will be created for you at:<br>/data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore<br><br>To access files for individual sequences use the datastore index:<br>/data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_master_datastore_index.log<br><br>STATUS: Now running MAKER...<br>MAKER WARNING: The file plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/28/scaffold#1.289.plant_repeatFinal%2Elib%2Empi%2E10%2E2.specific.out<br>did not finish on the last run and must be erased<br>WARNING: Multiple MAKER processes have been started in the<br>same directory.<br><br>STATUS: Processing and indexing input FASTA files...<br>examining contents of the fasta file and run log<br><br><br><br>--Next Contig--<br><br>#---------------------------------------------------------------------<br>Another instance of maker is processing this contig!!<br>SeqID: scaffold#1<br>Length: 73580997<br>#---------------------------------------------------------------------<br><br><br>WARNING: Multiple MAKER processes have been started in the<br>same directory.<br><br>STATUS: Processing and indexing input FASTA files...<br>STATUS: Setting up database for any GFF3 input...<br>STATUS: Setting up database for any GFF3 input...<br>A data structure will be created for you at:<br>/data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore<br><br>To access files for individual sequences use the datastore index:<br>/data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_master_datastore_index.log<br><br>STATUS: Now running MAKER...<br>examining contents of the fasta file and run log<br><br><br><br>--Next Contig--<br><br>#---------------------------------------------------------------------<br>Another instance of maker is processing this contig!!<br>SeqID: scaffold#1<br>Length: 73580997<br>#---------------------------------------------------------------------<br><br><br><br><br><font color="#cc0000"><b>Maker is now finished!!!</b></font><br><br><br><br>Start_time: 1603000030<br>End_time: 1603000034<br>Elapsed: 4<br>#---------------------------------------------------------------------<br>Now starting the contig!!<br>SeqID: scaffold#1<br>Length: 73580997<br>#---------------------------------------------------------------------<br><br><br>examining contents of the fasta file and run log<br><br><br><br>--Next Contig--<br><br>#---------------------------------------------------------------------<br>Another instance of maker is processing this contig!!<br>SeqID: scaffold#1<br>Length: 73580997<br>#---------------------------------------------------------------------<br><br><br>examining contents of the fasta file and run log<br><br><br><br>--Next Contig--<br><br>#---------------------------------------------------------------------<br>Another instance of maker is processing this contig!!<br>SeqID: scaffold#1<br>Length: 73580997<br>#---------------------------------------------------------------------<br><br><br><br><br>Start_time: 1603000030<br>End_time: 1603000034<br>Elapsed: 4<br><br><br><font color="#cc0000"><b>Maker is now finished!!!</b></font><br><br>A data structure will be created for you at:<br>/data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore<br><br>To access files for individual sequences use the datastore index:<br>/data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_master_datastore_index.log<br><br>STATUS: Now running MAKER...<br>examining contents of the fasta file and run log</div><div>..........</div><div><br></div><div>Maker is now finished!!!<br><br><br><br>Start_time: 1602920407<br>End_time: 1602920580<br>Elapsed: 173<br>running repeat masker.<br>#--------- command -------------#<br>Widget::RepeatMasker:<br>cd /tmp/maker_VflRYH; ./maker3.01/exe/RepeatMasker/RepeatMasker /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/0/scaffold#1.0.plant_repeatFinal%2Elib%2Empi%2E10%2E0.specific -dir /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/0 -pa 1 -lib /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/mpi_blastdb/plant_repeatFinal%2Elib.mpi.10/plant_repeatFinal%2Elib.mpi.10.0<br>#-------------------------------#<br>running repeat masker.<br>#--------- command -------------#<br>Widget::RepeatMasker:<br>cd /tmp/maker_VflRYH; ./maker3.01/exe/RepeatMasker/RepeatMasker /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/0/scaffold#1.0.plant_repeatFinal%2Elib%2Empi%2E10%2E1.specific -dir /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/0 -pa 1 -lib /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/mpi_blastdb/plant_repeatFinal%2Elib.mpi.10/plant_repeatFinal%2Elib.mpi.10.1<br>#-------------------------------#<br>running repeat masker.<br>#--------- command -------------#<br>Widget::RepeatMasker:<br>cd /tmp/maker_VflRYH; ./maker3.01/exe/RepeatMasker/RepeatMasker /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/0/scaffold#1.0.plant_repeatFinal%2Elib%2Empi%2E10%2E2.specific -dir /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/0 -pa 1 -lib /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/mpi_blastdb/plant_repeatFinal%2Elib.mpi.10/plant_repeatFinal%2Elib.mpi.10.2<br>#-------------------------------#<br>running repeat masker.<br>#--------- command -------------#<br>Widget::RepeatMasker:<br>cd /tmp/maker_VflRYH; ./maker3.01/exe/RepeatMasker/RepeatMasker /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/0/scaffold#1.0.plant_repeatFinal%2Elib%2Empi%2E10%2E3.specific -dir /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/plant_datastore/B5/CD/scaffold#1//theVoid.scaffold#1/0 -pa 1 -lib /data/splitgenome/top1000/splitGenome/part1/plant.maker.output/mpi_blastdb/plant_repeatFinal%2Elib.mpi.10/plant_repeatFinal%2Elib.mpi.10.3<br>#-------------------------------#<br>running repeat masker.<br>#--------- command -------------#<br></div><div><br></div><div>.....</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Oct 19, 2020 at 10:32 PM Carson Holt <<a href="mailto:carsonhh@gmail.com">carsonhh@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Yes. It will divide contigs into chunks the same size as the max_dna_len parameter.<br>
<br>
—Carson<br>
<br>
Sent from my iPhone<br>
<br>
> On Oct 17, 2020, at 12:48 AM, Xu, taosheng <<a href="mailto:taosheng.x@gmail.com" target="_blank">taosheng.x@gmail.com</a>> wrote:<br>
> <br>
> <br>
> Dear Maker Development Team,<br>
> I wonder whether maker supports parallel processing for a single long genome sequence?<br>
> <br>
> When I submit my maker task using openMPI with multiple cpus (like, mpiexec -n 40 maker) to annotate a single long genome sequence, always only one maker with 4 rmblast run. The other cpus is on idle.<br>
> <br>
> I want to use the maker parallel processing with openMPI to speed up a single ultra-long genome sequence annotation.<br>
> <br>
> Best regards,<br>
> Taosheng<br>
> _______________________________________________<br>
> maker-devel mailing list<br>
> <a href="mailto:maker-devel@yandell-lab.org" target="_blank">maker-devel@yandell-lab.org</a><br>
> <a href="http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org" rel="noreferrer" target="_blank">http://yandell-lab.org/mailman/listinfo/maker-devel_yandell-lab.org</a><br>
</blockquote></div>