[maker-devel] questions on master_datastore_index.log file
Carson Holt
carsonhh at gmail.com
Tue Jan 9 10:14:05 MST 2018
Your contig names may create issues. Specifically the ‘;’ character, but you should also remove the ‘=‘ character. However, I believe your problem may be IO. If you are running under MPI or are running multiple jobs, the disk one of the machines may have that location unmounted, it may be full, you may have hit a system file quota limit, or the IO load is slowing it is not actually finished writing the file when MAKER tries to read it. If IO load, is the issue, then you just need to run fewer processes. The other possibilities would mean you need to make space, fix the mount, or raise any quotas on your systems.
Thanks,
Carson
> On Jan 6, 2018, at 4:09 PM, Qihua Liang <qlian003 at ucr.edu> wrote:
>
> Hi Carson,
>
> I am pasting more lines of error messages. I notice an error of "ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq”, the seq name of “ScsGwly” is ">ScsGwly_6124;HRSCAF=6247”, is it because of the seq naming that makes the temp file name weird?
>
> Thanks
> Qihua
>
> #--------- command -------------#
> Widget::blastx:
> /24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_nJDkCL/te_proteins%2Efasta.mpi.10.9 -query /tmp/maker_nJDkCL/0/ScsG
> wly_5932%3BHRSCAF=6050.0 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes
> -soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/D7/4
> A/ScsGwly_5932%3BHRSCAF=6050//theVoid.ScsGwly_5932%3BHRSCAF=6050/0/ScsGwly_5932%3BHRSCAF=6050.0.te_proteins%2Efasta.repeatrunner.temp_dir/te_
> proteins%2Efasta.mpi.10.9.repeatrunner
> #-------------------------------#
> deleted:0 hits
> collecting blastx repeatmasking
> processing all repeats
> in cluster::shadow_cluster...
> ...finished clustering.
> preparing masked sequence
> preparing ab-inits
> running snap.
> #--------- command -------------#
> Widget::snap:
> /24-2/home/qliang/0.soft/maker/exe/snap/snap /home/qliang/cowpea/annotation/09.tingting/4.Abintio/2.CEGMA/3.maker/maker1.hmm/maker1.snap.hmm
> /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.maker1%2Esnap%2Eh
> mm.snap
> #-------------------------------#
> scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
> scoring....decoding.10.20.30.40.50.60.70.80.90.100 done
> running augustus.
> #--------- command -------------#
> Widget::augustus:
> /usr/local/augustus.2.7/bin/augustus --species=cowpea_new --UTR=off /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0 > /tmp/maker
> _nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_masked.0.cowpea_new.augustus
> #-------------------------------#
> deleted:0 hits
> collecting blastx repeatmasking
> processing all repeats
> in cluster::shadow_cluster...
> ...finished clustering.
> deleted:0 hits
> doing blastx repeats
> running blast search.
> #--------- command -------------#
> Widget::blastx:
> /24-2/home/qliang/0.soft/maker/bin/../exe/blast/bin/blastx -db /tmp/maker_mvdRkd/te_proteins%2Efasta.mpi.10.6 -query /tmp/maker_mvdRkd/0/chr10.75 -num_alignments 10000 -num_descriptions 10000 -evalue 1e-06 -dbsize 300 -searchsp 500000000 -num_threads 1 -seg yes -soft_masking true -lcase_masking -show_gis -out /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/09/chr10//theVoid.chr10/7/chr10.75.te_proteins%2Efasta.repeatrunner.temp_dir/te_proteins%2Efasta.mpi.10.6.repeatrunner
> #-------------------------------#
> doing blastx repeats
> re reading blast report.
> /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/0/ScsGwly_6124%3BHRSCAF=6247.0.te_proteins%2Efasta.repeatrunner
> deleted:0 hits
> doing blastx repeats
> doing blastx repeats
> doing blastx repeats
> doing blastx repeats
> doing blastx repeats
> doing blastx repeats
> doing blastx repeats
> doing blastx repeats
> doing blastx repeats
> collecting blastx repeatmasking
> processing all repeats
> in cluster::shadow_cluster...
> ...finished clustering.
> ERROR: Can't open seq file: /24-2/home/qliang/cowpea/annotation/22.dovetail.assembly/map.maker.output/map_datastore/ED/F1/ScsGwly_6124%3BHRSCAF=6247//theVoid.ScsGwly_6124%3BHRSCAF=6247/query.masked.gff.seq
> No such file or directory
>
> at /24-2/home/qliang/0.soft/maker/bin/../lib/Dumper/GFF/GFFV3.pm line 199.
> Dumper::GFF::GFFV3::finalize(Dumper::GFF::GFFV3=HASH(0x5000ab8)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 700
> Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
> eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
> Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
> Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
> Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
> Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
> eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
> Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
> Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179
> Process::MpiTiers::next_chunk(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 286
> Process::MpiTiers::run_all(Process::MpiTiers=HASH(0x4fb3350), 0) called at /home/qliang/0.soft/maker/bin/maker line 695
> --> rank=NA, hostname=H4
> ERROR: Failed while builing masking tiers
> --> rank=NA, hostname=H4
> --> rank=NA, hostname=H4
> ERROR: Can not get next level
> running genemark.
> #--------- command -------------#
> Widget::genemark:
> /24-2/home/qliang/0.soft/PerlPackages/ActivePerl-5.22/bin/perl-static /24-2/home/qliang/0.soft/maker/bin/../lib/Widget/genemark/gmhmm_wrap -m /home/qliang/cowpea/annotation/05.CEGMA/2.genemask/output/gmhmm.mod -g /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/gmhmme3 -p /24-2/home/qliang/0.soft/makerPackages/gm_et_linux_64/gmes_petap/probuild -o /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0.gmhmm%2Emod.genemark /tmp/maker_nJDkCL/ScsGwly_5932%3BHRSCAF=6050.abinit_nomask.0
> #-------------------------------#
> FAILED CONTIG:ScsGwly_6124;HRSCAF=6247
>
> examining contents of the fasta file and run log
>
>
>
> --Next Contig--
>
> #---------------------------------------------------------------------
> Now starting the contig!!
> SeqID: ScsGwly_6140;HRSCAF=6263
> Length: 1247
> #---------------------------------------------------------------------
>
>
>
>
>> On Jan 5, 2018, at 7:22 PM, Carson Holt <carsonhh at gmail.com> wrote:
>>
>> That’s the stack trace. The error is going to be a few lines further back. It would be best to get a few hundred lines right around the area you are showing.
>>
>> —Carson
>>
>>> On Jan 4, 2018, at 2:36 PM, Qihua Liang <qlian003 at ucr.edu> wrote:
>>>
>>> Hi Ence,
>>>
>>> When I searched for “E/error” in the output file, here is what first showed up:
>>> Process::MpiChunk::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
>>> eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
>>> Error::subs::try(CODE(0x502bbb0), HASH(0x5007788)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 4269
>>> Process::MpiChunk::_go(Process::MpiChunk=HASH(0x50a1a18), "flow", HASH(0x50ad0f0), 2, 0) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiChunk.pm line 378
>>> Process::MpiChunk::_flow(Process::MpiChunk=HASH(0x50a1a18), HASH(0x50ad0f0), 2, 0, Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 318
>>> Process::MpiTiers::__ANON__() called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 415
>>> eval {...} called at /24-2/home/qliang/0.soft/maker/bin/../lib/Error.pm line 407
>>> Error::subs::try(CODE(0x50a9348), HASH(0x4ff0ec0)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 338
>>> Process::MpiTiers::_next_level(Process::MpiTiers=HASH(0x4fb3350)) called at /24-2/home/qliang/0.soft/maker/bin/../lib/Process/MpiTiers.pm line 179
>>>
>>> Is this what you may need?
>>>
>>> Qihua
>>>
>>>> On Jan 4, 2018, at 6:16 AM, Ence,daniel <d.ence at ufl.edu> wrote:
>>>>
>>>> Hi, Before we can give any help to debug it, we need the error messages. These should be in the same file that the “maker is finished” message is in. Look for the first error message (the one closest to the top of the file) and send that to the mailing list.
>>>>
>>>> Thanks,
>>>> Daniel
>>>>
>>>>
>>>>> On Jan 3, 2018, at 8:52 PM, Qihua Liang <qlian003 at ucr.edu> wrote:
>>>>>
>>>>> Dear Maker Develop Team,
>>>>>
>>>>> I have successfully run Maker for several times before. But I came across a strange thing days ago when I ran Maker again on a different assembly with the same input files and settings.
>>>>>
>>>>> I saw the message of "Maker is now finished!!!” but got empty GFF3 and no fasta files. And then I checked the master_datastore_index.log and realized that there are a lot of “failed”s and “retry”s and “failed” again. What does this mean? Since I used same inputs as previous successful runs, could you provide some instructions on how to debug and solve it?
>>>>>
>>>>> Thank you so much
>>>>> Qihua
>>>>> _______________________________________________
>>>>> maker-devel mailing list
>>>>> maker-devel at box290.bluehost.com
>>>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__box290.bluehost.com_mailman_listinfo_maker-2Ddevel-5Fyandell-2Dlab.org&d=DwIGaQ&c=pZJPUDQ3SB9JplYbifm4nt2lEVG5pWx2KikqINpWlZM&r=12jzlNvGVD0AlPJ4E7cTlw1Dvu6n9cb4kMCobJ28XPs&m=nUDCP_0kFOhDYlTHgOpWtf_zdL77aQFeQwYOGIQwP8c&s=9Z4T1hdtxOyIjpn6f70qhrQRuGsZxXdV-oLSJF1zGkY&e=
>>>>
>>>
>>> _______________________________________________
>>> maker-devel mailing list
>>> maker-devel at box290.bluehost.com
>>> http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20180109/311e718f/attachment-0002.html>
More information about the maker-devel
mailing list