<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; "><div>Done.</div><div><br></div><div>Thanks,</div><div>Carson</div><div><br></div><div><br></div><span id="OLK_SRC_BODY_SECTION"><div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span> "Krishnakumar, Vivek" <<a href="mailto:vkrishnakumar@jcvi.org">vkrishnakumar@jcvi.org</a>><br><span style="font-weight:bold">Date: </span> Thursday, 24 January, 2013 4:29 PM<br><span style="font-weight:bold">To: </span> Carson Holt <<a href="mailto:carsonhh@gmail.com">carsonhh@gmail.com</a>><br><span style="font-weight:bold">Cc: </span> "<a href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>" <<a href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>><br><span style="font-weight:bold">Subject: </span> Re: [maker-devel] DBD::SQLite::db do failed: database is locked (GFFDB.pm lines 407, 408, 497) error - Using MAKER 2.27b MPI<br></div><div><br></div><div>
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000">
Ok. Seems to be working fine. I pointed the TMP parameter to our
scratch area.<br>
<br>
Thanks for your suggestion.<br>
Should I forward this conversation thread to the maker-devel list ?<br>
<br>
Vivek<br>
<br>
<br>
<div class="moz-cite-prefix">On 01/24/2013 04:08 PM, Carson Holt
wrote:<br>
</div>
<blockquote cite="mid:CD270651.17FB2%25carsonhh@gmail.com" type="cite">
<div>SQLlite will choke almost immediately if it's not set up with
the correct flags, so you will know right away.</div>
<div><br>
</div>
<div>--Carson</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt;
text-align:left; color:black; BORDER-BOTTOM: medium none;
BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT:
0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid;
BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span> "Krishnakumar,
Vivek" <<a moz-do-not-send="true" href="mailto:vkrishnakumar@jcvi.org">vkrishnakumar@jcvi.org</a>><br>
<span style="font-weight:bold">Date: </span> Thursday, 24
January, 2013 3:35 PM<br>
<span style="font-weight:bold">To: </span> Carson Holt <<a moz-do-not-send="true" href="mailto:carsonhh@gmail.com">carsonhh@gmail.com</a>><br>
<span style="font-weight:bold">Cc: </span> "<a moz-do-not-send="true" href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>"
<<a moz-do-not-send="true" href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>><br>
<span style="font-weight:bold">Subject: </span> Re:
[maker-devel] DBD::SQLite::db do failed: database is locked
(GFFDB.pm lines 407, 408, 497) error - Using MAKER 2.27b MPI<br>
</div>
<div><br>
</div>
<div>
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000"> Okay. I will try doing
that. But how can I find out that my NFS mounted area has
-nolock set? The filesystem areas that we work on are
managed by our IT department.<br>
<br>
Thanks.<br>
Vivek<br>
<br>
<div class="moz-cite-prefix">On 01/24/2013 03:27 PM, Carson
Holt wrote:<br>
</div>
<blockquote cite="mid:CD2702CC.17F93%25carsonhh@gmail.com" type="cite">
<div>If you don't have locally mounted space big enough,
then just use an NFS mounted /TMP. You will take a
slight performance hit, but it may still work if it is
mounted with -nolock set so SQLite stops complaining.</div>
<div><br>
</div>
<div>--Carson </div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt;
text-align:left; color:black; BORDER-BOTTOM: medium
none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in;
PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP:
#b5c4df 1pt solid; BORDER-RIGHT: medium none;
PADDING-TOP: 3pt"><span style="font-weight:bold">From:
</span> "Krishnakumar, Vivek" <<a moz-do-not-send="true" href="mailto:vkrishnakumar@jcvi.org">vkrishnakumar@jcvi.org</a>><br>
<span style="font-weight:bold">Date: </span>
Thursday, 24 January, 2013 3:24 PM<br>
<span style="font-weight:bold">To: </span> Carson
Holt <<a moz-do-not-send="true" href="mailto:carsonhh@gmail.com">carsonhh@gmail.com</a>><br>
<span style="font-weight:bold">Cc: </span> "<a moz-do-not-send="true" href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>"
<<a moz-do-not-send="true" href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>><br>
<span style="font-weight:bold">Subject: </span> Re:
[maker-devel] DBD::SQLite::db do failed: database is
locked (GFFDB.pm lines 407, 408, 497) error - Using
MAKER 2.27b MPI<br>
</div>
<div><br>
</div>
<div>
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000"> I no longer see
that warning.<br>
<br>
The only thing I see is that MAKER now complains
that the DB is corrupt, and that is because I only
have 5GB of /tmp space on the server I currently
have access to and that is not enough for the whole
genome FASTA + est2genome + protein2genome +
model_gff (EVM).<br>
<br>
Thanks,<br>
Vivek<br>
<br>
<br>
<div class="moz-cite-prefix">On 01/24/2013 03:20 PM,
Carson Holt wrote:<br>
</div>
<blockquote cite="mid:CD27013E.17F86%25carsonhh@gmail.com" type="cite">
<div>You should be able to run on an NFS drive.
After creating the initial SQLite database,
maker now gives each node it's own copy that is
stored in the TMP directory (should be local to
each node). Could you send me the GFF3 file
that gives the warning, so I can look at it.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Carson</div>
<div><br>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt;
text-align:left; color:black; BORDER-BOTTOM:
medium none; BORDER-LEFT: medium none;
PADDING-BOTTOM: 0in; PADDING-LEFT: 0in;
PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt
solid; BORDER-RIGHT: medium none; PADDING-TOP:
3pt"><span style="font-weight:bold">From: </span>
"Krishnakumar, Vivek" <<a moz-do-not-send="true" href="mailto:vkrishnakumar@jcvi.org">vkrishnakumar@jcvi.org</a>><br>
<span style="font-weight:bold">Date: </span>
Thursday, 24 January, 2013 3:17 PM<br>
<span style="font-weight:bold">To: </span>
Carson Holt <<a moz-do-not-send="true" href="mailto:carsonhh@gmail.com">carsonhh@gmail.com</a>><br>
<span style="font-weight:bold">Cc: </span> "<a moz-do-not-send="true" href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>"
<<a moz-do-not-send="true" href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>><br>
<span style="font-weight:bold">Subject: </span>
Re: [maker-devel] DBD::SQLite::db do failed:
database is locked (GFFDB.pm lines 407, 408,
497) error - Using MAKER 2.27b MPI<br>
</div>
<div><br>
</div>
<div>
<meta content="text/html; charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000"> Hi
Carson,<br>
<br>
I did make sure that every time I reran the
pipeline, I deleted the maker.output
directory and the temporary files in /tmp.
That way I know that the DBs get created
fresh.<br>
<br>
When it was openMPI based, I noticed that
irrespective of whether I enabled or
disabled the `-TMP` parameter, I would get
the database locked error. At this point,
mpicc was openMPI based and mpiexec (hydra)
was MPICH2 based.<br>
<br>
On installing MPICH2 as a non-root user and
rebuilding `maker`, this issue vanished.
Just an observation.<br>
<br>
The issue I have now with scaling up is that
I do not currently have access to a server
that has a large amount of non-NFS mounted
storage space (we do have such a server but
it has been reserved for a different project
by a group in our institute). Thus, the
creation of the SQLite db is halted mid way
and now MAKER complains that the database is
corrupt. I can either run the chromosomes
individually (serially) and then the
scaffolds as one group or wait to get a time
slot of that server.<br>
<br>
Will let you know if I have any more issues.<br>
<br>
Thank you.<br>
Vivek<br>
<br>
<br>
<div class="moz-cite-prefix">On 01/24/2013
03:02 PM, Carson Holt wrote:<br>
</div>
<blockquote cite="mid:CD26FB98.17F6C%25carsonhh@gmail.com" type="cite">
<div>If you still see issue, let me know.
I have a hard time seeing how just
moving to MPICH2 would have solved the
issue. It may also have been an initial
problem with the first GFF# db being
created and running on a new job
recreated it correctly. Either way, let
me know if it comes up on your bigger
run.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Carson</div>
<div><br>
</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri;
font-size:11pt; text-align:left;
color:black; BORDER-BOTTOM: medium
none; BORDER-LEFT: medium none;
PADDING-BOTTOM: 0in; PADDING-LEFT:
0in; PADDING-RIGHT: 0in; BORDER-TOP:
#b5c4df 1pt solid; BORDER-RIGHT:
medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span>
"Krishnakumar, Vivek" <<a moz-do-not-send="true" href="mailto:vkrishnakumar@jcvi.org">vkrishnakumar@jcvi.org</a>><br>
<span style="font-weight:bold">Date: </span>
Thursday, 24 January, 2013 2:49 PM<br>
<span style="font-weight:bold">To: </span>
<<a moz-do-not-send="true" href="mailto:maker-devel@yandell-lab.org">maker-devel@yandell-lab.org</a>>,
<<a moz-do-not-send="true" href="mailto:mcampbell@genetics.utah.edu">mcampbell@genetics.utah.edu</a>><br>
<span style="font-weight:bold">Subject:
</span> Re: [maker-devel]
DBD::SQLite::db do failed: database is
locked (GFFDB.pm lines 407, 408, 497)
error - Using MAKER 2.27b MPI<br>
</div>
<div><br>
</div>
<div>
<meta content="text/html;
charset=ISO-8859-1" http-equiv="Content-Type">
<div bgcolor="#FFFFFF" text="#000000">
Hi Everybody,<br>
<br>
Sorry for jumping the gun and
writing to the mailing list before
completely troubleshooting this
issue. I was able to figure out the
problem and get MAKER 2.27b working
fine in MPI mode. <br>
<br>
The issue was with the incorrect
version of mpicc (it was openMPI
based and not mpich2). I was able to
successfully run through the steps
described below on one chromosome in
under a few hours (~2). Now I'm
scaling it to the whole genome (8
chromosomes and ~2200 unanchored
scaffolds).<br>
<br>
Thank you.<br>
Vivek<br>
<br>
<div class="moz-cite-prefix">On
01/23/2013 02:42 PM, Krishnakumar,
Vivek wrote:<br>
</div>
<blockquote cite="mid:51003D32.4080908@jcvi.org" type="cite">
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
Hello,<br>
<br>
I'm trying to use the MAKER
pipeline to calculate AED scores
for the gene models coming out of
our annotation pipeline (EVidence
Modeler). We then want to compare
these computed AED scores with
those computed for the gene models
coming out of the MAKER2 pipeline.<br>
<br>
For this purpose, I configured the
maker_opts.ctl file like so:<br>
1) Specify gff file coming from
our annotation pipeline using
`model_gff` param <br>
2) Specify gff file containing
est2genome results (computed using
MAKER2) using the `est_gff` param<br>
3) Specify gff file containing
protein2genome results (computed
using MAKER2) using the
`protein_gff` param<br>
4) Specify the genome fasta
sequence file using the `genome`
param<br>
<br>
I recently configured MAKER 2.27b
to work using MPI and I tried
launching one contig (chromosome)
on a compute node spawning 12
processes like so:<br>
<big><tt>mpiexec -n 12 maker -R
-TMP /tmp/maker maker_opts.ctl
maker_bopts.ctl maker_exe.ctl
2>&1 | tee maker.err</tt></big><br>
<br>
Side note: Without MPI, it works
fine but takes very very long to
complete running on one chromosome
(since it is going through the
chunks serially).<br>
<br>
On inspecting the `maker.err`
file, I see server error messages
that say `DBD::SQLite::db do
failed: database is locked`. On
referring to a previous email
thread on the `maker-devel`
mailing list, I see that Carson
had replied to the person with a
similar issue asking them to check
if the working directory and TMP
directory are being mounted on an
NFS filesystem because this is a
potential problem with SQLite
databases.<br>
<br>
Based on that suggestion, I
switched the TMP directory to use
a local filesytem but the working
directory is still NFS mounted.
Despite that, I still get this
error. The previous email thread
had no resolution for this issue.
My question is, should the working
directory be local as well? I see
that despite setting the `TMP`
variable, the SQLite DB is getting
created in the working directory.<br>
<br>
What might be the issue?<br>
<br>
Also, apart from this one error, I
notice another error:<br>
<big><tt>ERROR: Non-unique top
level ID for chr5:hit:1454:4_0<br>
While this is technically
legal in GFF3, it usually<br>
indicates a poorly fomatted
GFF3 file (perhaps you<br>
tried to merge two GFF3 files
without accounting for<br>
unique IDs). MAKER will not
handle these correctly.</tt></big><br>
<br>
On inspecting the input GFF file
(protein2genome), I see that this
ID is never repeated. So I have no
idea whats going on here either.
Any suggestions?<br>
<br>
Below is my `maker.err` file from
the run. (Dropbox link)<br>
<br>
Thank you very much.<br>
Best,<br>
<br>
Vivek<br>
<br>
--<br>
Vivek Krishnakumar<br>
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:vkrishnakumar@jcvi.org">vkrishnakumar@jcvi.org</a><br>
<br>
<div style="padding: 15px;
background-color: rgb(217, 237,
255);">
<div style="margin-bottom:
15px;">I've linked 1 file to
this email:</div>
<div style="background-color:
rgb(255, 255, 255); padding:
15px;">
<div style="border: 1px solid
rgb(205, 205, 205);
border-radius: 5px 5px 5px
5px; margin-top: 10px;
margin-bottom: 10px;
padding: 15px;" class="cloudAttachmentItem"><img src="cid:part17.07020406.00030802@jcvi.org" style="margin-right: 5px;
float: left; width: 24px;
height: 24px;"><a moz-do-not-send="true" style="color: rgb(15, 126,
219) ! important;" href="http://db.tt/2tYyKCD7">maker.err</a><span style="margin-left: 5px;
font-size: small; color:
grey;">(97.7 MB)</span><span style="float: right;"><img style="margin-right:
5px;" src="cid:part19.00010506.01070403@jcvi.org"><a moz-do-not-send="true" style="color: rgb(15,
126, 219) ! important;" href="https://www.dropbox.com/">Dropbox</a></span><a moz-do-not-send="true" style="color: rgb(15, 126,
219) ! important;
font-size: small; display:
block;" href="http://db.tt/2tYyKCD7">http://db.tt/2tYyKCD7</a></div>
</div>
<div style="color: rgb(68, 68,
68); font-size: small;
margin-top: 15px;"><a moz-do-not-send="true" style="color: rgb(15, 126,
219) ! important;" href="http://www.getthunderbird.com">Mozilla
Thunderbird</a> makes it
easy to share large files over
email.</div>
</div>
<br>
</blockquote>
<br>
</div>
</div>
_______________________________________________
maker-devel mailing list <a moz-do-not-send="true" href="mailto:maker-devel@box290.bluehost.com">maker-devel@box290.bluehost.com</a>
<a moz-do-not-send="true" href="http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org">http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org</a>
</span> </blockquote>
<br>
</div>
</div>
</span> </blockquote>
<br>
</div>
</div>
</span> </blockquote>
<br>
</div>
</div>
</span>
</blockquote>
<br>
</div></div></span></body></html>