[maker-devel] errors in final gff

[Carson Holt]

Thanks. For the first two --> scaffold00002:hit:1026:1.3.0.12

The value 1026 is held in a global iterator, so it cannot repeat the same
value during the life of the process. And 1.3.0.12 is generated from the
point in the code the ID is being generated.  This means that two distinct
processses had to write to the same file at the same point in the code,
which should normally be impossible.

However, there are ways to make this happen.  First if you turn file locks
off (-nolock) option and then run MAKER multiple times on the same dataset
you can get process collisions (because you disabled the locks that stop
this).  If your NFS file system does not support hard links (FhGFS for
example) then you cannot lock the files (which is the same as setting
-nolock).  Or you have other serious IO failures over NFS. Note that NFS
is your Network Mounted Storage.

The last example you give shows the preceding line being truncated.  This
suggests that two processes are trying to write to the same file
simultaneously (inserting lines in between other lines), or serious IO
failures are occurring where writes are not completing but true is being
returned for the operations (can happen on unreliable NFS implementations).

So in summary either your NFS storage implementation is giving IO errors,
you have run MAKER with -nolock set and then started MAKER multiple times
in the same directory (process collisions), or your NFS implementation
doesn't support hardlinks and won't allow MAKER to lock files (process
collisions).  If it is one of the latter two, you will have to make sure
you never start MAKER more than once simultaneously on the same dataset.
You can still run via MPI fro parallelization, but you won't be able to
start a second MPI process while the first one is still running.

Thanks,
Carson


On 6/24/14, 12:56 PM, "Anurag Priyam" <a.priyam at qmul.ac.uk> wrote:

>I am sorry. I have updated the gist -
>https://gist.github.com/yeban/ffaf5cd419639dd073a7.
>1. The first two chunks contain the annotations with duplicate ids. (4
>rows)
>2. The last chunk contains the annotations that refer to a
>non-existent parent. And what looks like an incomplete line of
>annotation (I forgot to state this in my original email).
>
>No, I didn't use est_forward. I am not passing in any old data via GFF3.
>
>-- Priyam
>
>On Sat, Jun 21, 2014 at 3:26 AM, Carson Holt <carsonhh at gmail.com> wrote:
>> Also note that ID= must be unique. Name= does not have to be, and won't
>>be
>> if the same protein or repeat element aligns to more than one location
>>for
>> example.
>>
>> Thanks,
>> Carson
>>
>>
>> On 6/20/14, 3:50 PM, "Carson Holt" <carsonhh at gmail.com> wrote:
>>
>>>did you use est_forward?  Also in the example you showed all the IDs are
>>>unique (one says hit and the other hsp in the ID, so they are
>>>different)?
>>>Could you find the non-uunique IDs causing the error?
>>>
>>>--Carson
>>>
>>>
>>>On 6/19/14, 2:05 AM, "Anurag Priyam" <anurag08priyam at gmail.com> wrote:
>>>
>>>>I used est_gff= option, which refers to a GFF file generated by
>>>>cufflinks2gff3. The erroneous annotations didn't come from this GFF.
>>>>
>>>>-- Priyam
>>>>
>>>>On Thu, Jun 19, 2014 at 3:03 AM, Carson Holt <carsonhh at gmail.com>
>>>>wrote:
>>>>> Are you passing in old data via GFF3?
>>>>>
>>>>> --Carson
>>>>>
>>>>>
>>>>> On 6/18/14, 12:15 PM, "Anurag Priyam" <anurag08priyam at gmail.com>
>>>>>wrote:
>>>>>
>>>>>>It's version 2.31.
>>>>>>
>>>>>>-- Priyam
>>>>>>
>>>>>>On Wed, Jun 18, 2014 at 11:41 PM, Carson Holt <carsonhh at gmail.com>
>>>>>>wrote:
>>>>>>> What MAKER version are you using?
>>>>>>>
>>>>>>> --Carson
>>>>>>>
>>>>>>>
>>>>>>> On 6/18/14, 11:44 AM, "Anurag Priyam" <a.priyam at qmul.ac.uk> wrote:
>>>>>>>
>>>>>>>>Hi,
>>>>>>>>
>>>>>>>>I compiled all annotations generated by MAKER into a single GFF
>>>>>>>>file
>>>>>>>>using the gff3_merge script distributed with MAKER. While
>>>>>>>>formatting
>>>>>>>>this GFF for use with JBrowse, I found a few errors:
>>>>>>>>
>>>>>>>>1. Three instances where two features were assigned the same id.
>>>>>>>>2. One instance where a group of three subfeatures refer to a
>>>>>>>>non-existent parent.
>>>>>>>>
>>>>>>>>Here is the relevant portion of the GFF file:
>>>>>>>>https://gist.github.com/yeban/ffaf5cd419639dd073a7
>>>>>>>>
>>>>>>>>I worked around the issue temporarily for the job at hand, but I am
>>>>>>>>left wondering why would these errors creep in.
>>>>>>>>
>>>>>>>>-- Priyam
>>>>>>>>
>>>>>>>>_______________________________________________
>>>>>>>>maker-devel mailing list
>>>>>>>>maker-devel at box290.bluehost.com
>>>>>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab
>>>>>>>>.o
>>>>>>>>r
>>>>>>>>g
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>>
>>