[maker-devel] keep_preds values

[Carson Holt]

Sorry meant to say "3 predictors and 2 make perfectly
matching calls and 1 makes no call."


On 3/10/14, 9:16 AM, "Carson Holt" <carsonhh at gmail.com> wrote:

>There is a value called abAED being calculated, which somewhat captures
>the concordance among the predictors.  It is not currently printed in the
>GFF3, but it is used to identify the best non-overlapping ab initio
>predictor to put in the non-overlapping fasta file.  There are a couple of
>things I still need to do with it to though.  It’s not yet normalized to
>take into account the absence of a predictor in the cluster of overlapping
>predictions. For example, if I have 2 predictors and 2 make perfectly
>matching calls and 1 makes no call, they get a score of 0 before I have
>perfect concordance between what’s there, but I really should make it 0.33
>because the abscence of the third predictor is meaningful.  The
>unnormalized concordance value is fine for deciding which overlapping
>model to keep in the file, but not for global comparison.
>
>—Carson
>
>
>
>On 3/10/14, 8:08 AM, "Mikael Brandström Durling" <mikael.durling at slu.se>
>wrote:
>
>>Ok. But that is not implemented no as far as I can tell from the source,
>>right? Or is it reflected in the AED for the unsupported models?
>>
>>Mikael
>>
>>10 mar 2014 kl. 16:59 skrev Carson Holt <carsonhh at gmail.com>:
>>
>>> Yes.  It will eventually perform an AED like calculation between
>>>multiple
>>> predictors (i.e. if you use 3 predictors it, then you require support
>>>by
>>> at least 2 predictors across all exons to get a value of 0.33).  A
>>>value
>>> of 0 would be perfect concordance across all 3 predictors.
>>> 
>>> —Carson
>>> 
>>> 
>>> 
>>> 
>>> On 3/10/14, 7:57 AM, "Mikael Brandström Durling"
>>><mikael.durling at slu.se>
>>> wrote:
>>> 
>>>> Hi Carson and Daniel,
>>>> 
>>>> That sounds more logical to me.  Then it would be appropriate to
>>>>change
>>>> the comment of keep_preds in the generated config files.
>>>> 
>>>> Would it make sense to make keep_preds a non-binary value to evaluate
>>>>the
>>>> concordance between ab initio models obtained from different
>>>>predictors?
>>>> That would assume that it is less likely to be a false positive when
>>>>two
>>>> or more predictors suggest the same unsported model?
>>>> 
>>>> Mikael
>>>> 
>>>> 
>>>> 10 mar 2014 kl. 16:51 skrev Carson Holt <carsonhh at gmail.com>:
>>>> 
>>>>> Actually that is false. The keep_preds option is still binary.  Any
>>>>> value
>>>>> other than 0 sets it to true.  There was discussion about making it a
>>>>> non-binary value, but that has not been implemented.
>>>>> 
>>>>> —Carson
>>>>> 
>>>>> 
>>>>> On 3/10/14, 7:47 AM, "Daniel Ence" <dence at genetics.utah.edu> wrote:
>>>>> 
>>>>>> Hi Mikael, 
>>>>>> 
>>>>>> The keep_preds parameter is often used the same as a binary
>>>>>>parameter,
>>>>>> but it doesn't have to be. The concordance that is mentioned in the
>>>>>> comment line is the AED for that prediction. AED is a measurement of
>>>>>> how
>>>>>> well a prediction is supported by the evidence and ranges from 0 -
>>>>>>1. A
>>>>>> prediction with an AED of 0 matches the evidence exactly while a
>>>>>> prediction with an AED of 1 isn't overlapped by any evidence.
>>>>>> 
>>>>>> The default behavior for MAKER is to make a gene model out of a
>>>>>> prediction with any AED <1. When you change the keep_preds option
>>>>>>from
>>>>>> 0
>>>>>> to 1, then MAKER will make a gene model out of any prediction that
>>>>>> matches the other parameters (like single_exon, min_exon, etc).
>>>>>>Setting
>>>>>> the keep_preds option to somewhere in between 0 and 1 will set a
>>>>>> ceiling
>>>>>> on the AED required for promoting a prediction to a gene model.
>>>>>> 
>>>>>> From a user standpoint, when you will almost certainly lose gene
>>>>>>models
>>>>>> when you set AED at an intermediate value, but you might benefit by
>>>>>> knowing that all your models will now have an AED of at least a
>>>>>>certain
>>>>>> value. 
>>>>>> 
>>>>>> I hope that helps; let me know if it didn't.
>>>>>> 
>>>>>> ~Daniel
>>>>>> 
>>>>>> PS The original paper that described the AED is Eilbeck et al in BMC
>>>>>> Bioinformatics 2009. It's also discussed in more detail in the
>>>>>>MAKER2
>>>>>> paper, the MAKER-P paper, and the Yandell and Ence Nature Reviews
>>>>>> Genetics paper from 2012.
>>>>>> 
>>>>>> Daniel Ence
>>>>>> Graduate Student
>>>>>> Eccles Institute of Human Genetics
>>>>>> University of Utah
>>>>>> 15 North 2030 East, Room 2100
>>>>>> Salt Lake City, UT 84112-5330
>>>>>> ________________________________________
>>>>>> From: maker-devel [maker-devel-bounces at yandell-lab.org] on behalf of
>>>>>> Mikael Brandström Durling [mikael.durling at slu.se]
>>>>>> Sent: Monday, March 10, 2014 4:27 AM
>>>>>> To: maker-devel at yandell-lab.org
>>>>>> Subject: [maker-devel] keep_preds values
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> Can someone, please, explain the keep_preds parameter, as it works
>>>>>>now
>>>>>> with a value between 1 and 0? It used to be binary, but now it seems
>>>>>>to
>>>>>> test concordance towards something. The maker wiki doesn’t explain
>>>>>>it
>>>>>> any
>>>>>> further either.
>>>>>> 
>>>>>> Thanks,
>>>>>> Mikael
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> maker-devel mailing list
>>>>>> maker-devel at box290.bluehost.com
>>>>>> 
>>>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.o
>>>>>>r
>>>>>>g
>>>>>> 
>>>>>> _______________________________________________
>>>>>> maker-devel mailing list
>>>>>> maker-devel at box290.bluehost.com
>>>>>> 
>>>>>>http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.o
>>>>>>r
>>>>>>g
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>>
>
>