<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;"><blockquote style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; margin: 0px 0px 0px 40px; border: none; padding: 0px;"><div><span style="font-size: 12px;">Wouldn’t it be more sensible to rely on the evidence over probabilistic models?</span></div></blockquote><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">Yes. Infact that is the backbone of MAKER. The evidence is used to derive hints that are passed back into the predictors and reviewed in light of the evidence to decide on final models (no longer strictly probabalistic). Take a look at the MAKER2 paper (Table 2 and Figure 1) and you will see that eve when you use the wrong species parameters in the predictor (I.e. A. thaliana to annotate C. elegant) you get as much as a 3 fold increase in exon level accuracy by using the hint feedback from MAKER. With est2genome option you don’t get that hint feedback (normally probabilistic models, EST evidence, and protein evidence would all work together), and the models are overall poorer and contain more false positives (we have looked at this a lot).</div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><blockquote style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; margin: 0px 0px 0px 40px; border: none; padding: 0px;"><div><span style="font-size: 12px;">The annotation would be partial, but on the other hand the chance of incorporating false signals are smaller (assuming I can generate a clean set of transcripts from RNA-seq data)?</span></div></blockquote><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">False signals are abundant. It’s just the nature of how ESTs and especially mRNAseq reads are generated and anchored back to the assembly. By letting there be feedback between the probabilistic model and the evidence (both protein and EST/mRNAseq) a lot of this is eliminated.</div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><blockquote style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; margin: 0px 0px 0px 40px; border: none; padding: 0px;"><div><span style="font-size: 12px;">As an example, using SNAP and Augustus on a bird genome - with augustus achieving nucleotide and exon sensitivities in the 70-90% range gave a host if false exons that were simply not supported by the RNAseq data, yet made it into the final gene build.</span></div></blockquote><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">You will get false positives from est2genome alone approach as well. Models will be more partial, and false negative rate will be very high (often 30-70% false negative rate). Also look at the MAKER2 paper Figure 1. The false positive rate from ab initio alone can be quite high, but with the evidence feedback it is substantially reduced (especially for poorly trained predictors).</div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><blockquote style="margin:0 0 0 40px; border:none; padding:0px;"><div><font face="Calibri,sans-serif" style="font-size: 12px;">Is it possible to get some more details on how Maker uses ab-inito predictions and reconciles them with evidence alignments? At the moment it seems to me that maker gives higher weight to the ab-initio predictions, which to me seems problematic. </font></div></blockquote><div><font face="Calibri,sans-serif"><br></font></div><div>Take a look at the MAKER, MAKER2, and MAKER-P papers. Final genes are chosen based off of evidence overlap using AED (completely evidence based). It is the model generation that leverages the hint based feedback. The names of MAKER genes can let you know what the source of the model is. Any time hint based models match the evidence better the name will have hame like this —></div><div>maker-<contig>-<predictor>-gene-<ID> (I.e. maker-chr1-snap-gene-0.4)</div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">When the ab initio model matches better than the hint based model the name is like this —></div><div><font face="Calibri,sans-serif"><predictor>-<contig>-abinit-gene-<ID> </font>(I.e. <span style="font-family: Calibri, sans-serif;">snap-chr1-abinit-gene-0.2</span>)</div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">In summary, using est2genome alone (while good for generating training sets) undercuts the power of the evidence feedback together with the probabilistic models.</div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">Thanks,</div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">Carson</div><div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><br></div><span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;"><div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span> Marc Höppner <<a href="mailto:marc.hoeppner@imbim.uu.se">marc.hoeppner@imbim.uu.se</a>><br><span style="font-weight:bold">Date: </span> Thursday, March 6, 2014 at 12:26 AM<br><span style="font-weight:bold">To: </span> Carson Holt <<a href="mailto:carsonhh@gmail.com">carsonhh@gmail.com</a>><br><span style="font-weight:bold">Cc: </span> "<a href="mailto:maker-devel@yandell-lab.org">maker-devel@yandell-lab.org</a>" <<a href="mailto:maker-devel@yandell-lab.org">maker-devel@yandell-lab.org</a>><br><span style="font-weight:bold">Subject: </span> Re: [maker-devel] FW: maker-control file<br></div><div><br></div><div><meta http-equiv="Content-Type" content="text/html; charset=Windows-1252"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div>Hi,</div><div><br></div><div>I think this is an interesting comment that I would like a few more information on:</div><div><br><blockquote type="cite"><div style="font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br>
correct_est_fusion should not be used together with est2genome. It won’t<br>
fail, you just get odd results. Actually est2genome should not ever be<br>
used to generate the final annotation set. It is a convenience method<br>
that allows you to generate rough models for training gene predictors like<br>
SNAP and Augustus. But once they are trained it should be turned off,<br>
because the models it produces will be partial (Ests rarely cover the<br>
whole transcript) and the results will have many false potties from<br>
background transcription events from your EST data. These models are good<br>
enough to train with, but make very poor final annotations. So in the end<br>
you should be using correct_est_fusion=1 with the SNAP pr Augustus set and<br>
not est2genome (which should already have been turned off by then).<br><br></div></blockquote></div><br><div>My experience has been that the process of training gene finders, especially for complex genomes like vertebrates, is a very slow and painful process. And ultimately, the results are far from accurate, even with a sizeable, manually curated training set.
Wouldn’t it be more sensible to rely on the evidence over probabilistic models? The annotation would be partial, but on the other hand the chance of incorporating false signals are smaller (assuming I can generate a clean set of transcripts from RNA-seq data)?
And I’d rather underestimate the exon inventory slightly than putting out an annotation with ~ 10% false exon calls. </div><div><br></div><div>As an example, using SNAP and Augustus on a bird genome - with augustus achieving nucleotide and exon sensitivities in the 70-90% range gave a host if false exons that were simply not supported by the RNAseq data, yet made it into the final gene build.
Not sure what to think about that to be honest. Is it possible to get some more details on how Maker uses ab-inito predictions and reconciles them with evidence alignments? At the moment it seems to me that maker gives higher weight to the ab-initio predictions,
which to me seems problematic. </div><div><br></div><div><br></div><div>/Marc</div></div></div></span></body></html>