<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Hi Carson,
<div><br>
</div>
<div>
<div apple-content-edited="true">Thanks for the detailed feedback, this has cleared up a few things. I don’t necessarily share your view on the problematic nature of RNA-seq data - especially with newer protocols near-perfect strandedness. We work a lot on
transcriptome assembly and with a stringent approach to transcript assembly I think I got better results with est2genome than trying to let Maker work with a semi-refined ab-initio model. But it can be a bit tricky to hit that sweet spot (we did validate >
4000 models manually in order to make that sort of assessment tho).</div>
<div apple-content-edited="true"><br>
</div>
<div apple-content-edited="true">But I will have another look at this and see if I can get Maker to do what I need with the approach you describe. That reminds me, I think it would be fantastic if you guys could put together a Wiki for Maker. This is such a
useful and powerful tool, but clearly there are many things that people should get a proper explanation on that has only ever been discussed on this list here - best practices, experimental features etc. </div>
<div apple-content-edited="true"><br>
</div>
<div apple-content-edited="true">Regards,</div>
<div apple-content-edited="true"><br>
</div>
<div apple-content-edited="true">Marc<br>
<div><br>
</div>
<br class="Apple-interchange-newline">
</div>
<br>
<div>
<div>On 06 Mar 2014, at 15:29, Carson Holt <<a href="mailto:carsonhh@gmail.com">carsonhh@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14px; font-family: Calibri, sans-serif;">
<blockquote style="font-family: Calibri, sans-serif; margin: 0px 0px 0px 40px; border: none; padding: 0px;" type="cite">
<span style="font-size: 12px;">Wouldn’t it be more sensible to rely on the evidence over probabilistic models?</span></blockquote>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">Yes. Infact that is the backbone of MAKER. The evidence is used to derive hints that are passed back into the predictors and reviewed in light of the evidence to decide on final models (no longer
strictly probabalistic). Take a look at the MAKER2 paper (Table 2 and Figure 1) and you will see that eve when you use the wrong species parameters in the predictor (I.e. A. thaliana to annotate C. elegant) you get as much as a 3 fold increase in exon level
accuracy by using the hint feedback from MAKER. With est2genome option you don’t get that hint feedback (normally probabilistic models, EST evidence, and protein evidence would all work together), and the models are overall poorer and contain more false positives
(we have looked at this a lot).</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<blockquote style="font-family: Calibri, sans-serif; margin: 0px 0px 0px 40px; border: none; padding: 0px;" type="cite">
<span style="font-size: 12px;">The annotation would be partial, but on the other hand the chance of incorporating false signals are smaller (assuming I can generate a clean set of transcripts from RNA-seq data)?</span></blockquote>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">False signals are abundant. It’s just the nature of how ESTs and especially mRNAseq reads are generated and anchored back to the assembly. By letting there be feedback between the probabilistic
model and the evidence (both protein and EST/mRNAseq) a lot of this is eliminated.</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<blockquote style="font-family: Calibri, sans-serif; margin: 0px 0px 0px 40px; border: none; padding: 0px;" type="cite">
<span style="font-size: 12px;">As an example, using SNAP and Augustus on a bird genome - with augustus achieving nucleotide and exon sensitivities in the 70-90% range gave a host if false exons that were simply not supported by the RNAseq data, yet made it
into the final gene build.</span></blockquote>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">You will get false positives from est2genome alone approach as well. Models will be more partial, and false negative rate will be very high (often 30-70% false negative rate). Also look at the
MAKER2 paper Figure 1. The false positive rate from ab initio alone can be quite high, but with the evidence feedback it is substantially reduced (especially for poorly trained predictors).</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<blockquote style="margin:0 0 0 40px; border:none; padding:0px;" type="cite"><font face="Calibri,sans-serif" style="font-size: 12px;">Is it possible to get some more details on how Maker uses ab-inito predictions and reconciles them with evidence alignments?
At the moment it seems to me that maker gives higher weight to the ab-initio predictions, which to me seems problematic. </font></blockquote>
<div><font face="Calibri,sans-serif"><br>
</font></div>
<div>Take a look at the MAKER, MAKER2, and MAKER-P papers. Final genes are chosen based off of evidence overlap using AED (completely evidence based). It is the model generation that leverages the hint based feedback. The names of MAKER genes can let you
know what the source of the model is. Any time hint based models match the evidence better the name will have hame like this —></div>
<div>maker-<contig>-<predictor>-gene-<ID> (I.e. maker-chr1-snap-gene-0.4)</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">When the ab initio model matches better than the hint based model the name is like this —></div>
<div><font face="Calibri,sans-serif"><predictor>-<contig>-abinit-gene-<ID> </font>(I.e. <span style="font-family: Calibri, sans-serif;">snap-chr1-abinit-gene-0.2</span>)</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">In summary, using est2genome alone (while good for generating training sets) undercuts the power of the evidence feedback together with the probabilistic models.</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">Thanks,</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;">Carson</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;"><br>
</div>
<span id="OLK_SRC_BODY_SECTION" style="font-family: Calibri, sans-serif; font-size: 14px;">
<div style="font-family: Calibri; font-size: 11pt; text-align: left; border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-top-color: rgb(181, 196, 223);">
<span style="font-weight:bold">From: </span>Marc Höppner <<a href="mailto:marc.hoeppner@imbim.uu.se">marc.hoeppner@imbim.uu.se</a>><br>
<span style="font-weight:bold">Date: </span>Thursday, March 6, 2014 at 12:26 AM<br>
<span style="font-weight:bold">To: </span>Carson Holt <<a href="mailto:carsonhh@gmail.com">carsonhh@gmail.com</a>><br>
<span style="font-weight:bold">Cc: </span>"<a href="mailto:maker-devel@yandell-lab.org">maker-devel@yandell-lab.org</a>" <<a href="mailto:maker-devel@yandell-lab.org">maker-devel@yandell-lab.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [maker-devel] FW: maker-control file<br>
</div>
<div><br>
</div>
<div>
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>Hi,</div>
<div><br>
</div>
<div>I think this is an interesting comment that I would like a few more information on:</div>
<div><br>
<blockquote type="cite">
<div style="font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
correct_est_fusion should not be used together with est2genome. It won’t<br>
fail, you just get odd results. Actually est2genome should not ever be<br>
used to generate the final annotation set. It is a convenience method<br>
that allows you to generate rough models for training gene predictors like<br>
SNAP and Augustus. But once they are trained it should be turned off,<br>
because the models it produces will be partial (Ests rarely cover the<br>
whole transcript) and the results will have many false potties from<br>
background transcription events from your EST data. These models are good<br>
enough to train with, but make very poor final annotations. So in the end<br>
you should be using correct_est_fusion=1 with the SNAP pr Augustus set and<br>
not est2genome (which should already have been turned off by then).<br>
<br>
</div>
</blockquote>
</div>
<br>
<div>My experience has been that the process of training gene finders, especially for complex genomes like vertebrates, is a very slow and painful process. And ultimately, the results are far from accurate, even with a sizeable, manually curated training set.
Wouldn’t it be more sensible to rely on the evidence over probabilistic models? The annotation would be partial, but on the other hand the chance of incorporating false signals are smaller (assuming I can generate a clean set of transcripts from RNA-seq data)?
And I’d rather underestimate the exon inventory slightly than putting out an annotation with ~ 10% false exon calls. </div>
<div><br>
</div>
<div>As an example, using SNAP and Augustus on a bird genome - with augustus achieving nucleotide and exon sensitivities in the 70-90% range gave a host if false exons that were simply not supported by the RNAseq data, yet made it into the final gene build.
Not sure what to think about that to be honest. Is it possible to get some more details on how Maker uses ab-inito predictions and reconciles them with evidence alignments? At the moment it seems to me that maker gives higher weight to the ab-initio predictions,
which to me seems problematic. </div>
<div><br>
</div>
<div><br>
</div>
<div>/Marc</div>
</div>
</div>
</span></div>
</blockquote>
</div>
<br>
</div>
</body>
</html>