[maker-devel] Maker in the cloud

[Carson Holt]

Hello Jasmin,

I haven't used MAKER in parralel on the cloud before (just tutorial images);
however, I believe there is an iPlant atmosphere image available through
iPlant with MAKER version 2.27.  You can get maximum 16 cpus per instance
there.
--> http://www.iplantcollaborative.org/discover/atmosphere

Alternatively if you have any US based collaborators you can apply for a
startup allocation on the Lonestar cluster via XSEDE (allocation can be
requested by any US based researcher and only takes a few days to approve)
--> https://www.xsede.org/
That cluster was used recently to process the largest genome ever annotated
(the pine genome).  Total run time will be less than a day on that cluster,
because you can request thousands of CPUs for your job with very short queue
wait times.

There is also a work in progress to give access to MAKER on the same cluster
via the iPlant discovery environment. I've CC'd Joshua Stein who can correct
me if I'm wrong, but I believe that resource would be available to non-US
based researchers as well, and will be available in the very very near
future (potentially within the next month or less).

Perhaps someone else on the mailing list may want to share their experience
using MAKER on the cloud?

Thanks,
Carson


From:  Jasmin Zohren <j.zohren at qmul.ac.uk>
Date:  Thursday, September 5, 2013 11:58 AM
To:  <maker-devel at yandell-lab.org>
Subject:  [maker-devel] Maker in the cloud

Dear Maker developers,
 
I¹ve already contacted you a while ago about my annotation of the birch
genome (Betula nana). As I am constantly running into problems using our
cluster facilities at QMUL I thought of moving into the cloud. As I am
rather inexperienced in cloud computing I have several questions:
 
1.       To me it seems that there are two different Maker images on EC2 ­
ami-ea661f83 and ami-b10abed8 ­ which one is ³the right one²?

2.       Can I use this Maker AMI for the annotation of a whole genome or is
it only suitable for the tutorial tasks?

3.       Also, when I followed the steps outlined in the tutorial, there
seemed to be a problem with RepeatMasker. Although Maker would run and
produce output files, the log file stated that the contig had failed after
the second attempt. I launched the image on a T1.micro instance, maybe that
wasn¹t enough computing power? Or do you have another explanation for this?

4.       Would it be possible to run the annotation in parallel (e.g. using
MPICH2) in the cloud? I¹ve also recently heard about a parallelisation
module for use in the cloud developed by Era7, called ³nispero². But I am
not sure whether it is publicly available yet.

5.       Do you have any experience of how long an annotation task in the
cloud would take and also what the expected costs would be? The birch genome
is only 500 MB in size and currently I am simply annotating it with a SNAP
trained HMM. However, in the future I will feed it with RNAseq data as well.

 
Many thanks in advance and kind regards,
Jasmin
                   
-----------------------------
Jasmin Zohren
PhD student in the INTERCROSSING ITN
Queen Mary University of London
 
intercrossing.wikispaces.com <http://intercrossing.wikispaces.com/>
evolve.sbcs.qmul.ac.uk <http://evolve.sbcs.qmul.ac.uk/buggs/jasmin-zohren/>
 
_______________________________________________ maker-devel mailing list
maker-devel at box290.bluehost.com
http://box290.bluehost.com/mailman/listinfo/maker-devel_yandell-lab.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://yandell-lab.org/pipermail/maker-devel_yandell-lab.org/attachments/20130905/efc0f8fb/attachment-0003.html>