Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Pig and Storm


+
Pradeep Gollakota 2013-07-23, 16:52
+
Russell Jurney 2013-07-24, 06:18
+
Alan Gates 2013-07-24, 18:36
Copy link to this message
-
Re: Pig and Storm
I've added a wiki page for a "Pig on Storm Proposal" at
https://cwiki.apache.org/confluence/display/PIG/Pig+on+Storm+Proposal

I've included a primer on Storm (and Trident) as well as some of the
challenges I foresee. Please read though my proposal and let me know what
your thoughts are.
On Wed, Jul 24, 2013 at 2:36 PM, Alan Gates <[EMAIL PROTECTED]> wrote:

> This sounds exciting.  The next question is how do you plan to do it?
>  Would a physical plan be translated to a Storm job (or jobs)?  Would it
> need a different physical plan?  Or would you just have the connection at
> the language layer and all the planning separate?  Do you envision needing
> extensions/changes to the language to support Storm?  Feel free to add a
> page to Pig's wiki with your thoughts on an approach.
>
> Alan.
>
> On Jul 23, 2013, at 9:52 AM, Pradeep Gollakota wrote:
>
> > Hi Pig Developers,
> >
> > I wanted to reach out to you all and ask for you opinion on something.
> >
> > As a Pig user, I have come to love Pig as a framework. Pig provides a
> great
> > set of abstractions that make working with large datasets easy. Currently
> > Pig is only backed by hadoop. However, with the new rise of Twitter Storm
> > as a distributed real time processing engine, Pig users are missing out
> on
> > a great opportunity to be able to work with Pig in Storm. As a user of
> Pig,
> > Hadoop and Storm, and keeping with the Pig philosophy of "Pigs live
> > anywhere," I'd like to get your thoughts on starting the implementation
> of
> > a Pig backend for Storm.
> >
> > Thanks
> > Pradeep
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB