-Re: How is pig so much faster than my java MR job?
Anoop John 2013-09-03, 07:03
You are using Scan caching in ur MR java code? How many mapper and
reducers in case of pig? How is ur Java MR job written.. a bit more on
its logic pls.
On Tue, Sep 3, 2013 at 11:33 AM, Pavan Sudheendra <[EMAIL PROTECTED]>wrote:
> Hi all,
> I'm doing a kind of table join across 3 tables in the MR job ( plus doing
> some computation).. It took nearly 19 hours to run with 21 mappers and 21
> reducers.. But with pig it ran in less than 2 hours..
> We are using HBase both as source and sink.. Is this normal?
> On Mon, Sep 2, 2013 at 7:20 PM, Adrien Mogenet <[EMAIL PROTECTED]
> > You should have a kind of debug/explain mode in Pig, and will show you
> > it does clever things to optimize its excution path.
> > On Mon, Sep 2, 2013 at 3:45 PM, Dhaval Shah <[EMAIL PROTECTED]
> > >wrote:
> > > Java MR code is not optimized/efficiently written while Pig is highly
> > > optimized? Can you give us more details on what exactly you are trying
> > > do and how your Java MR code is written, how many MR jobs for Java vs
> > > and so on
> > >
> > > Sent from Yahoo! Mail on Android
> > >
> > >
> > --
> > Adrien Mogenet
> > http://www.borntosegfault.com