-Re: pig 0.9 slower in local mode?
Dexin Wang 2011-12-22, 01:00
Cool. Looking forward to trying out 0.9.2+patch.
On Mon, Dec 19, 2011 at 2:31 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
> Yes, starting with 0.7 local mode was moved to sit on top of Hadoop's
> local mode (instead of being a completely separate implementation).
> This had two effects, one desirable, one not so much:
> 1) get rid of surprises / bugs caused by having to support 2
> completely separate implementations of the runtime
> 2) slow the thing down a lot.
> Julien sped up local mode quite a bit in 0.9.2, and we have another
> patch coming that further removes unnecessary slowness, but I'm afraid
> hadoop-based local mode will never be quite as fast as the old
> On Mon, Dec 19, 2011 at 2:23 PM, Dexin Wang <[EMAIL PROTECTED]> wrote:
> > I recently switched to pig 0.9.1 and noticed it runs slower than previous
> > version (like 0.6 which was only recent version supported on Amazon
> > of months ago) in local mode. Haven't tried the timing in hadoop mode
> > I figure it is probably due to some extra debugging or some parameter.
> > Anything I can do to make it faster?
> > Thanks,
> > Dexin