I recently switched to pig 0.9.1 and noticed it runs slower than previous
version (like 0.6 which was only recent version supported on Amazon couple
of months ago) in local mode. Haven't tried the timing in hadoop mode yet.

I figure it is probably due to some extra debugging or some parameter.
Anything I can do to make it faster?

Thanks,
Dexin

https://issues.apache.org/jira/browse/PIG-2128
could be related.

Koji
On 12/19/11 2:23 PM, "Dexin Wang" <[EMAIL PROTECTED]> wrote:

> I recently switched to pig 0.9.1 and noticed it runs slower than previous
> version (like 0.6 which was only recent version supported on Amazon couple
> of months ago) in local mode. Haven't tried the timing in hadoop mode yet.
>
> I figure it is probably due to some extra debugging or some parameter.
> Anything I can do to make it faster?
>
> Thanks,
> Dexin

Yes, starting with 0.7 local mode was moved to sit on top of Hadoop's
local mode (instead of being a completely separate implementation).
This had two effects, one desirable, one not so much:

1) get rid of surprises / bugs caused by having to support 2
completely separate implementations of the runtime
2) slow the thing down a lot.

Julien sped up local mode quite a bit in 0.9.2, and we have another
patch coming that further removes unnecessary slowness, but I'm afraid
hadoop-based local mode will never be quite as fast as the old
local-mode...

D

On Mon, Dec 19, 2011 at 2:23 PM, Dexin Wang <[EMAIL PROTECTED]> wrote:
> I recently switched to pig 0.9.1 and noticed it runs slower than previous
> version (like 0.6 which was only recent version supported on Amazon couple
> of months ago) in local mode. Haven't tried the timing in hadoop mode yet.
>
> I figure it is probably due to some extra debugging or some parameter.
> Anything I can do to make it faster?
>
> Thanks,
> Dexin

Cool. Looking forward to trying out 0.9.2+patch.

On Mon, Dec 19, 2011 at 2:31 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Yes, starting with 0.7 local mode was moved to sit on top of Hadoop's
> local mode (instead of being a completely separate implementation).
> This had two effects, one desirable, one not so much:
>
> 1) get rid of surprises / bugs caused by having to support 2
> completely separate implementations of the runtime
> 2) slow the thing down a lot.
>
> Julien sped up local mode quite a bit in 0.9.2, and we have another
> patch coming that further removes unnecessary slowness, but I'm afraid
> hadoop-based local mode will never be quite as fast as the old
> local-mode...
>
> D
>
> On Mon, Dec 19, 2011 at 2:23 PM, Dexin Wang <[EMAIL PROTECTED]> wrote:
> > I recently switched to pig 0.9.1 and noticed it runs slower than previous
> > version (like 0.6 which was only recent version supported on Amazon
> couple
> > of months ago) in local mode. Haven't tried the timing in hadoop mode
> yet.
> >
> > I figure it is probably due to some extra debugging or some parameter.
> > Anything I can do to make it faster?
> >
> > Thanks,
> > Dexin
>