Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Large pig script takes forever to parse?


Copy link to this message
-
Re: Large pig script takes forever to parse?
Thejas Nair 2011-10-25, 16:20
This is not expected. Daniel had some experiments done with large
queries (> 2k lines) and the parsing was actually faster with them.
There was some small slow down for smaller queries.
If you can replicate with a script that you can share, that would be great.

-Thejas
On 10/21/11 3:37 PM, Jonathan Coveney wrote:
> Pig is version 0.9. I have a script that under version 0.8, begins running
> quickly...but under version 0.9, it takes 10 minutes to parse (I did -debug
> ALL and it builds a big old AST). I am curious if there is anything that
> could cause this difference? It is a very long script (almost 2k lines!),
> but I'm wondering why the difference.
>
> I'm going to try and replicate with a script I can share, and then run it
> against trunk....just curious if anyone has seen this before.
>