Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Large pig script takes forever to parse?


Copy link to this message
-
Re: Large pig script takes forever to parse?
This is not expected. Daniel had some experiments done with large
queries (> 2k lines) and the parsing was actually faster with them.
There was some small slow down for smaller queries.
If you can replicate with a script that you can share, that would be great.

-Thejas
On 10/21/11 3:37 PM, Jonathan Coveney wrote:
> Pig is version 0.9. I have a script that under version 0.8, begins running
> quickly...but under version 0.9, it takes 10 minutes to parse (I did -debug
> ALL and it builds a big old AST). I am curious if there is anything that
> could cause this difference? It is a very long script (almost 2k lines!),
> but I'm wondering why the difference.
>
> I'm going to try and replicate with a script I can share, and then run it
> against trunk....just curious if anyone has seen this before.
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB