-Re: How to run pig batch?
Haitao Yao 2012-02-21, 01:53
the pig script is generated by a system which is more than 4000 lines.
The pig client compile it for a long time.
I have solved the problem by dividing the big script into small ones and implementing a new pig client which will concurrently submit multiple pig scripts for computation.
Is there any other way?
在 2012-2-21，上午5:02， Dmitriy Ryaboy 写道�
> Is something preventing you from calling them via a simple bash script?
> On Sun, Feb 19, 2012 at 10:58 PM, Haitao Yao <[EMAIL PROTECTED]> wrote:
>> hi, all
>> I have tens of simple pig scripts to run. While there's no parameter name collision, I merged them into a large pig script which is about 4000 lines.
>> But the merged pig script takes pig a lot of time merging and analyzing.
>> How can I quickly submit tens of scripts in parallel?
>> BTW, I do not want to start tens of pig process for memory issue.
>> thanks very much.