Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Execute multiple PIG scripts parallely


Copy link to this message
-
Re: Execute multiple PIG scripts parallely
If you write your scripts as one large Pig script Pig will execute them in parallel.  You can keep from confusing your individual scripts by writing one master script that has imports (see http://pig.apache.org/docs/r0.11.1/cont.html#import-macros ).  You just need to make sure your various scripts don't share variable names, as imports don't maintain a namespace.

Alan.

On Jul 22, 2013, at 3:34 AM, Bhavesh Shah wrote:

> Hello All,
>
>
>
> I have multiple PIG Script with and currently I am executing it in sequential manner using command
>
> pig -x mapreduce /path/to/Script/Script1.pig && /path/to/Script/Script2.pig && /path/to/Script/Script3.pig
>
>
>
> But now I am looking for executing those scripts in parallel as all are independent of each other. I searched for it but not getting exactly.
>
>
>
> So is there any way through which I can execute my all scripts parallely?
>
>
>
>
>
> Thanks,
>
> Bhavesh Shah
>    
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB