Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> pig in local mode


Copy link to this message
-
Re: pig in local mode
Thanks Alan for the reply.  So does this mean one will have to setup pig and hadoop (manually) on the same box or will pig have all the hadoop dependencies built in (probably running hadoop pseudo-distributed mode)?

On Apr 21, 2010, at 2:53 PM, Alan Gates wrote:

> Local mode is reasonable for initial testing.  However, since the execution is currently different the tests should be considered incomplete.
>
> As of Pig 0.7 local mode will also run on Hadoop.  This means the speed advantage will vanish, but it will be a more true test of your scripts.
>
> Alan.
>
> On Apr 15, 2010, at 9:55 AM, Corbin Hoenes wrote:
>
>> Hi there,
>>
>> I've got a bunch of pig scripts that produce some output that I would like to assert based on some known good data.  I've found that running the scripts in local mode (via bash scripts) on my local box is quite fast.  Would this be a good testing mechanism or should I be running only under mapred mode?
>>
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB