Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Pig package supporting both hadoop 1 and 2


Copy link to this message
-
Re: Pig package supporting both hadoop 1 and 2
Hi Julien,
   We use a perl script internally instead of bin/pig shell script which
has some Y! deployment stuff. That is why changes have not been done to
bin/pig to support both versions already. We have two lib directories - one
for hadoop20, other for hadoop23(hadoop2) and choose the one to have in
classpath based on presence of hadoop-core.jar.

my $hadoopClasspath = `$hadoop_cmd classpath`;

 my $jarDir = "$pigJarRoot/lib-hadoop23";

        if ($hadoopClasspath =~ m/hadoop-core-/) {

           $jarDir = "$pigJarRoot/lib-hadoop20";

        }

Regards,
Rohini

On Fri, May 3, 2013 at 3:40 PM, Julien Le Dem <[EMAIL PROTECTED]> wrote:

> Hi Pig developers,
> I'm looking into having a Pig package that works both for Hadoop 1.0 and
> Hadoop 2.0
> That means have both pig*.jar and pig*-h2.jar in the package and choosing
> the right one dynamically.
> In particular I created this JIRA as a first step:
> https://issues.apache.org/jira/browse/PIG-3311
> I'm curious to know how others do it.
> Yahoo! for example? (Rohini?)
> Thanks,
> Julien
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB