Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Pig package supporting both hadoop 1 and 2


Copy link to this message
-
Re: Pig package supporting both hadoop 1 and 2
Rohini Palaniswamy 2013-05-06, 14:06
Hi Julien,
   We use a perl script internally instead of bin/pig shell script which
has some Y! deployment stuff. That is why changes have not been done to
bin/pig to support both versions already. We have two lib directories - one
for hadoop20, other for hadoop23(hadoop2) and choose the one to have in
classpath based on presence of hadoop-core.jar.

my $hadoopClasspath = `$hadoop_cmd classpath`;

 my $jarDir = "$pigJarRoot/lib-hadoop23";

        if ($hadoopClasspath =~ m/hadoop-core-/) {

           $jarDir = "$pigJarRoot/lib-hadoop20";

        }

Regards,
Rohini

On Fri, May 3, 2013 at 3:40 PM, Julien Le Dem <[EMAIL PROTECTED]> wrote:

> Hi Pig developers,
> I'm looking into having a Pig package that works both for Hadoop 1.0 and
> Hadoop 2.0
> That means have both pig*.jar and pig*-h2.jar in the package and choosing
> the right one dynamically.
> In particular I created this JIRA as a first step:
> https://issues.apache.org/jira/browse/PIG-3311
> I'm curious to know how others do it.
> Yahoo! for example? (Rohini?)
> Thanks,
> Julien
>
>