Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Pig 0.10 on YARN


+
Konstantin Boudnik 2013-03-29, 06:32
+
Dmitriy Ryaboy 2013-03-29, 16:08
+
Konstantin Boudnik 2013-03-29, 17:38
+
Dmitriy Ryaboy 2013-03-29, 18:52
+
Konstantin Boudnik 2013-03-29, 21:54
Copy link to this message
-
Re: Pig 0.10 on YARN
Hi, cos:
I met the same problem before. Can you try below in your Bigtop pig
maven execution .pom ? It will actually resolve other problem as well (for
example, you may see Pig cannot get kerberos principle in secure YARN
cluster)
      <dependencies>

        <dependency>

          <groupId>org.apache.pig</groupId>

          <artifactId>pigsmoke</artifactId>

          <version>${pig-smoke.version}</version>

          <exclusions>

            <exclusion>

              <groupId>org.apache.pig</groupId>

              <artifactId>pig</artifactId>

            </exclusion>

          </exclusions>

        </dependency>

        <dependency>

          <groupId>org.apache.hadoop</groupId>

          <artifactId>hadoop-common</artifactId>

          <version>${hadoop.version}</version>

        </dependency>

        <dependency>

          <groupId>org.apache.hadoop</groupId>

          <artifactId>hadoop-hdfs</artifactId>

          <version>${hadoop.version}</version>

        </dependency>

        <dependency>

          <groupId>org.apache.hadoop</groupId>

          <artifactId>hadoop-mapreduce-client-core</artifactId>

          <version>${hadoop.version}</version>

        </dependency>

        <dependency>

          <groupId>org.apache.hadoop</groupId>

          <artifactId>hadoop-mapreduce-client-jobclient</artifactId>

          <version>${hadoop.version}</version>

        </dependency>

        <dependency>

          <groupId>org.apache.hadoop</groupId>

          <artifactId>hadoop-mapreduce-client-hs</artifactId>

          <version>${hadoop.version}</version>

        </dependency>
Johnny

On Fri, Mar 29, 2013 at 2:54 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:

> I certainly agree with you. I have managed to narrow it down to the
> difference
> between pig-0.10.0-withouthadoop.jar and maven pig-0.10.0.jar.
>
> Basically, I can run:
>
> java -cp \
> /etc/hadoop/conf:\
> /etc/pig/conf:\
> /root/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:\
>
> /root/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:\
>
> /root/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:\
> /root/.m2/repository/org/slf4j/slf4j-api/1.6.1/slf4j-api-1.6.1.jar:\
> /root/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:\
> /usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:\
> /root/.m2/repository/org/apache/avro/avro/1.5.3/avro-1.5.3.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-common/2.0.3.1-SNAPSHOT/hadoop-common-2.0.3.1-SNAPSHOT.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-auth/2.0.3.1-SNAPSHOT/hadoop-auth-2.0.3.1-SNAPSHOT.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.0.3.1-SNAPSHOT/hadoop-hdfs-2.0.3.1-SNAPSHOT.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-yarn-api/2.0.3.1-SNAPSHOT/hadoop-yarn-api-2.0.3.1-SNAPSHOT.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-yarn-client/2.0.3.1-SNAPSHOT/hadoop-yarn-client-2.0.3.1-SNAPSHOT.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-yarn-common/2.0.3.1-SNAPSHOT/hadoop-yarn-common-2.0.3.1-SNAPSHOT.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.0.3.1-SNAPSHOT/hadoop-mapreduce-client-core-2.0.3.1-SNAPSHOT.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-common/2.0.3.1-SNAPSHOT/hadoop-mapreduce-client-common-2.0.3.1-SNAPSHOT.jar:\
>
> /root/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.0.3.1-SNAPSHOT/hadoop-mapreduce-client-jobclient-2.0.3.1-SNAPSHOT.jar:\
> /usr/lib/pig/pig-0.10.0-withouthadoop.jar:\
> /root/.m2/repository/com/google/guava/guava/11.0/guava-11.0.jar:\
> /root/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:\
> /root/.m2/repository/org/antlr/antlr-runtime/3.4/antlr-runtime-3.4.jar:\
>
> /root/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:\
>
> /root/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:\
+
Konstantin Boudnik 2013-03-29, 22:45
+
Konstantin Boudnik 2013-04-09, 06:41
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB