Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Building pig and piggybank from CDH 4.2.0 sources?


+
Niels Basjes 2013-04-24, 20:45
Copy link to this message
-
Re: Building pig and piggybank from CDH 4.2.0 sources?
Hi,

Please try:
ant clean jar-all piggybank -Dhadoopversion=23

Thanks,
Cheolsoo
On Wed, Apr 24, 2013 at 1:45 PM, Niels Basjes <[EMAIL PROTECTED]> wrote:

> Hi,
>
> [ I know this question is probably CDH specific, yet I'm hoping one of you
> may be able to point me in the right direction. ]
>
> I want to make a small change to the piggybank for pig 0.10 that is in CDH
> 4.2.0 and recreate an RPM from that for easier deployment.
>
> So I downloaded
> http://archive.cloudera.com/cdh4/cdh/4/pig-0.10.0-cdh4.2.0.tar.gz and
> tried
> to simply build pig using ant.
>
> When I do that on CentOS 6.4 x86_64 it starts downloading and compiling and
> then it fails with:
>
> *compile-sources:*
> *    [javac] /home/niels/tmp/pig-0.10.0-cdh4.2.0/build.xml:575: warning:
> 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set
> to false for repeatable builds*
> *    [javac] Compiling 788 source files to
> /home/niels/tmp/pig-0.10.0-cdh4.2.0/build/classes*
> *    [javac]
> /home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:23:
> cannot find symbol*
> *    [javac] symbol  : class WritableComparable*
> *    [javac] location: package org.apache.hadoop.io*
> *    [javac] import org.apache.hadoop.io.WritableComparable;*
> *    [javac]                            ^*
> *    [javac]
> /home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:47:
> cannot find symbol*
> *    [javac] symbol: class WritableComparable*
> *    [javac] public interface Tuple extends WritableComparable,
> Serializable, Iterable<Object> {*
> *    [javac]                                ^*
>
> Apparently it is unable to find the most basic Hadoop dependencies
>
> This is strange because during the download phase I see:
> *[ivy:resolve]   found org.apache.hadoop#hadoop-annotations;2.0.0-cdh4.2.0
> in cloudera-releases*
> *[ivy:resolve]   found org.apache.hadoop#hadoop-auth;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found org.apache.hadoop#hadoop-common;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found org.apache.hadoop#hadoop-hdfs;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-mapreduce-client-core;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-yarn-server-tests;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-mapreduce-client-app;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-mapreduce-client-shuffle;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-mapreduce-client-common;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-api;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-client;2.0.0-cdh4.2.0
> in cloudera-releases*
> *[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-common;2.0.0-cdh4.2.0
> in cloudera-releases*
> *[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-server;2.0.0-cdh4.2.0
> in cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-yarn-server-web-proxy;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-yarn-server-common;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-yarn-server-nodemanager;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-yarn-server-resourcemanager;2.0.0-cdh4.2.0 in
> cloudera-releases*
> *[ivy:resolve]   found
> org.apache.hadoop#hadoop-mapreduce-client-hs;2.0.0-cdh4.2.0 in
> cloudera-releases*
>
> I've also tried to just build the piggybank in the contrib folder and when
> I do that I get similar (dependency) errors:
>
> *    [javac]
>
> /home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:23:
+
Niels Basjes 2013-04-25, 21:01
+
Cheolsoo Park 2013-04-25, 23:20
+
Niels Basjes 2013-05-08, 14:58
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB