Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Building pig and piggybank from CDH 4.2.0 sources?


Copy link to this message
-
Building pig and piggybank from CDH 4.2.0 sources?
Hi,

[ I know this question is probably CDH specific, yet I'm hoping one of you
may be able to point me in the right direction. ]

I want to make a small change to the piggybank for pig 0.10 that is in CDH
4.2.0 and recreate an RPM from that for easier deployment.

So I downloaded
http://archive.cloudera.com/cdh4/cdh/4/pig-0.10.0-cdh4.2.0.tar.gz and tried
to simply build pig using ant.

When I do that on CentOS 6.4 x86_64 it starts downloading and compiling and
then it fails with:

*compile-sources:*
*    [javac] /home/niels/tmp/pig-0.10.0-cdh4.2.0/build.xml:575: warning:
'includeantruntime' was not set, defaulting to build.sysclasspath=last; set
to false for repeatable builds*
*    [javac] Compiling 788 source files to
/home/niels/tmp/pig-0.10.0-cdh4.2.0/build/classes*
*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:23:
cannot find symbol*
*    [javac] symbol  : class WritableComparable*
*    [javac] location: package org.apache.hadoop.io*
*    [javac] import org.apache.hadoop.io.WritableComparable;*
*    [javac]                            ^*
*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:47:
cannot find symbol*
*    [javac] symbol: class WritableComparable*
*    [javac] public interface Tuple extends WritableComparable,
Serializable, Iterable<Object> {*
*    [javac]                                ^*

Apparently it is unable to find the most basic Hadoop dependencies

This is strange because during the download phase I see:
*[ivy:resolve]   found org.apache.hadoop#hadoop-annotations;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-auth;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-hdfs;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-core;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-tests;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-app;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-shuffle;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-api;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-client;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-common;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-server;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-web-proxy;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-nodemanager;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-resourcemanager;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-hs;2.0.0-cdh4.2.0 in
cloudera-releases*

I've also tried to just build the piggybank in the contrib folder and when
I do that I get similar (dependency) errors:

*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:23:
cannot find symbol*
*    [javac] symbol  : class Accumulator*
*    [javac] location: package org.apache.pig*
*    [javac] import org.apache.pig.Accumulator;*
*    [javac]                      ^*
*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:24:
cannot find symbol*
*    [javac] symbol  : class Algebraic*
*    [javac] location: package org.apache.pig*
*    [javac] import org.apache.pig.Algebraic;*
*    [javac]                      ^*
*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:25:
cannot find symbol*
*    [javac] symbol  : class EvalFunc*
*    [javac] location: package org.apache.pig*
*    [javac] import org.apache.pig.EvalFunc;*
*    [javac]                      ^*

What should I do to fix this?
Is there a 'better' manual on building pig/piggybank than what is described
in the RELEASE_NOTES.txt ?

Best regards / Met vriendelijke groeten,

Niels Basjes
+
Cheolsoo Park 2013-04-24, 21:53
+
Niels Basjes 2013-04-25, 21:01
+
Cheolsoo Park 2013-04-25, 23:20
+
Niels Basjes 2013-05-08, 14:58
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB