Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Building pig and piggybank from CDH 4.2.0 sources?


Copy link to this message
-
Building pig and piggybank from CDH 4.2.0 sources?
Hi,

[ I know this question is probably CDH specific, yet I'm hoping one of you
may be able to point me in the right direction. ]

I want to make a small change to the piggybank for pig 0.10 that is in CDH
4.2.0 and recreate an RPM from that for easier deployment.

So I downloaded
http://archive.cloudera.com/cdh4/cdh/4/pig-0.10.0-cdh4.2.0.tar.gz and tried
to simply build pig using ant.

When I do that on CentOS 6.4 x86_64 it starts downloading and compiling and
then it fails with:

*compile-sources:*
*    [javac] /home/niels/tmp/pig-0.10.0-cdh4.2.0/build.xml:575: warning:
'includeantruntime' was not set, defaulting to build.sysclasspath=last; set
to false for repeatable builds*
*    [javac] Compiling 788 source files to
/home/niels/tmp/pig-0.10.0-cdh4.2.0/build/classes*
*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:23:
cannot find symbol*
*    [javac] symbol  : class WritableComparable*
*    [javac] location: package org.apache.hadoop.io*
*    [javac] import org.apache.hadoop.io.WritableComparable;*
*    [javac]                            ^*
*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/src/org/apache/pig/data/Tuple.java:47:
cannot find symbol*
*    [javac] symbol: class WritableComparable*
*    [javac] public interface Tuple extends WritableComparable,
Serializable, Iterable<Object> {*
*    [javac]                                ^*

Apparently it is unable to find the most basic Hadoop dependencies

This is strange because during the download phase I see:
*[ivy:resolve]   found org.apache.hadoop#hadoop-annotations;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-auth;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-hdfs;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-core;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-jobclient;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-tests;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-app;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-shuffle;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-api;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-client;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-common;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve]   found org.apache.hadoop#hadoop-yarn-server;2.0.0-cdh4.2.0
in cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-web-proxy;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-common;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-nodemanager;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-yarn-server-resourcemanager;2.0.0-cdh4.2.0 in
cloudera-releases*
*[ivy:resolve]   found
org.apache.hadoop#hadoop-mapreduce-client-hs;2.0.0-cdh4.2.0 in
cloudera-releases*

I've also tried to just build the piggybank in the contrib folder and when
I do that I get similar (dependency) errors:

*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:23:
cannot find symbol*
*    [javac] symbol  : class Accumulator*
*    [javac] location: package org.apache.pig*
*    [javac] import org.apache.pig.Accumulator;*
*    [javac]                      ^*
*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:24:
cannot find symbol*
*    [javac] symbol  : class Algebraic*
*    [javac] location: package org.apache.pig*
*    [javac] import org.apache.pig.Algebraic;*
*    [javac]                      ^*
*    [javac]
/home/niels/tmp/pig-0.10.0-cdh4.2.0/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/evaluation/ExtremalTupleByNthField.java:25:
cannot find symbol*
*    [javac] symbol  : class EvalFunc*
*    [javac] location: package org.apache.pig*
*    [javac] import org.apache.pig.EvalFunc;*
*    [javac]                      ^*

What should I do to fix this?
Is there a 'better' manual on building pig/piggybank than what is described
in the RELEASE_NOTES.txt ?

Best regards / Met vriendelijke groeten,

Niels Basjes
+
Cheolsoo Park 2013-04-24, 21:53
+
Niels Basjes 2013-04-25, 21:01
+
Cheolsoo Park 2013-04-25, 23:20
+
Niels Basjes 2013-05-08, 14:58