|
|
-
Re: WholeFileInputFormat with streamingFatih Haltas 2013-03-03, 04:14
Hi Ring,
Can you write output of "jar tf filename" command to see package organization of jar, you created? 2 Mart 2013 Cumartesi tarihinde springring adlı kullanıcı şöyle yazdı: > Hi, > > I want to use: > hadoop jar <my dir>/hadoop-streaming-0.20.2-cdh3u3.jar -inputformat > org.apache.hadoop.streaming.WholeFileInputFormat > > so, I download code from : > https://github.com/tomwhite/hadoop-book/tree/master/ch07/src/main/java > WholeFileInputFormat.java<https://github.com/tomwhite/hadoop-book/blob/master/ch07/src/main/java/WholeFileInputFormat.java> > WholeFil! eRecordRe ader.java<https://github.com/tomwhite/hadoop-book/blob/master/ch07/src/main/java/WholeFileRecordReader.java> > > and package the java file with : > package org.apache.hadoop.streaming; > > solution A: > copy WholeFileInputFormat.java<https://github.com/tomwhite/hadoop-book/blob/master/ch07/src/main/java/WholeFileInputFormat.java>, > WholeFileRecordReader.java<https://github.com/tomwhite/hadoop-book/blob/master/ch07/src/main/java/WholeFileRecordReader.java> > to > hadoop-0.20.2-cdh3u3/src/contrib/streaming/src/java/org/apache/hadoop/streaming/ > then > javac -classpath > /usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.jar:/usr/lib/hadoop-0.20:/usr/lib/hadoop-0.20/lib/* > -d ! WFInputFo rmatClassNew > hadoop-0.20.2-cdh3u3/src/contrib/streaming/src/java/org/apache/hadoop/streaming/*.java > > there is a lot of error > > solution B: > compile the java file WholeFileInputFormat.java<https://github.com/tomwhite/hadoop-book/blob/master/ch07/src/main/java/WholeFileInputFormat.java>, > WholeFileRecordReader.java<https://github.com/tomwhite/hadoop-book/blob/master/ch07/src/main/java/WholeFileRecordReader.java> > : > javac -classpath > /usr/lib/hadoop-0.20/hadoop-0.20.2-cdh3u3-core.jar:/usr/lib/hadoop-0.20/*:/usr/lib/hadoop-0.20/lib/* > -d WFInputFormatClass <two java file> > copy /usr/l! ib/hadoop > /contrib/streaming/hadoop-streaming-0.20.2-cdh3u3.jar to <my dir> > then : > jar uf hadoop-streaming-0.20.2-cdh3u3.jar > WFInputFormatClass/org/apache/hadoop/streaming/WholeFileRecordReader.class > jar uf hadoop-streaming-0.20.2-cdh3u3.jar > WFInputFormatClass/org/apache/hadoop/streaming/WholeFileInputFormat.class > there is no error, but when I run: > hadoop jar <my dir>/hadoop-streaming-0.20.2-cdh3u3.jar -inputformat > org.apache.hadoop.streaming.WholeFileInputFormat ....... > there is error: > -inputformat : class not found : > org.apache.hadoop.streaming.WholeFileInputFormat > > what's wrong with the two solution? or is there any new solution? > > thx. > > Ring > > > |