Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> problem with streaming and libjars


Copy link to this message
-
Re: problem with streaming and libjars
I would try the following:

hadoop -libjars /home/ayon/jars/MultiOutput.jar jar
/usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar
-libjars /home/ayon/jars/MultiOutput.jar -input
/user/ayon/streaming_test_input -output
/user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat
-outputformat MultiFileOutput

Another thing to note, I'm pretty sure you need to specify the full
package with the name of the output format class (e.g.
my.package.MultiFileOutput).

-Joey

On Tue, Jun 14, 2011 at 11:31 PM, Ayon Sinha <[EMAIL PROTECTED]> wrote:
> None of my options are working. Any help is appreciated.
>
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar -libjars /home/ayon/jars/MultiOutput.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar-archives /home/ayon/jars/MultiOutput.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar-libjars /home/ayon/jars/MultiOutput.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar -libjars /home/ayon/jars/MultiOutput.jar -files /home/ayon/jars/MultiOutput.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
>
>
> -Ayon
> See My Photos on Flickr
> Also check out my Blog for answers to commonly asked questions.
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434