Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> problem with streaming and libjars


Copy link to this message
-
Re: problem with streaming and libjars
I would try the following:

hadoop -libjars /home/ayon/jars/MultiOutput.jar jar
/usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar
-libjars /home/ayon/jars/MultiOutput.jar -input
/user/ayon/streaming_test_input -output
/user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat
-outputformat MultiFileOutput

Another thing to note, I'm pretty sure you need to specify the full
package with the name of the output format class (e.g.
my.package.MultiFileOutput).

-Joey

On Tue, Jun 14, 2011 at 11:31 PM, Ayon Sinha <[EMAIL PROTECTED]> wrote:
> None of my options are working. Any help is appreciated.
>
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar -libjars /home/ayon/jars/MultiOutput.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar-archives /home/ayon/jars/MultiOutput.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar-libjars /home/ayon/jars/MultiOutput.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
> [ayon@dwdb002 ~]$ hadoop jar /usr/lib/hadoop-0.20/contrib/streaming/hadoop-streaming-0.20.2-cdh3u0.jar -libjars /home/ayon/jars/MultiOutput.jar -files /home/ayon/jars/MultiOutput.jar -input /user/ayon/streaming_test_input -output /user/ayon/streaming_test_output -mapper /bin/cat -reducer /bin/cat -outputformat MultiFileOutput -file /home/ayon/jars/MultiOutput.jar
> -outputformat : class not found : MultiFileOutput
> Streaming Command Failed!
>
>
> -Ayon
> See My Photos on Flickr
> Also check out my Blog for answers to commonly asked questions.
>

--
Joseph Echeverria
Cloudera, Inc.
443.305.9434
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB