Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - how to define new InputFormat with streaming?


Copy link to this message
-
Re:Re: how to define new InputFormat with streaming?
springring 2013-03-18, 01:52
thanks
I modify the java file with old "mapred" API, but there is still error

 javac -classpath /usr/lib/hadoop/hadoop-core-0.20.2-cdh3u3.jar:/usr/lib/hadoop/lib/* -d class9 ./*.java
./WholeFileInputFormat.java:16: error: package org.apache.hadoop.mapred.lib.input does not exist
import org.apache.hadoop.mapred.lib.input.*;

does it because hadoop-0.20.2-cdh3u3 not include "mapred" API?
At 2013-03-17 14:22:43,"Harsh J" <[EMAIL PROTECTED]> wrote:
>The issue is that Streaming expects the old/stable MR API
>(org.apache.hadoop.mapred.InputFormat) as its input format class, but your
>WholeFileInputFormat is using the new MR API
>(org.apache.hadoop.mapreduce.lib.input.InputFormat). Using the older form
>will let you pass.
>
>This has nothing to do with your version/distribution of Hadoop.
>
>
>On Fri, Mar 15, 2013 at 4:28 PM, Steve Loughran <[EMAIL PROTECTED]>wrote:
>
>> On 15 March 2013 09:18, springring <[EMAIL PROTECTED]> wrote:
>>
>> >  Hi,
>> >
>> >      my hadoop version is Hadoop 0.20.2-cdh3u3 and I want to define new
>> > InputFormat in hadoop book , but there is error
>> > "class org.apache.hadoop.streaming.WholeFileInputFormat not
>> > org.apache.hadoop.mapred.InputFormat"
>> >
>> > Hadoop version is 0.20, but the streaming still depend on 0.10 mapred
>> api?
>> >
>>
>>
>> 1. please don't spam all the lists
>> 2. grab a later version of the apache releases if you want help on them on
>> these mailing lists, or go to the cloudera lists, where they will probably
>> say "upgrade to CDH 4.x" before asking questions.
>>
>> thanks
>>
>
>
>
>--
>Harsh J