Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - how to define new InputFormat with streaming?


Copy link to this message
-
Re:Re: Re: how to define new InputFormat with streaming?
springring 2013-03-18, 02:48
you are right!

Now the import path is all right.
At 2013-03-18 09:57:33,"Harsh J" <[EMAIL PROTECTED]> wrote:
>It isn't as easy as changing that import line:
>
>> package org.apache.hadoop.mapred.lib.input does not exist
>
>The right package is package org.apache.hadoop.mapred.
>
>On Mon, Mar 18, 2013 at 7:22 AM, springring <[EMAIL PROTECTED]> wrote:
>> thanks
>> I modify the java file with old "mapred" API, but there is still error
>>
>>  javac -classpath /usr/lib/hadoop/hadoop-core-0.20.2-cdh3u3.jar:/usr/lib/hadoop/lib/* -d class9 ./*.java
>> ./WholeFileInputFormat.java:16: error: package org.apache.hadoop.mapred.lib.input does not exist
>> import org.apache.hadoop.mapred.lib.input.*;
>>
>> does it because hadoop-0.20.2-cdh3u3 not include "mapred" API?
>>
>>
>>
>>
>>
>>
>> At 2013-03-17 14:22:43,"Harsh J" <[EMAIL PROTECTED]> wrote:
>>>The issue is that Streaming expects the old/stable MR API
>>>(org.apache.hadoop.mapred.InputFormat) as its input format class, but your
>>>WholeFileInputFormat is using the new MR API
>>>(org.apache.hadoop.mapreduce.lib.input.InputFormat). Using the older form
>>>will let you pass.
>>>
>>>This has nothing to do with your version/distribution of Hadoop.
>>>
>>>
>>>On Fri, Mar 15, 2013 at 4:28 PM, Steve Loughran <[EMAIL PROTECTED]>wrote:
>>>
>>>> On 15 March 2013 09:18, springring <[EMAIL PROTECTED]> wrote:
>>>>
>>>> >  Hi,
>>>> >
>>>> >      my hadoop version is Hadoop 0.20.2-cdh3u3 and I want to define new
>>>> > InputFormat in hadoop book , but there is error
>>>> > "class org.apache.hadoop.streaming.WholeFileInputFormat not
>>>> > org.apache.hadoop.mapred.InputFormat"
>>>> >
>>>> > Hadoop version is 0.20, but the streaming still depend on 0.10 mapred
>>>> api?
>>>> >
>>>>
>>>>
>>>> 1. please don't spam all the lists
>>>> 2. grab a later version of the apache releases if you want help on them on
>>>> these mailing lists, or go to the cloudera lists, where they will probably
>>>> say "upgrade to CDH 4.x" before asking questions.
>>>>
>>>> thanks
>>>>
>>>
>>>
>>>
>>>--
>>>Harsh J
>
>
>
>--
>Harsh J