Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - how to define new InputFormat with streaming?


Copy link to this message
-
Re: Re: how to define new InputFormat with streaming?
Harsh J 2013-03-18, 01:57
It isn't as easy as changing that import line:

> package org.apache.hadoop.mapred.lib.input does not exist

The right package is package org.apache.hadoop.mapred.

On Mon, Mar 18, 2013 at 7:22 AM, springring <[EMAIL PROTECTED]> wrote:
> thanks
> I modify the java file with old "mapred" API, but there is still error
>
>  javac -classpath /usr/lib/hadoop/hadoop-core-0.20.2-cdh3u3.jar:/usr/lib/hadoop/lib/* -d class9 ./*.java
> ./WholeFileInputFormat.java:16: error: package org.apache.hadoop.mapred.lib.input does not exist
> import org.apache.hadoop.mapred.lib.input.*;
>
> does it because hadoop-0.20.2-cdh3u3 not include "mapred" API?
>
>
>
>
>
>
> At 2013-03-17 14:22:43,"Harsh J" <[EMAIL PROTECTED]> wrote:
>>The issue is that Streaming expects the old/stable MR API
>>(org.apache.hadoop.mapred.InputFormat) as its input format class, but your
>>WholeFileInputFormat is using the new MR API
>>(org.apache.hadoop.mapreduce.lib.input.InputFormat). Using the older form
>>will let you pass.
>>
>>This has nothing to do with your version/distribution of Hadoop.
>>
>>
>>On Fri, Mar 15, 2013 at 4:28 PM, Steve Loughran <[EMAIL PROTECTED]>wrote:
>>
>>> On 15 March 2013 09:18, springring <[EMAIL PROTECTED]> wrote:
>>>
>>> >  Hi,
>>> >
>>> >      my hadoop version is Hadoop 0.20.2-cdh3u3 and I want to define new
>>> > InputFormat in hadoop book , but there is error
>>> > "class org.apache.hadoop.streaming.WholeFileInputFormat not
>>> > org.apache.hadoop.mapred.InputFormat"
>>> >
>>> > Hadoop version is 0.20, but the streaming still depend on 0.10 mapred
>>> api?
>>> >
>>>
>>>
>>> 1. please don't spam all the lists
>>> 2. grab a later version of the apache releases if you want help on them on
>>> these mailing lists, or go to the cloudera lists, where they will probably
>>> say "upgrade to CDH 4.x" before asking questions.
>>>
>>> thanks
>>>
>>
>>
>>
>>--
>>Harsh J

--
Harsh J