Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> how to define new InputFormat with streaming?


Copy link to this message
-
Re:Re: Re: how to define new InputFormat with streaming?
you are right!

Now the import path is all right.
At 2013-03-18 09:57:33,"Harsh J" <[EMAIL PROTECTED]> wrote:
>It isn't as easy as changing that import line:
>
>> package org.apache.hadoop.mapred.lib.input does not exist
>
>The right package is package org.apache.hadoop.mapred.
>
>On Mon, Mar 18, 2013 at 7:22 AM, springring <[EMAIL PROTECTED]> wrote:
>> thanks
>> I modify the java file with old "mapred" API, but there is still error
>>
>>  javac -classpath /usr/lib/hadoop/hadoop-core-0.20.2-cdh3u3.jar:/usr/lib/hadoop/lib/* -d class9 ./*.java
>> ./WholeFileInputFormat.java:16: error: package org.apache.hadoop.mapred.lib.input does not exist
>> import org.apache.hadoop.mapred.lib.input.*;
>>
>> does it because hadoop-0.20.2-cdh3u3 not include "mapred" API?
>>
>>
>>
>>
>>
>>
>> At 2013-03-17 14:22:43,"Harsh J" <[EMAIL PROTECTED]> wrote:
>>>The issue is that Streaming expects the old/stable MR API
>>>(org.apache.hadoop.mapred.InputFormat) as its input format class, but your
>>>WholeFileInputFormat is using the new MR API
>>>(org.apache.hadoop.mapreduce.lib.input.InputFormat). Using the older form
>>>will let you pass.
>>>
>>>This has nothing to do with your version/distribution of Hadoop.
>>>
>>>
>>>On Fri, Mar 15, 2013 at 4:28 PM, Steve Loughran <[EMAIL PROTECTED]>wrote:
>>>
>>>> On 15 March 2013 09:18, springring <[EMAIL PROTECTED]> wrote:
>>>>
>>>> >  Hi,
>>>> >
>>>> >      my hadoop version is Hadoop 0.20.2-cdh3u3 and I want to define new
>>>> > InputFormat in hadoop book , but there is error
>>>> > "class org.apache.hadoop.streaming.WholeFileInputFormat not
>>>> > org.apache.hadoop.mapred.InputFormat"
>>>> >
>>>> > Hadoop version is 0.20, but the streaming still depend on 0.10 mapred
>>>> api?
>>>> >
>>>>
>>>>
>>>> 1. please don't spam all the lists
>>>> 2. grab a later version of the apache releases if you want help on them on
>>>> these mailing lists, or go to the cloudera lists, where they will probably
>>>> say "upgrade to CDH 4.x" before asking questions.
>>>>
>>>> thanks
>>>>
>>>
>>>
>>>
>>>--
>>>Harsh J
>
>
>
>--
>Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB