Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Writing intermediate key,value pairs to file and read it again


Copy link to this message
-
Re: Writing intermediate key,value pairs to file and read it again
ya i looked @ chain reducer but it will not help in my case

Now i am able to write each key value pair received @ reducer(using
IFile.Writer )
but when i try to read it using IFile.Reader it reads all data not single
key value pair

Example:

*IFile.Reader reader  =  new* IFile.Reader<KEYIN,
VALUEIN>(context.getConfiguration(), localFs, file, *null*, myReadCounter);

DataInputBuffer keyRead = new DataInputBuffer();

DataInputBuffer valueRead = new DataInputBuffer();

reader.next(keyRead, valueRead);

when i say keyRead.getData()

 it print all key value pair in that buffer i want single  key value pair.

On Sun, Apr 21, 2013 at 9:53 AM, Azuryy Yu <[EMAIL PROTECTED]> wrote:

> you would look at chain reducer java doc, which meet your requirement.
>
> --Send from my Sony mobile.
> On Apr 20, 2013 11:43 PM, "Vikas Jadhav" <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>> Can anyone help me in following issue
>> Writing intermediate key,value pairs to file and read it again
>>
>> let us say i have to write each intermediate pair received @reducer to a
>> file and again read that as key value pair again and use it for processing
>>
>> I found IFile.java file which has reader and writer but i am not able
>> understand how to use it for example. I dont understand Counter value as
>> last parameter "spilledRecordsCounter"
>>
>>
>> Thanks.
>>
>>
>> --
>> *
>> *
>> *
>>
>>   Regards,*
>> *   Vikas *
>>
>
--
*
*
*

  Regards,*
*   Vikas *