Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Writing intermediate key,value pairs to file and read it again


Copy link to this message
-
Re: Writing intermediate key,value pairs to file and read it again
ya i looked @ chain reducer but it will not help in my case

Now i am able to write each key value pair received @ reducer(using
IFile.Writer )
but when i try to read it using IFile.Reader it reads all data not single
key value pair

Example:

*IFile.Reader reader  =  new* IFile.Reader<KEYIN,
VALUEIN>(context.getConfiguration(), localFs, file, *null*, myReadCounter);

DataInputBuffer keyRead = new DataInputBuffer();

DataInputBuffer valueRead = new DataInputBuffer();

reader.next(keyRead, valueRead);

when i say keyRead.getData()

 it print all key value pair in that buffer i want single  key value pair.

On Sun, Apr 21, 2013 at 9:53 AM, Azuryy Yu <[EMAIL PROTECTED]> wrote:

> you would look at chain reducer java doc, which meet your requirement.
>
> --Send from my Sony mobile.
> On Apr 20, 2013 11:43 PM, "Vikas Jadhav" <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>> Can anyone help me in following issue
>> Writing intermediate key,value pairs to file and read it again
>>
>> let us say i have to write each intermediate pair received @reducer to a
>> file and again read that as key value pair again and use it for processing
>>
>> I found IFile.java file which has reader and writer but i am not able
>> understand how to use it for example. I dont understand Counter value as
>> last parameter "spilledRecordsCounter"
>>
>>
>> Thanks.
>>
>>
>> --
>> *
>> *
>> *
>>
>>   Regards,*
>> *   Vikas *
>>
>
--
*
*
*

  Regards,*
*   Vikas *
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB