Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How can I sort a file with pairs <Key Value> in reverse order?


Copy link to this message
-
Re: How can I sort a file with pairs <Key Value> in reverse order?
Thanks for your help. Now I get it.
On 11 June 2013 14:21, Bhasker Allene <[EMAIL PROTECTED]> wrote:

>  Mapper Input
>
> 7 vim
> 2 emacs
> 9 firefox
>
> Mapper output ( new key = Integer.MAX_VALUE - key value)
> 2147483640 vim
> 2147483645 emacs
> 2147483636 firefox
>
> Note :Integer.MAX_VALUE is 2147483647 (which would be 2^31 - 1)
>
> Hadoop will sort the records for you.
> If you are using single reducer, reducer input would be
>
> 2147483636 firefox
> 2147483640 vim
> 2147483645 emacs
>
> reducer output ( this time subtract key from Integer.MAX_VALUE to get
> back original value)
> 9 firefox
> 7 vim
> 2 emacs
>
>
>  On 11/06/2013 13:05, Pedro Sá da Costa wrote:
>
> Even with  your answer I can't see how can I sort the data in reverse
> order. I forgot to mention that, the output result is produced by one
> reduce task. This means that, at any point of the execution of the job,
>  the data must be grouped and sorted in descendent order.
>
>
> On 11 June 2013 13:57, Bhasker Allene <[EMAIL PROTECTED]> wrote:
>
>>  One way to approach is emit Integer.MAX_VALUE - your key as output of
>> mapper.
>> Example
>> Mapper input
>>  7 vim
>> 2 emacs
>> 9 firefox
>>
>>  Mapper output
>> (Integer.MAX_VALUE - 7) vim
>> (Integer.MAX_VALUE - 2) emacs
>> (Integer.MAX_VALUE - 9) firefox
>>
>> If you need secondary sorting on second part, you have to use composite
>> key  and write  your own petitioner, comparator.
>>
>> Regards,
>> Bhasker
>>
>> On 11/06/2013 11:10, Pedro Sá da Costa wrote:
>>
>>  I created a MapReduce job example that that uses the sort mechanism of
>> hadoop to sort a file by the key in ascending order. This is an example of
>> the data:
>>
>>  7    vim
>>  2    emacs
>>  9    firefox
>>
>>  At the end, I get the result:
>>
>> 2    emacs
>> 7    vim
>>  9    firefox
>>
>>
>>
>> Now I want to sort in reverse order, for the result be:
>> 9    firefox
>> 7    vim
>> 2    emacs
>>
>>
>>
>> How can I sort a file with pairs <Key Value> in reverse order?
>>
>> --
>> Best regards,
>>
>>
>>
>>   --
>> Thanks & Regards,
>> Bhasker Allene
>>
>>
>
>
>  --
> Best regards,
>
>
> --
> Thanks & Regards,
> Bhasker Allene
>
>
--
Best regards,