Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> How can I sort a file with pairs <Key Value> in reverse order?


Copy link to this message
-
Re: How can I sort a file with pairs <Key Value> in reverse order?
Mapper Input
7 vim
2 emacs
9 firefox

Mapper output ( new key = Integer.MAX_VALUE - key value)
2147483640 vim
2147483645 emacs
2147483636 firefox

Note :Integer.MAX_VALUE is 2147483647 (which would be 2^31 - 1)

Hadoop will sort the records for you.
If you are using single reducer, reducer input would be

2147483636 firefox
2147483640 vim
2147483645 emacs

reducer output ( this time subtract key from Integer.MAX_VALUE to get
back original value)
9 firefox
7 vim
2 emacs
On 11/06/2013 13:05, Pedro S� da Costa wrote:
> Even with  your answer I can't see how can I sort the data in reverse
> order. I forgot to mention that, the output result is produced by one
> reduce task. This means that, at any point of the execution of the
> job,  the data must be grouped and sorted in descendent order.
>
>
> On 11 June 2013 13:57, Bhasker Allene <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     One way to approach is emit |Integer.MAX_VALUE - your key ||as
>     output of mapper.|
>     Example
>     Mapper input
>     7 vim
>     2 emacs
>     9 firefox
>
>     Mapper output
>     (Integer.MAX_VALUE - 7) vim
>     (Integer.MAX_VALUE - 2) emacs
>     (Integer.MAX_VALUE - 9) firefox
>
>     If you need secondary sorting on second part, you have to use
>     composite key  and write  your own petitioner, comparator.
>
>     Regards,
>     Bhasker
>
>     On 11/06/2013 11:10, Pedro S� da Costa wrote:
>>     I created a MapReduce job example that that uses the sort
>>     mechanism of hadoop to sort a file by the key in ascending order.
>>     This is an example of the data:
>>
>>     7    vim
>>     2    emacs
>>     9    firefox
>>
>>     At the end, I get the result:
>>
>>     2    emacs
>>     7    vim
>>     9    firefox
>>
>>
>>
>>     Now I want to sort in reverse order, for the result be:
>>     9    firefox
>>     7    vim
>>     2    emacs
>>
>>
>>
>>     How can I sort a file with pairs <Key Value> in reverse order?
>>
>>     --
>>     Best regards,
>
>
>     --
>     Thanks & Regards,
>     Bhasker Allene
>
>
>
>
> --
> Best regards,

--
Thanks & Regards,
Bhasker Allene