-Re: Partitioners - How to know if they are working
David Rosenstrauch 2012-02-17, 04:15
On 02/16/2012 12:49 PM, [EMAIL PROTECTED] wrote:
> Hello All,
> I wrote my own partitioner and I would like to see if it's working.
> By printing the return of method getPartition I could see that the partitions were different, but were they really working? To answer that I got the keys that every reducer task processed and that was what I expected. It seems my partitioner is working properly. But not easy to discover though.
> Does anyone know if there is an easier way to see if your customized partitioner is working? For instance, a counter that shows how many partitioners a map generated or a reducer received?
> Thanks in advance,
> Fabio Almeida
At my last job we wrote a custom partitioner, and we tested it out
completely outside of Hadoop using standard JUnit unit tests.