|
|
-
Custom Partitioner is now working in CDH4samir das mohapatra 2013-01-23, 16:58
Hi All
Just we migrated from CDH3 to CDH4 , But after that custom Partitioner is note working under CDH4. Can you tell me why it is like that but in CDH3 same code is working fine. PARTITONER CODE ------------------------------------------- public static class AgePartitioner extends Partitioner<Text, Text> { static{ System.out.println("#############################AgePartitioner++++++++++++++++++++++"); } @Override public int getPartition(Text key, Text value, int numReduceTasks) { System.out.println("#############################AgePartitioner"+value.toString()); String [] nameAgeScore = value.toString().split("\t"); String age = nameAgeScore[1]; int ageInt = Integer.parseInt(age); //this is done to avoid performing mod with 0 if(numReduceTasks == 0) return 0; //if the age is <20, assign partition 0 if(ageInt <=20){ return 0; } //else if the age is between 20 and 50, assign partition 1 if(ageInt >20 && ageInt <=50){ return 1 % numReduceTasks; } //otherwise assign partition 2 else return 2 % numReduceTasks; } } DRIVER LEVEL CODE ------------------------------------------- job.setPartitionerClass(AgePartitioner.class); job.setNumReduceTasks(3); |