Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Partition column on an Alpha Numeric Column


Copy link to this message
-
Partition column on an Alpha Numeric Column
All,

One of the primary key columns in a Relational table has alpha numberic of 6 characters - varchar(6).

The first three characters has this pattern -
1st one - 1 to 9   

2nd one - 1 to 9 or a -z
3rd one - 1 to 9 or a -z

Is this a good idea for performing queries ( can be any queries based on other columns of the table )

Partition the data based on the first three characters summing upto a total of 10 * 36 * 36 which is 12,960 partitons.
12960 partitons - Is it too much ? Impossible or never heard ? or can we consider this design ?

I know NameNode should have a powerful RAM. But how much ? How do we determine the limitation of the number of files a Name Node can handle?

Thanks,
Raj