One of the primary key columns in a Relational table has alpha numberic of 6 characters - varchar(6).
The first three characters has this pattern -
1st one - 1 to 9
2nd one - 1 to 9 or a -z
3rd one - 1 to 9 or a -z
Is this a good idea for performing queries ( can be any queries based on other columns of the table )
Partition the data based on the first three characters summing upto a total of 10 * 36 * 36 which is 12,960 partitons.
12960 partitons - Is it too much ? Impossible or never heard ? or can we consider this design ?
I know NameNode should have a powerful RAM. But how much ? How do we determine the limitation of the number of files a Name Node can handle?