Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Does TotalOrderPartitioner refresh its partitions selection tree


+
chris Behin 2012-10-08, 17:33
Copy link to this message
-
Re: Does TotalOrderPartitioner refresh its partitions selection tree
Chris,

In this case nothing scared actually happens.

* If partitions are the same, then HBase simply copies all your HFiles
during bulkloading procedure.
* If partitions are changed, then it still copies them, but in addition,
some of these files (according to number of split regions) would be also
split and the whole procedure would be slightly longer.

To understand the idea, imagine you have only one HFile, but  some regions
in HBase.
On Mon, Oct 8, 2012 at 9:33 PM, chris Behin <[EMAIL PROTECTED]> wrote:

> Hil all,
>
> I am reading TotalOrderPartitioner code to understand its structure,
>
> As I understood ( correct me if I am wrong ) TotalOrderPartitioner.setConf
> method reads splitPoints from partFile via readPartitions method
> ( TotalOrderPartitioner.java line 82 )
> and creates a buildTrie or BinarySearchNode according to this partition
> information ( TotalOrderPartitioner.java line 97 and 108 )
>
> I wonder what happens if regions are changed during the execution of
> mapreduce job. Is it a problem?
> Because TotalOrderPartitioner does not refresh its partitioner during its
> execution ( as I understand ).
>
> You may say that this is an exception case and the possibility is low,
> I just ask for understanding the mechanism..
>
> best regards...
>

--
Evgeny Morozov
Developer Grid Dynamics
Skype: morozov.evgeny
www.griddynamics.com
[EMAIL PROTECTED]