Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - HBase split policy


+
Jean-Marc Spaggiari 2013-01-22, 11:42
+
Anoop Sam John 2013-01-22, 12:24
+
ramkrishna vasudevan 2013-01-22, 13:38
+
Jean-Marc Spaggiari 2013-01-22, 13:47
+
ramkrishna vasudevan 2013-01-22, 14:02
+
Jean-Marc Spaggiari 2013-01-22, 14:10
+
Jean-Marc Spaggiari 2013-01-23, 02:39
+
Anoop Sam John 2013-01-23, 06:17
+
Jean-Marc Spaggiari 2013-01-23, 12:26
+
ramkrishna vasudevan 2013-01-23, 18:09
Copy link to this message
-
Re: HBase split policy
Jean-Marc Spaggiari 2013-01-23, 18:24
It's all VERY small data... It's 4 bytes followed by a less thatn 256
bytes string, and there is no data. (one byte data).

I merged the regions again and this time the split went well. I looked
in the logs and did not find anything special.

Now, I have a MR running for few hours, so I can't retry. But later I
will. I will activate the debug logs for the split classes and give it
another try.

JM

2013/1/23, ramkrishna vasudevan <[EMAIL PROTECTED]>:
>>>This morning, I have some very big regions, still over the 100MB, and
>>>some very small. And the big regions are at least hundred times bigger
>>>than the small one.
>
> The region that was bigger than 100 MB (much bigger) what was the data in
> them.  Were there any hefty rows in them.  Check them.
>
> Regarding the problem like the major_compact did not trigger split, could
> you check out the logs.  May be the logs could give us some idea and based
> on that the calculations can be done.
>
> Regards
> Ram
>
> On Wed, Jan 23, 2013 at 5:56 PM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Anoop,
>>
>> I ran another major_compact and the split is now totally done.
>>
>> Question is, why has it not been done initially when I ran the first
>> major_compact. No idea.
>>
>> I will re-merge the file into one single region and re-compact to see
>> if I can reproduce that.
>>
>> JM
>>
>> 2013/1/23, Anoop Sam John <[EMAIL PROTECTED]>:
>> >>What will trigger the split?
>> > The things which can trigger a split
>> > 1. Explicit split call from the client side using admin API
>> > 2. A memstore flush
>> > 3. A compaction
>> >
>> > So even though there is no write operations happening on the region (no
>> > flushes) still a compaction performed for that region can trigger
>> > split.
>> > May be in your case compaction happened for some of the regions and
>> resulted
>> > in split...
>> >
>> > -Anoop-
>> > ________________________________________
>> > From: Jean-Marc Spaggiari [[EMAIL PROTECTED]]
>> > Sent: Wednesday, January 23, 2013 8:09 AM
>> > To: [EMAIL PROTECTED]
>> > Subject: Re: HBase split policy
>> >
>> > Another related question.
>> >
>> > What will trigger the split?
>> >
>> > I mean, I merge all the regions in a single one, split that in 4 2.5GB
>> > regions, alter it to set maxsize to 300MB and enable the table. I
>> > don't do anything. No put, no get. What will trigger the regions
>> > split?
>> >
>> > I have one small table, about 1.2GB with 8M lines. I merged it in a
>> > single region, and setup the maxsize to the 12MB. It got almost
>> > split... All the regions got split except one.
>> >
>> > Here is the screenshot:
>> > http://imageshack.us/photo/my-images/834/hannibalb.png/
>> >
>> > It's not the first region, not the last. There is nothing specific
>> > with this region, and it's not getting split.
>> >
>> > Any idea why, and how I can trigger the split without putting any data
>> > into the date?
>> >
>> > Thanks,
>> >
>> > JM
>>
>