Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase split policy


Copy link to this message
-
Re: HBase split policy
It's all VERY small data... It's 4 bytes followed by a less thatn 256
bytes string, and there is no data. (one byte data).

I merged the regions again and this time the split went well. I looked
in the logs and did not find anything special.

Now, I have a MR running for few hours, so I can't retry. But later I
will. I will activate the debug logs for the split classes and give it
another try.

JM

2013/1/23, ramkrishna vasudevan <[EMAIL PROTECTED]>:
>>>This morning, I have some very big regions, still over the 100MB, and
>>>some very small. And the big regions are at least hundred times bigger
>>>than the small one.
>
> The region that was bigger than 100 MB (much bigger) what was the data in
> them.  Were there any hefty rows in them.  Check them.
>
> Regarding the problem like the major_compact did not trigger split, could
> you check out the logs.  May be the logs could give us some idea and based
> on that the calculations can be done.
>
> Regards
> Ram
>
> On Wed, Jan 23, 2013 at 5:56 PM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Anoop,
>>
>> I ran another major_compact and the split is now totally done.
>>
>> Question is, why has it not been done initially when I ran the first
>> major_compact. No idea.
>>
>> I will re-merge the file into one single region and re-compact to see
>> if I can reproduce that.
>>
>> JM
>>
>> 2013/1/23, Anoop Sam John <[EMAIL PROTECTED]>:
>> >>What will trigger the split?
>> > The things which can trigger a split
>> > 1. Explicit split call from the client side using admin API
>> > 2. A memstore flush
>> > 3. A compaction
>> >
>> > So even though there is no write operations happening on the region (no
>> > flushes) still a compaction performed for that region can trigger
>> > split.
>> > May be in your case compaction happened for some of the regions and
>> resulted
>> > in split...
>> >
>> > -Anoop-
>> > ________________________________________
>> > From: Jean-Marc Spaggiari [[EMAIL PROTECTED]]
>> > Sent: Wednesday, January 23, 2013 8:09 AM
>> > To: [EMAIL PROTECTED]
>> > Subject: Re: HBase split policy
>> >
>> > Another related question.
>> >
>> > What will trigger the split?
>> >
>> > I mean, I merge all the regions in a single one, split that in 4 2.5GB
>> > regions, alter it to set maxsize to 300MB and enable the table. I
>> > don't do anything. No put, no get. What will trigger the regions
>> > split?
>> >
>> > I have one small table, about 1.2GB with 8M lines. I merged it in a
>> > single region, and setup the maxsize to the 12MB. It got almost
>> > split... All the regions got split except one.
>> >
>> > Here is the screenshot:
>> > http://imageshack.us/photo/my-images/834/hannibalb.png/
>> >
>> > It's not the first region, not the last. There is nothing specific
>> > with this region, and it's not getting split.
>> >
>> > Any idea why, and how I can trigger the split without putting any data
>> > into the date?
>> >
>> > Thanks,
>> >
>> > JM
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB