Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Evenly splitting the table


Copy link to this message
-
Re: Evenly splitting the table
@Aaron,

You said you're using a salt, which would imply that your number is random and not derived from the base key.  (Where base key is the key prior to being hashed. )

Is that the case, or do you mean that Kiji is taking the first two bytes of the hash and prepending it to the key?
On Mar 20, 2013, at 6:55 PM, Aaron Kimball <[EMAIL PROTECTED]> wrote:

> Hi Cole,
>
> How are your keys structured? In Kiji, we default to using hashed row keys
> where each key starts with two bytes of salt. This makes it a lot easier to
> pre-split the table since you can make stronger guarantees about the key
> distribution.
>
> If your keys are "raw" text like, say, plaintext email addresses, it is
> significantly more difficult to guess the right splits a priori.
>
> cheers,
> - Aaron
>
>
>
> On Wed, Mar 20, 2013 at 3:43 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Take a look at TestAdmin#testCreateTableRPCTimeOut() where
>> hbaseadmin.createTable() is called.
>>
>> bq. Is there a way to go about splitting the entire table without having
>> specific start and end keys?
>>
>> I don't think so.
>>
>> On Wed, Mar 20, 2013 at 3:32 PM, Cole <[EMAIL PROTECTED]> wrote:
>>
>>> I was wondering how I can go about evenly splitting an entire table in
>>> HBase during table creation[1]. I tried providing the empty byte arrays
>>> HConstants.EMPTY_START_ROW and HConstants.EMPTY_END_ROW
>>> as parameters to the method I linked below, and got an error: "Start
>>> key must be smaller than end key". Is there a way to go about splitting
>>> the entire table without having specific start and end keys? Thanks in
>>> advance.
>>>
>>>
>>> [1]
>>>
>>>
>> http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html
>>> #createTable(org.apache.hadoop.hbase.HTableDescriptor, byte[], byte[],
>> int)
>>>
>>>
>>