Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Cluster load


Copy link to this message
-
Re: Cluster load
Suraj Varma 2012-07-29, 04:38
You can also do an online merge to merge the regions together and then
resplit it ... https://issues.apache.org/jira/browse/HBASE-1621
--S

On Sat, Jul 28, 2012 at 11:07 AM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
> On Fri, Jul 27, 2012 at 6:03 PM, Alex Baranau <[EMAIL PROTECTED]>wrote:
>
>> Yeah, your row keys start with \x00 which is = (byte) 0. This is not the
>> same as "0" (which is = (byte) 48). You know what to fix now ;)
>>
>>
> Thanks for checking! I'll make the required changes to my split. Is it
> possible to alter splits or only way is to re-create the tables?
>
>
>> Alex Baranau
>> ------
>> Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
>> Solr
>>
>>
>> On Fri, Jul 27, 2012 at 8:43 PM, Mohit Anchlia <[EMAIL PROTECTED]
>> >wrote:
>>
>> > On Fri, Jul 27, 2012 at 4:51 PM, Alex Baranau <[EMAIL PROTECTED]
>> > >wrote:
>> >
>> > > Can you scan your table and show one record?
>> > >
>> > > I guess you might be confusing Bytes.toBytes("0") vs byte[] {(byte) 0}
>> > that
>> > > I mentioned in the other thread. I.e. looks like first region holds
>> > records
>> > > which key starts with any byte up to "0", which is (byte) 48. Hence, if
>> > you
>> > > set first byte of your key to anything from (byte) 0 - (byte) 9, all of
>> > > them will fall into first regions which holds records with prefixes
>> > (byte)
>> > > 0 - (byte) 48.
>> > >
>> > > Could you check that?
>> > >
>> > >
>> > I thought that if I give Bytes.toBytes("0") it really means that the row
>> > keys starting with "0" will go in that region. Here is my code that
>> creates
>> > a row key and splits using admin util. I also am including the output of
>> > hbase shell scan after the code.
>> >
>> > public static byte[][] splitRegionsSessionTimeline(int start, int end) {
>> >  byte[][] splitKeys = new byte[end][];
>> >  // the first region starting with empty key will be created
>> >  // automatically
>> >  for (int i = 0; i < splitKeys.length; i++) {
>> >   splitKeys[i] = Bytes.toBytes(String.valueOf(i));
>> >  }
>> >  return splitKeys;
>> > }
>> >  public static byte [] getRowKey(MetricType metricName, Long timestamp,
>> > Short bucketNo, char rowDelim){
>> >   byte [] result = null;
>> >   int rowSize = getRowSize();
>> >   ByteBuffer b = ByteBuffer.allocate(rowSize);
>> >   //Bucket No 0-9 randomely
>> >   b.putShort(bucketNo);
>> >   //Row Delimiter
>> >   b.putChar(rowDelim);
>> >   b.putShort(metricName.getId());
>> >   long reverseOrderEpoch = getReverseBaseTimeStamp(metricName,timestamp);
>> >   b.putLong(reverseOrderEpoch);
>> >   result = b.array();
>> >   return result;
>> > }
>> >
>> > from hbase shell scan table:
>> >
>> >   \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9
>> > column=S_T_MTX:\x00\x00gK,
>> > timestamp=1343350528865, value=1343350646443
>> >  F
>> >  \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9
>> column=S_T_MTX:\x00\x00gL,
>> > timestamp=1343350528866, value=1343350646444
>> >  F
>> >  \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9
>> column=S_T_MTX:\x00\x00gU,
>> > timestamp=1343350528874, value=1343350646453
>> >  F
>> >  \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9
>> column=S_T_MTX:\x00\x00gZ,
>> > timestamp=1343350528880, value=1343350646458
>> >  F
>> >
>> > > Alex Baranau
>> > > ------
>> > > Sematext :: http://blog.sematext.com/ :: Hadoop - HBase -
>> ElasticSearch
>> > -
>> > > Solr
>> > >
>> > > On Fri, Jul 27, 2012 at 7:24 PM, Mohit Anchlia <[EMAIL PROTECTED]
>> > > >wrote:
>> > >
>> > > > On Fri, Jul 27, 2012 at 11:48 AM, Alex Baranau <
>> > [EMAIL PROTECTED]
>> > > > >wrote:
>> > > >
>> > > > > You can read metrics [0] from JMX directly [1] or use Ganglia [2]
>> or
>> > > > other
>> > > > > third-party tools like [3] (I'm a little biased here;)).
>> > > > >
>> > > > > [0] http://hbase.apache.org/book.html#hbase_metrics
>> > > > > [1] http://hbase.apache.org/metrics.html
>> > > > > [2] http://wiki.apache.org/hadoop/GangliaMetrics
>> > > > > [3]
>> http://sematext.com/spm/hbase-performance-monitoring/index.html