Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Cluster load


Copy link to this message
-
Re: Cluster load
On Fri, Jul 27, 2012 at 4:51 PM, Alex Baranau <[EMAIL PROTECTED]>wrote:

> Can you scan your table and show one record?
>
> I guess you might be confusing Bytes.toBytes("0") vs byte[] {(byte) 0} that
> I mentioned in the other thread. I.e. looks like first region holds records
> which key starts with any byte up to "0", which is (byte) 48. Hence, if you
> set first byte of your key to anything from (byte) 0 - (byte) 9, all of
> them will fall into first regions which holds records with prefixes (byte)
> 0 - (byte) 48.
>
> Could you check that?
>
>
I thought that if I give Bytes.toBytes("0") it really means that the row
keys starting with "0" will go in that region. Here is my code that creates
a row key and splits using admin util. I also am including the output of
hbase shell scan after the code.

public static byte[][] splitRegionsSessionTimeline(int start, int end) {
 byte[][] splitKeys = new byte[end][];
 // the first region starting with empty key will be created
 // automatically
 for (int i = 0; i < splitKeys.length; i++) {
  splitKeys[i] = Bytes.toBytes(String.valueOf(i));
 }
 return splitKeys;
}
 public static byte [] getRowKey(MetricType metricName, Long timestamp,
Short bucketNo, char rowDelim){
  byte [] result = null;
  int rowSize = getRowSize();
  ByteBuffer b = ByteBuffer.allocate(rowSize);
  //Bucket No 0-9 randomely
  b.putShort(bucketNo);
  //Row Delimiter
  b.putChar(rowDelim);
  b.putShort(metricName.getId());
  long reverseOrderEpoch = getReverseBaseTimeStamp(metricName,timestamp);
  b.putLong(reverseOrderEpoch);
  result = b.array();
  return result;
}

from hbase shell scan table:

  \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9 column=S_T_MTX:\x00\x00gK,
timestamp=1343350528865, value=1343350646443
 F
 \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9 column=S_T_MTX:\x00\x00gL,
timestamp=1343350528866, value=1343350646444
 F
 \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9 column=S_T_MTX:\x00\x00gU,
timestamp=1343350528874, value=1343350646453
 F
 \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9 column=S_T_MTX:\x00\x00gZ,
timestamp=1343350528880, value=1343350646458
 F

> Alex Baranau
> ------
> Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
> Solr
>
> On Fri, Jul 27, 2012 at 7:24 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > On Fri, Jul 27, 2012 at 11:48 AM, Alex Baranau <[EMAIL PROTECTED]
> > >wrote:
> >
> > > You can read metrics [0] from JMX directly [1] or use Ganglia [2] or
> > other
> > > third-party tools like [3] (I'm a little biased here;)).
> > >
> > > [0] http://hbase.apache.org/book.html#hbase_metrics
> > > [1] http://hbase.apache.org/metrics.html
> > > [2] http://wiki.apache.org/hadoop/GangliaMetrics
> > > [3] http://sematext.com/spm/hbase-performance-monitoring/index.html
> > >
> > > Note, that metrics values may seem a bit ugly/weird: as they say, you
> > have
> > > to refer to Lars' book HBase in Action to understand how some of them
> > > calculated. There's an ongoing work towards revising metrics, they
> should
> > > look much better in next releases.
> > >
> > >
> >
> > I did flush still what I am seeing is that all my keys are still going to
> > the first region even though my keys have 0-9 as the first character. Is
> > there a easy way to see why that might be? hbase shell scan only shows
> > value in hex.
> >
> >   SESSION_TIMELINE1,,1343334722986.0a5f6fadd0435 column=info:regioninfo,
> > timestamp=1343334723073, value=REGION => {NAME =>
> > 'SESSION_TIMELINE1,,1343334722986.0a5f6fadd0435898c6f4cf11daa989
> >  898c6f4cf11daa9895a.                           5a.', STARTKEY => '',
> > ENDKEY => '0', ENCODED => 0a5f6fadd0435898c6f4cf11daa9895a, TABLE =>
> {{NAME
> > => 'SESSION_TIMELINE1', FAMILIES => [{NA
> >                                                 ME => 'S_T_MTX',
> > BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'GZ',
> > VERSIONS => '1', TTL => '2147483647', BLOCKSIZE =>
> >                                                 '65536', IN_MEMORY =>