Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Cluster load


Copy link to this message
-
Re: Cluster load
On Fri, Jul 27, 2012 at 4:51 PM, Alex Baranau <[EMAIL PROTECTED]>wrote:

> Can you scan your table and show one record?
>
> I guess you might be confusing Bytes.toBytes("0") vs byte[] {(byte) 0} that
> I mentioned in the other thread. I.e. looks like first region holds records
> which key starts with any byte up to "0", which is (byte) 48. Hence, if you
> set first byte of your key to anything from (byte) 0 - (byte) 9, all of
> them will fall into first regions which holds records with prefixes (byte)
> 0 - (byte) 48.
>
> Could you check that?
>
>
I thought that if I give Bytes.toBytes("0") it really means that the row
keys starting with "0" will go in that region. Here is my code that creates
a row key and splits using admin util. I also am including the output of
hbase shell scan after the code.

public static byte[][] splitRegionsSessionTimeline(int start, int end) {
 byte[][] splitKeys = new byte[end][];
 // the first region starting with empty key will be created
 // automatically
 for (int i = 0; i < splitKeys.length; i++) {
  splitKeys[i] = Bytes.toBytes(String.valueOf(i));
 }
 return splitKeys;
}
 public static byte [] getRowKey(MetricType metricName, Long timestamp,
Short bucketNo, char rowDelim){
  byte [] result = null;
  int rowSize = getRowSize();
  ByteBuffer b = ByteBuffer.allocate(rowSize);
  //Bucket No 0-9 randomely
  b.putShort(bucketNo);
  //Row Delimiter
  b.putChar(rowDelim);
  b.putShort(metricName.getId());
  long reverseOrderEpoch = getReverseBaseTimeStamp(metricName,timestamp);
  b.putLong(reverseOrderEpoch);
  result = b.array();
  return result;
}

from hbase shell scan table:

  \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9 column=S_T_MTX:\x00\x00gK,
timestamp=1343350528865, value=1343350646443
 F
 \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9 column=S_T_MTX:\x00\x00gL,
timestamp=1343350528866, value=1343350646444
 F
 \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9 column=S_T_MTX:\x00\x00gU,
timestamp=1343350528874, value=1343350646453
 F
 \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9 column=S_T_MTX:\x00\x00gZ,
timestamp=1343350528880, value=1343350646458
 F

> Alex Baranau
> ------
> Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
> Solr
>
> On Fri, Jul 27, 2012 at 7:24 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > On Fri, Jul 27, 2012 at 11:48 AM, Alex Baranau <[EMAIL PROTECTED]
> > >wrote:
> >
> > > You can read metrics [0] from JMX directly [1] or use Ganglia [2] or
> > other
> > > third-party tools like [3] (I'm a little biased here;)).
> > >
> > > [0] http://hbase.apache.org/book.html#hbase_metrics
> > > [1] http://hbase.apache.org/metrics.html
> > > [2] http://wiki.apache.org/hadoop/GangliaMetrics
> > > [3] http://sematext.com/spm/hbase-performance-monitoring/index.html
> > >
> > > Note, that metrics values may seem a bit ugly/weird: as they say, you
> > have
> > > to refer to Lars' book HBase in Action to understand how some of them
> > > calculated. There's an ongoing work towards revising metrics, they
> should
> > > look much better in next releases.
> > >
> > >
> >
> > I did flush still what I am seeing is that all my keys are still going to
> > the first region even though my keys have 0-9 as the first character. Is
> > there a easy way to see why that might be? hbase shell scan only shows
> > value in hex.
> >
> >   SESSION_TIMELINE1,,1343334722986.0a5f6fadd0435 column=info:regioninfo,
> > timestamp=1343334723073, value=REGION => {NAME =>
> > 'SESSION_TIMELINE1,,1343334722986.0a5f6fadd0435898c6f4cf11daa989
> >  898c6f4cf11daa9895a.                           5a.', STARTKEY => '',
> > ENDKEY => '0', ENCODED => 0a5f6fadd0435898c6f4cf11daa9895a, TABLE =>
> {{NAME
> > => 'SESSION_TIMELINE1', FAMILIES => [{NA
> >                                                 ME => 'S_T_MTX',
> > BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0', COMPRESSION => 'GZ',
> > VERSIONS => '1', TTL => '2147483647', BLOCKSIZE =>
> >                                                 '65536', IN_MEMORY =>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB