Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Cluster load


+
Mohit Anchlia 2012-07-26, 22:32
+
syed kather 2012-07-27, 00:53
+
Mohit Anchlia 2012-07-27, 01:07
+
Khang Pham 2012-07-27, 07:16
+
Alex Baranau 2012-07-27, 14:21
+
syed kather 2012-07-27, 14:52
+
Alex Baranau 2012-07-27, 18:07
+
syed kather 2012-07-27, 18:21
+
Alex Baranau 2012-07-27, 18:48
+
Mohit Anchlia 2012-07-27, 23:24
+
Alex Baranau 2012-07-27, 23:51
+
Mohit Anchlia 2012-07-28, 00:43
+
Alex Baranau 2012-07-28, 01:03
+
Mohit Anchlia 2012-07-28, 18:07
+
Suraj Varma 2012-07-29, 04:38
Copy link to this message
-
Re: Cluster load
On Fri, Jul 27, 2012 at 6:03 PM, Alex Baranau <[EMAIL PROTECTED]>wrote:

> Yeah, your row keys start with \x00 which is = (byte) 0. This is not the
> same as "0" (which is = (byte) 48). You know what to fix now ;)
>
>

I made required changes and it seems to be load balancing it pretty well. I
do have a follow up question around how to intrepret the output of hbase
shell. If I want to visually calculate the length of the row key can I
assume that \x00\x00 is equal to 2 bytes? I am just trying to get my head
around understanding hex format displayed on the shell.

 \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7'\x05\x11
column=S_T_MTX:\x00\x00?\xB8, timestamp=1343670017892, value=1343670136312
 \xBF
> Alex Baranau
> ------
> Sematext :: http://blog.sematext.com/ :: Hadoop - HBase - ElasticSearch -
> Solr
>
>
> On Fri, Jul 27, 2012 at 8:43 PM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > On Fri, Jul 27, 2012 at 4:51 PM, Alex Baranau <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Can you scan your table and show one record?
> > >
> > > I guess you might be confusing Bytes.toBytes("0") vs byte[] {(byte) 0}
> > that
> > > I mentioned in the other thread. I.e. looks like first region holds
> > records
> > > which key starts with any byte up to "0", which is (byte) 48. Hence, if
> > you
> > > set first byte of your key to anything from (byte) 0 - (byte) 9, all of
> > > them will fall into first regions which holds records with prefixes
> > (byte)
> > > 0 - (byte) 48.
> > >
> > > Could you check that?
> > >
> > >
> > I thought that if I give Bytes.toBytes("0") it really means that the row
> > keys starting with "0" will go in that region. Here is my code that
> creates
> > a row key and splits using admin util. I also am including the output of
> > hbase shell scan after the code.
> >
> > public static byte[][] splitRegionsSessionTimeline(int start, int end) {
> >  byte[][] splitKeys = new byte[end][];
> >  // the first region starting with empty key will be created
> >  // automatically
> >  for (int i = 0; i < splitKeys.length; i++) {
> >   splitKeys[i] = Bytes.toBytes(String.valueOf(i));
> >  }
> >  return splitKeys;
> > }
> >  public static byte [] getRowKey(MetricType metricName, Long timestamp,
> > Short bucketNo, char rowDelim){
> >   byte [] result = null;
> >   int rowSize = getRowSize();
> >   ByteBuffer b = ByteBuffer.allocate(rowSize);
> >   //Bucket No 0-9 randomely
> >   b.putShort(bucketNo);
> >   //Row Delimiter
> >   b.putChar(rowDelim);
> >   b.putShort(metricName.getId());
> >   long reverseOrderEpoch = getReverseBaseTimeStamp(metricName,timestamp);
> >   b.putLong(reverseOrderEpoch);
> >   result = b.array();
> >   return result;
> > }
> >
> > from hbase shell scan table:
> >
> >   \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9
> > column=S_T_MTX:\x00\x00gK,
> > timestamp=1343350528865, value=1343350646443
> >  F
> >  \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9
> column=S_T_MTX:\x00\x00gL,
> > timestamp=1343350528866, value=1343350646444
> >  F
> >  \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9
> column=S_T_MTX:\x00\x00gU,
> > timestamp=1343350528874, value=1343350646453
> >  F
> >  \x00\x00\x00:\x00\x01\x7F\xFF\xFE\xC7:\x10@\x9
> column=S_T_MTX:\x00\x00gZ,
> > timestamp=1343350528880, value=1343350646458
> >  F
> >
> > > Alex Baranau
> > > ------
> > > Sematext :: http://blog.sematext.com/ :: Hadoop - HBase -
> ElasticSearch
> > -
> > > Solr
> > >
> > > On Fri, Jul 27, 2012 at 7:24 PM, Mohit Anchlia <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > On Fri, Jul 27, 2012 at 11:48 AM, Alex Baranau <
> > [EMAIL PROTECTED]
> > > > >wrote:
> > > >
> > > > > You can read metrics [0] from JMX directly [1] or use Ganglia [2]
> or
> > > > other
> > > > > third-party tools like [3] (I'm a little biased here;)).
> > > > >
> > > > > [0] http://hbase.apache.org/book.html#hbase_metrics
> > > > > [1] http://hbase.apache.org/metrics.html
> > > > > [2] http://wiki.apache.org/hadoop/GangliaMetrics
> > > > > [3]
+
Alex Baranau 2012-07-30, 18:58
+
Mohit Anchlia 2012-07-30, 19:37