Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Re: HBase - Secondary Index


+
anil gupta 2012-12-14, 08:41
+
Anoop Sam John 2012-12-14, 08:54
+
ramkrishna vasudevan 2012-12-14, 11:34
+
anil gupta 2012-12-14, 18:01
+
Anoop Sam John 2012-12-17, 04:02
+
anil gupta 2012-12-18, 08:28
+
Anoop Sam John 2012-12-18, 09:27
+
anil gupta 2012-12-19, 08:24
+
Michel Segel 2012-12-18, 09:02
+
Anoop Sam John 2012-12-18, 09:35
+
anil gupta 2012-12-19, 08:39
+
Shengjie Min 2012-12-27, 11:23
+
Anoop Sam John 2012-12-27, 11:30
+
Shengjie Min 2012-12-27, 13:07
+
Anoop John 2012-12-27, 15:54
+
ramkrishna vasudevan 2012-12-27, 16:11
+
Shengjie Min 2012-12-27, 16:29
+
Anoop Sam John 2012-12-28, 03:33
+
Mohit Anchlia 2012-12-28, 03:42
+
Anoop Sam John 2012-12-28, 04:14
+
Shengjie Min 2012-12-28, 10:55
+
Adrien Mogenet 2013-01-06, 20:30
+
Anoop Sam John 2013-01-07, 03:48
+
Mohit Anchlia 2013-01-07, 04:17
+
Anoop Sam John 2013-01-07, 13:49
+
Michael Segel 2013-01-08, 14:33
+
lars hofhansl 2013-01-09, 00:30
+
Michel Segel 2013-01-09, 01:30
+
anil gupta 2013-01-09, 01:28
+
Anoop Sam John 2013-01-09, 03:22
+
ramkrishna vasudevan 2013-01-09, 04:11
+
Mohit Anchlia 2013-01-09, 01:50
+
Asaf Mesika 2013-01-08, 23:00
Copy link to this message
-
Re: HBase - Secondary Index
Does anyone has any links or information to the new prefix encoding feature
in HBase that's being referred to in this mail?

On Sun, Jan 6, 2013 at 12:30 PM, Adrien Mogenet <[EMAIL PROTECTED]>wrote:

> Nice topic, perhaps one of the most important for 2013 :-)
> I still don't get how you're ensuring consistency between index table and
> main table, without an external component (such as bookkeeper/zookeeper).
> What's the exact write path in your situation when inserting data ?
> (WAL/RegionObserver, pre/post put/WALedit...)
>
> The underlying question is about how you're ensuring that WALEdit in Index
> and Main tables are perfectly sync'ed, and how you 're able to rollback in
> case of issue in both WAL ?
>
>
> On Fri, Dec 28, 2012 at 11:55 AM, Shengjie Min <[EMAIL PROTECTED]>
> wrote:
>
> > >Yes as you say when the no of rows to be returned is becoming more and
> > more the latency will be becoming more.  seeks within an HFile block is
> > some what expensive op now. (Not much but still)  The new encoding
> >prefix
> > trie will be a huge bonus here. There the seeks will be flying.. [Ted
> also
> > presented this in the Hadoop China]  Thanks to Matt... :)  I am trying to
> > measure the scan performance with this new encoding . Trying to >back
> port
> > a simple patch for 94 version just for testing...   Yes when the no of
> > results to be returned is more and more any index will become less
> > performing as per my study  :)
> >
> > yes, you are right, I guess it's just a drawback of any index approach.
> > Thanks for the explanation.
> >
> > Shengjie
> >
> > On 28 December 2012 04:14, Anoop Sam John <[EMAIL PROTECTED]> wrote:
> >
> > > > Do you have link to that presentation?
> > >
> > > http://hbtc2012.hadooper.cn/subject/track4TedYu4.pdf
> > >
> > > -Anoop-
> > >
> > > ________________________________________
> > > From: Mohit Anchlia [[EMAIL PROTECTED]]
> > > Sent: Friday, December 28, 2012 9:12 AM
> > > To: [EMAIL PROTECTED]
> > > Subject: Re: HBase - Secondary Index
> > >
> > > On Thu, Dec 27, 2012 at 7:33 PM, Anoop Sam John <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > Yes as you say when the no of rows to be returned is becoming more
> and
> > > > more the latency will be becoming more.  seeks within an HFile block
> is
> > > > some what expensive op now. (Not much but still)  The new encoding
> > prefix
> > > > trie will be a huge bonus here. There the seeks will be flying.. [Ted
> > > also
> > > > presented this in the Hadoop China]  Thanks to Matt... :)  I am
> trying
> > to
> > > > measure the scan performance with this new encoding . Trying to back
> > > port a
> > > > simple patch for 94 version just for testing...   Yes when the no of
> > > > results to be returned is more and more any index will become less
> > > > performing as per my study  :)
> > > >
> > > > Do you have link to that presentation?
> > >
> > >
> > > > >btw, quick question- in your presentation, the scale there is
> seconds
> > or
> > > > mill-seconds:)
> > > >
> > > > It is seconds.  Dont consider the exact values. What is the % of
> > increase
> > > > in latency is important :) Those were not high end machines.
> > > >
> > > > -Anoop-
> > > > ________________________________________
> > > > From: Shengjie Min [[EMAIL PROTECTED]]
> > > > Sent: Thursday, December 27, 2012 9:59 PM
> > > > To: [EMAIL PROTECTED]
> > > > Subject: Re: HBase - Secondary Index
> > > >
> > > >  >Didnt follow u completely here. There wont be any get() happening..
> > As
> > > > the
> > > > >exact rowkey in a region we get from the index table, we can seek to
> > the
> > > > >exact position and return that row.
> > > >
> > > > Sorry, When I misused "get()" here, I meant seeking. Yes, if it's
> just
> > > > small number of rows returned, this works perfect. As you said you
> will
> > > get
> > > > the exact rowkey positions per region, and simply seek them. I was
> > trying
> > > > to work out the case that when the number of result rows increases
+
Adrien Mogenet 2013-01-06, 20:40
+
anil gupta 2013-01-06, 22:12
+
Anoop Sam John 2012-12-20, 03:33
+
Farah Karim 2012-12-25, 10:14
+
David Arthur 2012-12-20, 02:47
+
Anoop Sam John 2012-12-20, 03:44
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB