Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Coprocessor Increments


Copy link to this message
-
Re: Coprocessor Increments
anil gupta 2013-10-15, 03:57
Hi Ted,

Sure, I would like to revive it. My bad that i didnt wrap up the patch. I
am also in the middle of making this coprocessor handle "nulls first" and
"nulls last" clause.  I am targeting to do that in a month or so. Thanks
for reminding me.

~Anil

On Mon, Oct 14, 2013 at 3:34 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Anil:
> bq. We also use CP's wherever they are appropriate(like HBASE-7474).
>
> HBASE-7474 has been dormant for several months. Do you want to revive it ?
>
> Cheers
>
>
> On Mon, Oct 14, 2013 at 3:25 PM, anil gupta <[EMAIL PROTECTED]> wrote:
>
> > Inline.
> >
> >
> > On Mon, Oct 14, 2013 at 7:50 AM, Michael Segel <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > Anil,
> > >
> > > I wasn't suggesting that you can't do what you're doing, but you end up
> > > running in to the risks which coprocessors are supposed to remove. The
> > > standard YMMV always applies.
> > >
> > Agree with you. But, as per my knowledge and experience with
> coprocessors,
> > they are meant to be used for operations that are local to RS. Otherwise,
> > you are in danger of running into deadlocks, scalability issues.
> >
> > >
> > > You have a cluster… another team in your company wants to use the
> > cluster.
> > > So instead of the cluster being a single resource for your app/team, it
> > now
> > > becomes a shared resource. So now you have people accessing HBase for
> > > multiple apps.
> > >
> > Well, its a separation of responsibility in this case. We don't want
> teams
> > to step each other toes and at the same time work well as an ecosystem.
> > Rule: Other teams can use same cluster. But they cannot write directly
> into
> > the tables that we own/control.  If they want to write into our tables
> then
> > they have to use our HBase Client.
> >
> > >
> > > You could then run multiple HBase HMasters with different locations for
> > > files, however… this can get messy.
> > > HOYA seems to suggest this as the future.  If so, then you have to
> wonder
> > > about data locality.
> > >
> > HOYA is not even in beta at present. So, right now we are not thinking
> > about it.
> >
> > >
> > > Having your app update the primary table and then the secondary index
> is
> > > always a good fallback, however you need to ensure that you understand
> > the
> > > risks.
> > >
> > Agree, i understand that there is risk. But, you have to bite the bullet
> > when you are doing something that is not supported out of the box.  We
> also
> > use CP's wherever they are appropriate(like HBASE-7474).
> >
> > >
> > > With respect to secondary indexes… if you decouple the writes… you can
> > get
> > > better throughput. Note that the code becomes a bit more complex
> because
> > > you're going to have to introduce a couple of different things.  But
> > thats
> > > something for a different discussion…
> > >
> > Whether to use CP or not, depends on the use case. In my opinion, CP's
> are
> > really powerful and an awesome feature in HBase. But, sometimes if not
> used
> > properly(like creating a Cyclic Graph as per Tom's example), they might
> be
> > problematic.
> >
> >
> > >
> > > On Oct 13, 2013, at 10:15 AM, anil gupta <[EMAIL PROTECTED]>
> wrote:
> > >
> > > > Inline.
> > > >
> > > > On Sun, Oct 13, 2013 at 6:02 AM, Michael Segel <
> > > [EMAIL PROTECTED]>wrote:
> > > >
> > > >> Ok…
> > > >>
> > > >> Sure you can have your app update the secondary index table.
> > > >> The only issue with that is if someone updates the base table
> outside
> > of
> > > >> your app,
> > > >> they may or may not increment the secondary index.
> > > >>
> > > > Anil: We dont allow people to write data into HBase from their own
> > HBase
> > > > client. We control the writes into HBase. So, we dont have the
> problem
> > of
> > > > secondary index not getting written.
> > > > For example, If you expose a restful web service you can easily
> control
> > > the
> > > > writes to HBase. Even, if user requests to write one row in "main
> > table",
> > > > you application can have the logic to writing in "Secondary index"

Thanks & Regards,
Anil Gupta