Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> increment-related bug Was: HBASE-7114 Increment does not extend Mutation but probably should


+
Ted Yu 2013-01-26, 16:56
+
Asaf Mesika 2013-01-26, 17:18
+
Ted Yu 2013-01-26, 17:50
+
Mesika, Asaf 2013-01-30, 17:25
+
Andrew Purtell 2013-01-30, 18:28
+
Mesika, Asaf 2013-01-30, 18:43
Copy link to this message
-
Re: increment-related bug Was: HBASE-7114 Increment does not extend Mutation but probably should
If increasing the timeout reduces the magnitude of the error, then this is
probably it.

The solution is IMHO to introduce a nonce (probably internally generated by
the client) on non-idempotent operations to convert them into idempotent
ones. I know this has been discussed before but am not sure about the
outcome of those discussions, if any. I've updated HBASE-3787 to raise the
priority and target 0.96. Should restart things.
On Wed, Jan 30, 2013 at 10:43 AM, Mesika, Asaf <[EMAIL PROTECTED]>wrote:

> So if this bug you mentioned (3787) is correct, there is no workaround.
> Once you reach 60 seconds timeout, you have no way of knowing if the
> server finished processing this Increment or not, so you'll know whether to
> send it or not.
>
> On Jan 30, 2013, at 8:28 PM, Andrew Purtell wrote:
>
> > This may be an old one: https://issues.apache.org/jira/browse/HBASE-3787
> >
> >
> >
> > On Wed, Jan 30, 2013 at 9:25 AM, Mesika, Asaf <[EMAIL PROTECTED]>
> wrote:
> >
> >> Hi,
> >>
> >> We ran the QA test again, this time with INFO message on at the client
> >> side (HTable).
> >> We saw many retry attempts which failed on RPC timeouts (we use the
> >> default of 60 seconds).
> >>
> >> I guess when this error occurs, the increment shouldn't really happen,
> >> right?
> >>
> >> This may explain the diff we see from the expected value.
> >>
> >> We will re-run this test again, this time with a large timeouts and see
> >> what happens.
> >>
> >>
> >>
> >> On Jan 26, 2013, at 6:56 PM, Ted Yu wrote:
> >>
> >>> Asaf:
> >>> Were you referring to HBASE-6291: Don't retry increments on an invalid
> >> cell
> >>> ?
> >>> That was fixed in 0.94.2
> >>>
> >>> Or maybe: HBASE-6195 Increment data will be lost when the memstore is
> >>> flushed
> >>> The above was fixed in 0.94.1
> >>>
> >>> Cheers
> >>>
> >>> On Sat, Jan 26, 2013 at 8:40 AM, Asaf Mesika <[EMAIL PROTECTED]>
> >> wrote:
> >>>
> >>>> The all counters is on the same row?
> >>>>
> >>>> By the way, did you guys handle the hbase bug that when an increment
> is
> >>>> sent to region server and fails it still does it but throws an
> >> exception to
> >>>> the client which causes it to do that increment again?
> >>>>
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>> On 26 בינו 2013, at 17:32, Amit Sela <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>> Well, I increment counters where the row key is a keyword and the
> >> qualifier
> >>>> is a country code, and in the post increment region observer I
> >> increment an
> >>>> "all countries" aggregative counter. These counters are divided to
> >> families
> >>>> such as daily, weekly, hourly etc.
> >>>> So I get the family map to know which aggregative counter should I
> >>>> increment, then I piggyback onto the Result the "all countries"
> current
> >>>> count.
> >>>> On Jan 26, 2013 2:39 AM, "Ted Yu" <[EMAIL PROTECTED]> wrote:
> >>>>
> >>>> Amit:
> >>>>
> >>>> Can you tell us what operation you perform on the returned family map
> ?
> >>>>
> >>>>
> >>>> Thanks
> >>>>
> >>>>
> >>>> On Thu, Jan 24, 2013 at 3:37 AM, Amit Sela <[EMAIL PROTECTED]>
> wrote:
> >>>>
> >>>>
> >>>> I'm using Increment.getFamilyMap in a postIncrement Observer.
> >>>>
> >>>> I'm running with HBase 0.94.2.
> >>>>
> >>>>
> >>>> Amit.
> >>>>
> >>>>
> >>>> On Thu, Jan 24, 2013 at 4:23 AM, lars hofhansl <[EMAIL PROTECTED]>
> >> wrote:
> >>>>
> >>>>
> >>>> The reason was that Increment was serialized differently (compared to
> >>>>
> >>>> all
> >>>>
> >>>> other mutations).
> >>>>
> >>>> In trunk that is no longer an issue, since the serialization logic is
> >>>>
> >>>> no
> >>>>
> >>>> longer part of the object to be serialized.
> >>>>
> >>>>
> >>>>
> >>>> -- Lars
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> ________________________________
> >>>>
> >>>> From: Ted Yu <[EMAIL PROTECTED]>
> >>>>
> >>>> To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
> >>>>
> >>>> Sent: Wednesday, January 23, 2013 10:25 AM
> >>>>
> >>>> Subject: HBASE-7114 Increment does not extend Mutation but probably
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)