Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> Accumulo Junit Concurrency/Latency issues ( Accumulo 1.3 )


+
Joe Berk 2012-11-29, 14:59
+
Adam Fuchs 2012-11-29, 15:38
+
Eric Newton 2012-11-29, 15:51
+
Josh Berk 2012-11-29, 16:10
Copy link to this message
-
Re: Accumulo Junit Concurrency/Latency issues ( Accumulo 1.3 )
I misspoke about the the timestamp not being associated with the key, but
am still positive that the timestamp is not the issue. Because I re-create
the same key. When i said i used the same key, I meant i used the same
(row,family,qualifer) to build the key. so, the timestamp is taken care of
away from me.

Also again, my biggest clue is that the same method and conditions does not
fail when repeatedly ran from a regular class, but intermittently fails in
a JUnit test.

-Josh

On Thu, Nov 29, 2012 at 11:10 AM, Josh Berk <[EMAIL PROTECTED]> wrote:

> Sorry Adam, I can't give my source code :/.
>
> and Eric, I'm positive that the timestamp is not the issue. When I said I
> use the same key, i meant only the key and not the entire same Entry.  The
> "timestamp" and "visibility" are associated with the entry and not the key.
> so, the timestamp is auto-updated. Because, if it weren't auto-updated then
> the problem i'm experiencing would occur for the same method outside of
> JUnit.
>
> I think the problem I'm experiencing has something to do with JUnit
> running it's test methods concurrently..  Each of my JUnit methods are
> independent from each other. So, no other method could influence the values
> of the objects that i'm storing/accessing with Accumulo in any particular
> method.
>
> -All JUnit methods are independent of each other.
> -The problem I'm experiencing is that when i update a value & retrieve it,
> I can intermittently receive "stale" data, which causes my test to fail.
> -The same method ran in a normal class (non-JUnit) succeeds every single
> time.
>
> I know that JUnit concurrently runs its test methods . I'm wondering if
> more than one thread maybe executes for a single test method and some parts
> are getting ahead of others in the same method. I won't say more about that
> theory in fear of being labeled a crazy person. I only know that the
> problem is JUnit-Accumulo specific & am wondering if anyone else has
> experienced the same issues?
>
> -Josh
>
>
>
>
>
>
> On Thu, Nov 29, 2012 at 10:51 AM, Eric Newton <[EMAIL PROTECTED]>wrote:
>
>> "I am definitely using the same key to update and retrieve the data."
>>
>> At least update the timestamp to the current time (or old timestamp + 1).
>>
>> -Eric
>>
>>
>> On Thu, Nov 29, 2012 at 10:38 AM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>>
>>> Josh,
>>>
>>> Can you share your junit test code so I can replicate this behavior?
>>>
>>> Adam
>>>
>>>
>>>
>>> On Thu, Nov 29, 2012 at 9:59 AM, Joe Berk <[EMAIL PROTECTED]>wrote:
>>>
>>>> Good morning all,
>>>>
>>>> I'm experiencing some "weirdness" when executing JUnit tests for my
>>>> classes that operate with Accumulo. I can best describe it as latency.
>>>> Basically, when I write my object to Accumulo & then immediately
>>>> retrieve it to inspect the values, the values are not always updated to
>>>> what I just saved them as.
>>>>
>>>> Problem:
>>>>   part 1:
>>>>     - I create an object that has some primitive types.
>>>>     - I set the primitive variables to acceptable values.
>>>>     - I serialize the object (the "Value")
>>>>     - I write the Value to Accumulo ( Entry )
>>>>     - I retrieve the Object from Accumulo & inspect. The primitive
>>>> values are equal to what they were set to.
>>>>
>>>>   part 2:
>>>>     - I retrieve the object from Accumulo
>>>>     - I set the primitive variables to different values
>>>>     - I serialize the object
>>>>     - I write the Value to Accumlo ( Entry )
>>>>     - I retrieve the Object from Accumulo & inspect. The primitive
>>>> values are *not equal* to what they were just set to
>>>>
>>>> This only seems to be happening during the JUnit.
>>>>
>>>> I have a method that performs the above task, in a JUnit test, and when
>>>> I repeatedly run the JUnit test, it will intermittently fail.
>>>> I have the same exact method, but it is in a regular class, and I can
>>>> run it as much as I want, with no failure.
>>>>
>>>> for the non-JUnit test, MockInstances and  "real" instances succeed
+
Adam Fuchs 2012-11-29, 18:45