Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # user - Accumulo Junit Concurrency/Latency issues ( Accumulo 1.3 )


Copy link to this message
-
Re: Accumulo Junit Concurrency/Latency issues ( Accumulo 1.3 )
Adam Fuchs 2012-11-29, 18:45
Sounds like you might need to introduce some synchronization to serialize
your JUnit tests. If you can recreate these symptoms in a small test case
that is representative of your code, maybe you can share that?

Adam

On Thu, Nov 29, 2012 at 11:10 AM, Josh Berk <[EMAIL PROTECTED]> wrote:

> Sorry Adam, I can't give my source code :/.
>
> and Eric, I'm positive that the timestamp is not the issue. When I said I
> use the same key, i meant only the key and not the entire same Entry.  The
> "timestamp" and "visibility" are associated with the entry and not the key.
> so, the timestamp is auto-updated. Because, if it weren't auto-updated then
> the problem i'm experiencing would occur for the same method outside of
> JUnit.
>
> I think the problem I'm experiencing has something to do with JUnit
> running it's test methods concurrently..  Each of my JUnit methods are
> independent from each other. So, no other method could influence the values
> of the objects that i'm storing/accessing with Accumulo in any particular
> method.
>
> -All JUnit methods are independent of each other.
> -The problem I'm experiencing is that when i update a value & retrieve it,
> I can intermittently receive "stale" data, which causes my test to fail.
> -The same method ran in a normal class (non-JUnit) succeeds every single
> time.
>
> I know that JUnit concurrently runs its test methods . I'm wondering if
> more than one thread maybe executes for a single test method and some parts
> are getting ahead of others in the same method. I won't say more about that
> theory in fear of being labeled a crazy person. I only know that the
> problem is JUnit-Accumulo specific & am wondering if anyone else has
> experienced the same issues?
>
> -Josh
>
>
>
>
>
>
> On Thu, Nov 29, 2012 at 10:51 AM, Eric Newton <[EMAIL PROTECTED]>wrote:
>
>> "I am definitely using the same key to update and retrieve the data."
>>
>> At least update the timestamp to the current time (or old timestamp + 1).
>>
>> -Eric
>>
>>
>> On Thu, Nov 29, 2012 at 10:38 AM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>>
>>> Josh,
>>>
>>> Can you share your junit test code so I can replicate this behavior?
>>>
>>> Adam
>>>
>>>
>>>
>>> On Thu, Nov 29, 2012 at 9:59 AM, Joe Berk <[EMAIL PROTECTED]>wrote:
>>>
>>>> Good morning all,
>>>>
>>>> I'm experiencing some "weirdness" when executing JUnit tests for my
>>>> classes that operate with Accumulo. I can best describe it as latency.
>>>> Basically, when I write my object to Accumulo & then immediately
>>>> retrieve it to inspect the values, the values are not always updated to
>>>> what I just saved them as.
>>>>
>>>> Problem:
>>>>   part 1:
>>>>     - I create an object that has some primitive types.
>>>>     - I set the primitive variables to acceptable values.
>>>>     - I serialize the object (the "Value")
>>>>     - I write the Value to Accumulo ( Entry )
>>>>     - I retrieve the Object from Accumulo & inspect. The primitive
>>>> values are equal to what they were set to.
>>>>
>>>>   part 2:
>>>>     - I retrieve the object from Accumulo
>>>>     - I set the primitive variables to different values
>>>>     - I serialize the object
>>>>     - I write the Value to Accumlo ( Entry )
>>>>     - I retrieve the Object from Accumulo & inspect. The primitive
>>>> values are *not equal* to what they were just set to
>>>>
>>>> This only seems to be happening during the JUnit.
>>>>
>>>> I have a method that performs the above task, in a JUnit test, and when
>>>> I repeatedly run the JUnit test, it will intermittently fail.
>>>> I have the same exact method, but it is in a regular class, and I can
>>>> run it as much as I want, with no failure.
>>>>
>>>> for the non-JUnit test, MockInstances and  "real" instances succeed
>>>> every time
>>>> for the JUnit test, MockInstances and "real" instances both fail
>>>> intermittently.
>>>>
>>>> sidenotes:
>>>> - I am definitely using the same key to update and retrieve the data. I
>>>> also inspected the entries that I was writing to Accumulo, every time, and