Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo, mail # user - Accumulo Junit Concurrency/Latency issues ( Accumulo 1.3 )


+
Joe Berk 2012-11-29, 14:59
+
Adam Fuchs 2012-11-29, 15:38
+
Eric Newton 2012-11-29, 15:51
Copy link to this message
-
Re: Accumulo Junit Concurrency/Latency issues ( Accumulo 1.3 )
Josh Berk 2012-11-29, 16:10
Sorry Adam, I can't give my source code :/.

and Eric, I'm positive that the timestamp is not the issue. When I said I
use the same key, i meant only the key and not the entire same Entry.  The
"timestamp" and "visibility" are associated with the entry and not the key.
so, the timestamp is auto-updated. Because, if it weren't auto-updated then
the problem i'm experiencing would occur for the same method outside of
JUnit.

I think the problem I'm experiencing has something to do with JUnit running
it's test methods concurrently..  Each of my JUnit methods are independent
from each other. So, no other method could influence the values of the
objects that i'm storing/accessing with Accumulo in any particular method.

-All JUnit methods are independent of each other.
-The problem I'm experiencing is that when i update a value & retrieve it,
I can intermittently receive "stale" data, which causes my test to fail.
-The same method ran in a normal class (non-JUnit) succeeds every single
time.

I know that JUnit concurrently runs its test methods . I'm wondering if
more than one thread maybe executes for a single test method and some parts
are getting ahead of others in the same method. I won't say more about that
theory in fear of being labeled a crazy person. I only know that the
problem is JUnit-Accumulo specific & am wondering if anyone else has
experienced the same issues?

-Josh
On Thu, Nov 29, 2012 at 10:51 AM, Eric Newton <[EMAIL PROTECTED]> wrote:

> "I am definitely using the same key to update and retrieve the data."
>
> At least update the timestamp to the current time (or old timestamp + 1).
>
> -Eric
>
>
> On Thu, Nov 29, 2012 at 10:38 AM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>
>> Josh,
>>
>> Can you share your junit test code so I can replicate this behavior?
>>
>> Adam
>>
>>
>>
>> On Thu, Nov 29, 2012 at 9:59 AM, Joe Berk <[EMAIL PROTECTED]>wrote:
>>
>>> Good morning all,
>>>
>>> I'm experiencing some "weirdness" when executing JUnit tests for my
>>> classes that operate with Accumulo. I can best describe it as latency.
>>> Basically, when I write my object to Accumulo & then immediately
>>> retrieve it to inspect the values, the values are not always updated to
>>> what I just saved them as.
>>>
>>> Problem:
>>>   part 1:
>>>     - I create an object that has some primitive types.
>>>     - I set the primitive variables to acceptable values.
>>>     - I serialize the object (the "Value")
>>>     - I write the Value to Accumulo ( Entry )
>>>     - I retrieve the Object from Accumulo & inspect. The primitive
>>> values are equal to what they were set to.
>>>
>>>   part 2:
>>>     - I retrieve the object from Accumulo
>>>     - I set the primitive variables to different values
>>>     - I serialize the object
>>>     - I write the Value to Accumlo ( Entry )
>>>     - I retrieve the Object from Accumulo & inspect. The primitive
>>> values are *not equal* to what they were just set to
>>>
>>> This only seems to be happening during the JUnit.
>>>
>>> I have a method that performs the above task, in a JUnit test, and when
>>> I repeatedly run the JUnit test, it will intermittently fail.
>>> I have the same exact method, but it is in a regular class, and I can
>>> run it as much as I want, with no failure.
>>>
>>> for the non-JUnit test, MockInstances and  "real" instances succeed
>>> every time
>>> for the JUnit test, MockInstances and "real" instances both fail
>>> intermittently.
>>>
>>> sidenotes:
>>> - I am definitely using the same key to update and retrieve the data. I
>>> also inspected the entries that I was writing to Accumulo, every time, and
>>> can confirm that they are being "sent"/"written" to Accumulo as I intend
>>> them to be. In summary, I am positive that I am sending the correct data to
>>> be written. This is doubly verified by my ability to intermittently succeed
>>> when JUnit and 100% succeed in a "normal" class.
>>>
>>> Any assistance would be greatly appreciated.
>>>
>>> Best Regards,
+
Josh Berk 2012-11-29, 16:15
+
Adam Fuchs 2012-11-29, 18:45