Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # dev - VFS class reloading?


Copy link to this message
-
Re: VFS class reloading?
John Vines 2013-04-17, 16:43
The issue is hadoop 1.0.1 and earlier do not have commons-io at all. So
there is no jar to pick up on the classpath. So when we made commons-io
provided, we lost the jar for the previous releases.
On Wed, Apr 17, 2013 at 12:20 PM, Josh Elser <[EMAIL PROTECTED]> wrote:

> Actually, I think I may renege for a couple of points:
>
> 1) This is really a minimal change on us to support 0.20.x-based Hadoop
> versions. The patch (or something similar) I made last night and then also
> figuring out the correct commons-io jar to add to that patch.
>
> 2) I re-read Dave's email and noted that he was saying CDH3 is still based
> off of 0.20.x, so that makes the affected community much larger than Apache
> Hadoop 0.20.x.
>
> John Vines, reading back through ACCUMULO-1244:
>
> For the record, this breaks compatibility for releases at least at and
> before hadoop-1.0.1.
>
> Do we just need to set the commons-io dependency back to 1.4 for
> <Hadoop-1.0.1? Is 1.5 now dependent on new functionality that is only in
> >commons-io-1.4? Is there a way that we can try to smooth the fracturing of
> Hadoop distributions without being overbearing on Accumulo?
>
>
> On 4/16/13 10:08 PM, Josh Elser wrote:
>
>> I think I'm ok with dropping 0.20.205.0 support, but, as Dave pointed
>> out, we need to update the README to reflect such (really needs to happen
>> before 1.5.0 drops).
>>
>> I'll drop a poll to see if anyone who's only subscribed to [EMAIL PROTECTED] any concerns on the matter.
>>
>> On 04/16/2013 09:59 PM, Billie Rinaldi wrote:
>>
>>> On Tue, Apr 16, 2013 at 5:39 PM, <[EMAIL PROTECTED]> wrote:
>>>
>>>  Updated my local 1.5 branch and tried to build with "mvn clean package
>>>> -P
>>>> assemble -Dhadoop.version=0.20.205.0". I'm running CDH3 Update 3
>>>> locally,
>>>> so that should work. A bunch of tests failed in core with:
>>>>
>>>> Caused by: java.lang.**ClassNotFoundException:
>>>> org.codehaus.jackson.map.**JsonMappingException
>>>> at java.net.URLClassLoader$1.run(**URLClassLoader.java:202)
>>>> at java.security.**AccessController.doPrivileged(**Native Method)
>>>> at java.net.URLClassLoader.**findClass(URLClassLoader.java:**190)
>>>> at java.lang.ClassLoader.**loadClass(ClassLoader.java:**306)
>>>> at sun.misc.Launcher$**AppClassLoader.loadClass(**Launcher.java:301)
>>>> at java.lang.ClassLoader.**loadClass(ClassLoader.java:**247)
>>>>
>>>>
>>>> Has anyone else seen this?
>>>>
>>>>  Yep, I broke that in ACCUMULO-730.  See the first comment where I ask
>>> if
>>> anyone still needs 0.20.205.0 and what we should do to fix it if they do.
>>> It's not too late to weigh in.
>>>
>>> Billie
>>>
>>>
>>>
>>>  -- Dave
>>>>
>>>> ----- Original Message -----
>>>> From: "Eric Newton" <[EMAIL PROTECTED]>
>>>> To: [EMAIL PROTECTED]
>>>> Sent: Tuesday, April 16, 2013 7:52:09 PM
>>>> Subject: Re: VFS class reloading?
>>>>
>>>> We have tests for dynamic loading of classes so I'm pretty sure it
>>>> works.
>>>>
>>>> John, can you repeat the failure?
>>>>
>>>> -Eric
>>>>
>>>>
>>>>
>>>> On Tue, Apr 16, 2013 at 6:50 PM, Dave Marion <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>>  Looking at the code, it should work. Keith and I had several
>>>>>
>>>> conversations
>>>>
>>>>> about what the new classloader should do. I believe that he wanted it
>>>>> to
>>>>> behave like the old one and what I see in the code supports that. If it
>>>>>
>>>> is
>>>>
>>>>> not working, then I would say create a ticket for it for now. I'll try
>>>>> to
>>>>> replicate it tonight if I have time.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Dave Marion [mailto:[EMAIL PROTECTED]]
>>>>> Sent: Tuesday, April 16, 2013 6:41 PM
>>>>> To: [EMAIL PROTECTED]
>>>>> Subject: RE: VFS class reloading?
>>>>>
>>>>> The implementation changed several times, so the pre-1.5 layout may not
>>>>> work. In 1.5, using the bootstrap script, it should put the accumulo
>>>>> jars
>>>>> into HDFS and dynamic loading from there should occur. I'll try and
>>>>> test
>>>>> tonight if I have time.