Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo, mail # dev - Hadoop 2 compatibility issues

Copy link to this message
Re: Hadoop 2 compatibility issues
Christopher 2013-05-15, 01:01
I'm very much partial to the "First" option, as it's far less effort
for approximately the same value (in my opinion, but in light of the
enthusiasm above for hadoop2, I could be very wrong on my assessment
of the value).

I'm going to upload a patch to ACCUMULO-1402 soon (tiny polishing
left), to demonstrate a way to push redundant jars, with an extra
classifier (though I still have to build twice, to avoid
maven-invoker-plugin complexity) for hadoop2-compatible binaries. If
you don't mind, I'll tag you with a request to review that patch, as
I'd like more details about the classifier issues you mention, in

Christopher L Tubbs II
On Tue, May 14, 2013 at 8:27 PM, Benson Margulies <[EMAIL PROTECTED]> wrote:
> Maven will malfunction in various entertaining ways if you try to
> change the GAV of the output of the build using a profile.
> Maven will malfunction in various entertaining ways if you use
> classifiers on real-live-JAR files that get used as
> real-live-dependencies, because it has no concept of a
> pom-per-classifier.
> Where does this leave you/us? (I'm not sure that I've earned an 'us'
> recently around here.)
> First, I note that 'Apache releases are source releases'. So, one
> resort of scoundrels here would be to support only one hadoop in the
> convenience binaries that get pushed to Maven Central, and let other
> hadoop users take the source release and build for themselves.
> Second, I am reduced to suggesting an elaboration of the build in
> which some tool edits poms and runs builds. The maven-invoker-plugin
> could be used to run that, but a plain old script in a plain old
> language might be less painful.
> I appreciate that this may not be an appealing contribution to where
> things are, but it might be the best of the evil choices.
> On Tue, May 14, 2013 at 7:50 PM, John Vines <[EMAIL PROTECTED]> wrote:
>> The compiled code is compiled code. There are no concerns of dependency
>> resolution. So I see no issues in using the profile to define the gav if
>> that is feasible.
>> Sent from my phone, please pardon the typos and brevity.
>> On May 14, 2013 7:47 PM, "Christopher" <[EMAIL PROTECTED]> wrote:
>>> Response to Benson inline, but additional note here:
>>> It should be noted that the situation will be made worse for the
>>> solution I was considering for ACCUMULO-1402, which would move the
>>> accumulo artifacts, classified by the hadoop2 variant, into the
>>> profiles... meaning they will no longer resolve transitively when they
>>> did before. Can go into details on that ticket, if needed.
>>> On Tue, May 14, 2013 at 7:41 PM, Benson Margulies <[EMAIL PROTECTED]>
>>> wrote:
>>> > On Tue, May 14, 2013 at 7:36 PM, Christopher <[EMAIL PROTECTED]>
>>> wrote:
>>> >> Benson-
>>> >>
>>> >> They produce different byte-code. That's why we're even considering
>>> >> this. ACCUMULO-1402 is the ticket under which our intent is to add
>>> >> classifiers, so that they can be distinguished.
>>> >
>>> > whoops, missed that.
>>> >
>>> > Then how do people succeed in just fixing up their dependencies and
>>> using it?
>>> The specific differences are things like changes from abstract class
>>> to an interface. Apparently an import of these do not produce
>>> compatible byte-code, even though the method signature looks the same.
>>> > In any case, speaking as a Maven-maven, classifiers are absolutely,
>>> > positively, a cure worse than the disease. If you want the details
>>> > just ask.
>>> Agreed. I just don't see a good alternative here.
>>> >>
>>> >> All-
>>> >>
>>> >> To Keith's point, I think perhaps all this concern is a non-issue...
>>> >> because as Keith points out, the dependencies in question are marked
>>> >> as "provided", and dependency resolution doesn't occur for provided
>>> >> dependencies anyway... so even if we leave off the profiles, we're in
>>> >> the same boat. Maybe not the boat we should be in... but certainly not