-Re: problems with protoc versions / anyone tried under hadoop 2.0?
Lisen Mu 2013-08-04, 15:26
I think I might have put together a way to make things work as you said,
with maven-shade-plugin. Will verify tomorrow.
The idea is to rename classes in protobuf-java and references to pb-java in
'common' into renamed package '250.com.google.protobuf.*' and to package
together into an alternative jar: common-1.0-SNAPSHOT-rebuffed.jar
The compilation of java-exec would depend on normal common-1.0-SNAPSHOT.jar
and normal protobuf-java without a problem. java-exec also packages an
alternative jar: java-exec-1.0-SNAPSHOT-rebuffed.jar, with all references
to pb-java renamed, but without renamed pb-java itself.
In this way nothing changed under normal workflow; If one wants to run with
hadoop 2.0, he/she can have dependency on rebuffed jars with
the patch is at: https://issues.apache.org/jira/browse/DRILL-161
Is there any problem? Or any better way to do this?
On Fri, Aug 2, 2013 at 3:00 AM, Jacques Nadeau <[EMAIL PROTECTED]> wrote:
> It's painful that 2.5 and 2.4.1 don't coeexist better together. It
> seems like they should simply separate the classes that are bw/fw
> compatible and those that aren't. C'est la vie.
> My current thinking is that we generate an alternate 2.5 jar that
> modifies all the package names and then use a mvn filter on the
> generated protoc code as part of the compile part of the build. I
> much prefer doing it compile time rather than trying to fight through
> classloader issues. The last piece is modifying the protoc builds to
> be allowed to be pointed at an alternative path for the protoc
> generator so people can have both on their system (if people want to
> build a combination of Drill and Hadoop or HBase).
> But I haven't gotten to it yet. We should get a jira in for it and
> see if a maven maestro can help us out.
> On Wed, Jul 31, 2013 at 12:17 AM, Lisen Mu <[EMAIL PROTECTED]> wrote:
> > Hi drillers,
> > Has anyone tried scan under hadoop 2.0?
> > We've built a HBase scanner, with hadoop 2.0.x.
> > The problem is, drill depends on protoc 2.5.0, while hadoop 2.0.x
> > depends on 2.4.x.
> > Is there any suggestion to keep both drill and hadoop 2.0 working
> > Keeping a branch of drill with protoc downgraded to 2.4 is just painful.
> > would be great if the drill master could downgrade to 2.4, however, the
> > most promising solution is likely to be some class loading techniques to
> > keep both happy. Additional requirement is to keep 'mvn test' happy, too.
> > Any suggestion on this? What's the easiest way?
> > Thanks,
> > Lisen