-Re: TableMapReduceUtil addDependencyJars question
Amit Sela 2013-08-22, 10:20
So the zookeeper and protobuf classes are sent to cluster to determine
version compatibility ?
And about the other jars, the client is sending them anyway in the job jar,
right ? So isn't that a duplication ?
On Aug 21, 2013 8:02 PM, "Ted Yu" <[EMAIL PROTECTED]> wrote:
> bq. you are supposed to have ZooKeeper deployed in your cluster, right?
> We need to consider whether the versions (of ZooKeeper) in the two clusters
> match or not.
> On Wed, Aug 21, 2013 at 2:00 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
> > Hi all,
> > I'm using HBase 0.94.2.
> > Looking at TableMapReduceUtil.addDependencyJars(Job job) I see that
> > org.apache.zookeeper.ZooKeeper.class and
> > are hard-coded as jars to send to cluster.
> > I can understand why all the others (map output key/value class, etc.)
> > sent to cluster but if you are using HBase you are supposed to have
> > ZooKeeper deployed in your cluster, right? and protobuf is a part of that
> > installation, isn't it ?
> > Are these two jars necessary ?
> > In general, If I have all the classes I need deployed in the cluster
> > (including map output key/value class, etc.), can I skip
> > "addDependencyJars" ? I'm using an extended HFileOutputFormat I wrote...
> > Thanks,
> > Amit.