Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> TableMapReduceUtil addDependencyJars question


Copy link to this message
-
Re: TableMapReduceUtil addDependencyJars question
So the zookeeper and protobuf classes are sent to cluster to determine
version compatibility ?
And about the other jars, the client is sending them anyway in the job jar,
right ? So isn't that a duplication ?

Thanks.
On Aug 21, 2013 8:02 PM, "Ted Yu" <[EMAIL PROTECTED]> wrote:

> bq. you are supposed to have ZooKeeper deployed in your cluster, right?
>
> We need to consider whether the versions (of ZooKeeper) in the two clusters
> match or not.
>
> Cheers
>
> On Wed, Aug 21, 2013 at 2:00 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> > I'm using HBase 0.94.2.
> > Looking at TableMapReduceUtil.addDependencyJars(Job job) I see that
> > org.apache.zookeeper.ZooKeeper.class and
> com.google.protobuf.Message.class
> > are hard-coded as jars to send to cluster.
> > I can understand why all the others (map output key/value class, etc.)
> are
> > sent to cluster but if you are using HBase you are supposed to have
> > ZooKeeper deployed in your cluster, right? and protobuf is a part of that
> > installation, isn't it ?
> > Are these two jars necessary ?
> >
> > In general, If I have all the classes I need deployed in the cluster
> > (including map output key/value class, etc.), can I skip
> > "addDependencyJars" ? I'm using an extended HFileOutputFormat I wrote...
> >
> > Thanks,
> >
> > Amit.
> >
>