Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> TableMapReduceUtil addDependencyJars question


Copy link to this message
-
Re: TableMapReduceUtil addDependencyJars question
So the zookeeper and protobuf classes are sent to cluster to determine
version compatibility ?
And about the other jars, the client is sending them anyway in the job jar,
right ? So isn't that a duplication ?

Thanks.
On Aug 21, 2013 8:02 PM, "Ted Yu" <[EMAIL PROTECTED]> wrote:

> bq. you are supposed to have ZooKeeper deployed in your cluster, right?
>
> We need to consider whether the versions (of ZooKeeper) in the two clusters
> match or not.
>
> Cheers
>
> On Wed, Aug 21, 2013 at 2:00 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> > I'm using HBase 0.94.2.
> > Looking at TableMapReduceUtil.addDependencyJars(Job job) I see that
> > org.apache.zookeeper.ZooKeeper.class and
> com.google.protobuf.Message.class
> > are hard-coded as jars to send to cluster.
> > I can understand why all the others (map output key/value class, etc.)
> are
> > sent to cluster but if you are using HBase you are supposed to have
> > ZooKeeper deployed in your cluster, right? and protobuf is a part of that
> > installation, isn't it ?
> > Are these two jars necessary ?
> >
> > In general, If I have all the classes I need deployed in the cluster
> > (including map output key/value class, etc.), can I skip
> > "addDependencyJars" ? I'm using an extended HFileOutputFormat I wrote...
> >
> > Thanks,
> >
> > Amit.
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB