Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase M/R with M/R and HBase not on same cluster


Copy link to this message
-
HBase M/R with M/R and HBase not on same cluster
Hello,

Is it possible to run a M/R on cluster A over a table that resides on
cluster B with output to a table on cluster A? If so, how?

I am interested in doing this for the purpose of copying part of a table
from B to A. Cluster B is a production environment, cluster A is a slow
test platform. I do not want the M/R to run on B since it would block
precious slots on this cluster. Otherwise I could just run CopyTable on
cluster B and specify cluster A as output quorum.

Could this work by pointing the client configuration at the mapred-site.xml
of cluster A and the hdfs-site.xml and hbase-site.xml of cluster B? In this
scenario - in order to output to cluster A I guess I'd have to set
TableOutputFormat.QUORUM_ADDRESS to cluster A.

I use a client configuration generated by CDH4 and there are some other
files floating around - such as core-site.xml, not sure what to do with
that.

Thank you,

/David
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB