Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HBase M/R with M/R and HBase not on same cluster


+
David Koch 2013-03-18, 22:09
Copy link to this message
-
Re: HBase M/R with M/R and HBase not on same cluster
Checkout how CopyTable does it:
https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/CopyTable.java

J-D

On Mon, Mar 18, 2013 at 3:09 PM, David Koch <[EMAIL PROTECTED]> wrote:
> Hello,
>
> Is it possible to run a M/R on cluster A over a table that resides on
> cluster B with output to a table on cluster A? If so, how?
>
> I am interested in doing this for the purpose of copying part of a table
> from B to A. Cluster B is a production environment, cluster A is a slow
> test platform. I do not want the M/R to run on B since it would block
> precious slots on this cluster. Otherwise I could just run CopyTable on
> cluster B and specify cluster A as output quorum.
>
> Could this work by pointing the client configuration at the mapred-site.xml
> of cluster A and the hdfs-site.xml and hbase-site.xml of cluster B? In this
> scenario - in order to output to cluster A I guess I'd have to set
> TableOutputFormat.QUORUM_ADDRESS to cluster A.
>
> I use a client configuration generated by CDH4 and there are some other
> files floating around - such as core-site.xml, not sure what to do with
> that.
>
> Thank you,
>
> /David
+
David Koch 2013-03-24, 12:49
+
Michael Segel 2013-03-25, 16:12
+
David Koch 2013-03-26, 14:49