Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Mapreduce outputs to a different cluster?


Copy link to this message
-
Re: Mapreduce outputs to a different cluster?
You can specify the HDFS path as follows:
FileOutputFormat.setOutputPath(conf, new Path(args[1]));
where Path object is of course the location of your output dir.

See this for details
http://www.rohitmenon.com/index.php/introducing-mapreduce-part-i/
Regards,
Shahab
On Thu, Oct 24, 2013 at 11:25 PM, S. Zhou <[EMAIL PROTECTED]> wrote:

> Thanks Shahab & Yong. If cluster B (in which I want to dump output) has
> url "hdfs://machine.domain:8080" and data folder "/tmp/myfolder", what
> should I specify as the output path for MR job?
> Thanks
>
>
>   On Thursday, October 24, 2013 5:31 PM, java8964 java8964 <
> [EMAIL PROTECTED]> wrote:
>  Just specify the output location using the URI to another cluster. As
> long as the network is accessible, you should be fine.
>
> Yong
>
> ------------------------------
> Date: Thu, 24 Oct 2013 15:28:27 -0700
> From: [EMAIL PROTECTED]
> Subject: Mapreduce outputs to a different cluster?
> To: [EMAIL PROTECTED]
>
> The scenario is: I run mapreduce job on cluster A (all source data is in
> cluster A) but I want the output of the job to cluster B. Is it possible?
> If yes, please let me know how to do it.
>
> Here are some notes of my mapreduce job:
> 1. the data source is an HBase table
> 2. It only has mapper no reducer.
>
> Thanks
> Senqiang
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB