Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> setNumReduceTasks(1)


Copy link to this message
-
Re: setNumReduceTasks(1)
*See my comments below*

On Mon, Jan 25, 2010 at 3:22 PM, Something Something <
[EMAIL PROTECTED]> wrote:

> If I set # of reduce tasks to 1 using setNumReduceTasks(1), would the class
> be instantiated only on one machine.. always?  I mean if I have a cluster
> of
> say 1 master, 10 workers & 3 zookeepers, is the Reducer class guaranteed to
> be instantiated only on 1 machine?
>
> *--Yes*
> If answer is yes, then I will use static variable as a counter to see how
> may rows have been added to my HBase table so far.  In my use case, I want
> to write only N number of rows to a table.  Is there a better way to do
> this?  Please let me know.  Thanks.
>

*--Maybe you can use Counter to track the number of rows you add to HBase,
then you do not need to limit the reduce task as 1*
--
Best Regards

Jeff Zhang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB