Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> ReflectionUtils.setConf would configure anything Configurable twice?


Copy link to this message
-
Re: ReflectionUtils.setConf would configure anything Configurable twice?
Anyone have the same partition problem using Streaming in 0.21.0?

On Fri, Jun 24, 2011 at 10:22 AM, steven zhuang <[EMAIL PROTECTED]>wrote:

> hello, list,
>
>               Recently I have upgraded our Hadoop cluster from 0.20.2 to
> 0.21.0, and I found there is something wrong introduced by the upgrade.
>
>               In the setConf method of
> org.apache.hadoop.util.ReflectionUtils, any instance of Configurable would
> be configured twice, and this may cause trouble.
>               For example, in 0.21.0, KeyFieldBasedPartitioner implements
> the Configurable interface. When configured twice, it get two KeyDescription
> and gives out wrong partition number.
>
> I have created a ticket for this:
> https://issues.apache.org/jira/browse/HADOOP-7425
>
>
> Paste the source code in 0.21.0 (and 0.20.2 too) below:
>
> public static void setConf(Object theObject, Configuration conf) {
>     if (conf != null) {
>        if (theObject instanceof Configurable) {
>            ((Configurable) theObject).setConf(conf);
>        }
>       setJobConf(theObject, conf);
>    }
> }
>
>
>
> --
>         best wishes.
>                              steven
>
>
--
        best wishes.
                             steven
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB