Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - ReflectionUtils.setConf would configure anything Configurable twice?


Copy link to this message
-
Re: ReflectionUtils.setConf would configure anything Configurable twice?
steven zhuang 2011-06-29, 07:05
Anyone have the same partition problem using Streaming in 0.21.0?

On Fri, Jun 24, 2011 at 10:22 AM, steven zhuang <[EMAIL PROTECTED]>wrote:

> hello, list,
>
>               Recently I have upgraded our Hadoop cluster from 0.20.2 to
> 0.21.0, and I found there is something wrong introduced by the upgrade.
>
>               In the setConf method of
> org.apache.hadoop.util.ReflectionUtils, any instance of Configurable would
> be configured twice, and this may cause trouble.
>               For example, in 0.21.0, KeyFieldBasedPartitioner implements
> the Configurable interface. When configured twice, it get two KeyDescription
> and gives out wrong partition number.
>
> I have created a ticket for this:
> https://issues.apache.org/jira/browse/HADOOP-7425
>
>
> Paste the source code in 0.21.0 (and 0.20.2 too) below:
>
> public static void setConf(Object theObject, Configuration conf) {
>     if (conf != null) {
>        if (theObject instanceof Configurable) {
>            ((Configurable) theObject).setConf(conf);
>        }
>       setJobConf(theObject, conf);
>    }
> }
>
>
>
> --
>         best wishes.
>                              steven
>
>
--
        best wishes.
                             steven