Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> merging the size of the reduce output

Copy link to this message
Re: merging the size of the reduce output
Looking at
hive.merge.mapredfiles is effective if there is a reducer for your job.
Otherwise you should have set hive.merge.mapfiles to true.

On Sat, Jun 12, 2010 at 11:22 PM, Sammy Yu <[EMAIL PROTECTED]> wrote:

> Hi,
>    I'm running the latest version of trunk r953172.  I'm doing doing a
> dynamic partition insert overwrite query which generates a lot of small
> files in each of the partition.  I was hoping this could be solved by
> setting hive.merge.mapredfiles to true.  However, it seems like whenever the
> job is submitted it is always set to false, thus it doesnt seem to have any
> effect.  I also tried to modified this property in the hive-default.xml, but
> it didn't work either.
> Thanks,
> Sammy