-Re: Maps split size
Mark Olimpiati 2012-10-29, 04:25
Well, when I said I found a solution this link was one of them :). Even
though I set :
dfs.block.size = mapred.min.split.size = mapred.max.split.size = 14MB the
job is still running maps with 64MB !
I don't see what else can I change :(
On Fri, Oct 26, 2012 at 2:23 PM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:
> Hi Mark,
> I think http://wiki.apache.org/hadoop/HowManyMapsAndReduces might interest
> If you require more information, feel free to ask after reading it.
> On Fri, Oct 26, 2012 at 10:47 PM, Mark Olimpiati <[EMAIL PROTECTED]
> > Hi,
> > I've found that the solution to control the split size per mapper is to
> > modify the following configurations:
> > mapred.min.split.size and mapred.max.split.size, but when I set them both
> > to 14MB with dfs.block.size = 64MB, the splits are still = 64MB.
> > So, is there a relation between them that I should consider?
> > Thank you,
> > Mark
> Bertrand Dechoux