-Configuring # task slots
David Rosenstrauch 2010-08-19, 18:46
Was reading up a bit today on configuring the settings for # task slots,
Was just wondering: couldn't (shouldn't?) this be done dynamically by
default? i.e., couldn't/shouldn't a slave node be able to compute these
values programmatically based on the # of cores in the machine?
(Perhaps in conjunction with a mappers-to-reducers ratio, and a %
Obviously there'd be times where you'd want to manually override that,
but I'd think there could be a simple algorithm for computing this
(e.g., based on the info in slide #8 of this presentation:
that would cover most users' main use case.
Thoughts? Is there something I'm overlooking here that would make this