In pig 0.8 the default number of reducers changed from 1 to a value computed based on input data size -
On 5/27/11 6:46 AM, "Jonathan Coveney" <[EMAIL PROTECTED]> wrote:
SET default_parallel X; will set the PARALLEL keyword for all parallel
functions (ie set the reducers for the job)
I am not sure how the default is calculated...for a while it was set to 1 I
believe, ostensibly to force people to set it to something more reasonable.
2011/5/27 Harsh J <[EMAIL PROTECTED]>
> The PARALLEL keyword controls the number of reducers used in the job.
> If unspecified, a default number is applied. Is this what you're
> looking for?
> On Fri, May 27, 2011 at 3:46 PM, Sudharsan Sampath
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> > Is there a reference on how the number of reducers required for a job is
> > Thanks
> > Sudharsan S
> Harsh J