| clear query|facets|time |
Search criteria: .
Results from 71 to 76 from
76 (0.265s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: how to control the number of mappers? - Pig - [mail # user]
|
|
...ok, I see, I was using pig 0.5 tried 0.9, works now thanks! On Tue, Jan 17, 2012 at 1:20 PM, Yang wrote: ...
|
|
|
Author: Yang,
2012-01-17, 21:28
|
|
|
Re: how to control the number of mappers? - Pig - [mail # user]
|
|
...weird I tried # head a.pg set job.name 'blah'; SET mapred.map.tasks.speculative.execution false; set mapred.min.split.size 10000; set mapred.tasktracker.map.tasks.max...
|
|
|
Author: Yang,
2012-01-17, 21:20
|
|
|
Re: how to control the number of mappers? - Pig - [mail # user]
|
|
...Prashant: I tried splitting the input files, yes that worked, and multiple mappers were indeed created. but then I would have to create a separate stage simply to split the input...
|
|
|
Author: Yang,
2012-01-17, 20:53
|
|
|
Re: how to control the number of mappers? - Pig - [mail # user]
|
|
...thanks, but from http://pig.apache.org/docs/r0.7.0/piglatin_ref2.html#set it looks the params that can be 'set' is very limited, and does not contain the min split size and mapper coun...
|
|
|
Author: Yang,
2012-01-17, 20:46
|
|
|
Re: how to control the number of mappers? - Pig - [mail # user]
|
|
...Prashant: thanks. by "reducing the block size", do you mean split size ? ---- block size is fixed on a hadoop hdfs. my application is not really data heavy, each line of in...
|
|
|
Author: Yang,
2012-01-12, 02:27
|
|
|
how to control the number of mappers? - Pig - [mail # user]
|
|
...I have a pig script that does basically a map-only job: raw = LOAD 'input.txt' ; processed = FOREACH raw GENERATE convert_somehow($1,$2...); store processed into 'out...
|
|
|
Author: Yang,
2012-01-12, 02:12
|
|
|
|
