| clear query|facets|time |
Search criteria: .
Results from 11 to 20 from
60 (0.053s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: How can I split the data with more reducers? - Pig - [mail # user]
|
|
...The pie chart is generated by MemoryAnalyzer(http://www.eclipse.org/mat/) from the heap dump when OOME happened. I've increased all the parallelisms and set default_paralle...
|
|
|
Author: Haitao Yao,
2012-09-17, 09:26
|
|
|
Re: How can I split the data with more reducers? - Pig - [mail # user]
|
|
...Thank you very much for the reply. I've checked the latest heapdump again,and you're right: the OOME is not caused by split, but DefaultDataBag. Sorry for the misleading, I've go...
|
|
|
Author: Haitao Yao,
2012-09-17, 08:53
|
|
|
Re: How to force the script finish the job and continue the follow script? - Pig - [mail # user]
|
|
...I just wanna to spilt the big job into smaller ones but still write into one script file. I still does not know why the first one of the compiled job have only 1 reducer. ...
|
|
|
Author: Haitao Yao,
2012-09-17, 00:00
|
|
|
Re: How can I split the data with more reducers? - Pig - [mail # user]
|
|
...The map output of the first MR job is over 500MB, and only 1 reducer processes it. So OutOfMemoryError is caused. After set the child memory to 1GB, the first job succeeded. But ...
|
|
|
Author: Haitao Yao,
2012-09-16, 09:18
|
|
|
Re: How can I split the data with more reducers? - Pig - [mail # user]
|
|
...here's the explain result compressed.(The apache mail server does not allow big attachments.) Haitao Yao [EMAIL PROTECTED] weibo: @haitao_yao Skype: haitao.yao.final...
|
|
|
Author: Haitao Yao,
2012-09-16, 09:05
|
|
|
How to force the script finish the job and continue the follow script? - Pig - [mail # user]
|
|
...Hi, all I forgot the keyword which force Pig to finish the job and then continue the following script. My job failed because of OOME, so I want to split the jobs into smaller o...
|
|
|
Author: Haitao Yao,
2012-09-16, 02:52
|
|
|
Re: How can I split the data with more reducers? - Pig - [mail # user]
|
|
...No, I also thought it is a mapper , but It surely is a reducer. all the mappers succeeded and the reducer failed. Haitao Yao [EMAIL PROTECTED] weibo: @haitao_yao Skype: &n...
|
|
|
Author: Haitao Yao,
2012-09-16, 02:50
|
|
|
How can I split the data with more reducers? - Pig - [mail # user]
|
|
...Hi, I 'v encountered a problem: the job failed because of POSplit retained too much memory in the reducer. How can I specify more reducers for the spill? Here's the scree...
|
|
|
Author: Haitao Yao,
2012-09-16, 02:08
|
|
|
[PIG-2916] Add script file name for logging with log4j MDC - Pig - [issue]
|
|
...We often execute several scripts in parallel and log the messages into one log file.If got error , we can not identify which pig script is causing it. So add a MDC with name "pig_script_file...
|
|
|
http://issues.apache.org/jira/browse/PIG-2916
Author: Haitao Yao,
2012-09-13, 03:11
|
|
|
Add file command in Pig - Pig - [mail # user]
|
|
...hi, all I want to add GeoIP.dat to my pig scripts. Does Pig have the "add file XXX" command like hive? I want to distribute the data file GeoIP.dat with Pig. Or is there any ot...
|
|
|
Author: Haitao Yao,
2012-08-28, 07:20
|
|
|
|
