| clear query|facets|time |
Search criteria: .
Results from 1 to 10 from
15 (0.286s).
|
|
|
Loading phrases to help you
refine your search...
|
|
error in escaping "/" and "+" in hadoop s3://key:secret@bucket/ - Hadoop - [mail # user]
|
|
...previously this does work, by escaping "/" with "%2F" (not sure about the "+" case) now hadoop gives me error saying that signature is wrong. I went back and generated a new key...
|
|
|
Author: Yang,
2013-01-18, 23:35
|
|
|
out of memory for Reducer possible? - Hadoop - [mail # user]
|
|
...if I do a X = GROUP ALL my_relation; in pig. and let's say X is very big, and contains 1billion records, in the reducer,would it be possible for the reducer to get OOM or s...
|
|
|
Author: Yang,
2012-08-24, 01:43
|
|
|
Re: can't disable speculative execution? - Hadoop - [mail # user]
|
|
...Thanks Harsh I did set mapred.map.tasks = 1 but still I can consistently see 3 mappers being invoked and the order is always like this: ****_00002_0 ***_00000_0 ***_0...
|
|
|
Author: Yang,
2012-07-12, 06:39
|
|
|
Re: can't disable speculative execution? - Hadoop - [mail # user]
|
|
...yes, let me try that changing the max mapper slot actually requires changing the hadoop config, since I just found that it's "final" param On Wed, Jul 11, 2012 at 10:05 PM...
|
|
|
Author: Yang,
2012-07-12, 05:07
|
|
|
Re: can't disable speculative execution? - Hadoop - [mail # user]
|
|
...Thanks Harsh I see then there seems to be some small problems with the Splitter / InputFormat. I'm just reading a 1-line text file through pig: A = LOAD 'myinput.txt'...
|
|
|
Author: Yang,
2012-07-12, 05:06
|
|
|
can't disable speculative execution? - Hadoop - [mail # user]
|
|
...I set the following params to be false in my pig script (0.10.0) SET mapred.map.tasks.speculative.execution false; SET mapred.reduce.tasks.speculative.execution false; I also ve...
|
|
|
Author: Yang,
2012-07-12, 03:46
|
|
|
force minimum number of nodes to run a job? - Hadoop - [mail # user]
|
|
...let's say my job can run on 4 mapper slots, but if there is only 1 slot available, I don't want them to run one by one, and have to wait till the time that at least 4 slots are available. &n...
|
|
|
Author: Yang,
2012-07-03, 00:18
|
|
|
bug in streaming? - Hadoop - [mail # user]
|
|
...if I set input to a file that contains just 1 line (which does not even contain "\n") and the mapper is -mapper " bash -c './a.sh' " and a.sh is echo -n "|" cat echo ...
|
|
|
Author: Yang,
2012-07-02, 21:16
|
|
|
S3-->HDFS -->MR or S3-->MR ? - Hadoop - [mail # user]
|
|
...our raw data is in S3, and we need to process them, and dump the output to S3. so there is no need to store the raw data or output on HDFS. but since MR works on hdfs, we n...
|
|
|
Author: Yang,
2012-06-20, 04:56
|
|
|
ulimit -v changed for child job? - Hadoop - [mail # user]
|
|
...the Child of our jobs are launched using user hdfs we got OOM in some jobs when they try to mmap() a big file. I manually checked the ulimit -v of hdfs on the slave nodes, they a...
|
|
|
Author: Yang,
2012-06-13, 22:41
|
|
|
|
