| clear query|facets|time |
Search criteria: .
Results from 11 to 20 from
20 (0.476s).
|
|
|
Loading phrases to help you
refine your search...
|
|
RE: How can I limit reducers to one-per-node? - HDFS - [mail # user]
|
|
...I guess the FairScheduler is doing multiple assignments per heartbeat, hence the behavior of multiple reduce tasks per node even when they should otherwise be full distributed. ...
|
|
|
Author: David Parks,
2013-02-11, 02:29
|
|
|
RE: Skipping entire task - HDFS - [mail # user]
|
|
...Thinking here... if you submitted the task programmatically you should be able to capture the failure of the task and gracefully move past it to your next tasks. To say it in a long-wi...
|
|
|
Author: David Parks,
2013-01-05, 14:14
|
|
|
RE: What does mapred.map.tasksperslot do? - HDFS - [mail # user]
|
|
...Ah this is on AWS EMR, hadoop 1.0.3. This could be an AWS feature based on my reading of the AWS docs, but I thought it was hadoop. From: Hemanth Yamijala ...
|
|
|
Author: David Parks,
2012-12-27, 09:42
|
|
|
RE: How to submit Tool jobs programatically in parallel? - HDFS - [mail # user]
|
|
...Can I do that with s3distcp / distcp? The job is being configured in the run() method of s3distcp (as it implements Tool). So I think I can't use this approach. I use this for th...
|
|
|
Author: David Parks,
2012-12-14, 06:14
|
|
|
Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException - HDFS - [mail # user]
|
|
...I'm having exactly this problem, and it's causing my job to fail when I try to process a larger amount of data (I'm attempting to process 30GB of compressed CSVs and the entire job fails eve...
|
|
|
Author: David Parks,
2012-12-13, 04:22
|
|
|
RE: Hadoop 101 - HDFS - [mail # user]
|
|
...Nothing that I'm aware of for text files, I'd just use standard unix utils to process it outside of Hadoop. As to getting a reader from any of the Input Formats, here's the typical exa...
|
|
|
Author: David Parks,
2012-12-13, 04:16
|
|
|
Can we declare some HDFS nodes "primary" - HDFS - [mail # user]
|
|
...Assume for a moment that you have a large cluster of 500 AWS spot instance servers running. And you want to keep the bid price low, so at some point it's likely that the whole cluster will g...
|
|
|
Author: David Parks,
2012-12-11, 11:39
|
|
|
RE: When reduce function is used as combiner? - HDFS - [mail # user]
|
|
...The map task may use a combiner 0+ times. Basically that means (as far as I understand), if the map output data is below some internal hadoop threshold, it'll just send it to the reducer, if...
|
|
|
Author: David Parks,
2012-12-11, 11:32
|
|
|
RE: [Bulk] Re: Failed To Start SecondaryNameNode in Secure Mode - HDFS - [mail # user]
|
|
...I'm curious about profiling, I see some documentation about it (1.0.3 on AWS), but the references to JobConf seem to be for the "old api" and I've got everything running on the "new api". &n...
|
|
|
Author: David Parks,
2012-12-04, 08:00
|
|
|
How do map tasks get assigned efficiently? - HDFS - [mail # user]
|
|
...Even after reading O'reillys book on hadoop I don't feel like I have a clear vision of how the map tasks get assigned. They depend on splits right? But I have 3 j...
|
|
|
Author: David Parks,
2012-10-24, 06:10
|
|
|
|
