| clear query|facets|time |
Search criteria: .
Results from 21 to 30 from
38 (0.15s).
|
|
|
Loading phrases to help you
refine your search...
|
|
RE: Fastest way to transfer files - MapReduce - [mail # user]
|
|
...Here’s an example of running distcp (actually in this case s3distcp, but it’s about the same, just new DistCp()) from java: ToolRunner.run(getConf(), new S3DistCp(), new String...
|
|
|
Author: David Parks,
2012-12-29, 10:29
|
|
|
What does mapred.map.tasksperslot do? - MapReduce - [mail # user]
|
|
...I didn't come up with much in a google search. In particular, what are the side effects of changing this setting? Memory? Sort process? I'm guessing it means that...
|
|
|
Author: David Parks,
2012-12-27, 08:21
|
|
|
How to troubleshoot OutOfMemoryError - MapReduce - [mail # user]
|
|
...I'm pretty consistently seeing a few reduce tasks fail with OutOfMemoryError (below). It doesn't kill the job, but it slows it down. In my current case the reducer is pretty da...
|
|
|
Author: David Parks,
2012-12-22, 04:33
|
|
|
OutOfMemory in ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory - MapReduce - [mail # user]
|
|
...I've got 15 boxes in a cluster, 7.5GB of ram each on AWS (m1.large), 1 reducer per node. I'm seeing this exception sometimes. It's not stopping the job from completing, it's ju...
|
|
|
Author: David Parks,
2012-12-17, 05:36
|
|
|
How to submit Tool jobs programatically in parallel? - MapReduce - [mail # user]
|
|
...I'm submitting unrelated jobs programmatically (using AWS EMR) so they run in parallel. I'd like to run an s3distcp job in parallel as well, but the interface to that job is a Tool, e....
|
|
|
Author: David Parks,
2012-12-14, 04:39
|
|
|
RE: Shuffle's getMapOutput() fails with EofException, followed by IllegalStateException - MapReduce - [mail # user]
|
|
...If anyone follows this thread in the future, it turns out that I was being lead astray by these errors, they weren't the cause of the problem. This was the resolution: http://stackover...
|
|
|
Author: David Parks,
2012-12-14, 04:25
|
|
|
RE: Map output copy failure - MapReduce - [mail # user]
|
|
...I had the same problem yesterday, it sure does look to be dead on that issue. I found another forum discussion on AWS that suggested more memory as a stop-gap way to deal with it, or apply t...
|
|
|
Author: David Parks,
2012-12-12, 06:15
|
|
|
RE: Hadoop Deployment usecases - MapReduce - [mail # user]
|
|
...You're likely to find answers to your questions here, but you'll need specific questions and some rudimentary subject matter knowledge. I'd suggest starting off with a good book on Hadoop, y...
|
|
|
Author: David Parks,
2012-12-12, 06:10
|
|
|
RE: Hadoop 101 - MapReduce - [mail # user]
|
|
...You use TextInputFormat, you'll get the following key, value pairs in your mapper: file_position, your_input Example: 0, "0\t[356:0.3481597,359:0.3481597,358:0.3481597,361:0.3481...
|
|
|
Author: David Parks,
2012-12-12, 06:04
|
|
|
XMLOutputFormat, anything in the works? - MapReduce - [mail # user]
|
|
...Is there an XMLOutputFormat in existence somewhere? I need to output Solr XML change docs, I'm betting I'm not the first. David...
|
|
|
Author: David Parks,
2012-11-19, 12:22
|
|
|
|
