| clear query|facets|time |
Search criteria: .
Results from 1 to 10 from
10 (0.096s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Multi-threaded map task - Hadoop - [mail # user]
|
|
...Never mind, depends on plantform, in my case would work fine. Thanks guys! Mark On Mon, Jan 14, 2013 at 12:23 PM, Mark Olimpiati wrote: ...
|
|
|
Author: Mark Olimpiati,
2013-01-14, 20:44
|
|
|
Re: Multi-threaded map task - Hadoop - [mail # user]
|
|
...Thanks Bertrand, I shall try it and hope to gain some speed. One last question though, do you think the threads used are user-level or kernel-level threads in MultithreadedMapper ? Mar...
|
|
|
Author: Mark Olimpiati,
2013-01-14, 20:23
|
|
|
Re: Multi-threaded map task - Hadoop - [mail # user]
|
|
...Thanks for the reply Nitin, but I don't see what's the bottleneck of having it distributed with multi-threaded maps ? I see your point in that each map is processing different splits, ...
|
|
|
Author: Mark Olimpiati,
2013-01-14, 07:22
|
|
|
Multi-threaded map task - Hadoop - [mail # user]
|
|
...Hi, this is a simple question, but why wasn't map or reduce tasks programmed to be multi-threaded ? ie. instead of spawning 6 map tasks for 6 cores, run one map task with 6 parallel threads....
|
|
|
Author: Mark Olimpiati,
2013-01-14, 05:57
|
|
|
Hadoop I/O buffer size - Hadoop - [mail # user]
|
|
...Hi guys, Need help figuring out Hadoop I/O buffer size. This link: http://developer.yahoo.com/blogs/hadoop/posts/2009/08/the_anatomy_of_hadoop_io_pipel/, implies that *io.file....
|
|
|
Author: Mark Olimpiati,
2012-12-07, 02:35
|
|
|
Re: Maps split size - Hadoop - [mail # user]
|
|
...Well, when I said I found a solution this link was one of them :). Even though I set : dfs.block.size = mapred.min.split.size = mapred.max.split.size = 14MB the job is still running ma...
|
|
|
Author: Mark Olimpiati,
2012-10-29, 04:25
|
|
|
Maps split size - Hadoop - [mail # user]
|
|
...Hi, I've found that the solution to control the split size per mapper is to modify the following configurations: mapred.min.split.size and mapred.max.split.size, but when ...
|
|
|
Author: Mark Olimpiati,
2012-10-26, 20:47
|
|
|
Re: Reading Sequence File from Hadoop Distributed Cache .. - Hadoop - [mail # user]
|
|
...I'll try that thanks for the suggestion Steve! Mark On Fri, Oct 12, 2012 at 11:27 AM, Steve Loughran wrote: ...
|
|
|
Author: Mark Olimpiati,
2012-10-15, 17:15
|
|
|
Re: Metrics .. - Hadoop - [mail # user]
|
|
...Hi David, I enabled the "jvm.class" of the hadoop-metrics.properties, you're output seems to be from something else (dfs.class or mapred.class) which reports hadoop deamo...
|
|
|
Author: Mark Olimpiati,
2012-08-29, 20:47
|
|
|
Metrics .. - Hadoop - [mail # user]
|
|
...Hi, I enabled the "metrics.properties" to use FileContext, in which jvm metrics values are written to a file as follows: jvm.metrics: hostName= localhost, processName=MAP,...
|
|
|
Author: Mark Olimpiati,
2012-08-29, 19:54
|
|
|
|
