| clear query|facets|time |
Search criteria: .
Results from 1 to 10 from
11 (0.11s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Large static structures in M/R heap - MapReduce - [mail # user]
|
|
...We actually use CDBs a good bit outside of M/R. This is something worth looking into, but the big structure we're currently using is a giant tree-based lookup table whose access patter...
|
|
|
Author: Adam Phelps,
2013-02-27, 19:40
|
|
|
Large static structures in M/R heap - MapReduce - [mail # user]
|
|
...We have a job that uses a large lookup structure that gets created as a static class during the map setup phase (and we have the JVM reused so this only takes place once). However of l...
|
|
|
Author: Adam Phelps,
2013-02-27, 18:42
|
|
|
Re: Getting (or setting) a job ID - MapReduce - [mail # user]
|
|
...We could make a change of that nature. We currently use job.waitForCompletion() to submit jobs, if we switch to job.submit() is there an alternate blocking call to wait for...
|
|
|
Author: Adam Phelps,
2011-05-10, 23:47
|
|
|
Getting (or setting) a job ID - MapReduce - [mail # user]
|
|
...We have intermittently seen cases where a job will "freeze" for some as yet unknown reason, and thereby block other processes waiting for that job to complete. I'm trying t...
|
|
|
Author: Adam Phelps,
2011-05-10, 22:45
|
|
|
Re: Choosing number of map/reduce slots (with hyperthreading) - MapReduce - [mail # user]
|
|
...I'll attempt some tests on this later this week and report back once I've done so. - Adam On 1/10/11 12:06 AM, Eric wrote:...
|
|
|
Author: Adam Phelps,
2011-01-10, 22:26
|
|
|
Choosing number of map/reduce slots (with hyperthreading) - MapReduce - [mail # user]
|
|
...By scouring various web pages and lists via google I've found some general recommendations when it comes to setting the number of map and reduce slots for a cluster. It see...
|
|
|
Author: Adam Phelps,
2011-01-07, 01:51
|
|
|
Re: Duplicated entries with map job reading from HBase - MapReduce - [mail # user]
|
|
...That had been my initial thought, however dumping the data from hbase shell only found single entries. Further with the experiment I ran yesterday (generating the data to a...
|
|
|
Author: Adam Phelps,
2010-11-09, 18:29
|
|
|
Re: Duplicated entries with map job reading from HBase - MapReduce - [mail # user]
|
|
...Ok, poked around at this a little more with a few experiments. The most interesting one is that I ran a a couple of the jobs that generate this data in HBase, one for the existin...
|
|
|
Author: Adam Phelps,
2010-11-09, 00:29
|
|
|
Re: Duplicated entries with map job reading from HBase - MapReduce - [mail # user]
|
|
...Yeah, it wasn't the combiner. The repeated entries are actually seen by the mapper, so before the combiner comes into play. Is there some other info that would be use...
|
|
|
Author: Adam Phelps,
2010-11-05, 23:01
|
|
|
Re: Duplicated entries with map job reading from HBase - MapReduce - [mail # user]
|
|
...No, the system actually is much larger than two nodes. But the number of mappers used here tends to be fairly small (I suspect based on the HBase regions being accessed but...
|
|
|
Author: Adam Phelps,
2010-11-05, 18:35
|
|
|
|
