| clear query|facets|time |
Search criteria: .
Results from 71 to 80 from
99 (3.085s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Algorithm for cross product - MapReduce - [mail # user]
|
|
...The approach you suggest is similar to what I am currently doing but it requires you to size the partitions to the memory available on the reducer. This is a non-trivial task and is not nece...
|
|
|
Author: Steve Lewis,
2011-06-23, 17:07
|
|
|
Algorithm for cross product - MapReduce - [mail # user]
|
|
...Assume I have two data sources A and B Assume I have an input format and can generate key values for both A and B I want an algorithm which will generate the cross product of all values in A...
|
|
|
Author: Steve Lewis,
2011-06-22, 22:16
|
|
|
Is there any way for the reducer to determine the total number of reduce tasks? - MapReduce - [mail # user]
|
|
...Also is there a good way in code to determine whether job is running on a cluster or in local mode. I want certain debugging information to log only in local mode Steven M. Lewis...
|
|
|
Author: Steve Lewis,
2011-06-22, 22:12
|
|
|
Re: Custom input format query - MapReduce - [mail # user]
|
|
...1) add @Override public Text getCurrentKey() { return key; ...
|
|
|
Author: Steve Lewis,
2011-05-20, 00:57
|
|
|
Re: How to create a SequenceFile more faster? - MapReduce - [mail # user]
|
|
...Even for a single machine (and there may be reasons to use a single machine if the original data is not splittable) Our experience suggests it should take about an hour to process 32 GB on a...
|
|
|
Author: Steve Lewis,
2011-05-12, 15:55
|
|
|
Re: Passing an Object to All Reducers - MapReduce - [mail # user]
|
|
...If possible serialize the object as XML then add it as a set of lines to the config - alternatively serialize it (maybe xml) to a known spot in HDFS and read it in in the setup code in the r...
|
|
|
Author: Steve Lewis,
2011-05-06, 17:55
|
|
|
Re: Creating custom input split.l - MapReduce - [mail # user]
|
|
...This is a custom splitter for extracting XML Tags from XML documents - to use is subclass as follows. It assumes the document is pretty printed with the start and end tags on separate lines....
|
|
|
Author: Steve Lewis,
2011-04-09, 16:20
|
|
|
Re: question for understanding partitioning - MapReduce - [mail # user]
|
|
...1) you need not have 26 reducers but you want 26 partitions - you might send to int reducer = character % Math.min(26,nreducers); // this insures that all items with A go...
|
|
|
Author: Steve Lewis,
2011-01-18, 20:33
|
|
|
Re: Mapper runs only on one machine - MapReduce - [mail # user]
|
|
...Are you sure your input file is splittable - many files (say gzip) are not and such files must be processed on a single machine On Tue, Nov 16, 2010 at 9:24 AM, wrote: &nb...
|
|
|
Author: Steve Lewis,
2010-11-16, 17:33
|
|
|
Re: Big split file to Partitioner - MapReduce - [mail # user]
|
|
...It is a good idea to ask what the meaning of ths split is. Typically a split is one per line but I have written splits which return the entire file for a small file - say an xml document &nb...
|
|
|
Author: Steve Lewis,
2010-08-22, 17:47
|
|
|
|
