Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Binary Search in map reduce

jamal sasha 2013-01-07, 23:21
Copy link to this message
RE: Binary Search in map reduce
It depends.  What data is going into the table, and what keys will drive the lookup?

Let's suppose that you have a single JSON file that has some reasonable number of key/value tuples.  You could easily load a Hashtable to associate the integer keys with the values (which appear to be lists of integers).  Each task in your MapReduce could process each input tuple, doing a lookup by key and appending values to the output records, and that is a perfectly fine thing to do in MapReduce.  In this model, the JSON file is effectively a constant singleton table for the entire MapReduce job.  You can just load it from HDFS or any file system.  Specifying it as a cached file may improve performance somewhat.

If you explain your intent we might be able to help better.


From: jamal sasha [mailto:[EMAIL PROTECTED]]
Sent: Monday, January 07, 2013 4:21 PM
Subject: Binary Search in map reduce

 I have data in json format like:

key, values are longints.
Now, I want to do a fast lookup of a key.
How would I implement a binary search in map reduce abstraction.

Or am i not thinking about this correctly?
Any suggestions/advices?
jamal sasha 2013-01-07, 23:42
John Lilley 2013-01-08, 00:11
jamal sasha 2013-01-08, 00:17
Mahesh Balija 2013-01-08, 03:26
John Lilley 2013-01-08, 04:05
Pamecha, Abhishek 2013-01-07, 23:31
jamal sasha 2013-01-08, 09:14