jamal sasha 2013-01-07, 23:21
-RE: Binary Search in map reduce
John Lilley 2013-01-07, 23:35
It depends. What data is going into the table, and what keys will drive the lookup?
Let's suppose that you have a single JSON file that has some reasonable number of key/value tuples. You could easily load a Hashtable to associate the integer keys with the values (which appear to be lists of integers). Each task in your MapReduce could process each input tuple, doing a lookup by key and appending values to the output records, and that is a perfectly fine thing to do in MapReduce. In this model, the JSON file is effectively a constant singleton table for the entire MapReduce job. You can just load it from HDFS or any file system. Specifying it as a cached file may improve performance somewhat.
If you explain your intent we might be able to help better.
From: jamal sasha [mailto:[EMAIL PROTECTED]]
Sent: Monday, January 07, 2013 4:21 PM
To: [EMAIL PROTECTED]
Subject: Binary Search in map reduce
I have data in json format like:
key, values are longints.
Now, I want to do a fast lookup of a key.
How would I implement a binary search in map reduce abstraction.
Or am i not thinking about this correctly?
jamal sasha 2013-01-07, 23:42
John Lilley 2013-01-08, 00:11
jamal sasha 2013-01-08, 00:17
Mahesh Balija 2013-01-08, 03:26
John Lilley 2013-01-08, 04:05
Pamecha, Abhishek 2013-01-07, 23:31
jamal sasha 2013-01-08, 09:14