Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Binary Search in map reduce


+
jamal sasha 2013-01-07, 23:21
+
John Lilley 2013-01-07, 23:35
Copy link to this message
-
Re: Binary Search in map reduce
jamal sasha 2013-01-07, 23:42
Hi
 Thanks for the reply. So here is the intent.
I process some data and output of that processing is this set of json
documents outputting {key:[values]}  (This is essentially a form of graph
where each entry is an edge)
Now.. I process a different set of data and the idea is to modify the
existing document based on this new data.
If the key is present then add/modify values.
Else... create new key:[values] json object and save.

So, the first step is checking whether the key is present or not..
So thats why I thought of doing the binary search.
Any suggestions?
On Mon, Jan 7, 2013 at 3:35 PM, John Lilley <[EMAIL PROTECTED]>wrote:

>  It depends.  What data is going into the table, and what keys will drive
> the lookup?****
>
> ** **
>
> Let’s suppose that you have a single JSON file that has some reasonable
> number of key/value tuples.  You could easily load a Hashtable to associate
> the integer keys with the values (which appear to be lists of integers).
> Each task in your MapReduce could process each input tuple, doing a lookup
> by key and appending values to the output records, and that is a perfectly
> fine thing to do in MapReduce.  In this model, the JSON file is effectively
> a constant singleton table for the entire MapReduce job.  You can just load
> it from HDFS or any file system.  Specifying it as a cached file may
> improve performance somewhat.****
>
> ** **
>
> If you explain your intent we might be able to help better.****
>
> ** **
>
> john****
>
> ** **
>
> *From:* jamal sasha [mailto:[EMAIL PROTECTED]]
> *Sent:* Monday, January 07, 2013 4:21 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Binary Search in map reduce****
>
> ** **
>
> Hi,****
>
>  I have data in json format like:****
>
> ** **
>
> {key:[values.....]}****
>
> key, values are longints.****
>
> Now, I want to do a fast lookup of a key.****
>
> How would I implement a binary search in map reduce abstraction.****
>
> ** **
>
> Or am i not thinking about this correctly?****
>
> Any suggestions/advices?****
>
> Thanks****
>
+
John Lilley 2013-01-08, 00:11
+
jamal sasha 2013-01-08, 00:17
+
Mahesh Balija 2013-01-08, 03:26
+
John Lilley 2013-01-08, 04:05
+
Pamecha, Abhishek 2013-01-07, 23:31
+
jamal sasha 2013-01-08, 09:14