Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Binary Search in map reduce


Copy link to this message
-
Re: Binary Search in map reduce
Hi
 Thanks for the reply. So here is the intent.
I process some data and output of that processing is this set of json
documents outputting {key:[values]}  (This is essentially a form of graph
where each entry is an edge)
Now.. I process a different set of data and the idea is to modify the
existing document based on this new data.
If the key is present then add/modify values.
Else... create new key:[values] json object and save.

So, the first step is checking whether the key is present or not..
So thats why I thought of doing the binary search.
Any suggestions?
On Mon, Jan 7, 2013 at 3:35 PM, John Lilley <[EMAIL PROTECTED]>wrote:

>  It depends.  What data is going into the table, and what keys will drive
> the lookup?****
>
> ** **
>
> Let’s suppose that you have a single JSON file that has some reasonable
> number of key/value tuples.  You could easily load a Hashtable to associate
> the integer keys with the values (which appear to be lists of integers).
> Each task in your MapReduce could process each input tuple, doing a lookup
> by key and appending values to the output records, and that is a perfectly
> fine thing to do in MapReduce.  In this model, the JSON file is effectively
> a constant singleton table for the entire MapReduce job.  You can just load
> it from HDFS or any file system.  Specifying it as a cached file may
> improve performance somewhat.****
>
> ** **
>
> If you explain your intent we might be able to help better.****
>
> ** **
>
> john****
>
> ** **
>
> *From:* jamal sasha [mailto:[EMAIL PROTECTED]]
> *Sent:* Monday, January 07, 2013 4:21 PM
> *To:* [EMAIL PROTECTED]
> *Subject:* Binary Search in map reduce****
>
> ** **
>
> Hi,****
>
>  I have data in json format like:****
>
> ** **
>
> {key:[values.....]}****
>
> key, values are longints.****
>
> Now, I want to do a fast lookup of a key.****
>
> How would I implement a binary search in map reduce abstraction.****
>
> ** **
>
> Or am i not thinking about this correctly?****
>
> Any suggestions/advices?****
>
> Thanks****
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB