Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> How to build multiple inverted indexes?


Copy link to this message
-
Re: How to build multiple inverted indexes?
Hadoop does not prevent you from writing key value pair multiple times in
the same map iteration if that is what is your roadblock.

You can call collector.collect() multiple times with same or distinct key
value pairs within a single map iteration.

-Rahul

On Thu, Jul 29, 2010 at 8:10 AM, ailinykh <[EMAIL PROTECTED]> wrote:

>
> Hello, everybody!
> I have a bunch of records. Each record has key, and two fields A,B - R(k,
> A,B)
> I want to build two inverted indexes, one per each field. As output I
> expect
> two files
> IndexA =(A1- [k1,k2,k3..]),(A2 -[k1,k2,k4...]) ...
> IndexB =(B1- [k1,k2,k3..]),(B2-[k1,k2,k4...]) ...
>
> Hadoop mapper allows to write out only one key value pair. Of cause, I can
> run it twice, but I don't want to read the same record twice. What is the
> best way to handle this situation?
>
> Thank you,
>  Andrey
> --
> View this message in context:
> http://old.nabble.com/How-to-build-multiple-inverted-indexes--tp29297476p29297476.html
> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB