Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Collecting MAP output in a Iterator


Copy link to this message
-
RE: Collecting MAP output in a Iterator

Hi Bertrand,

You got me spot on,
I wanted to do a cross Join.
I have some data related to a particular type of payment card, we need to find all types of txn user did during the day. So in one file I have something linke below :-

USER MERCHANT ANMT TXN TS LAT LONG   -- actual big log

so we want a report want a report which shows how many events happened at what day.
so that way we can see how active was teh user in aparticular hour on a particular day

*------------------------*

Cheers !!!

Siddharth Tiwari

Have a refreshing day !!!
"Every duty is holy, and devotion to duty is the highest form of worship of God.”

"Maybe other people will try to limit me but I don't limit myself"
Date: Mon, 20 Aug 2012 22:28:50 +0200
Subject: Re: Collecting MAP output in a Iterator
From: [EMAIL PROTECTED]
To: [EMAIL PROTECTED]

The question is why do you want to do that? How will you use the output?You really want a cross join? This a very expensive operation (in terms of memory).
Could you provide more context?

Regards
Bertrand

On Mon, Aug 20, 2012 at 10:24 PM, Lance Norskog <[EMAIL PROTECTED]> wrote:

The reducer has to open MAP2 explicitly. If it is a regular file, it

has to be in distributed cache.

On Mon, Aug 20, 2012 at 12:56 PM, Siddharth Tiwari

<[EMAIL PROTECTED]> wrote:

>

> Hi

>

> I have two mappers MAP1 and MAP2, which collect data from two different

> files,

> In reducer I want to traverse all keys and values of MAP2 for each key and

> value of  MAP1.

> How can I achieve it in one Reducer.

>

> Please suggest

>

>

> *------------------------*

> Cheers !!!

> Siddharth Tiwari

> Have a refreshing day !!!

> "Every duty is holy, and devotion to duty is the highest form of worship of

> God.”

> "Maybe other people will try to limit me but I don't limit myself"

--

Lance Norskog

[EMAIL PROTECTED]

--
Bertrand Dechoux

     
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB