Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: What happens when I do not output anything from my mapper - Solution


+
murat migdisoglu 2012-06-04, 15:42
+
murat migdisoglu 2012-06-04, 12:22
+
praveenesh kumar 2012-06-04, 12:27
Copy link to this message
-
RE: What happens when I do not output anything from my mapper
Hi Murat,

As Praveenesh explained, you can control the map outputs as you want.

map() function will be called for each input i.e map() function invokes multiple times with different inputs in the same mapper. You can check by having the logs in the map function what is happening in it.
  

Thanks
Devaraj

________________________________________
From: praveenesh kumar [[EMAIL PROTECTED]]
Sent: Monday, June 04, 2012 5:57 PM
To: [EMAIL PROTECTED]
Subject: Re: What happens when I do not output anything from my mapper

You can control your map outputs based on any condition you want. I have
done that - it worked for me.
It could be your code problem that its not working for you.
Can you please share your map code or cross-check whether your conditions
are correct ?

Regards,
Praveenesh

On Mon, Jun 4, 2012 at 5:52 PM, murat migdisoglu <[EMAIL PROTECTED]
> wrote:

> Hi,
> I have a small application where I have only mapper class defined(no
> reducer, no combiner).
> Within the mapper class, I have an if condition according to which I decide
> If I want to put something in the context or not.
> If my condition is not match, I want that mapper does not give any output
> to the hdfs.
> But apparently, this does not worj as I expected. Once I run my job, a file
> per mapper in the hdfs with 87 kb of size.
>
> the if block that I'm using in the map method is as following:
> if (ip == null || ip.equals(cip)) {
>            Text value = new Text(mwrapper.toJson());
>            word.set(ip);
>            context.write( word, value);
>        } else {
>            log.info("ip not match [" + ip + "]");
>        }
> }
> }//end of mapper method
>
> How can I manage that? Does mapper always need to have an output?
>
> --
> "Find a job you enjoy, and you'll never work a day in your life."
> Confucius
>
+
murat migdisoglu 2012-06-04, 12:48
+
Devaraj k 2012-06-05, 05:22
+
murat migdisoglu 2012-06-05, 10:18