Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Setting number of mappers according to number of TextInput lines


Copy link to this message
-
Re: Setting number of mappers according to number of TextInput lines
Bejoy KS 2012-06-16, 09:27
Hi Ondrej

You can use NLineInputFormat with n set to 10.

------Original Message------
From: Ondřej Klimpera
To: [EMAIL PROTECTED]
ReplyTo: [EMAIL PROTECTED]
Subject: Setting number of mappers according to number of TextInput lines
Sent: Jun 16, 2012 14:31

Hello,

I have very small input size (kB), but processing to produce some output
takes several minutes. Is there a way how to say, file has 100 lines, i
need 10 mappers, where each mapper node has to process 10 lines of input
file?

Thanks for advice.
Ondrej Klimpera
Regards
Bejoy KS

Sent from handheld, please excuse typos.