|
|
-
executing map/reduce and different file's format
Oleg Ruchovets 2011-07-12, 08:31
Hi ,
I have to run map/reduce job. Inputs to this job will be files with different formats . question: processing line from file , how can I check (know) from which file this line was read?
Thanks in advance Oleg.
-
RE: executing map/reduce and different file's format
Devaraj K 2011-07-12, 10:41
Hi Oleg,
It is not required to know the each line read from which file. If you want to process multiple input formats of files with same job you can use any of these below API's from MultipleInputs class. You can specify different formats for different files.
1. public static void addInputPath(JobConf conf, Path path, Class<? extends InputFormat> inputFormatClass) 2. public static void addInputPath(JobConf conf, Path path, Class<? extends InputFormat> inputFormatClass, Class<? extends Mapper> mapperClass)
Devaraj K ---------------------------------------------------------------------------- --------------------------------------------------------- This e-mail and its attachments contain confidential information from HUAWEI, which is intended only for the person or entity whose address is listed above. Any use of the information contained herein in any way (including, but not limited to, total or partial disclosure, reproduction, or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender by phone or email immediately and delete it!ss -----Original Message----- From: Oleg Ruchovets [mailto:[EMAIL PROTECTED]] Sent: Tuesday, July 12, 2011 2:01 PM To: [EMAIL PROTECTED] Subject: executing map/reduce and different file's format
Hi ,
I have to run map/reduce job. Inputs to this job will be files with different formats . question: processing line from file , how can I check (know) from which file this line was read?
Thanks in advance Oleg.
|
|
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by
Sematext