Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Provide context to map function


+
Abhinav M Kulkarni 2013-04-02, 03:55
+
Azuryy Yu 2013-04-02, 04:02
Copy link to this message
-
Re: Provide context to map function
I supposed your input splits are FileSplit, if not, you need to:

InputSplit split = context.getInputSplit();

if (split instanceof FileSplit){
  Path path = ((FileSplit)split).getPath();
}
On Tue, Apr 2, 2013 at 12:02 PM, Azuryy Yu <[EMAIL PROTECTED]> wrote:

> In your map function add following:
>
> Path currentInput = ((FileSplit)context.getInputSplit()).getPath();
>
> then:
>
> if (currentInput is first ){
> ................
> }
> else{
> ..................
> }
>
>
>
>
> On Tue, Apr 2, 2013 at 11:55 AM, Abhinav M Kulkarni <
> [EMAIL PROTECTED]> wrote:
>
>>  Hi,
>>
>> I have a following scenario:
>>
>>
>>    - Two mappers (acting on two different files) and one reducer
>>    - The mapper code for two different files is the same, except for
>>    minor change which depends on which file is being read
>>    - Essentially assume there is an if statement - if first file is
>>    being read do this else do this
>>
>> So how do I provide this context to map function i.e. file name or say a
>> boolean flag variable indicating the file being read?
>>
>> Thanks,
>> Abhinav
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB