Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Mapper input as argument


Copy link to this message
-
Re: Mapper input as argument
If you dont need line by line but you want to get a number of lines
together, use NLineInputFormat. If you dont want to split at all, override
isSplitable in FileInputFormat. Or you can use FileInputFormat, get each
line as key/value and compute over it, saving the results and emitting only
as necessary.

I am not sure what your use case is, but I hope the above helps.

Best Regards,
Sonal
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>
On Thu, Nov 7, 2013 at 11:44 AM, unmesha sreeveni <[EMAIL PROTECTED]>wrote:

> Am i able to get the entire split data from mapper. i dnt need as line by
> line.
>
> my input is of say 50 lines.so these files can be splited into different
> mappers right. how to get each split data. are we able to get that data?
>
>
> On Thu, Nov 7, 2013 at 11:39 AM, Sonal Goyal <[EMAIL PROTECTED]>wrote:
>
>> Hi Unmesha,
>>
>> What is the computation you are trying to do? If you are interested in
>> computing over multiple lines instead of a single line, have a look at
>> NLineInputFormat.
>>
>> Best Regards,
>> Sonal
>> Nube Technologies <http://www.nubetech.co>
>>
>> <http://in.linkedin.com/in/sonalgoyal>
>>
>>
>>
>>
>> On Thu, Nov 7, 2013 at 11:35 AM, unmesha sreeveni <[EMAIL PROTECTED]>wrote:
>>
>>> one more doubt : how to copy each input split entering into mapper into
>>> a file for computation?
>>>
>>>
>>> On Thu, Nov 7, 2013 at 10:35 AM, unmesha sreeveni <[EMAIL PROTECTED]
>>> > wrote:
>>>
>>>> My driver code is
>>>>                 FileInputFormat.setInputPaths(job, new Path(args[0]));
>>>>  FileOutputFormat.setOutputPath(job,new Path(args[1]));
>>>>
>>>> and My mapper is
>>>>         public void map(Object key, Text value, Context context)
>>>> throws IOException, InterruptedException {
>>>> where value.tostring() contains my input data.
>>>>
>>>> is that a better way to copy all the data s coming into a file and do
>>>> computations. OR read each line and do the calculation.
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>>
>>>> Unmesha Sreeveni U.B
>>>>
>>>> *Junior Developer*
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>>
>>> *Junior Developer*
>>>
>>> *Amrita Center For Cyber Security *
>>>
>>>
>>> * Amritapuri.www.amrita.edu/cyber/ <http://www.amrita.edu/cyber/>*
>>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
> *Amrita Center For Cyber Security *
>
>
> * Amritapuri.www.amrita.edu/cyber/ <http://www.amrita.edu/cyber/>*
>