Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Mapper input as argument


Copy link to this message
-
Re: Mapper input as argument
If you dont need line by line but you want to get a number of lines
together, use NLineInputFormat. If you dont want to split at all, override
isSplitable in FileInputFormat. Or you can use FileInputFormat, get each
line as key/value and compute over it, saving the results and emitting only
as necessary.

I am not sure what your use case is, but I hope the above helps.

Best Regards,
Sonal
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>
On Thu, Nov 7, 2013 at 11:44 AM, unmesha sreeveni <[EMAIL PROTECTED]>wrote:

> Am i able to get the entire split data from mapper. i dnt need as line by
> line.
>
> my input is of say 50 lines.so these files can be splited into different
> mappers right. how to get each split data. are we able to get that data?
>
>
> On Thu, Nov 7, 2013 at 11:39 AM, Sonal Goyal <[EMAIL PROTECTED]>wrote:
>
>> Hi Unmesha,
>>
>> What is the computation you are trying to do? If you are interested in
>> computing over multiple lines instead of a single line, have a look at
>> NLineInputFormat.
>>
>> Best Regards,
>> Sonal
>> Nube Technologies <http://www.nubetech.co>
>>
>> <http://in.linkedin.com/in/sonalgoyal>
>>
>>
>>
>>
>> On Thu, Nov 7, 2013 at 11:35 AM, unmesha sreeveni <[EMAIL PROTECTED]>wrote:
>>
>>> one more doubt : how to copy each input split entering into mapper into
>>> a file for computation?
>>>
>>>
>>> On Thu, Nov 7, 2013 at 10:35 AM, unmesha sreeveni <[EMAIL PROTECTED]
>>> > wrote:
>>>
>>>> My driver code is
>>>>                 FileInputFormat.setInputPaths(job, new Path(args[0]));
>>>>  FileOutputFormat.setOutputPath(job,new Path(args[1]));
>>>>
>>>> and My mapper is
>>>>         public void map(Object key, Text value, Context context)
>>>> throws IOException, InterruptedException {
>>>> where value.tostring() contains my input data.
>>>>
>>>> is that a better way to copy all the data s coming into a file and do
>>>> computations. OR read each line and do the calculation.
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>>
>>>> Unmesha Sreeveni U.B
>>>>
>>>> *Junior Developer*
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>>
>>> *Junior Developer*
>>>
>>> *Amrita Center For Cyber Security *
>>>
>>>
>>> * Amritapuri.www.amrita.edu/cyber/ <http://www.amrita.edu/cyber/>*
>>>
>>
>>
>
>
> --
> *Thanks & Regards*
>
> Unmesha Sreeveni U.B
>
> *Junior Developer*
>
> *Amrita Center For Cyber Security *
>
>
> * Amritapuri.www.amrita.edu/cyber/ <http://www.amrita.edu/cyber/>*
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB