Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: What is the preferred way to pass a small number of configuration parameters to a mapper or reducer


Copy link to this message
-
Re: What is the preferred way to pass a small number of configuration parameters to a mapper or reducer
Niels Basjes 2012-12-30, 19:38
F. put a mongodb replica set on all hadoop workernodes and let the tasks
query the mongodb at localhost.

(this is what I did recently with a multi GiB dataset)

--
Met vriendelijke groet,
Niels Basjes
(Verstuurd vanaf mobiel )
Op 30 dec. 2012 20:01 schreef "Jonathan Bishop" <[EMAIL PROTECTED]> het
volgende:

> E. Store them in hbase...
>
>
> On Sun, Dec 30, 2012 at 12:24 AM, Hemanth Yamijala <
> [EMAIL PROTECTED]> wrote:
>
>> If it is a small number, A seems the best way to me.
>>
>> On Friday, December 28, 2012, Kshiva Kps wrote:
>>
>>>
>>> Which one is current ..
>>>
>>>
>>> What is the preferred way to pass a small number of configuration
>>> parameters to a mapper or reducer?
>>>
>>>
>>>
>>>
>>>
>>> *A.  *As key-value pairs in the jobconf object.
>>>
>>> * *
>>>
>>> *B.  *As a custom input key-value pair passed to each mapper or
>>> reducer.
>>>
>>> * *
>>>
>>> *C.  *Using a plain text file via the Distributedcache, which each
>>> mapper or reducer reads.
>>>
>>> * *
>>>
>>> *D.  *Through a static variable in the MapReduce driver class (i.e.,
>>> the class that submits the MapReduce job).
>>>
>>>
>>>
>>> *Answer: B*
>>>
>>>
>>>
>>
>