Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: What is the preferred way to pass a small number of configuration parameters to a mapper or reducer


Copy link to this message
-
Re: What is the preferred way to pass a small number of configuration parameters to a mapper or reducer
Z. Implement passing simple small objects in the most complicated manner
possible try JPOX, ontop of hbase, configured by smartfrog and puppet with
environments, on heroku.

On Mon, Dec 31, 2012 at 3:17 AM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:

> *G*. Use cascading so that way you don't have to actually provide the
> parameters yourself because there is a transparent serialization of what
> will become the mapper and the reducer. (but it is really a hidden kind-of
> *A*).
>
> http://www.cascading.org/
>
> About certifications, of course, cheating is not allowed. And if you are
> indeed cheating, you are open the 'retributions' you agreed on.
> But at the same time, you can find online resources which are plain wrong.
> I think the only good answers are
> *1)* use the API and figure it yourself
> *2)* Do not trust everybody (even well intentioned people can be wrong
> and the same can be said about public opinion)
> *3)* read a good reference (like http://hadoopbook.com/)
>
> The mailing list could have a rule stating the such post are not allowed.
> It really looks like a copy-and-paste from somewhere. Any author should
> provide more context if there really is a point which is not understood.
>
> Bertrand
>
>
> On Sun, Dec 30, 2012 at 8:38 PM, Niels Basjes <[EMAIL PROTECTED]> wrote:
>
>> F. put a mongodb replica set on all hadoop workernodes and let the tasks
>> query the mongodb at localhost.
>>
>> (this is what I did recently with a multi GiB dataset)
>>
>> --
>> Met vriendelijke groet,
>> Niels Basjes
>> (Verstuurd vanaf mobiel )
>> Op 30 dec. 2012 20:01 schreef "Jonathan Bishop" <[EMAIL PROTECTED]>
>> het volgende:
>>
>> E. Store them in hbase...
>>>
>>>
>>> On Sun, Dec 30, 2012 at 12:24 AM, Hemanth Yamijala <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> If it is a small number, A seems the best way to me.
>>>>
>>>> On Friday, December 28, 2012, Kshiva Kps wrote:
>>>>
>>>>>
>>>>> Which one is current ..
>>>>>
>>>>>
>>>>> What is the preferred way to pass a small number of configuration
>>>>> parameters to a mapper or reducer?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *A.  *As key-value pairs in the jobconf object.
>>>>>
>>>>> * *
>>>>>
>>>>> *B.  *As a custom input key-value pair passed to each mapper or
>>>>> reducer.
>>>>>
>>>>> * *
>>>>>
>>>>> *C.  *Using a plain text file via the Distributedcache, which each
>>>>> mapper or reducer reads.
>>>>>
>>>>> * *
>>>>>
>>>>> *D.  *Through a static variable in the MapReduce driver class (i.e.,
>>>>> the class that submits the MapReduce job).
>>>>>
>>>>>
>>>>>
>>>>> *Answer: B*
>>>>>
>>>>>
>>>>>
>>>>
>>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB