Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Set the number of mapper in hive


Copy link to this message
-
Re: Set the number of mapper in hive
Philips Kokoh Prasetyo 2012-12-07, 11:44
Thank you very much. I will check on that.

On Fri, Dec 7, 2012 at 5:48 PM, Nitin Pawar <[EMAIL PROTECTED]> wrote:

> Yes
> On Dec 7, 2012 3:09 PM, "Philips Kokoh Prasetyo" <[EMAIL PROTECTED]>
> wrote:
>
>> Hi Nitin,
>>
>> Thanks for the reply.
>> Do you mean using fair scheduler to separate job queue?
>> http://hadoop.apache.org/docs/r0.20.2/fair_scheduler.html
>>
>>
>> Regards,
>> Philips
>>
>>
>> On Fri, Dec 7, 2012 at 5:15 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>
>>> ways to handle this
>>> 1) create separate job queues for hive and hbase users on jt and
>>> allocate resources according to your needs
>>> 2) you can not actually limit how many maps can be launched as its
>>> decided on run time by looking at split size. If you want less number of
>>> maps to be launched then increase split size...also keep in mind you will
>>> need larger memory for maps
>>>  On Dec 7, 2012 2:32 PM, "Philips Kokoh Prasetyo" <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> My cluster also runs HBase for real time processing. Hive query (on a
>>>> big table) occupies all the map tasks so that the other service cannot run
>>>> properly.
>>>> Does anyone know how to limit the number of running map in hive?
>>>> I see mapred.reduce.tasks in the configuration properties, but I don't
>>>> see the configuration for mapper.
>>>>
>>>> Thank you.
>>>>
>>>> Regards,
>>>> Philips
>>>>
>>>
>>