Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Set the number of mapper in hive


Copy link to this message
-
Re: Set the number of mapper in hive
Philips Kokoh Prasetyo 2012-12-07, 09:38
Hi Nitin,

Thanks for the reply.
Do you mean using fair scheduler to separate job queue?
http://hadoop.apache.org/docs/r0.20.2/fair_scheduler.html
Regards,
Philips
On Fri, Dec 7, 2012 at 5:15 PM, Nitin Pawar <[EMAIL PROTECTED]> wrote:

> ways to handle this
> 1) create separate job queues for hive and hbase users on jt and allocate
> resources according to your needs
> 2) you can not actually limit how many maps can be launched as its decided
> on run time by looking at split size. If you want less number of maps to be
> launched then increase split size...also keep in mind you will need larger
> memory for maps
> On Dec 7, 2012 2:32 PM, "Philips Kokoh Prasetyo" <[EMAIL PROTECTED]>
> wrote:
>
>> Hi everyone,
>>
>> My cluster also runs HBase for real time processing. Hive query (on a big
>> table) occupies all the map tasks so that the other service cannot run
>> properly.
>> Does anyone know how to limit the number of running map in hive?
>> I see mapred.reduce.tasks in the configuration properties, but I don't
>> see the configuration for mapper.
>>
>> Thank you.
>>
>> Regards,
>> Philips
>>
>