Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Working with Capacity Scheduler


Copy link to this message
-
Re: Working with Capacity Scheduler
I don't believe the distro should matter.

Could you confirm that you've got the following in capacity-scheduler.xml ?
( The permission are hierarchical, so you need to restrict the root queue )
  <property>
      <name>yarn.scheduler.capacity.root.acl_submit_applications</name>
      <value> </value>
  </property>

  <property>
      <name>yarn.scheduler.capacity.root.acl_administer_queue</name>
      <value> </value>

Thanks,
Olivier

On 26 November 2013 22:51, Munna <[EMAIL PROTECTED]> wrote:

> Hi Olivier,
>
> "yarn.acl.enable"  is enabled earlier, for your information i am using
> Cloudera Manager to manage the cluster. again same problem :(
>
>
>
>
> On Wed, Nov 27, 2013 at 3:37 AM, Olivier Renault <[EMAIL PROTECTED]
> > wrote:
>
>> Here is a working configuration.
>>
>> In yarn-site.xml, you'll need to enable
>>     <name>yarn.acl.enable</name>
>>     <value>true</value>
>>
>> Thanks,
>> Olivier
>>
>>
>> On 26 November 2013 21:48, Munna <[EMAIL PROTECTED]> wrote:
>>
>>> Hi,
>>>
>>> Can i able to get the solution on Capacity Scheduler ?
>>>
>>>
>>> On Wed, Nov 27, 2013 at 2:24 AM, Munna <[EMAIL PROTECTED]> wrote:
>>>
>>>> yes... acl's are not enforced.
>>>>
>>>>
>>>> On Wed, Nov 27, 2013 at 2:16 AM, Olivier Renault <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> Do you mean that acl are not being enforced?
>>>>>
>>>>> Olivier
>>>>> On 26 Nov 2013 20:18, "Munna" <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> Hi Olivier,
>>>>>>
>>>>>> have u hold any solution, why user are not identifying their queues?
>>>>>>
>>>>>> Thanks
>>>>>> Munna
>>>>>>
>>>>>>
>>>>>> On Tue, Nov 26, 2013 at 11:17 PM, Munna <[EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> *mapred queue -list* result:
>>>>>>>
>>>>>>> [user@host ~]$ mapred queue -list
>>>>>>> 13/11/26 09:37:48 INFO service.AbstractService:
>>>>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
>>>>>>> 13/11/26 09:37:48 INFO service.AbstractService:
>>>>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
>>>>>>> =====================>>>>>>> Queue Name : exploration
>>>>>>> Queue State : running
>>>>>>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>>>>> CurrentCapacity: 0.0
>>>>>>>     =====================>>>>>>>     Queue Name : a
>>>>>>>     Queue State : running
>>>>>>>     Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>>>>> CurrentCapacity: 0.0
>>>>>>>     =====================>>>>>>>     Queue Name : b
>>>>>>>     Queue State : running
>>>>>>>     Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>>>>> CurrentCapacity: 0.0
>>>>>>>     =====================>>>>>>>     Queue Name : c
>>>>>>>     Queue State : running
>>>>>>>     Scheduling Info : Capacity: 40.0, MaximumCapacity: 1.0,
>>>>>>> CurrentCapacity: 0.0
>>>>>>> =====================>>>>>>> Queue Name : production
>>>>>>> Queue State : running
>>>>>>> Scheduling Info : Capacity: 70.0, MaximumCapacity: 1.0,
>>>>>>> CurrentCapacity: 0.0
>>>>>>>
>>>>>>> and I saw in RM logs, mapred user try to find default queue, logs
>>>>>>> are below:
>>>>>>>
>>>>>>> 2013-11-26 05:22:59,804 INFO
>>>>>>> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application
>>>>>>> with id 1 submitted by user mapred
>>>>>>> 2013-11-26 05:22:59,808 INFO
>>>>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred
>>>>>>> IP=10.10.10.1  OPERATION=Submit Application Request
>>>>>>> TARGET=ClientRMService  RESULT=SUCCESS  APPID=application_1385472086605_0001
>>>>>>> 2013-11-26 05:22:59,828 INFO
>>>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
>>>>>>> application_1385472086605_0001 State change from NEW to SUBMITTED
>>>>>>> 2013-11-26 05:22:59,829 INFO
>>>>>>> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService:
>>>>>>> Registering appattempt_1385472086605_0001_000001
>>>>>>> 2013-11-26 05:22:59,830 INFO
>>>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
>>>>>>
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB