Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Working with Capacity Scheduler


Copy link to this message
-
Re: Working with Capacity Scheduler
Here is a working configuration.

In yarn-site.xml, you'll need to enable
    <name>yarn.acl.enable</name>
    <value>true</value>

Thanks,
Olivier
On 26 November 2013 21:48, Munna <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Can i able to get the solution on Capacity Scheduler ?
>
>
> On Wed, Nov 27, 2013 at 2:24 AM, Munna <[EMAIL PROTECTED]> wrote:
>
>> yes... acl's are not enforced.
>>
>>
>> On Wed, Nov 27, 2013 at 2:16 AM, Olivier Renault <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Do you mean that acl are not being enforced?
>>>
>>> Olivier
>>> On 26 Nov 2013 20:18, "Munna" <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi Olivier,
>>>>
>>>> have u hold any solution, why user are not identifying their queues?
>>>>
>>>> Thanks
>>>> Munna
>>>>
>>>>
>>>> On Tue, Nov 26, 2013 at 11:17 PM, Munna <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> *mapred queue -list* result:
>>>>>
>>>>> [user@host ~]$ mapred queue -list
>>>>> 13/11/26 09:37:48 INFO service.AbstractService:
>>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
>>>>> 13/11/26 09:37:48 INFO service.AbstractService:
>>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
>>>>> =====================>>>>> Queue Name : exploration
>>>>> Queue State : running
>>>>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>>> CurrentCapacity: 0.0
>>>>>     =====================>>>>>     Queue Name : a
>>>>>     Queue State : running
>>>>>     Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>>> CurrentCapacity: 0.0
>>>>>     =====================>>>>>     Queue Name : b
>>>>>     Queue State : running
>>>>>     Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>>> CurrentCapacity: 0.0
>>>>>     =====================>>>>>     Queue Name : c
>>>>>     Queue State : running
>>>>>     Scheduling Info : Capacity: 40.0, MaximumCapacity: 1.0,
>>>>> CurrentCapacity: 0.0
>>>>> =====================>>>>> Queue Name : production
>>>>> Queue State : running
>>>>> Scheduling Info : Capacity: 70.0, MaximumCapacity: 1.0,
>>>>> CurrentCapacity: 0.0
>>>>>
>>>>> and I saw in RM logs, mapred user try to find default queue, logs are
>>>>> below:
>>>>>
>>>>> 2013-11-26 05:22:59,804 INFO
>>>>> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application
>>>>> with id 1 submitted by user mapred
>>>>> 2013-11-26 05:22:59,808 INFO
>>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred
>>>>> IP=10.10.10.1  OPERATION=Submit Application Request
>>>>> TARGET=ClientRMService  RESULT=SUCCESS  APPID=application_1385472086605_0001
>>>>> 2013-11-26 05:22:59,828 INFO
>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
>>>>> application_1385472086605_0001 State change from NEW to SUBMITTED
>>>>> 2013-11-26 05:22:59,829 INFO
>>>>> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService:
>>>>> Registering appattempt_1385472086605_0001_000001
>>>>> 2013-11-26 05:22:59,830 INFO
>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
>>>>> appattempt_1385472086605_0001_000001 State change from NEW to SUBMITTED
>>>>> 2013-11-26 05:22:59,832 INFO
>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
>>>>> appattempt_1385472086605_0001_000001 State change from SUBMITTED to FAILED
>>>>> 2013-11-26 05:22:59,836 INFO
>>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
>>>>> application_1385472086605_0001 State change from SUBMITTED to FAILED
>>>>> 2013-11-26 05:22:59,837 WARN
>>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred
>>>>> OPERATION=Application Finished - Failed TARGET=RMAppManager
>>>>> RESULT=FAILURE  DESCRIPTION=App failed with state: FAILED
>>>>> PERMISSIONS=Application appattempt_1385472086605_0001_000001 submitted by
>>>>> user mapred to unknown queue: default APPID=application_1385472086605_0001
>>>>> 2013-11-26 05:22:59,841 INFO
>>>>> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:

   * Olivier Renault *       Solution Engineer

    Phone:        +44 7500 933 036
  Email:      [EMAIL PROTECTED]
  Website:   http://www.hortonworks.com/

      * Follow Us: *
<http://facebook.com/hortonworks/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://twitter.com/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>
<http://www.linkedin.com/company/hortonworks?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>

 [image: photo]

  Latest From Our Blog:  SAP HANA + Hadoop: A Perfect Match
<http://hortonworks.com/blog/sap-hana-hadoop-a-perfect-match/?utm_source=WiseStamp&utm_medium=email&utm_term=&utm_content=&utm_campaign=signature>

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB