Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Working with Capacity Scheduler


Copy link to this message
-
Re: Working with Capacity Scheduler
Hi,

Can i able to get the solution on Capacity Scheduler ?
On Wed, Nov 27, 2013 at 2:24 AM, Munna <[EMAIL PROTECTED]> wrote:

> yes... acl's are not enforced.
>
>
> On Wed, Nov 27, 2013 at 2:16 AM, Olivier Renault <[EMAIL PROTECTED]
> > wrote:
>
>> Do you mean that acl are not being enforced?
>>
>> Olivier
>> On 26 Nov 2013 20:18, "Munna" <[EMAIL PROTECTED]> wrote:
>>
>>> Hi Olivier,
>>>
>>> have u hold any solution, why user are not identifying their queues?
>>>
>>> Thanks
>>> Munna
>>>
>>>
>>> On Tue, Nov 26, 2013 at 11:17 PM, Munna <[EMAIL PROTECTED]> wrote:
>>>
>>>> *mapred queue -list* result:
>>>>
>>>> [user@host ~]$ mapred queue -list
>>>> 13/11/26 09:37:48 INFO service.AbstractService:
>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
>>>> 13/11/26 09:37:48 INFO service.AbstractService:
>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
>>>> =====================>>>> Queue Name : exploration
>>>> Queue State : running
>>>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>>     =====================>>>>     Queue Name : a
>>>>     Queue State : running
>>>>     Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>>     =====================>>>>     Queue Name : b
>>>>     Queue State : running
>>>>     Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>>     =====================>>>>     Queue Name : c
>>>>     Queue State : running
>>>>     Scheduling Info : Capacity: 40.0, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>> =====================>>>> Queue Name : production
>>>> Queue State : running
>>>> Scheduling Info : Capacity: 70.0, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>>
>>>> and I saw in RM logs, mapred user try to find default queue, logs are
>>>> below:
>>>>
>>>> 2013-11-26 05:22:59,804 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application
>>>> with id 1 submitted by user mapred
>>>> 2013-11-26 05:22:59,808 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred
>>>> IP=10.10.10.1  OPERATION=Submit Application Request
>>>> TARGET=ClientRMService  RESULT=SUCCESS  APPID=application_1385472086605_0001
>>>> 2013-11-26 05:22:59,828 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
>>>> application_1385472086605_0001 State change from NEW to SUBMITTED
>>>> 2013-11-26 05:22:59,829 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService:
>>>> Registering appattempt_1385472086605_0001_000001
>>>> 2013-11-26 05:22:59,830 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
>>>> appattempt_1385472086605_0001_000001 State change from NEW to SUBMITTED
>>>> 2013-11-26 05:22:59,832 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
>>>> appattempt_1385472086605_0001_000001 State change from SUBMITTED to FAILED
>>>> 2013-11-26 05:22:59,836 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
>>>> application_1385472086605_0001 State change from SUBMITTED to FAILED
>>>> 2013-11-26 05:22:59,837 WARN
>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred
>>>> OPERATION=Application Finished - Failed TARGET=RMAppManager
>>>> RESULT=FAILURE  DESCRIPTION=App failed with state: FAILED
>>>> PERMISSIONS=Application appattempt_1385472086605_0001_000001 submitted by
>>>> user mapred to unknown queue: default APPID=application_1385472086605_0001
>>>> 2013-11-26 05:22:59,841 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
>>>> Removing info for app: application_1385472086605_0001
>>>> 2013-11-26 05:22:59,842 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary:
>>>> appId=application_1385472086605_0001,name=select count(*) from
>>>> crc_exchange_rate(Stage-1),user=mapred,queue=default,state=FAILED,trackingUrl=hostname:8088/proxy/application_1385472086605_0001/,appMasterHost=N/A,startTime=1385472179799,finishTime=1385472179836

*Regards*

*Munna*