Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Re: Working with Capacity Scheduler


+
Munna 2013-11-26, 15:33
+
Munna 2013-11-26, 20:14
+
Olivier Renault 2013-11-26, 20:46
+
Munna 2013-11-26, 20:54
Copy link to this message
-
Re: Working with Capacity Scheduler
Hi,

Can i able to get the solution on Capacity Scheduler ?
On Wed, Nov 27, 2013 at 2:24 AM, Munna <[EMAIL PROTECTED]> wrote:

> yes... acl's are not enforced.
>
>
> On Wed, Nov 27, 2013 at 2:16 AM, Olivier Renault <[EMAIL PROTECTED]
> > wrote:
>
>> Do you mean that acl are not being enforced?
>>
>> Olivier
>> On 26 Nov 2013 20:18, "Munna" <[EMAIL PROTECTED]> wrote:
>>
>>> Hi Olivier,
>>>
>>> have u hold any solution, why user are not identifying their queues?
>>>
>>> Thanks
>>> Munna
>>>
>>>
>>> On Tue, Nov 26, 2013 at 11:17 PM, Munna <[EMAIL PROTECTED]> wrote:
>>>
>>>> *mapred queue -list* result:
>>>>
>>>> [user@host ~]$ mapred queue -list
>>>> 13/11/26 09:37:48 INFO service.AbstractService:
>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
>>>> 13/11/26 09:37:48 INFO service.AbstractService:
>>>> Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
>>>> =====================>>>> Queue Name : exploration
>>>> Queue State : running
>>>> Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>>     =====================>>>>     Queue Name : a
>>>>     Queue State : running
>>>>     Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>>     =====================>>>>     Queue Name : b
>>>>     Queue State : running
>>>>     Scheduling Info : Capacity: 30.000002, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>>     =====================>>>>     Queue Name : c
>>>>     Queue State : running
>>>>     Scheduling Info : Capacity: 40.0, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>> =====================>>>> Queue Name : production
>>>> Queue State : running
>>>> Scheduling Info : Capacity: 70.0, MaximumCapacity: 1.0,
>>>> CurrentCapacity: 0.0
>>>>
>>>> and I saw in RM logs, mapred user try to find default queue, logs are
>>>> below:
>>>>
>>>> 2013-11-26 05:22:59,804 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application
>>>> with id 1 submitted by user mapred
>>>> 2013-11-26 05:22:59,808 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred
>>>> IP=10.10.10.1  OPERATION=Submit Application Request
>>>> TARGET=ClientRMService  RESULT=SUCCESS  APPID=application_1385472086605_0001
>>>> 2013-11-26 05:22:59,828 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
>>>> application_1385472086605_0001 State change from NEW to SUBMITTED
>>>> 2013-11-26 05:22:59,829 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService:
>>>> Registering appattempt_1385472086605_0001_000001
>>>> 2013-11-26 05:22:59,830 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
>>>> appattempt_1385472086605_0001_000001 State change from NEW to SUBMITTED
>>>> 2013-11-26 05:22:59,832 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl:
>>>> appattempt_1385472086605_0001_000001 State change from SUBMITTED to FAILED
>>>> 2013-11-26 05:22:59,836 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl:
>>>> application_1385472086605_0001 State change from SUBMITTED to FAILED
>>>> 2013-11-26 05:22:59,837 WARN
>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred
>>>> OPERATION=Application Finished - Failed TARGET=RMAppManager
>>>> RESULT=FAILURE  DESCRIPTION=App failed with state: FAILED
>>>> PERMISSIONS=Application appattempt_1385472086605_0001_000001 submitted by
>>>> user mapred to unknown queue: default APPID=application_1385472086605_0001
>>>> 2013-11-26 05:22:59,841 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
>>>> Removing info for app: application_1385472086605_0001
>>>> 2013-11-26 05:22:59,842 INFO
>>>> org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary:
>>>> appId=application_1385472086605_0001,name=select count(*) from
>>>> crc_exchange_rate(Stage-1),user=mapred,queue=default,state=FAILED,trackingUrl=hostname:8088/proxy/application_1385472086605_0001/,appMasterHost=N/A,startTime=1385472179799,finishTime=1385472179836

*Regards*

*Munna*
+
Olivier Renault 2013-11-26, 22:07
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB