Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Working with Capacity Scheduler


Copy link to this message
-
Re: Working with Capacity Scheduler
Hi olivier,

Sorry to again, same error: Job error given below

[mapred@host~]$ hive -e "use databse; select count(*) from table"
Logging initialized using configuration in
jar:file:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hive/lib/hive-common-0.10.0-cdh4.4.0.jar!/hive-log4j.properties
Hive history
file=/tmp/mapred/hive_job_log_868a2d7a-1df8-4a8a-b09d-4fad10d3f1b4_1410849451.txt
OK
Time taken: 1.616 seconds
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_1385510241043_0001, Tracking URL http://host:8088/proxy/application_1385510241043_0001/
Kill Command /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/bin/hadoop job
-kill job_1385510241043_0001
Ended Job = job_1385510241043_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL:
http://hpdl-R306-16:8088/proxy/application_1385510241043_0001/
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0:  HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

On Wed, Nov 27, 2013 at 5:01 AM, Olivier Renault
<[EMAIL PROTECTED]>wrote:

> Yes you need to forbid every users on your root queue
>
>   <property>
>       <name>yarn.scheduler.capacity.root.acl_submit_applications</name>
>       <value> </value>
>   </property>
>
>   <property>
>       <name>yarn.scheduler.capacity.root.acl_administer_queue</name>
>       <value> </value>
>   </property>
>
> Olivier
>
>
> On 26 November 2013 23:29, Munna <[EMAIL PROTECTED]> wrote:
>
>> I have configured acls like this:
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *<property>  <name>yarn.scheduler.capacity.root.
>> production.acl_submit_applications</name>
>> <value>yarn,mapred</value></property><property>    <value>userb</value>
>> <name>yarn.scheduler.capacity.root.exploration.a.acl_submit_applications</name>
>> </property><property>
>> <name>yarn.scheduler.capacity.root.exploration.b.acl_submit_applications</name>
>> <value>userb</value></property><property>
>> <name>yarn.scheduler.capacity.root.exploration.c.acl_submit_applications</name>
>> <value>userc</value></property>*
>>  Is any thing missing on this?
>>
>> Thanks
>>
>>
>> On Wed, Nov 27, 2013 at 4:31 AM, Olivier Renault <
>> [EMAIL PROTECTED]> wrote:
>>
>>> I don't believe the distro should matter.
>>>
>>> Could you confirm that you've got the following in
>>> capacity-scheduler.xml ? ( The permission are hierarchical, so you need to
>>> restrict the root queue )
>>>   <property>
>>>       <name>yarn.scheduler.capacity.root.acl_submit_applications</name>
>>>       <value> </value>
>>>   </property>
>>>
>>>   <property>
>>>       <name>yarn.scheduler.capacity.root.acl_administer_queue</name>
>>>       <value> </value>
>>>
>>> Thanks,
>>> Olivier
>>>
>>>
>>> On 26 November 2013 22:51, Munna <[EMAIL PROTECTED]> wrote:
>>>
>>>> Hi Olivier,
>>>>
>>>> "yarn.acl.enable"  is enabled earlier, for your information i am using
>>>> Cloudera Manager to manage the cluster. again same problem :(
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Nov 27, 2013 at 3:37 AM, Olivier Renault <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>> Here is a working configuration.
>>>>>
>>>>> In yarn-site.xml, you'll need to enable
>>>>>     <name>yarn.acl.enable</name>
>>>>>     <value>true</value>
>>>>>
>>>>> Thanks,
>>>>> Olivier
>>>>>
>>>>>
>>>>> On 26 November 2013 21:48, Munna <[EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Can i able to get the solution on Capacity Scheduler ?
>>>>>>
>>>>>>
>>>>>> On Wed, Nov 27, 2013 at 2:24 AM, Munna <[EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> yes... acl's are not enforced.

*Regards*

*Munna*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB