Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> issue about running hive MR job in hadoop


Copy link to this message
-
Re: issue about running hive MR job in hadoop
yes i checked the code ,and find the Exception from

     lfs.mkdir(userFileCacheDir, null, false);

also find the AM located in CHBM224 ,all will failed but,AM located in
CHBM223,all success

in CHBM224

# ls -l /data/mrlocal/1/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov  5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan  3 15:57 logs
# ls -l /data/mrlocal/2/yarn/
total 8
drwxrwxrwx 5 yarn yarn 4096 Nov  5 20:50 local
drwxr-xr-x 3 yarn yarn 4096 Jan  3 15:57 logs

in CHBM223

# ls /data/mrlocal/1/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov  5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan  3 15:46 logs
# ls /data/mrlocal/2/yarn/ -l
total 8
drwxr-xr-x 5 yarn yarn 4096 Nov  5 20:51 local
drwxr-xr-x 3 yarn yarn 4096 Jan  3 15:46 logs
i also find if i let abnormal node (CHBM224) run and shutdown the other
normal node ,when i submit a MR job use hive ,and the dir
"/data/mrlocal/2/yarn/local/usercache/hive/filecache" ,it's mode will flush
to 710 ,even i change the file to 755,but i test on a normal node (open one
normal node ,and shutdown others) ,the dir mode will not changed
# ls -l /data/mrlocal/2/yarn/local/usercache/hive/
total 16
drwx--x---   7 yarn yarn  4096 Jan  3 16:30 appcache
drwx--x--- 148 yarn yarn 12288 Jan  3 10:03 filecache
On Fri, Jan 3, 2014 at 3:52 PM, Bing Jiang <[EMAIL PROTECTED]> wrote:

>  Could you check your yarn-local directory authority? From the diagnosis,
> error occurs at mkdir in local directory.
> I guess something wrong with  local direcotry which is set as yarn local
> dir.
>
>
>
> 2014/1/3 ch huang <[EMAIL PROTECTED]>
>
>> hi, i submit a MR job through hive ,but when it run stage-2 ,it failed
>> why?
>>
>>  it seems permission problem ,but i do not know which dir cause the
>> problem
>>
>> Application application_1388730279827_0035 failed 1 times due to AM
>> Container for appattempt_1388730279827_0035_000001 exited with exitCode:
>> -1000 due to: EPERM: Operation not permitted at
>> org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method) at
>> org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:581)
>> at
>> org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:388)
>> at org.apache.hadoop.fs.FileSystem.primitiveMkdir(FileSystem.java:1041) at
>> org.apache.hadoop.fs.DelegateToFileSystem.mkdir(DelegateToFileSystem.java:150)
>> at org.apache.hadoop.fs.FilterFs.mkdir(FilterFs.java:190) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:698) at
>> org.apache.hadoop.fs.FileContext$4.next(FileContext.java:695) at
>> org.apache.hadoop.fs.FileContext$FSLinkResolver.resolve(FileContext.java:2325)
>> at org.apache.hadoop.fs.FileContext.mkdir(FileContext.java:695) at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:385)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:130)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
>> at
>> org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:861)
>> .Failing this attempt.. Failing the application.
>>
>
>
>
> --
> Bing Jiang
> Tel:(86)134-2619-1361
> weibo: http://weibo.com/jiangbinglover
> BLOG: www.binospace.com
> BLOG: http://blog.sina.com.cn/jiangbinglover
> Focus on distributed computing, HDFS/HBase
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB