Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Hive and MR2


Copy link to this message
-
Hive and MR2
Hi,
I am using hive-0.13.0 and hadoop-2.4.0,

why I must set 'mapreduce.jobtracker.address' in yarn-site.xml? otherwise,
there are exceptions and job failed.

And, 'mapreduce.jobtracker.address' can be set to any value.

The following messages are gened without set 'mapreduce.jobtracker.address'.

Job output on the console:
Execution log at:
/tmp/test/test_20140507180505_bcd4d89f-017c-4cf4-81a3-5fa619de0ad0.log
Job running in-process (local Hadoop)
Hadoop job information for null: number of mappers: 1; number of reducers: 1
2014-05-07 18:06:25,782 null map = 0%,  reduce = 0%
2014-05-07 18:06:33,699 null map = 100%,  reduce = 0%
2014-05-07 18:06:34,774 null map = 0%,  reduce = 0%
2014-05-07 18:06:49,222 null map = 100%,  reduce = 100%
Ended Job = job_1399453944131_0006 with errors
Error during job, obtaining debugging information...

Container error:
2014-05-07 18:06:33,634 INFO [main]
org.apache.hadoop.hive.ql.exec.Utilities: No plan file found:
file:/tmp/test/hive_2014-05-07_18-06-08_349_1526907284076641211-1/-mr-10001/0a1c9ebe-cdb0-4adc-9e93-8f176019f19a/map.xml
2014-05-07 18:06:33,635 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:437)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:430)
at
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:587)
at
org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:168)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:409)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:167)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)