Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - execute hive query error "Error: GC overhead limit exceeded"


Copy link to this message
-
execute hive query error "Error: GC overhead limit exceeded"
ch huang 2013-08-22, 02:06
HI,ALL:
i execute a query ,but error,any one know what happened? BTW i use yarn
framework

2013-08-22 09:47:09,893 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4140.64 sec
2013-08-22 09:47:10,952 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4140.72 sec
2013-08-22 09:47:12,008 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4140.8 sec
2013-08-22 09:47:13,066 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4147.85 sec
2013-08-22 09:47:14,122 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4162.26 sec
2013-08-22 09:47:15,181 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4162.33 sec
2013-08-22 09:47:16,238 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4162.33 sec
2013-08-22 09:47:17,297 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4162.41 sec
2013-08-22 09:47:18,353 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4162.5 sec
2013-08-22 09:47:19,410 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4162.5 sec
2013-08-22 09:47:20,470 Stage-1 map = 28%,  reduce = 1%, Cumulative CPU
4066.31 sec
MapReduce Total cumulative CPU time: 0 days 1 hours 7 minutes 46 seconds
310 msec
Ended Job = job_1376355583846_0115 with errors
Error during job, obtaining debugging information...
Examining task ID: task_1376355583846_0115_m_000088 (and more) from job
job_1376355583846_0115
Examining task ID: task_1376355583846_0115_m_000092 (and more) from job
job_1376355583846_0115
Examining task ID: task_1376355583846_0115_m_000013 (and more) from job
job_1376355583846_0115
Examining task ID: task_1376355583846_0115_m_000105 (and more) from job
job_1376355583846_0115
Examining task ID: task_1376355583846_0115_m_000021 (and more) from job
job_1376355583846_0115
Examining task ID: task_1376355583846_0115_m_000101 (and more) from job
job_1376355583846_0115

Task with the most failures(4):
-----
Task ID:
  task_1376355583846_0115_m_000018

URL:

http://CH22:8088/taskdetails.jsp?jobid=job_1376355583846_0115&tipid=task_1376355583846_0115_m_000018<http://ch22:8088/taskdetails.jsp?jobid=job_1376355583846_0115&tipid=task_1376355583846_0115_m_000018>
-----
Diagnostic Messages for this Task:
Error: GC overhead limit exceeded
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 149  Reduce: 41   Cumulative CPU: 4066.31 sec   HDFS Read:
13504652062 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 days 1 hours 7 minutes 46 seconds 310 msec

loginfo
# less hive-server2.log
2013-08-22 09:30:55,834 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,837 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,838 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,839 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,842 ERROR exec.Task (SessionState.java:printError(421))
- Examining task ID: task_1376355583846_0114_m_000009 (and more) from job
job_1376355583846_0114
2013-08-22 09:30:55,843 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,844 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,844 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,845 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,846 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,847 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,848 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,849 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,850 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,851 WARN  shims.HadoopShimsSecure
(Hadoop23Shims.java:getTaskAttemptLogUrl(45)) - Can't fetch tasklog:
TaskLogServlet is not supported in MR2 mode.
2013-08-22 09:30:55,854 ERROR exec.Task (SessionState.java:printError(421))
-
Task with the most failures(4):
Task ID:
  task_1376355583846_0114_m_000018
URL:

http://CH22:8088/taskdetails.jsp?jobid=job_1376355583846_0114&tipid=task_1376355583846_0114_m_000018<http://ch22:8088/taskdetails.jsp?jobid=job_1376355583846_0114&tipid=task_1376355583846_0114_m_000018>
Diagnostic Messages for this Task:
Error: GC overhead limit exceeded
2013-08-22 09:30:55,885 ERROR ql.Driver (SessionState.java:printError(421))
- FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.MapRedTask
2013-08-22 09:30:55,885 INFO  ql.Driver (PerfLogger.java:PerfLogEnd(127)) -
</PERFLOG method=Driver.execute start=1377134404557 end=1377135055885
duration=651328>
2013-08-22 09:30:55,886 INFO  ql.Driver (SessionState.java:printInfo(412))
- MapReduce Jobs Launched:
2013-08-22 09:30:55,886 INFO  ql.Driver (SessionState.java:printInfo(412))
- Job 0: Map: 149  Reduce: 41   Cumulative CPU: 3750.17 sec   HDFS Read:
12206464928