Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Regression in trunk? (RE: Insert overwrite error using hive trunk)


Copy link to this message
-
RE: Regression in trunk? (RE: Insert overwrite error using hive trunk)
Pradeep Kamath 2010-09-28, 17:30
Should I open a jira for this? So far it seems like a regression.

Pradeep
________________________________
From: Pradeep Kamath [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, September 28, 2010 9:32 AM
To: [EMAIL PROTECTED]
Subject: Re: Regression in trunk? (RE: Insert overwrite error using hive trunk)

With "hive -hiveconf hive.root.logger=DEBUG,DRFA -e ... " /tmp/<username>/hive.log seems to have pretty detailed log messages including debug msgs. I don't see the "initialization failed" message and the stack trace mentioned in HADOOP-5759 - is there any other place I need to check. On the UI I only see map task in pending state and no further information (this is with hadoop-0.20.1). With a more recent hadoop I see no tasks launched at all. This used to work a month before - am wondering if any changes in hive caused this.

Thanks,
Pradeep
Amareshwari Sri Ramadasu wrote:
Pradeep, you might be hitting HADOOP-5759 and the job is not getting initialized at all. Look in JobTracker logs for the jobid to confirm the same.

On 9/28/10 6:28 AM, "Pradeep Kamath" <[EMAIL PROTECTED]> wrote:
Here is some relevant stuff from /tmp/pradeepk/hive.logs - can't make much out of it:

2010-09-27 17:40:01,081 INFO  exec.MapRedTask (SessionState.java:printInfo(268)) - Starting Job = job_201009251752_1341, Tracking URL = http://<hostname>:50030/jobdetails.jsp?jobid=job_201009251752_1341<http://%3Chostname%3E:50030/jobdetails.jsp?jobid=job_201009251752_1341>
2010-09-27 17:40:01,081 INFO  exec.MapRedTask (SessionState.java:printInfo(268)) - Kill Command = /homes/pradeepk/hadoopcluster/hadoop/bin/../bin/hadoop job -Dmapred.job.tracker=<hostname>:50020 -kill job_201009251752_1341
2010-09-27 17:40:01,081 DEBUG ipc.Client (Client.java:sendParam(469)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk sending #129
2010-09-27 17:40:01,083 DEBUG ipc.Client (Client.java:receiveResponse(504)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk got value #129
2010-09-27 17:40:01,083 DEBUG ipc.RPC (RPC.java:invoke(225)) - Call: getJobStatus 2
2010-09-27 17:40:02,086 DEBUG ipc.Client (Client.java:sendParam(469)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk sending #130
2010-09-27 17:40:02,090 DEBUG ipc.Client (Client.java:receiveResponse(504)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk got value #130
2010-09-27 17:40:02,091 DEBUG ipc.RPC (RPC.java:invoke(225)) - Call: getJobStatus 5
2010-09-27 17:40:02,092 DEBUG ipc.Client (Client.java:sendParam(469)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk sending #131
2010-09-27 17:40:02,093 DEBUG ipc.Client (Client.java:receiveResponse(504)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk got value #131
2010-09-27 17:40:02,094 DEBUG ipc.RPC (RPC.java:invoke(225)) - Call: getJobStatus 2
2010-09-27 17:40:02,094 DEBUG ipc.Client (Client.java:sendParam(469)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk sending #132
2010-09-27 17:40:02,096 DEBUG ipc.Client (Client.java:receiveResponse(504)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk got value #132
2010-09-27 17:40:02,096 DEBUG ipc.RPC (RPC.java:invoke(225)) - Call: getJobProfile 2
2010-09-27 17:40:02,096 DEBUG ipc.Client (Client.java:sendParam(469)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk sending #133
2010-09-27 17:40:02,100 DEBUG ipc.Client (Client.java:receiveResponse(504)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk got value #133
2010-09-27 17:40:02,100 DEBUG ipc.RPC (RPC.java:invoke(225)) - Call: getJobCounters 4
2010-09-27 17:40:02,101 DEBUG mapred.Counters (Counters.java:<init>(151)) - Creating group org.apache.hadoop.hive.ql.exec.Operator$ProgressCounter with nothing
2010-09-27 17:40:02,101 DEBUG mapred.Counters (Counters.java:getCounterForName(277)) - Adding CREATED_FILES
2010-09-27 17:40:02,103 INFO  exec.MapRedTask (SessionState.java:printInfo(268)) - 2010-09-27 17:40:02,103 Stage-2 map = 100%,  reduce = 100%
2010-09-27 17:40:02,104 DEBUG ipc.Client (Client.java:sendParam(469)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk sending #134
2010-09-27 17:40:02,105 DEBUG ipc.Client (Client.java:receiveResponse(504)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk got value #134
2010-09-27 17:40:02,106 DEBUG ipc.RPC (RPC.java:invoke(225)) - Call: getJobStatus 2
2010-09-27 17:40:02,106 DEBUG ipc.Client (Client.java:sendParam(469)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk sending #135
2010-09-27 17:40:02,108 DEBUG ipc.Client (Client.java:receiveResponse(504)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk got value #135
2010-09-27 17:40:02,108 DEBUG ipc.RPC (RPC.java:invoke(225)) - Call: getJobCounters 2
2010-09-27 17:40:02,109 DEBUG mapred.Counters (Counters.java:<init>(151)) - Creating group org.apache.hadoop.hive.ql.exec.Operator$ProgressCounter with nothing
2010-09-27 17:40:02,109 DEBUG mapred.Counters (Counters.java:getCounterForName(277)) - Adding CREATED_FILES
2010-09-27 17:40:02,109 DEBUG ipc.Client (Client.java:sendParam(469)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk sending #136
2010-09-27 17:40:02,111 DEBUG ipc.Client (Client.java:receiveResponse(504)) - IPC Client (47) connection to <hostname>/216.252.118.203:50020 from pradeepk got value #136
2010-09-27 17:40:02,111 DEBUG ipc.RPC (RPC.java:invoke(225)) - Call: getJobStatus 2
2010-09-27 17:40:02,112 ERROR exec.MapRedTask (SessionState.java:printError(277)) - Ended Job = job_201009251752_1341 with errors

Ning Zhang wrote:
On Sep 27, 2010, at 1:11 PM, Steven Wong wrote: