Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> JobDebugger bug / wrong Hadoop version


Copy link to this message
-
JobDebugger bug / wrong Hadoop version
Hi all,

I'm running into a problem - I'm using Hive trunk (pretty recent, but
I see the bug's in trunk at time of writing as well), with Hadoop
0.20.205.0. I have a job which fails (for a reason which is entirely
my own fault), and when it does fail Hive dies with the following
exception:

Ended Job = job_201202291327_1399 with errors
Error during job, obtaining debugging information...
Examining task ID: task_201202291327_1399_m_000003 (and more) from job
job_201202291327_1399
Examining task ID: task_201202291327_1399_r_000002 (and more) from job
job_201202291327_1399
Exception in thread "Thread-342" java.lang.RuntimeException: Error
while reading from task log url
        at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
        at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.io.IOException: Server returned HTTP response code:
400 for URL: http://...:50060/tasklog?taskid=attempt_201202291327_1399_r_000002_2&start=-8193
        at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
        at java.net.URL.openStream(URL.java:1010)
        at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
        ... 3 more

When I point a web browser at the given URL, I get the following message:

HTTP ERROR 400

Problem accessing /tasklog. Reason:

    Argument attemptid is required

If I change taskid to attemptid, everything works perfectly. The code
which generates this URL appears to be in
org.apache.hadoop.hive.ql.exec.JobDebugger.java. I presume that this
code is correct for a given version of Hadoop. Which version is
currently in use in the Jenkins/Hudson build environment? I'd be happy
to change it and submit a patch to JIRA, but I guess that'd probably
break the other version of Hadoop, so perhaps some more profound
versioning type thing might be needed.

This is the issue that HIVE-1579 is referring to.

Cheers,

Phil.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB