Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Hive Query


Copy link to this message
-
RE: Hive Query
yogesh.kumar13@... 2012-07-24, 09:26
Thanks Bejoy :-)

I have an error Issue with

select count(*) from table;

it throws error

2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201207231123_0011 with errors
Error during job, obtaining debugging information...
Examining task ID: task_201207231123_0011_m_000002 (and more) from job job_201207231123_0011
Exception in thread "Thread-93" java.lang.RuntimeException: Error while reading from task log url
    at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
    at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Server returned HTTP response code: 407 for URL: http://10.203.33.81:50060/tasklog?taskid=attempt_201207231123_0011_r_000000_0&start=-8193
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
    at java.net.URL.openStream(URL.java:1010)
    at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
    ... 3 more
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1  Reduce: 1   HDFS Read: 24 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec

and I run query

SELECT count(*),sub.name FROM (Select * FROM sitealias JOIN site ON (sitealias.site_id = site.site_id) ) sub GROUP BY sub.name;

it went into loop and still Map-Reduce process going on.

Total MapReduce jobs = 2
Launching Job 1 out of 2
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201207231123_0018, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0018
Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0018
Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1
2012-07-24 14:42:03,824 Stage-1 map = 0%,  reduce = 0%
2012-07-24 14:42:09,850 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:43:10,030 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:44:10,177 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:45:10,358 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:46:10,516 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:47:10,672 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:48:10,882 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:49:11,016 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:50:11,152 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:51:11,409 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:52:11,550 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:53:11,679 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:54:11,807 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:55:11,935 Stage-1 map = 100%,  reduce = 0%
2012-07-24 14:56:12,060 Stage-1 map = 100%,  reduce = 0%
from past 10 minutes and still on...
Please suggest and Help

Thanks & Regards
Yogesh Kumar

________________________________
From: Bejoy Ks [[EMAIL PROTECTED]]
Sent: Tuesday, July 24, 2012 2:33 PM
To: [EMAIL PROTECTED]
Subject: Re: Hive Query

Hi Yogesh

Try out this query, it should work though it is little expensive

SELECT count(*),sub.name FROM (Select * FROM sitealias JOIN site ON (sitealias.site_id = site.site_id) ) sub GROUP BY sub.name;
Regards
Bejoy KS

________________________________
From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]; [EMAIL PROTECTED]
Sent: Tuesday, July 24, 2012 1:39 PM
Subject: RE: Hive Query

Hi Bejoy,

even If if perform count(*) operation on table it shows error,

select count(*) from dummysite;
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201207231123_0011, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201207231123_0011
Kill Command = /HADOOP/hadoop-0.20.2/bin/../bin/hadoop job  -Dmapred.job.tracker=localhost:9001 -kill job_201207231123_0011
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2012-07-24 13:38:18,928 Stage-1 map = 0%,  reduce = 0%
2012-07-24 13:38:21,938 Stage-1 map = 100%,  reduce = 0%
2012-07-24 13:39:22,170 Stage-1 map = 100%,  reduce = 0%
2012-07-24 13:39:25,181 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201207231123_0011 with errors
Error during job, obtaining debugging information...
Examining task ID: task_201207231123_0011_m_000002 (and more) from job job_201207231123_0011
Exception in thread "Thread-93" java.lang.RuntimeException: Error while reading from task log url
    at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:130)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:211)
    at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:81)
    at java.lang.Thread.run(Thread.java:680)
Caused by: java.io.IOException: Server returned HTTP response code: 407 for URL: http://10.203.33.81:50060/tasklog?taskid=attempt_201207231123_0011_r_000000_0&start=-8193
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1436)
    at java.net.URL.openStream(URL.java:1010)
    at org.apache.hadoop.hive.ql.exec.errors.TaskLogProcessor.getErrors(TaskLogProcessor.java:120)
    ... 3 more
FAILED: Exec