Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive 0.7.1 Query hands


Copy link to this message
-
Re: Hive 0.7.1 Query hands
Hi sir,
the root cause of your issues seems to be java.io.EOFException, that based on the java doc description means the following:

  "Signals that an end of file or end of stream has been reached unexpectedly during input."

What is the health status of the box with ip 10.6.0.55? Isn't it by any chance having some issues? Port 8021 is used for TaskTracker, so I would start by connecting to that box and checking the TT logs.

Jarcec

On Thu, Feb 21, 2013 at 02:44:09PM +0300, Павел Мезенцев wrote:
> Hello!
>
> I use Hive 0.7.1 over Hadoop 0.20.2 (CHD3u3) on 70 nodes cluster.
> I have a trouble with query like this:
>
> *FROM* (  *SELECT* *id*, {expressions} *FROM* table1   *WHERE*
> day='2013-02-16' *AND* ({conditions1})*UNION* *ALL*  *SELECT* *id*,
> {expressions} *FROM* table2   *WHERE* day='2013-02-16' *AND*
> (conditions)*UNION* *ALL*  *SELECT* *id*, {expressions} *FROM* table3
> *WHERE* day='2013-02-16' *AND* (conditions)*UNION* *ALL*  *SELECT*
> *id*, {expressions} *FROM* table4  *WHERE* day='2013-02-16' *AND*
> (conditions)) union_tmp*INSERT* OVERWRITE *table* result_table
> *PARTITION* (day='2013-02-16')*SELECT* *id*, transformations
> (expressions)*GROUP* *BY* *id*;
>
> it had 4865 map tasks and 100 reduce tasks.
> first 4780 map taks completed succefull and last
> 85 tasks hangs.
>
> All this tasks hands with no progress,
>
>
> and no tasks attempts for each:
>
> One hour later after hang situation starting, job failed with exeption:
>
> 2013-02-20 20:02:02,000 Stage-1 map = 0%,  reduce = 0%
> 2013-02-20 20:02:40,679 Stage-1 map = 1%,  reduce = 0%
> 2013-02-20 20:02:54,022 Stage-1 map = 2%,  reduce = 0%
> 2013-02-20 20:03:14,129 Stage-1 map = 3%,  reduce = 0%
> ......................................................
> 2013-02-20 21:18:00,361 Stage-1 map = 98%,  reduce = 22%
> 2013-02-20 21:18:05,691 Stage-1 map = 98%,  reduce = 23%
> java.io.IOException: Call to statlabjt/10.6.0.55:8021 failed on local
> exception: java.io.EOFException
> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1142)
> at org.apache.hadoop.ipc.Client.call(Client.java:1110)
> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:226)
> at org.apache.hadoop.mapred.$Proxy8.getJobStatus(Unknown Source)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:1053)
> at org.apache.hadoop.mapred.JobClient.getJob(JobClient.java:1065)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.progress(ExecDriver.java:351)
> at org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:672)
> at org.apache.hadoop.hive.ql.exec.MapRedTask.execute(MapRedTask.java:123)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:130)
> at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1063)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:900)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:748)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:164)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:241)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:425)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> Caused by: java.io.EOFException
> at java.io.DataInputStream.readInt(DataInputStream.java:375)
> at org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:815)
> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:724)
> Ended Job = job_201302152355_4764 with exception
> 'java.io.IOException(Call to statlabjt/10.6.0.55:8021 failed on local
> exception: java.io.EOFException)'
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.MapRedTask
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB