Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> hive jobs pending so long


+
Neil Guo 2012-12-19, 16:27
+
Nitin Pawar 2012-12-19, 17:03
+
Neil Guo 2012-12-19, 17:27
+
Nitin Pawar 2012-12-19, 17:47
+
Neil Guo 2012-12-19, 18:15
+
Nitin Pawar 2012-12-19, 18:58
+
Neil Guo 2012-12-20, 04:40
+
Neil Guo 2012-12-20, 07:21
Copy link to this message
-
Re: hive jobs pending so long
Neil,
It might be a transient problem with your single-node cluster. Was
something else running on the machine when you ran the long running
job. Can you monitor the usage of the machine when the hive query is
running. Have you tried rebooting the machine? Does the query remain
slow even after reboot?

Mark

On Wed, Dec 19, 2012 at 11:21 PM, Neil Guo <[EMAIL PROTECTED]> wrote:
> another log,
>
> cd $HADOOP_LOG_DIR && tail -n 0 -f *.log |tee log_merge.log
>
>
> ==> hadoop-neil-datanode-Ct-bj108.log <=>
> ==> hadoop-neil-jobtracker-Ct-bj108.log <=>
> ==> hadoop-neil-namenode-Ct-bj108.log <=>
> ==> hadoop-neil-secondarynamenode-Ct-bj108.log <=>
> ==> hadoop-neil-tasktracker-Ct-bj108.log <=>
> ==> hadoop-neil-datanode-Ct-bj108.log <=> 2012-12-20 15:10:10,425 INFO
> org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification
> succeeded for blk_7549818753225420678_5717
>
> ==> hadoop-neil-namenode-Ct-bj108.log <=> 2012-12-20 15:10:15,265 INFO
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of transactions:
> 1 Total time for transactions(ms): 0Number of transactions batched in Syncs:
> 0 Number of syncs: 0 SyncTimes(ms): 0
>
> ==> hadoop-neil-datanode-Ct-bj108.log <=> 2012-12-20 15:10:16,415 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> blk_-6072279734308701530_134821 src: /127.0.0.1:34080 dest: /127.0.0.1:50010
>
> ==> hadoop-neil-namenode-Ct-bj108.log <=> 2012-12-20 15:10:16,393 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.allocateBlock:
> /tmp/hive-neil/hive_2012-12-20_15-10-12_982_9050806798858515370/-mr-10003/418d928d-6cd4-485b-99b6-0355e9a20fd7.
> blk_-6072279734308701530_134821
>
> ==> hadoop-neil-datanode-Ct-bj108.log <=> 2012-12-20 15:10:16,564 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src:
> /127.0.0.1:34080, dest: /127.0.0.1:50010, bytes: 57401, op: HDFS_WRITE,
> cliID: DFSClient_-1778955690, offset: 0, srvID:
> DS-1688600142-127.0.0.1-50010-1318579377533, blockid:
> blk_-6072279734308701530_134821, duration: 84788000
> 2012-12-20 15:10:16,565 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block
> blk_-6072279734308701530_134821 terminating
> 2012-12-20 15:10:16,911 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> blk_5272124058034081804_134822 src: /127.0.0.1:34082 dest: /127.0.0.1:50010
> 2012-12-20 15:10:17,003 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src:
> /127.0.0.1:34082, dest: /127.0.0.1:50010, bytes: 3915, op: HDFS_WRITE,
> cliID: DFSClient_-1778955690, offset: 0, srvID:
> DS-1688600142-127.0.0.1-50010-1318579377533, blockid:
> blk_5272124058034081804_134822, duration: 56734000
> 2012-12-20 15:10:17,003 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block
> blk_5272124058034081804_134822 terminating
> 2012-12-20 15:10:17,122 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> blk_-5406425112482671769_134823 src: /127.0.0.1:34083 dest: /127.0.0.1:50010
> 2012-12-20 15:10:17,313 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src:
> /127.0.0.1:34083, dest: /127.0.0.1:50010, bytes: 449818, op: HDFS_WRITE,
> cliID: DFSClient_-1778955690, offset: 0, srvID:
> DS-1688600142-127.0.0.1-50010-1318579377533, blockid:
> blk_-5406425112482671769_134823, duration: 152658000
> 2012-12-20 15:10:17,313 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block
> blk_-5406425112482671769_134823 terminating
> 2012-12-20 15:10:17,474 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> blk_-6418269071912761245_134824 src: /127.0.0.1:34084 dest: /127.0.0.1:50010
>
> ==> hadoop-neil-namenode-Ct-bj108.log <=> 2012-12-20 15:10:16,565 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
> NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to
> blk_-6072279734308701530_134821 size 57401
> 2012-12-20 15:10:16,567 INFO org.apache.hadoop.hdfs.StateChange: DIR*
+
Neil Guo 2012-12-21, 06:22
+
Neil Guo 2012-12-21, 07:53
+
Neil Guo 2012-12-20, 10:10