Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> java.io.IOException: Bad connect ack with firstBadLink


Copy link to this message
-
java.io.IOException: Bad connect ack with firstBadLink
Hi , running hadoop map/reduce got such exception?

  1) Why does it happen?

  2) Job didn't failed and continue it's execution? Does this
exception cause losing data or map/reduce

uses recovery mechanism?

2010-11-09 05:10:08,735 INFO org.apache.hadoop.hdfs.DFSClient:
Exception in createBlockOutputStream java.io.IOException: Bad connect
ack with firstBadLink 10.11.87.65:50010
2010-11-09 05:10:08,735 INFO org.apache.hadoop.hdfs.DFSClient:
Abandoning block blk_-9208177033562590356_775948
2010-11-09 05:10:08,739 INFO org.apache.hadoop.hdfs.DFSClient: Waiting
to find target node: 10.11.87.61:50010
2010-11-09 05:11:23,743 INFO org.apache.hadoop.hdfs.DFSClient:
Exception in createBlockOutputStream java.net.SocketTimeoutException:
69000 millis timeout while waiting for channel to be ready for read.
ch : java.nio.channels.SocketChannel[connected
local=/10.11.87.61:55309 remote=/10.11.87.61:50010]
2010-11-09 05:11:23,743 INFO org.apache.hadoop.hdfs.DFSClient:
Abandoning block blk_-10251707095594311_775950
2010-11-09 05:11:23,744 INFO org.apache.hadoop.hdfs.DFSClient: Waiting
to find target node: 10.11.87.61:50010
2010-11-09 05:12:29,815 INFO org.apache.hadoop.hdfs.DFSClient:
Exception in createBlockOutputStream java.io.IOException: Bad connect
ack with firstBadLink 10.11.87.65:50010
2010-11-09 05:12:29,816 INFO org.apache.hadoop.hdfs.DFSClient:
Abandoning block blk_3509928762116143133_775950
2010-11-09 05:12:29,818 INFO org.apache.hadoop.hdfs.DFSClient: Waiting
to find target node: 10.11.87.61:50010
2010-11-09 05:13:35,949 INFO org.apache.hadoop.hdfs.DFSClient:
Exception in createBlockOutputStream java.io.IOException: Bad connect
ack with firstBadLink 10.11.87.65:50010
2010-11-09 05:13:35,949 INFO org.apache.hadoop.hdfs.DFSClient:
Abandoning block blk_3138002906377068146_775950
2010-11-09 05:13:35,950 INFO org.apache.hadoop.hdfs.DFSClient: Waiting
to find target node: 10.11.87.61:50010
2010-11-09 05:13:51,757 WARN org.apache.hadoop.hdfs.DFSClient:
DataStreamer Exception: java.io.IOException: Unable to create new
block.
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2812)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2076)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2262)

2010-11-09 05:13:51,757 WARN org.apache.hadoop.hdfs.DFSClient: Error
Recovery for block blk_3138002906377068146_775950 bad datanode[2]
nodes == null
2010-11-09 05:13:51,758 WARN org.apache.hadoop.hdfs.DFSClient: Could
not get block locations. Source file
"/user/hadoop/requests_logs/merged/2010-11-08/_temporary/_attempt_201011081008_0002_r_000008_0/part-r-00008"
- Aborting...
2010-11-09 05:13:51,760 WARN org.apache.hadoop.mapred.TaskTracker:
Error running child
java.io.IOException: Bad connect ack with firstBadLink 10.11.87.65:50010
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.createBlockOutputStream(DFSClient.java:2870)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2793)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2076)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2262)
2010-11-09 05:13:54,888 INFO org.apache.hadoop.mapred.TaskRunner:
Runnning cleanup for the task
Thanks

Oleg.

------------------------------
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB