Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - [0.8] exception running on windows when attempting to send to an uncreated topic


+
Rob Withers 2013-05-09, 06:04
+
Jun Rao 2013-05-09, 15:00
+
Rob Withers 2013-05-11, 00:38
Copy link to this message
-
Re: [0.8] exception running on windows when attempting to send to an uncreated topic
Jun Rao 2013-05-11, 04:54
It seems this is because we hardcoded "/" in HighwaterMarkCheckpoint. Could
you file a jira?

  val name = path + "/" + HighwaterMarkCheckpoint.highWatermarkFileName

Thanks,

Jun
On Fri, May 10, 2013 at 5:38 PM, Rob Withers <[EMAIL PROTECTED]> wrote:

> Jun, thanks!  I changed the properties for the broker to the following,
> worked, and I was able to produce data, for a while.  More on the next
> issue below.
>
> thanks,
> rob
>
> ...I changed the broker properties to the following:
>
> for broker 0:
> props.setProperty("log.dir", "\\tmp\\kafka0-logs");
> for broker 1:
> props.setProperty("log.dir", "\\tmp\\kafka1-logs");
>
> Additionally, it seems the zookeeper had no problems with the following
> property setting:
> props.setProperty("dataDir", "/tmp/zookeeper");
>
>
> Now here is the combined log for a zookeeper, 2 brokers, a producer, and 2
> consumers.  It produced for awhile, as evidenced by the log in the
> beginning, and
> - I have 10 MB in C:\tmp\kafka1-logs\robert_v_**
> 2x0-0\00000000000000000000.**index
> - I have 3.4 MB in C:\tmp\kafka1-logs\robert_v_**
> 2x0-0\00000000000000000000.log
>
> Due to the following FATAL line, both brokers shutdown (and their
> listening sockets go away), and so the producer fails:
> {2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-**thread1]
> (Logging.scala:109) - Attempt to swap the new high watermark file with the
> old one failed
>
> {2013-05-10 18:23:57,456} DEBUG [local-vat] (Logging.scala:51) - Getting
> broker partition info for topic robert_v_2x0
> {2013-05-10 18:23:57,457} DEBUG [local-vat] (Logging.scala:51) - Partition
> [robert_v_2x0,0] has leader 1
> {2013-05-10 18:23:57,458} DEBUG [local-vat] (Logging.scala:51) - Broker
> partitions registered for topic: robert_v_2x0 are 0
> {2013-05-10 18:23:57,459} DEBUG [local-vat] (Logging.scala:51) - Sending 1
> messages with no compression to [robert_v_2x0,0]
> {2013-05-10 18:23:57,461} DEBUG [local-vat] (Logging.scala:51) - Producer
> sending messages with correlation id 176 for topics [robert_v_2x0,0] to
> broker 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,466} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - [Kafka Request Handler 0 on Broker 1], handles request
> Request(1,sun.nio.ch.**SelectionKeyImpl@3d308bfd,**null,1368231837466,/
> 192.168.1.**100:56009 <http://192.168.1.100:56009>)
> {2013-05-10 18:23:57,468} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - Adding index entry 86 => 4281531 to
> 00000000000000000000.index.
> {2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - Partition [robert_v_2x0,0] on broker 1: Highwatermark
> for partition [robert_v_2x0,0] updated to 87
> {2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - [KafkaApi-1] Produce to local log in 2 ms
> {2013-05-10 18:23:57,475} DEBUG [local-vat] (Logging.scala:51) - Producer
> sent messages with correlation id 176 for topics [robert_v_2x0,0] to broker
> 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,497} DEBUG [local-vat] (Logging.scala:51) - Getting
> broker partition info for topic robert_v_2x0
> {2013-05-10 18:23:57,634} DEBUG [local-vat] (Logging.scala:51) - Partition
> [robert_v_2x0,0] has leader 1
> {2013-05-10 18:23:57,635} DEBUG [local-vat] (Logging.scala:51) - Broker
> partitions registered for topic: robert_v_2x0 are 0
> {2013-05-10 18:23:57,636} DEBUG [local-vat] (Logging.scala:51) - Sending 1
> messages with no compression to [robert_v_2x0,0]
> {2013-05-10 18:23:57,637} DEBUG [local-vat] (Logging.scala:51) - Producer
> sending messages with correlation id 178 for topics [robert_v_2x0,0] to
> broker 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-**thread1]
> (Logging.scala:109) - Attempt to swap the new high watermark file with the
> old one failed
> {2013-05-10 18:23:57,739}  INFO [Thread-4] (Logging.scala:67) - [Kafka
> Server 0], shutting down
> {2013-05-10 18:23:57,771} DEBUG [kafka-request-handler-1]
> (Logging.scala:51) - [Kafka Request Handler 1 on Broker 1], handles request

 
+
Rob Withers 2013-05-11, 04:57
+
Rob Withers 2013-05-10, 06:28
+
Jun Rao 2013-05-10, 15:58