Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> [0.8] exception running on windows when attempting to send to an uncreated topic


Copy link to this message
-
Re: [0.8] exception running on windows when attempting to send to an uncreated topic
Sweet!  That was an easy one!  Here's a jira for it:
https://issues.apache.org/jira/browse/KAFKA-903

thanks,
rob

-----Original Message-----
From: Jun Rao
Sent: Friday, May 10, 2013 10:53 PM
To: [EMAIL PROTECTED]
Subject: Re: [0.8] exception running on windows when attempting to send to
an uncreated topic

It seems this is because we hardcoded "/" in HighwaterMarkCheckpoint. Could
you file a jira?

  val name = path + "/" + HighwaterMarkCheckpoint.highWatermarkFileName

Thanks,

Jun
On Fri, May 10, 2013 at 5:38 PM, Rob Withers <[EMAIL PROTECTED]> wrote:

> Jun, thanks!  I changed the properties for the broker to the following,
> worked, and I was able to produce data, for a while.  More on the next
> issue below.
>
> thanks,
> rob
>
> ...I changed the broker properties to the following:
>
> for broker 0:
> props.setProperty("log.dir", "\\tmp\\kafka0-logs");
> for broker 1:
> props.setProperty("log.dir", "\\tmp\\kafka1-logs");
>
> Additionally, it seems the zookeeper had no problems with the following
> property setting:
> props.setProperty("dataDir", "/tmp/zookeeper");
>
>
> Now here is the combined log for a zookeeper, 2 brokers, a producer, and 2
> consumers.  It produced for awhile, as evidenced by the log in the
> beginning, and
> - I have 10 MB in C:\tmp\kafka1-logs\robert_v_**
> 2x0-0\00000000000000000000.**index
> - I have 3.4 MB in C:\tmp\kafka1-logs\robert_v_**
> 2x0-0\00000000000000000000.log
>
> Due to the following FATAL line, both brokers shutdown (and their
> listening sockets go away), and so the producer fails:
> {2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-**thread1]
> (Logging.scala:109) - Attempt to swap the new high watermark file with the
> old one failed
>
> {2013-05-10 18:23:57,456} DEBUG [local-vat] (Logging.scala:51) - Getting
> broker partition info for topic robert_v_2x0
> {2013-05-10 18:23:57,457} DEBUG [local-vat] (Logging.scala:51) - Partition
> [robert_v_2x0,0] has leader 1
> {2013-05-10 18:23:57,458} DEBUG [local-vat] (Logging.scala:51) - Broker
> partitions registered for topic: robert_v_2x0 are 0
> {2013-05-10 18:23:57,459} DEBUG [local-vat] (Logging.scala:51) - Sending 1
> messages with no compression to [robert_v_2x0,0]
> {2013-05-10 18:23:57,461} DEBUG [local-vat] (Logging.scala:51) - Producer
> sending messages with correlation id 176 for topics [robert_v_2x0,0] to
> broker 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,466} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - [Kafka Request Handler 0 on Broker 1], handles
> request
> Request(1,sun.nio.ch.**SelectionKeyImpl@3d308bfd,**null,1368231837466,/
> 192.168.1.**100:56009 <http://192.168.1.100:56009>)
> {2013-05-10 18:23:57,468} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - Adding index entry 86 => 4281531 to
> 00000000000000000000.index.
> {2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - Partition [robert_v_2x0,0] on broker 1: Highwatermark
> for partition [robert_v_2x0,0] updated to 87
> {2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - [KafkaApi-1] Produce to local log in 2 ms
> {2013-05-10 18:23:57,475} DEBUG [local-vat] (Logging.scala:51) - Producer
> sent messages with correlation id 176 for topics [robert_v_2x0,0] to
> broker
> 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,497} DEBUG [local-vat] (Logging.scala:51) - Getting
> broker partition info for topic robert_v_2x0
> {2013-05-10 18:23:57,634} DEBUG [local-vat] (Logging.scala:51) - Partition
> [robert_v_2x0,0] has leader 1
> {2013-05-10 18:23:57,635} DEBUG [local-vat] (Logging.scala:51) - Broker
> partitions registered for topic: robert_v_2x0 are 0
> {2013-05-10 18:23:57,636} DEBUG [local-vat] (Logging.scala:51) - Sending 1
> messages with no compression to [robert_v_2x0,0]
> {2013-05-10 18:23:57,637} DEBUG [local-vat] (Logging.scala:51) - Producer
> sending messages with correlation id 178 for topics [robert_v_2x0,0] to
> broker 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-**thread1]
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB