Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> [0.8] exception running on windows when attempting to send to an uncreated topic


Copy link to this message
-
Re: [0.8] exception running on windows when attempting to send to an uncreated topic
It seems this is because we hardcoded "/" in HighwaterMarkCheckpoint. Could
you file a jira?

  val name = path + "/" + HighwaterMarkCheckpoint.highWatermarkFileName

Thanks,

Jun
On Fri, May 10, 2013 at 5:38 PM, Rob Withers <[EMAIL PROTECTED]> wrote:

> Jun, thanks!  I changed the properties for the broker to the following,
> worked, and I was able to produce data, for a while.  More on the next
> issue below.
>
> thanks,
> rob
>
> ...I changed the broker properties to the following:
>
> for broker 0:
> props.setProperty("log.dir", "\\tmp\\kafka0-logs");
> for broker 1:
> props.setProperty("log.dir", "\\tmp\\kafka1-logs");
>
> Additionally, it seems the zookeeper had no problems with the following
> property setting:
> props.setProperty("dataDir", "/tmp/zookeeper");
>
>
> Now here is the combined log for a zookeeper, 2 brokers, a producer, and 2
> consumers.  It produced for awhile, as evidenced by the log in the
> beginning, and
> - I have 10 MB in C:\tmp\kafka1-logs\robert_v_**
> 2x0-0\00000000000000000000.**index
> - I have 3.4 MB in C:\tmp\kafka1-logs\robert_v_**
> 2x0-0\00000000000000000000.log
>
> Due to the following FATAL line, both brokers shutdown (and their
> listening sockets go away), and so the producer fails:
> {2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-**thread1]
> (Logging.scala:109) - Attempt to swap the new high watermark file with the
> old one failed
>
> {2013-05-10 18:23:57,456} DEBUG [local-vat] (Logging.scala:51) - Getting
> broker partition info for topic robert_v_2x0
> {2013-05-10 18:23:57,457} DEBUG [local-vat] (Logging.scala:51) - Partition
> [robert_v_2x0,0] has leader 1
> {2013-05-10 18:23:57,458} DEBUG [local-vat] (Logging.scala:51) - Broker
> partitions registered for topic: robert_v_2x0 are 0
> {2013-05-10 18:23:57,459} DEBUG [local-vat] (Logging.scala:51) - Sending 1
> messages with no compression to [robert_v_2x0,0]
> {2013-05-10 18:23:57,461} DEBUG [local-vat] (Logging.scala:51) - Producer
> sending messages with correlation id 176 for topics [robert_v_2x0,0] to
> broker 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,466} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - [Kafka Request Handler 0 on Broker 1], handles request
> Request(1,sun.nio.ch.**SelectionKeyImpl@3d308bfd,**null,1368231837466,/
> 192.168.1.**100:56009 <http://192.168.1.100:56009>)
> {2013-05-10 18:23:57,468} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - Adding index entry 86 => 4281531 to
> 00000000000000000000.index.
> {2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - Partition [robert_v_2x0,0] on broker 1: Highwatermark
> for partition [robert_v_2x0,0] updated to 87
> {2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
> (Logging.scala:51) - [KafkaApi-1] Produce to local log in 2 ms
> {2013-05-10 18:23:57,475} DEBUG [local-vat] (Logging.scala:51) - Producer
> sent messages with correlation id 176 for topics [robert_v_2x0,0] to broker
> 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,497} DEBUG [local-vat] (Logging.scala:51) - Getting
> broker partition info for topic robert_v_2x0
> {2013-05-10 18:23:57,634} DEBUG [local-vat] (Logging.scala:51) - Partition
> [robert_v_2x0,0] has leader 1
> {2013-05-10 18:23:57,635} DEBUG [local-vat] (Logging.scala:51) - Broker
> partitions registered for topic: robert_v_2x0 are 0
> {2013-05-10 18:23:57,636} DEBUG [local-vat] (Logging.scala:51) - Sending 1
> messages with no compression to [robert_v_2x0,0]
> {2013-05-10 18:23:57,637} DEBUG [local-vat] (Logging.scala:51) - Producer
> sending messages with correlation id 178 for topics [robert_v_2x0,0] to
> broker 1 on 192.168.1.100:9093
> {2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-**thread1]
> (Logging.scala:109) - Attempt to swap the new high watermark file with the
> old one failed
> {2013-05-10 18:23:57,739}  INFO [Thread-4] (Logging.scala:67) - [Kafka
> Server 0], shutting down
> {2013-05-10 18:23:57,771} DEBUG [kafka-request-handler-1]
> (Logging.scala:51) - [Kafka Request Handler 1 on Broker 1], handles request

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB