Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> [0.8] exception running on windows when attempting to send to an uncreated topic


Copy link to this message
-
Re: [0.8] exception running on windows when attempting to send to an uncreated topic
Jun, thanks!  I changed the properties for the broker to the following,
worked, and I was able to produce data, for a while.  More on the next issue
below.

thanks,
rob

...I changed the broker properties to the following:

for broker 0:
props.setProperty("log.dir", "\\tmp\\kafka0-logs");
for broker 1:
props.setProperty("log.dir", "\\tmp\\kafka1-logs");

Additionally, it seems the zookeeper had no problems with the following
property setting:
props.setProperty("dataDir", "/tmp/zookeeper");
Now here is the combined log for a zookeeper, 2 brokers, a producer, and 2
consumers.  It produced for awhile, as evidenced by the log in the
beginning, and
- I have 10 MB in
C:\tmp\kafka1-logs\robert_v_2x0-0\00000000000000000000.index
- I have 3.4 MB in
C:\tmp\kafka1-logs\robert_v_2x0-0\00000000000000000000.log

Due to the following FATAL line, both brokers shutdown (and their listening
sockets go away), and so the producer fails:
{2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-thread1]
(Logging.scala:109) - Attempt to swap the new high watermark file with the
old one failed

{2013-05-10 18:23:57,456} DEBUG [local-vat] (Logging.scala:51) - Getting
broker partition info for topic robert_v_2x0
{2013-05-10 18:23:57,457} DEBUG [local-vat] (Logging.scala:51) - Partition
[robert_v_2x0,0] has leader 1
{2013-05-10 18:23:57,458} DEBUG [local-vat] (Logging.scala:51) - Broker
partitions registered for topic: robert_v_2x0 are 0
{2013-05-10 18:23:57,459} DEBUG [local-vat] (Logging.scala:51) - Sending 1
messages with no compression to [robert_v_2x0,0]
{2013-05-10 18:23:57,461} DEBUG [local-vat] (Logging.scala:51) - Producer
sending messages with correlation id 176 for topics [robert_v_2x0,0] to
broker 1 on 192.168.1.100:9093
{2013-05-10 18:23:57,466} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - [Kafka Request Handler 0 on Broker 1], handles request
Request(1,sun.nio.ch.SelectionKeyImpl@3d308bfd,null,1368231837466,/192.168.1.100:56009)
{2013-05-10 18:23:57,468} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - Adding index entry 86 => 4281531 to
00000000000000000000.index.
{2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - Partition [robert_v_2x0,0] on broker 1: Highwatermark
for partition [robert_v_2x0,0] updated to 87
{2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - [KafkaApi-1] Produce to local log in 2 ms
{2013-05-10 18:23:57,475} DEBUG [local-vat] (Logging.scala:51) - Producer
sent messages with correlation id 176 for topics [robert_v_2x0,0] to broker
1 on 192.168.1.100:9093
{2013-05-10 18:23:57,497} DEBUG [local-vat] (Logging.scala:51) - Getting
broker partition info for topic robert_v_2x0
{2013-05-10 18:23:57,634} DEBUG [local-vat] (Logging.scala:51) - Partition
[robert_v_2x0,0] has leader 1
{2013-05-10 18:23:57,635} DEBUG [local-vat] (Logging.scala:51) - Broker
partitions registered for topic: robert_v_2x0 are 0
{2013-05-10 18:23:57,636} DEBUG [local-vat] (Logging.scala:51) - Sending 1
messages with no compression to [robert_v_2x0,0]
{2013-05-10 18:23:57,637} DEBUG [local-vat] (Logging.scala:51) - Producer
sending messages with correlation id 178 for topics [robert_v_2x0,0] to
broker 1 on 192.168.1.100:9093
{2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-thread1]
(Logging.scala:109) - Attempt to swap the new high watermark file with the
old one failed
{2013-05-10 18:23:57,739}  INFO [Thread-4] (Logging.scala:67) - [Kafka
Server 0], shutting down
{2013-05-10 18:23:57,771} DEBUG [kafka-request-handler-1]
(Logging.scala:51) - [Kafka Request Handler 1 on Broker 1], handles request
Request(1,sun.nio.ch.SelectionKeyImpl@3d308bfd,null,1368231837641,/192.168.1.100:56009)
{2013-05-10 18:23:57,772} DEBUG [kafka-request-handler-1]
(Logging.scala:51) - Partition [robert_v_2x0,0] on broker 1: Highwatermark
for partition [robert_v_2x0,0] updated to 88
{2013-05-10 18:23:57,773} DEBUG [kafka-request-handler-1]
(Logging.scala:51) - [KafkaApi-1] Produce to local log in 2 ms
{2013-05-10 18:23:57,785} DEBUG [local-vat] (Logging.scala:51) - Producer
sent messages with correlation id 178 for topics [robert_v_2x0,0] to broker
1 on 192.168.1.100:9093
{2013-05-10 18:23:57,794} DEBUG [local-vat] (Logging.scala:51) - Getting
broker partition info for topic robert_v_2x0
{2013-05-10 18:23:57,795} DEBUG [local-vat] (Logging.scala:51) - Partition
[robert_v_2x0,0] has leader 1
{2013-05-10 18:23:57,795} DEBUG [local-vat] (Logging.scala:51) - Broker
partitions registered for topic: robert_v_2x0 are 0
{2013-05-10 18:23:57,796} DEBUG [local-vat] (Logging.scala:51) - Sending 1
messages with no compression to [robert_v_2x0,0]
{2013-05-10 18:23:57,798} DEBUG [local-vat] (Logging.scala:51) - Producer
sending messages with correlation id 180 for topics [robert_v_2x0,0] to
broker 1 on 192.168.1.100:9093
{2013-05-10 18:23:57,802}  INFO [Thread-5] (Logging.scala:67) - [Kafka
Server 1], shutting down
{2013-05-10 18:23:57,803}  INFO [Thread-5] (Logging.scala:67) - [Socket
Server on Broker 1], shutting down
{2013-05-10 18:23:57,806} DEBUG [kafka-acceptor] (Logging.scala:51) -
Closing server socket and selector.
{2013-05-10 18:23:57,817} DEBUG [kafka-processor-9093-0]
(Logging.scala:51) - Closing selector.
{2013-05-10 18:23:57,820}  INFO [Thread-4] (Logging.scala:67) - [Socket
Server on Broker 0], shutting down
{2013-05-10 18:23:57,822} DEBUG [kafka-acceptor] (Logging.scala:51) -
Closing server socket and selector.
{2013-05-10 18:23:57,825} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - [Kafka Request Handler 0 on Broker 1], handles request
Request(1,sun.nio.ch.SelectionKeyImpl@3d308bfd,null,1368231837814,/192.168.1.100:56009)
{2013-05-10 18:23:57,826} DEBUG [kafka-processor-9092-0]
(Logging.scala:51) - Closing selector.
{2013-05-10 18:23:57,830} DEBUG [kafka-processor-9092-1]
(Logging.scala:51) - Closing selector.
{2013-05-10 18:23:57,834} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - Adding ind
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB