Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> [0.8] exception running on windows when attempting to send to an uncreated topic


Copy link to this message
-
Re: [0.8] exception running on windows when attempting to send to an uncreated topic
Jun, thanks!  I changed the properties for the broker to the following,
worked, and I was able to produce data, for a while.  More on the next issue
below.

thanks,
rob

...I changed the broker properties to the following:

for broker 0:
props.setProperty("log.dir", "\\tmp\\kafka0-logs");
for broker 1:
props.setProperty("log.dir", "\\tmp\\kafka1-logs");

Additionally, it seems the zookeeper had no problems with the following
property setting:
props.setProperty("dataDir", "/tmp/zookeeper");
Now here is the combined log for a zookeeper, 2 brokers, a producer, and 2
consumers.  It produced for awhile, as evidenced by the log in the
beginning, and
- I have 10 MB in
C:\tmp\kafka1-logs\robert_v_2x0-0\00000000000000000000.index
- I have 3.4 MB in
C:\tmp\kafka1-logs\robert_v_2x0-0\00000000000000000000.log

Due to the following FATAL line, both brokers shutdown (and their listening
sockets go away), and so the producer fails:
{2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-thread1]
(Logging.scala:109) - Attempt to swap the new high watermark file with the
old one failed

{2013-05-10 18:23:57,456} DEBUG [local-vat] (Logging.scala:51) - Getting
broker partition info for topic robert_v_2x0
{2013-05-10 18:23:57,457} DEBUG [local-vat] (Logging.scala:51) - Partition
[robert_v_2x0,0] has leader 1
{2013-05-10 18:23:57,458} DEBUG [local-vat] (Logging.scala:51) - Broker
partitions registered for topic: robert_v_2x0 are 0
{2013-05-10 18:23:57,459} DEBUG [local-vat] (Logging.scala:51) - Sending 1
messages with no compression to [robert_v_2x0,0]
{2013-05-10 18:23:57,461} DEBUG [local-vat] (Logging.scala:51) - Producer
sending messages with correlation id 176 for topics [robert_v_2x0,0] to
broker 1 on 192.168.1.100:9093
{2013-05-10 18:23:57,466} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - [Kafka Request Handler 0 on Broker 1], handles request
Request(1,sun.nio.ch.SelectionKeyImpl@3d308bfd,null,1368231837466,/192.168.1.100:56009)
{2013-05-10 18:23:57,468} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - Adding index entry 86 => 4281531 to
00000000000000000000.index.
{2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - Partition [robert_v_2x0,0] on broker 1: Highwatermark
for partition [robert_v_2x0,0] updated to 87
{2013-05-10 18:23:57,469} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - [KafkaApi-1] Produce to local log in 2 ms
{2013-05-10 18:23:57,475} DEBUG [local-vat] (Logging.scala:51) - Producer
sent messages with correlation id 176 for topics [robert_v_2x0,0] to broker
1 on 192.168.1.100:9093
{2013-05-10 18:23:57,497} DEBUG [local-vat] (Logging.scala:51) - Getting
broker partition info for topic robert_v_2x0
{2013-05-10 18:23:57,634} DEBUG [local-vat] (Logging.scala:51) - Partition
[robert_v_2x0,0] has leader 1
{2013-05-10 18:23:57,635} DEBUG [local-vat] (Logging.scala:51) - Broker
partitions registered for topic: robert_v_2x0 are 0
{2013-05-10 18:23:57,636} DEBUG [local-vat] (Logging.scala:51) - Sending 1
messages with no compression to [robert_v_2x0,0]
{2013-05-10 18:23:57,637} DEBUG [local-vat] (Logging.scala:51) - Producer
sending messages with correlation id 178 for topics [robert_v_2x0,0] to
broker 1 on 192.168.1.100:9093
{2013-05-10 18:23:57,689} FATAL [highwatermark-checkpoint-thread1]
(Logging.scala:109) - Attempt to swap the new high watermark file with the
old one failed
{2013-05-10 18:23:57,739}  INFO [Thread-4] (Logging.scala:67) - [Kafka
Server 0], shutting down
{2013-05-10 18:23:57,771} DEBUG [kafka-request-handler-1]
(Logging.scala:51) - [Kafka Request Handler 1 on Broker 1], handles request
Request(1,sun.nio.ch.SelectionKeyImpl@3d308bfd,null,1368231837641,/192.168.1.100:56009)
{2013-05-10 18:23:57,772} DEBUG [kafka-request-handler-1]
(Logging.scala:51) - Partition [robert_v_2x0,0] on broker 1: Highwatermark
for partition [robert_v_2x0,0] updated to 88
{2013-05-10 18:23:57,773} DEBUG [kafka-request-handler-1]
(Logging.scala:51) - [KafkaApi-1] Produce to local log in 2 ms
{2013-05-10 18:23:57,785} DEBUG [local-vat] (Logging.scala:51) - Producer
sent messages with correlation id 178 for topics [robert_v_2x0,0] to broker
1 on 192.168.1.100:9093
{2013-05-10 18:23:57,794} DEBUG [local-vat] (Logging.scala:51) - Getting
broker partition info for topic robert_v_2x0
{2013-05-10 18:23:57,795} DEBUG [local-vat] (Logging.scala:51) - Partition
[robert_v_2x0,0] has leader 1
{2013-05-10 18:23:57,795} DEBUG [local-vat] (Logging.scala:51) - Broker
partitions registered for topic: robert_v_2x0 are 0
{2013-05-10 18:23:57,796} DEBUG [local-vat] (Logging.scala:51) - Sending 1
messages with no compression to [robert_v_2x0,0]
{2013-05-10 18:23:57,798} DEBUG [local-vat] (Logging.scala:51) - Producer
sending messages with correlation id 180 for topics [robert_v_2x0,0] to
broker 1 on 192.168.1.100:9093
{2013-05-10 18:23:57,802}  INFO [Thread-5] (Logging.scala:67) - [Kafka
Server 1], shutting down
{2013-05-10 18:23:57,803}  INFO [Thread-5] (Logging.scala:67) - [Socket
Server on Broker 1], shutting down
{2013-05-10 18:23:57,806} DEBUG [kafka-acceptor] (Logging.scala:51) -
Closing server socket and selector.
{2013-05-10 18:23:57,817} DEBUG [kafka-processor-9093-0]
(Logging.scala:51) - Closing selector.
{2013-05-10 18:23:57,820}  INFO [Thread-4] (Logging.scala:67) - [Socket
Server on Broker 0], shutting down
{2013-05-10 18:23:57,822} DEBUG [kafka-acceptor] (Logging.scala:51) -
Closing server socket and selector.
{2013-05-10 18:23:57,825} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - [Kafka Request Handler 0 on Broker 1], handles request
Request(1,sun.nio.ch.SelectionKeyImpl@3d308bfd,null,1368231837814,/192.168.1.100:56009)
{2013-05-10 18:23:57,826} DEBUG [kafka-processor-9092-0]
(Logging.scala:51) - Closing selector.
{2013-05-10 18:23:57,830} DEBUG [kafka-processor-9092-1]
(Logging.scala:51) - Closing selector.
{2013-05-10 18:23:57,834} DEBUG [kafka-request-handler-0]
(Logging.scala:51) - Adding ind