Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - 0.8 protocol exception


Copy link to this message
-
Re: 0.8 protocol exception
Colin Blower 2013-07-08, 22:12
Hey Vinicius,

Feel free to contact me if you would like help with the protocol stuff.
I'm sure you will be able to figure it out from the wiki and scala code,
but it would be nice to make the wiki as detailed as possible for the
next set of developers.

Thanks for working on a new Kafka client. I would love to see the final
product.
On 07/08/2013 02:44 PM, Vinicius Carvalho wrote:
> Thanks Colin, yep you are right I was missing the ShortString way to
> represent arrays. All is fine now, I'm able to connect and execute
> MetadataRequest/Responses. Moving to ProducerRequest/Responses now :)
>
>
> On Mon, Jul 8, 2013 at 5:13 PM, Colin Blower <[EMAIL PROTECTED]> wrote:
>
>> I had the exact same problem when I started writing code for the new
>> protocol. This is an oddity with the way the protocol spec uses EBNF to
>> specify arrays.
>>
>> Checkout the section on protocol primitives, especially arrays.
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-ProtocolPrimitiveTypes
>>
>> Essentially, each array is preceded by its length as an int32.
>>
>> With regards to your specific buffer:
>> <Buffer 00 00 00 16 00 03 00 00 00 00 00 00 00 03 66 6f 6f 00 07 6d 79
>> 54 6f 70 69 63>
>>
>> You seem to be missing the clientId as well as the array length. For
>> comparison the buffer, with the client I wrote, for clientId
>> "perl-kafka" and topics "foo" and "myTopic":
>>
>> 00 00 00 26 00 03 00 00 00 00 00 2a 00 0a 70 65 72 6c 2d 6b 61 66 6b 61
>> 00 00 00 02 00 03 66 6f 6f 00 07 6d 79 54 6f 70 69 63
>>
>>
>> On 07/08/2013 04:12 AM, Vinicius Carvalho wrote:
>>> Ok, so I've found out the error: The documentation is outdated, the
>>> MetadataRequest BNF should be:
>>>
>>> NumberOfTopics [TopicList]
>>>

--
*Colin Blower*
/Software Engineer/
Barracuda Networks Inc.
+1 408-342-5576 (o)