|
|
-
Ruby or Python App -> Kafka -> S3
Russell Jurney 2011-12-16, 01:23
Can Kafka do this, without writing a lot of code?
Russell Jurney
twitter.com/rjurney
[EMAIL PROTECTED]
datasyndrome.com
-
Re: Ruby or Python App -> Kafka -> S3
Jun Rao 2011-12-16, 01:38
Russell,
We do have Ruby/Python clients under contrib (may not be well supported).
You will need to write a Kafka consumer that pipes to S3.
Thanks,
Jun
On Thu, Dec 15, 2011 at 5:23 PM, Russell Jurney <[EMAIL PROTECTED]>wrote:
> Can Kafka do this, without writing a lot of code?
>
> Russell Jurney
> twitter.com/rjurney
> [EMAIL PROTECTED]
> datasyndrome.com
>
-
Re: Ruby or Python App -> Kafka -> S3
Russell Jurney 2011-12-16, 02:04
Jun, thanks! Would you say that writing a Kafka consumer that pipes
to S3 is a little code, or a lot of code?
Russell Jurney
twitter.com/rjurney
[EMAIL PROTECTED]
datasyndrome.com
On Dec 15, 2011, at 5:39 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> Russell,
>
> We do have Ruby/Python clients under contrib (may not be well supported).
> You will need to write a Kafka consumer that pipes to S3.
>
> Thanks,
>
> Jun
>
> On Thu, Dec 15, 2011 at 5:23 PM, Russell Jurney <[EMAIL PROTECTED]>wrote:
>
>> Can Kafka do this, without writing a lot of code?
>>
>> Russell Jurney
>> twitter.com/rjurney
>> [EMAIL PROTECTED]
>> datasyndrome.com
>>
-
Re: Ruby or Python App -> Kafka -> S3
Neha Narkhede 2011-12-16, 02:15
Russell,
One way to do this, is use the ConsoleConsumer and plug in your
formatter. You can write your formatter to optionally deserialize the
message into an object, and either write it to S3 immediately, or
buffer and then write to S3. If you go with this approach, the
console consumer consumes data as fast as you can write to S3.
Thanks,
Neha
On Thu, Dec 15, 2011 at 6:04 PM, Russell Jurney
<[EMAIL PROTECTED]> wrote:
> Jun, thanks! Would you say that writing a Kafka consumer that pipes
> to S3 is a little code, or a lot of code?
>
> Russell Jurney
> twitter.com/rjurney
> [EMAIL PROTECTED]
> datasyndrome.com
>
> On Dec 15, 2011, at 5:39 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
>> Russell,
>>
>> We do have Ruby/Python clients under contrib (may not be well supported).
>> You will need to write a Kafka consumer that pipes to S3.
>>
>> Thanks,
>>
>> Jun
>>
>> On Thu, Dec 15, 2011 at 5:23 PM, Russell Jurney <[EMAIL PROTECTED]>wrote:
>>
>>> Can Kafka do this, without writing a lot of code?
>>>
>>> Russell Jurney
>>> twitter.com/rjurney
>>> [EMAIL PROTECTED]
>>> datasyndrome.com
>>>
|
|
