Congratulations!  Looking forward to reading more about what your
implementation.

On 11/18/2011 12:07 PM, Taylor Gautier wrote:
> I just wanted to let you guys know we are live with our first Kafka use case.
>
> You'll see more from me in the coming weeks about it.
>
> We released the feature for the public on Monday of this week and have
> been slowly rolling it out to our users.
>
> We will continue to gradually ramp up usage until we get to 100%.
>
> There is still a lot to do with our implementation and more use cases
> are coming soon.
>
> But so far I am very happy with Kafka it has met all of my
> expectations and lived up to all of the claims made by the LinkedIn
> team.
>
> Thank you guys so much for OSS'ing such a great piece of technology.

Taylor,

Just out of curiosity, do you guys have to deal with at-least-once message
processing?   If so, I'd be curious how you handle all the corner cases
with Kafka, such as what if something downstream fails and you need to
replay some logs.

thanks,
Evan
On Fri, Nov 18, 2011 at 9:07 AM, Taylor Gautier <[EMAIL PROTECTED]> wrote:

> I just wanted to let you guys know we are live with our first Kafka use
> case.
>
> You'll see more from me in the coming weeks about it.
>
> We released the feature for the public on Monday of this week and have
> been slowly rolling it out to our users.
>
> We will continue to gradually ramp up usage until we get to 100%.
>
> There is still a lot to do with our implementation and more use cases
> are coming soon.
>
> But so far I am very happy with Kafka it has met all of my
> expectations and lived up to all of the claims made by the LinkedIn
> team.
>
> Thank you guys so much for OSS'ing such a great piece of technology.
>

--
--
*Evan Chan*
Senior Software Engineer |
[EMAIL PROTECTED] | (650) 996-4600
www.ooyala.com | blog <http://www.ooyala.com/blog> |
@ooyala<http://www.twitter.com/ooyala>

Taylor,

Thanks for sharing this. Do you think that you can add your usage to our
wiki?

Look forward to various contributions (bug reports, patches, etc) from you
to make this project better.

Jun

On Fri, Nov 18, 2011 at 9:07 AM, Taylor Gautier <[EMAIL PROTECTED]> wrote:

> I just wanted to let you guys know we are live with our first Kafka use
> case.
>
> You'll see more from me in the coming weeks about it.
>
> We released the feature for the public on Monday of this week and have
> been slowly rolling it out to our users.
>
> We will continue to gradually ramp up usage until we get to 100%.
>
> There is still a lot to do with our implementation and more use cases
> are coming soon.
>
> But so far I am very happy with Kafka it has met all of my
> expectations and lived up to all of the claims made by the LinkedIn
> team.
>
> Thank you guys so much for OSS'ing such a great piece of technology.
>

Not to be left out, our first Kafka use case at Crowdflower went live
on Monday. We've completely replaced a rather large portion of
postgres-sweep-based statistics calculations with 2 kafka data
channels (handling a major fraction of all the data we touch) and a
handful of scala processes. Working well so far, and more uses cases
from this data will result over time.

The net result is a rather dramatic drop in write and read load to our
master postgres cluster. Our engineering blog will probably talk about
it publicly soon, because we're really quite happy with the resulting
system; it's simple, performs well, and is quite extensible thanks to
the low overhead of adding new consumer groups to existing channels.

- dlf

On Fri, Nov 18, 2011 at 9:07 AM, Taylor Gautier <[EMAIL PROTECTED]> wrote:
> I just wanted to let you guys know we are live with our first Kafka use case.
>
> You'll see more from me in the coming weeks about it.
>
> We released the feature for the public on Monday of this week and have
> been slowly rolling it out to our users.
>
> We will continue to gradually ramp up usage until we get to 100%.
>
> There is still a lot to do with our implementation and more use cases
> are coming soon.
>
> But so far I am very happy with Kafka it has met all of my
> expectations and lived up to all of the claims made by the LinkedIn
> team.
>
> Thank you guys so much for OSS'ing such a great piece of technology.
>

--
--
Dave Fayram
[EMAIL PROTECTED]

Dave,

Thanks for the update. Could you add your use case to our wiki too?

Look forward to feedback and contributions from you in the future.

Jun

Guys,

I've updated our "Powered By" wiki to include LinkedIn -
https://cwiki.apache.org/confluence/display/KAFKA/Powered+By

Please feel free to update it to include your respective companies and a
short description of the usage.

Thanks,
Neha

On Fri, Nov 18, 2011 at 10:37 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Dave,
>
> Thanks for the update. Could you add your use case to our wiki too?
>
> Look forward to feedback and contributions from you in the future.
>
> Jun
>

Great! I should say that we're now live with Kafka too - and we'll
soon have all our incoming data flow through it.

We love it so far, and it's absolutely perfect for our use cases. Great work!
Best,
Olivier.
--
Check us out at http://datadoghq.com
On Fri, Nov 18, 2011 at 4:31 PM, Neha Narkhede <[EMAIL PROTECTED]> wrote:
> Guys,
>
> I've updated our "Powered By" wiki to include LinkedIn -
> https://cwiki.apache.org/confluence/display/KAFKA/Powered+By
>
> Please feel free to update it to include your respective companies and a
> short description of the usage.
>
> Thanks,
> Neha
>
> On Fri, Nov 18, 2011 at 10:37 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
>> Dave,
>>
>> Thanks for the update. Could you add your use case to our wiki too?
>>
>> Look forward to feedback and contributions from you in the future.
>>
>> Jun
>>
>

I see that you removed the "Application and Usage" page. I was hoping we
would get some users of Kafka detailing their use cases rather than the
default high level "It powers my ..."

On 11/18/11 1:37 PM, Olivier Pomel wrote:
> Great! I should say that we're now live with Kafka too - and we'll
> soon have all our incoming data flow through it.
>
> We love it so far, and it's absolutely perfect for our use cases. Great work!
> Best,
> Olivier.
> --
> Check us out at http://datadoghq.com
>
>
>
>
> On Fri, Nov 18, 2011 at 4:31 PM, Neha Narkhede<[EMAIL PROTECTED]>  wrote:
>> Guys,
>>
>> I've updated our "Powered By" wiki to include LinkedIn -
>> https://cwiki.apache.org/confluence/display/KAFKA/Powered+By
>>
>> Please feel free to update it to include your respective companies and a
>> short description of the usage.
>>
>> Thanks,
>> Neha
>>
>> On Fri, Nov 18, 2011 at 10:37 AM, Jun Rao<[EMAIL PROTECTED]>  wrote:
>>
>>> Dave,
>>>
>>> Thanks for the update. Could you add your use case to our wiki too?
>>>
>>> Look forward to feedback and contributions from you in the future.
>>>
>>> Jun
>>>

Our second use case is live.  Traffic will be ramping soon.  First use case
is continuing to roll out to more of our user base.

On Fri, Nov 18, 2011 at 3:04 PM, Mark <[EMAIL PROTECTED]> wrote:

> I see that you removed the "Application and Usage" page. I was hoping we
> would get some users of Kafka detailing their use cases rather than the
> default high level "It powers my ..."
>
>
> On 11/18/11 1:37 PM, Olivier Pomel wrote:
>
>> Great! I should say that we're now live with Kafka too - and we'll
>> soon have all our incoming data flow through it.
>>
>> We love it so far, and it's absolutely perfect for our use cases. Great
>> work!
>> Best,
>> Olivier.
>> --
>> Check us out at http://datadoghq.com
>>
>>
>>
>>
>> On Fri, Nov 18, 2011 at 4:31 PM, Neha Narkhede<neha.narkhede@gmail.**com<[EMAIL PROTECTED]>>
>>  wrote:
>>
>>> Guys,
>>>
>>> I've updated our "Powered By" wiki to include LinkedIn -
>>> https://cwiki.apache.org/**confluence/display/KAFKA/**Powered+By<https://cwiki.apache.org/confluence/display/KAFKA/Powered+By>
>>>
>>> Please feel free to update it to include your respective companies and a
>>> short description of the usage.
>>>
>>> Thanks,
>>> Neha
>>>
>>> On Fri, Nov 18, 2011 at 10:37 AM, Jun Rao<[EMAIL PROTECTED]>  wrote:
>>>
>>>  Dave,
>>>>
>>>> Thanks for the update. Could you add your use case to our wiki too?
>>>>
>>>> Look forward to feedback and contributions from you in the future.
>>>>
>>>> Jun
>>>>
>>>>