Hi Kafka users and developers,
We are going to host our quarterly Stream Processing Meetup@LinkedIn on Dec
4. There will be three speakers from Slack, Uber and LinkedIn. Please check
the details below if you are interested.
Jiangjie (Becket) Qin
*Stream Processing with Apache Kafka & Apache Samza*
- Meetup Link: here
- When: Dec 4th 2017 @ 6:00pm
- Where: LinkedIn Building F , 605 West Maude Avenue, Sunnyvale, CA (edit
1. Stream processing using Samza-SQL @ LinkedIn
*Speaker: Srinivasulu Punuru, LinkedIn*
Imagine if you can develop and run a stream processing job in few minutes
and Imagine if a vast majority of your organization (business analysts,
Product manager, Data scientists) can do this on their own without a need
for a development team.
Need for real time insights into the big data is increasing at a rapid
pace. The traditional Java based development model of developing, deploying
and managing the stream processing application is becoming a huge
With Samza SQL we can simplify application development by enabling users to
create stream processing applications and get real time insights into their
business using SQL statements.
In this talk we try to answer the following questions
1. How SQL language can be used to perform stream processing?
2. How is Samza SQL implemented - Architecture?
3. How can you deploy Samza SQL in your company?
2. Streaming data pipeline @ Slack
*Speaker:- Ananth Packkildurai, Slack*
*Abstract: *Slack is a communication and collaboration platform for teams.
Our millions of users spend 10+ hrs connected to the service on a typical
working day. They expect reliability, low latency, and extraordinarily rich
client experiences across a wide variety of devices and network conditions.
It is crucial for the developers to get the realtime insights on Slack
In this talk, I will talk about how our data platform evolves from the
batch system to near realtime. I will also touch base on how Samza helps us
to build low latency data pipelines & Experimentation framework.
3. Improving Kafka at-least-once performance
*Speaker: Ying Zheng, Uber*
At Uber, we are seeing an increased demand for Kafka at-least-once
delivery. So far, we are running a dedicated at-least-once Kafka cluster
with special settings. With a very low workload, the dedicated
at-least-once cluster has been working well for more than a year. Now, when
we want to turn on at-least-once producing on all the Kafka clusters, the
at-least-once producing performance is one of the concerns. I have worked a
couple of months to investigate the Kafka performance issues. With Kafka
code changes and Kafka / Java configuration changes, I have reduced
at-least-once producing latency by about 60% to 70%. Some of those
improvements can also improve the general Kafka throughput or reducing
end-to-end Kafka latency, when ack = 0 or ack = 1.