Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume NG MySQL sink

Copy link to this message
Re: Flume NG MySQL sink
Well, like I mentioned, I'm still new to Flume so maybe I'm totally of
course here, but how about an implementation similar to HBaseSink ? replace
the list of Puts with a list of Strings of insert statements ? execute them
in batches ?
On Thu, Oct 10, 2013 at 7:32 PM, Steve Morin <[EMAIL PROTECTED]> wrote:

> I would be interested in knowing your use-case too
> On Thu, Oct 10, 2013 at 8:47 AM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
>> Hi Amit,
>> to my best knowledge, there is no MySQL sink that would store data from
>> Flume in MySQL. Would you mind describing your use case? I would be
>> interesting to hear how the MySQL fits into the use case. Flume is heavy
>> distributed system for collecting and injesting data, that can very easily
>> overload single MySQL server, so I'm wondering if you are planning to have
>> some sort of sharding or perhaps a mysql cluster to sustain the load?
>> Jarcec
>> On Thu, Oct 10, 2013 at 09:10:34AM +0200, Amit Sela wrote:
>> > Hi all,
>> >
>> > I'm new to Flume and I would like to know if anyone used it to aggregate
>> > logs and process them into MySQL (JDBC) inserts ?
>> >
>> > Thanks.