Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Flume C Client


Copy link to this message
-
Re: Flume C Client
Flume does not have any of its components or plugins implemented in C.

So there is no C client for it. It is not really designed in a
client-server architecture.

It was designed intentionally for collecting, aggregating, and transporting
large amounts of data as events within its internal set of components.

Nevertheless, there are components within this architecture (such as
sources, channels and sinks) that interact with external endpoints in a
client-server manner.

Be that as it may, since your objective is to stream data to Flume, you can
implement your own custom event-driven source for Flume and then connect to
this source from your code to bridge in and move data from your system into
flume.

You could set up HTTPSource as the source and then use libcurl within your
C code to send events to the HTTPSource for Flume.

http://flume.apache.org/FlumeUserGuide.html#http-source
http://curl.haxx.se/libcurl/c/

Apache Avro has a C api documented here
http://avro.apache.org/docs/current/api/c/index.html

I hope this helps.

On 24 April 2013 19:22, Neil Collier <[EMAIL PROTECTED]> wrote:

> I have setup flume in our organization using Java clients with the SDK.
>  There is a need for us to stream data to flume from our existing C code
> into our flume avro source.
>
> Is there an available C client or documentation somewhere?  Note: I am not
> a C programmer.
>
> Thanks,
>
> Neil
>
>
>
>
> This email and any files included with it may contain privileged,
> proprietary and/or confidential information that is for the sole use
> of the intended recipient(s).  Any disclosure, copying, distribution,
> posting, or use of the information contained in or attached to this
> email is prohibited unless permitted by the sender.  If you have
> received this email in error, please immediately notify the sender
> via return email, telephone, or fax and destroy this original transmission
> and its included files without reading or saving it in any manner.
> Thank you.
>