Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Re: FLUME AVRO


+
Harsh J 2012-08-12, 13:15
+
Mike Percy 2012-08-13, 00:29
Copy link to this message
-
Re: FLUME AVRO
Mohit Anchlia 2012-08-13, 00:34
On Sun, Aug 12, 2012 at 5:29 PM, Mike Percy <[EMAIL PROTECTED]> wrote:

> Assuming you are writing to a file or HDFS then look at the
> EventSerializer interface - there is an abstract class that implements that
> interface which you can use for writing Avro.
>
>
> https://cwiki.apache.org/confluence/display/FLUME/Flume+1.x+Event+Serializers
>
> http://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/serialization/EventSerializer.html
>
> This is an out-of-the-box Avro serializer that ships with Flume (its alias
> is AVRO_EVENT):
>
> http://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/serialization/FlumeEventAvroEventSerializer.html
>
> If you want to use your own Avro schema then you can just implement this
> abstract class and override the convert() method:
>
> http://flume.apache.org/releases/content/1.2.0/apidocs/org/apache/flume/serialization/AbstractAvroEventSerializer.html
>
>

Is the data that is writen in the hdfs file in avro format and uses avro
datafile? From what I understand data that is writen in HDFS is not in Avro
format and goes in the sequence file.

> Regards,
> Mike
>
>
> On Sun, Aug 12, 2012 at 6:15 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Abhishek,
>>
>> Moving this to user@flume lists, as it is Flume specific.
>>
>> P.s. Please do not cross post to multiple lists, it does not guarantee
>> you a faster response nor is mailing to a *-dev list relevant to your
>> question here. Help avoid additional inbox noise! :)
>>
>> On Thu, Aug 9, 2012 at 10:43 PM, abhiTowson cal
>> <[EMAIL PROTECTED]> wrote:
>> > hi all,
>> >
>> > can log data be converted into avro,when data is sent from source to
>> sink.
>> >
>> > Regards
>> > Abhishek
>>
>>
>>
>> --
>> Harsh J
>>
>
>
+
Mike Percy 2012-08-13, 00:50