Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Converting array<tinyint> in Flume avro default output to string in Hive


Copy link to this message
-
Converting array<tinyint> in Flume avro default output to string in Hive
Hi,

Anyone tried reading the default avro output from flume in Hive.

I am using Flume to generate events in the default flume avro output
format. Bytes in avro schema are stored as array<tinyint> in Hive when I
use avroserde for hive . How do I convert array<tinyint> to string to read
the flume body data. I am using hive version 0.10

CREATE  external TABLE flume_avro_test ROW FORMAT
    > SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe'
    > STORED AS
    > INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
    > OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
    > LOCATION '/testlogs/2013/11/08/17'
    > TBLPROPERTIES
('avro.schema.literal'='{"type":"record","name":"Event","fields":[{"name":"headers","type":{"type":"map","values":"string"}},{"name":"body","type":"bytes"}]}');
describe flume_avro_test
    > ;
OK
headers map<string,string> from deserializer
body array<tinyint> from deserializer

Thanks,
Deepak Subhramanian