Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> how to write a SerDe


Copy link to this message
-
how to write a SerDe
Hi,I am writing a SerDe class to be able to query some proprietary format we
have from hive.
The format is basically a sequence of records that are maps coded in binary
for which we have access libraries.
The file is also gzipped.

For what I understand, I need to
1 - write a FileInputFormat class to read the file and extract the single
records as Writables (but I am not clear how I tell hive to use this
fileformat since all I can use is STORED AS SEQUENCEFILE/TEXTFILE. How do I
plug my format in there? )
2 - Write a SerDe (Since I just need to read it I need just the deserializer
part) and an ObjectInspector to let hive understand how to find a column

is there any info around for these or somebody who's done something similar
?
Thanks in advance,
Roberto
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB