Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Writing Custom Serdes for Hive


+
John Omernik 2012-10-16, 15:30
+
Connell, Chuck 2012-10-16, 15:42
+
John Omernik 2012-10-16, 15:54
+
shrikanth shankar 2012-10-16, 16:09
+
John Omernik 2012-10-16, 16:29
Copy link to this message
-
Re: Writing Custom Serdes for Hive
Ariel Marcus 2012-10-16, 15:41
John,

This article is pretty good:
http://www.congiu.com/a-json-readwrite-serde-for-hive/

It looks like you can get access to the information you are looking for in
the Properties object passed to the SerDe's initialize method.

 public void initialize(Configuration conf, Properties tbl) throws
SerDeException

Look here for an example of usage:
https://github.com/rcongiu/Hive-JSON-Serde/blob/master/src/main/java/org/openx/data/jsonserde/JsonSerDe.java

Hope that helps.

Ariel

---------------------------------
Ariel Marcus, Consultant
www.openbi.com | [EMAIL PROTECTED]
150 N Michigan Avenue, Suite 2800, Chicago, IL 60601
Cell: 314-827-4356

On Tue, Oct 16, 2012 at 11:30 AM, John Omernik <[EMAIL PROTECTED]> wrote:

> We have a maybe obvious question about a serde. When a serde in invoked,
> does it have access to the original hive query?  Ideally the original query
> could provide the Serde some hints on how to access the data on the
> backend.
>
> Also, are there any good links/documention on how to write Serdes?  Kinda
> hard to google on for some reason.
>
>
>
> ------------------------------
>
> This transmission is confidential and intended solely for the use of the
> recipient named above. It may contain confidential, proprietary, or legally
> privileged information. If you are not the intended recipient, you are
> hereby notified that any unauthorized review, use, disclosure or
> distribution is strictly prohibited. If you have received this transmission
> in error, please contact the sender by reply e-mail and delete the original
> transmission and all copies from your system.
>

--

------------------------------

This transmission is confidential and intended solely for the use of the
recipient named above. It may contain confidential, proprietary, or legally
privileged information. If you are not the intended recipient, you are
hereby notified that any unauthorized review, use, disclosure or
distribution is strictly prohibited. If you have received this transmission
in error, please contact the sender by reply e-mail and delete the original
transmission and all copies from your system.