Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # dev >> Custom SerDe and getting column comments

Stephen Scaffid 2012-07-26, 23:17
Copy link to this message
Re: Custom SerDe and getting column comments
Currently you can't use comments with custom serde's. Look into
MetaStoreUtils.getFieldsFromDeserializer and you'll see "from
deserializer" is hard-coded as the comment.

This is definitely an area for improvement.

Is your serde reporting columns, or do you have them stored in the
metastore? If you "describe extended" your table do you see the
comments stored in the metastore?


On Thu, Jul 26, 2012 at 4:17 PM, Stephen Scaffid
> I have a custom SerDe we are using and it works well. However, we have one issue with it - an application used to allow users to maintain table and column descriptions does not work with it.
> We store the descriptions as "comments" on the tables and columns, via a python script that simply executes the necessary HQL to set the comment, but then uses the Hive class to retrieve it from metadata.
> The problem is that any comment we try to set ends up getting set to "from deserializer" on any table using our custom SerDe. This doesn't happen with tables using the LazySimpleSerDe.
> Is there a way to make our SerDe work with comments?
Stephen Scaffid 2012-07-27, 12:47