My team and I have been trying, with limited success, to use the COMMENT
feature of hive columns to maintain documentation for the tables and
columns in our data-warehouse built on hive. However, we use a number of
custom and non-native SerDes, and what happens to those tables is that
the comments always get overwritten with the string "from deserializer".

I've possibly found a way to work around this from within hive but I
want to get some insight from the hive-dev community to figure out
whether or not this is a patently bad idea and we are just setting
ourselves up for pain later on.

I won't go into all the details but it seems to work in our (so far)
limited testing. However, we are using hive 0.7.1 and the patch I am
sending is against master/HEAD.

Please let me know if this is an acceptable approach to preserving
column comments with non-native SerDes or not!

This was fixed in Hive 8 (https://issues.apache.org/jira/browse/HIVE-2171).
 Can you just apply that patch?

On Mon, Aug 6, 2012 at 2:15 PM, Stephen R. Scaffidi <
[EMAIL PROTECTED]> wrote:

> My team and I have been trying, with limited success, to use the COMMENT
> feature of hive columns to maintain documentation for the tables and
> columns in our data-warehouse built on hive. However, we use a number of
> custom and non-native SerDes, and what happens to those tables is that the
> comments always get overwritten with the string "from deserializer".
>
> I've possibly found a way to work around this from within hive but I want
> to get some insight from the hive-dev community to figure out whether or
> not this is a patently bad idea and we are just setting ourselves up for
> pain later on.
>
> I won't go into all the details but it seems to work in our (so far)
> limited testing. However, we are using hive 0.7.1 and the patch I am
> sending is against master/HEAD.
>
> Please let me know if this is an acceptable approach to preserving column
> comments with non-native SerDes or not!
>
>
>

Thanks! I'll see how it goes!

(better yet, this could be what it takes to convince the team to upgrade!)

On Aug 6, 2012, at 6:47 PM, Jakob Homan wrote:

> This was fixed in Hive 8 (https://issues.apache.org/jira/browse/HIVE-2171).  Can you just apply that patch?
>
> On Mon, Aug 6, 2012 at 2:15 PM, Stephen R. Scaffidi <[EMAIL PROTECTED]> wrote:
> My team and I have been trying, with limited success, to use the COMMENT feature of hive columns to maintain documentation for the tables and columns in our data-warehouse built on hive. However, we use a number of custom and non-native SerDes, and what happens to those tables is that the comments always get overwritten with the string "from deserializer".
>
> I've possibly found a way to work around this from within hive but I want to get some insight from the hive-dev community to figure out whether or not this is a patently bad idea and we are just setting ourselves up for pain later on.
>
> I won't go into all the details but it seems to work in our (so far) limited testing. However, we are using hive 0.7.1 and the patch I am sending is against master/HEAD.
>
> Please let me know if this is an acceptable approach to preserving column comments with non-native SerDes or not!
>
>
>

So, the patch doesn not seem to fix the problem we are having, but it
combined with the one I sent to the list seems to take care of it. I
will continue to study this issue and report back on any issues.

Thanks again!

On 08/06/2012 06:59 PM, Stephen Scaffid wrote:
> Thanks! I'll see how it goes!
>
> (better yet, this could be what it takes to convince the team to upgrade!)
>
> On Aug 6, 2012, at 6:47 PM, Jakob Homan wrote:
>
>> This was fixed in Hive 8 (https://issues.apache.org/jira/browse/HIVE-2171).  Can you just apply that patch?
>>
>> On Mon, Aug 6, 2012 at 2:15 PM, Stephen R. Scaffidi <[EMAIL PROTECTED]> wrote:
>> My team and I have been trying, with limited success, to use the COMMENT feature of hive columns to maintain documentation for the tables and columns in our data-warehouse built on hive. However, we use a number of custom and non-native SerDes, and what happens to those tables is that the comments always get overwritten with the string "from deserializer".
>>
>> I've possibly found a way to work around this from within hive but I want to get some insight from the hive-dev community to figure out whether or not this is a patently bad idea and we are just setting ourselves up for pain later on.
>>
>> I won't go into all the details but it seems to work in our (so far) limited testing. However, we are using hive 0.7.1 and the patch I am sending is against master/HEAD.
>>
>> Please let me know if this is an acceptable approach to preserving column comments with non-native SerDes or not!
>>
>>
>>
>
>

You'll need to update your serde to use the method call that takes comments.  See https://github.com/jghoman/haivvreo/commit/29ead1fe101baafa8e9844eaf92022cbe4846c6f for an example.  
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Tuesday, August 7, 2012 at 7:21 AM, Stephen R. Scaffidi wrote:

> So, the patch doesn not seem to fix the problem we are having, but it
> combined with the one I sent to the list seems to take care of it. I
> will continue to study this issue and report back on any issues.
>
> Thanks again!
>
> On 08/06/2012 06:59 PM, Stephen Scaffid wrote:
> > Thanks! I'll see how it goes!
> >
> > (better yet, this could be what it takes to convince the team to upgrade!)
> >
> > On Aug 6, 2012, at 6:47 PM, Jakob Homan wrote:
> >
> > > This was fixed in Hive 8 (https://issues.apache.org/jira/browse/HIVE-2171). Can you just apply that patch?
> > >
> > > On Mon, Aug 6, 2012 at 2:15 PM, Stephen R. Scaffidi <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])> wrote:
> > > My team and I have been trying, with limited success, to use the COMMENT feature of hive columns to maintain documentation for the tables and columns in our data-warehouse built on hive. However, we use a number of custom and non-native SerDes, and what happens to those tables is that the comments always get overwritten with the string "from deserializer".
> > >
> > > I've possibly found a way to work around this from within hive but I want to get some insight from the hive-dev community to figure out whether or not this is a patently bad idea and we are just setting ourselves up for pain later on.
> > >
> > > I won't go into all the details but it seems to work in our (so far) limited testing. However, we are using hive 0.7.1 and the patch I am sending is against master/HEAD.
> > >
> > > Please let me know if this is an acceptable approach to preserving column comments with non-native SerDes or not!