Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Table Wrapper


Copy link to this message
-
Re: Table Wrapper
Good luck, bro. :)   May i ask why are you doing this to yourself? I think
your instincts are correct going down the path you describe sounds a tad
more painful than just hitting yourself in the head with a hammer.
Different strokes for different folks though.

so can we back up? what - briefly if possible - do you want to achieve with
a "wrapper"? (i'm going to regret asking that i know.)
On Tue, Jun 25, 2013 at 7:29 AM, Peter Marron <
[EMAIL PROTECTED]> wrote:

>  Hi,****
>
> ** **
>
> Running Hive 0.11.0 over Hadoop 1.0.4.****
>
> ** **
>
> I would like to be able to “wrap” a Hive table.****
>
> ** **
>
> So, if I have table “X” which uses SerDe “s” and InputFormat “i”****
>
> then I would like to be able to create a table “Y” which has a ****
>
> SerDe “ws” which is a wrapper of “s” (and so can encapsulate an instance
> of “s”)****
>
> and an InputFormat “wi” which is a wrapper of “I” (and similarly
> encapsulates an****
>
> instance of “i”).  So far I have done this by creating a table like this**
> **
>
> ** **
>
> CREATE TABLE Y (… copy of underlying table’s columns...)****
>
> ROW FORMAT SERDE 'ws'****
>
> WITH SERDEPROPERTIES (…****
>
> 'wrapped.serde.name'='org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe',
> ****
>
> ‘wrapped.inputformat.name’=’TextInputFormat’,****
>
> 'serialization.format'='|',              'field.delim'='|'****
>
> )****
>
> STORED AS****
>
>   INPUTFORMAT 'wi'****
>
>   OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'
> ****
>
> TBLPROPERTIES (…);****
>
> ** **
>
> I have to add the names of the underlying classes “s” and “I”****
>
> into the table properties so that I know what to instantiate.****
>
> I also have to replicate all the column details of the wrapped table****
>
> to ensure the correct information is passed down to the underlying SerDe**
> **
>
> when I instantiate it. I also have to know the output format. I have to***
> *
>
> explicitly add the default SerDe properties to get it to work.****
>
> I have to explicitly provide the default output format too.****
>
> If any changes are made to the underlying table then I need to reflect****
>
> those changes in my “wrapper” table.****
>
> It’s a mess.****
>
> ** **
>
> What I’d like to be able to do is to just parameterise my wrapper****
>
> table with the name of the underlying table and using that name****
>
> be able to instantiate the correct SerDe and InputFormat.****
>
> ** **
>
> Is there an easier way to do this?****
>
> ** **
>
> Any pointers appreciated.****
>
> ** **
>
> Z****
>
> ** **
>