Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - table  from sequence file


Copy link to this message
-
Re: table from sequence file
Arvind Prabhakar 2010-04-15, 19:00
Hi Sagar,

Looks like your source file has custom writable types in it. If that is the
case, implementing a SerDe that works with that type may not be that
straight forward, although doable.

An alternative would be to implement a custom RecordReader that converts the
value of your custom writable to Struct type which can then be queried
directly.

Arvind

On Thu, Apr 15, 2010 at 1:06 AM, Sagar Naik <[EMAIL PROTECTED]> wrote:

> Hi
>
> My data is in the value field of a sequence file.
> The value field has subfields in it. I am trying to create table using
> these subfields.
> Example:
> <KEY> <VALUE>
> <KEY_FIELD1, KEYFIELD 2>  forms the key
> <VALUE_FIELD1, VALUE_FIELD2, VALUE_FIELD3>.
> So i am trying to create a table from VALUE_FIELD*
>
> CREATE EXTERNAL TABLE table_name (VALUE_FIELD1 as BIGINT, VALUE_FIELD2 as
> string, VALUE_FIELD3 as BIGINT ) STORED AS SEQUENCEFILE;
>
> I am planing to a write a custom SerDe implementation and custom
> SequenceFileReader
> Pl let me knw if I am on the right track.
>
>
> -Sagar