Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive to read sequencefile


Copy link to this message
-
Re: Hive to read sequencefile
yes you need an input format. I have one that turns key plus value to a
pairwritable

On Monday, July 9, 2012, Daning Wang <[EMAIL PROTECTED]> wrote:
> Thanks Navis.
>
> The problem is that the key and value are custom classes. e.g, key is
class A, and value is class B. we want to show certain data in Class A and
B from getters. so we do need a SerDe to deserialize the data.
>
> Since SerDe interface can only pass one parameter, so I need to write a
InputFormat to combine key and value to one object, right?
>
>     public Object deserialize(Writable arg0) throws SerDeException {
>
>
> Daning
>
>
>
>
> On Mon, Jul 9, 2012 at 4:56 PM, Navis류승우 <[EMAIL PROTECTED]> wrote:
>>
>> You can try 'STORED AS SEQUENCEFILE' when creating table.
>>
https://cwiki.apache.org/Hive/languagemanual-ddl.html#LanguageManualDDL-Create%252FDropTable
>>
>>
>> 2012/7/10 Daning Wang <[EMAIL PROTECTED]>
>>>
>>> Hi all,
>>>
>>> New to here. I have sequence files  and I want to read in Hive, I did
some search on the net. looks the way I need to do
>>>
>>> 1. Write new FileInputFormat so Hive can read both key and value.
>>> (ref http://osdir.com/ml/hive-user-hadoop-apache/2009-10/msg00034.html)
>>>
>>> 2. Write a SerDe to deserialize custom class data for key and value in
sequence file, I don't need serialize.
>>>
>>> Is this the right way to do? any simple way to do this?
>>>
>>> Thanks,
>>>
>>> Daning
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB