Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> SerDe issue


Copy link to this message
-
Re: SerDe issue
Thanks Zheng,
Didn't think of that.
that worked!
R.

On Fri, Feb 12, 2010 at 1:19 PM, Zheng Shao <[EMAIL PROTECTED]> wrote:

> Hi Roberto,
>
> The reason that Text is passed in is because the table is defined as
> TextFile format (the default).
>
> There are some examples (*.q files) of using SequenceFile format (
> CREATE TABLE xxx .... STORED AS SEQUENCEFILE).
> SEQUENCEFILE will return BytesWritable by default.
>
> Please have a try.
>
> Zheng
>
> On Fri, Feb 12, 2010 at 1:05 PM, Roberto Congiu
> <[EMAIL PROTECTED]> wrote:
> > Hey guys,
> > I wrote a SerDe to support lwes (http://lwes.org) using
> BinarySortableSerDe
> > as a model.
> > The code is very similar, and I serialize an lwes event to a
> BytesWritable,
> > and deserialize from it.
> > Serialization is fine...however, when I run an insert into... select, the
> > Deserialize methods is passed a Text object instead of a BytesWritable
> > object like expected.
> > Hive generates 2 jobs, and it fails on the mapper in the second.
> > getSerializedClass() is set correctly:
> >  public Class<? extends Writable> getSerializedClass() {
> >         LOG.debug("JournalSerDe::getSerializedClass()");
> >         return BytesWritable.class;
> >     }
> > And I don't see any relevant difference between BinarySortableSerDe and
> my
> > code.
> > Does anybody have a hint on what may be happening ?
> > Thanks,
> > Roberto
>
>
>
> --
> Yours,
> Zheng
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB