Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - SerDe issue


Copy link to this message
-
SerDe issue
Roberto Congiu 2010-02-12, 21:05
Hey guys,
I wrote a SerDe to support lwes (http://lwes.org) using BinarySortableSerDe
as a model.

The code is very similar, and I serialize an lwes event to a BytesWritable,
and deserialize from it.

Serialization is fine...however, when I run an insert into... select, the
Deserialize methods is passed a Text object instead of a BytesWritable
object like expected.

Hive generates 2 jobs, and it fails on the mapper in the second.

getSerializedClass() is set correctly:

 public Class<? extends Writable> getSerializedClass() {
        LOG.debug("JournalSerDe::getSerializedClass()");
        return BytesWritable.class;
    }

And I don't see any relevant difference between BinarySortableSerDe and my
code.
Does anybody have a hint on what may be happening ?

Thanks,
Roberto