Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: custom writablecomparable with complex fields


Copy link to this message
-
Re: custom writablecomparable with complex fields
The idea behind write(…) and readFields(…) is simply that of ordering.
You need to write your custom objects (i.e. a representation of them)
in order, and read them back in the same order.

An example way of serializing a list would be to first serialize the
length (so you know how many you'll be needed to read back), and then
serialize each item appropriately, using delimiters, or using
length-prefixes just like lists.

Mainly, you're required to tackle the serialization/deserialization on your own.

This is one of the reasons I highly recommend using a library like
Apache Avro instead. Its more powerful, faster, and yet simple to use:
http://avro.apache.org/docs/current/gettingstartedjava.html and
http://avro.apache.org/docs/current/mr.html. It is also popular and
carries first-grade support on several other hadoop-ecosystem
projects, such as Flume and Crunch.

On Sun, Sep 1, 2013 at 1:23 AM, Adeel Qureshi <[EMAIL PROTECTED]> wrote:
> I want to write a custom writablecomparable object with two List objects
> within it ..
>
> public class CompositeKey implements WritableComparable {
>
> private List<JsonKey> groupBy;
> private List<JsonKey> sortBy;
> ...
> }
>
> what I am not sure about is how to write
>
> readFields and write methods for this object. Any help would be appreciated.
>
> Thanks
> Adeel

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB