Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Re: Custom output value for map function


Copy link to this message
-
Re: Custom output value for map function
That's right, the date needs to be written and read in the same order.

On Wed, Feb 27, 2013 at 11:04 AM, Paul van Hoven <
[EMAIL PROTECTED]> wrote:

> Great! Thank you.
>
> I guess the order for writing and reading the data this way is
> important. I mean, for
>
> out.writeUTF("blabla")
> out.writeInt(12)
>
> the following would be correct
>
> text = in.readUTF();
> number = in.readInt();
>
> and this would fail:
>
> number = in.readInt();
> text = in.readUTF();
>
> ?
>
> 2013/2/27 Sandy Ryza <[EMAIL PROTECTED]>:
> > Hi Paul,
> >
> > To do this, you need to make your Dog class implement Hadoop's Writable
> > interface, so that it can be serialized to and deserialized from bytes.
> >
> http://hadoop.apache.org/docs/r1.1.1/api/org/apache/hadoop/io/Writable.html
> >
> > The methods you implement would look something like this:
> >
> > public void write(DataOutput out) {
> >   out.writeDouble(weight);
> >   out.writeUTF(name);
> >   out.writeLong(date.toTimeInMillis());
> > }
> >
> > public void readFields(DataInput in) {
> >   weight = in.readDouble();
> >   name = in.readUTF();
> >   date = new Date(in.readLong());
> > }
> >
> > hope that helps,
> > Sandy
> >
> > On Wed, Feb 27, 2013 at 10:34 AM, Paul van Hoven
> > <[EMAIL PROTECTED]> wrote:
> >>
> >> The output value in the map function is in most examples for hadoop
> >> something like this:
> >>
> >> public static class Map extends Mapper<LongWritable, Text, outputKey,
> >> outputValue>
> >>
> >> Normally outputValue is something like Text or IntWriteable.
> >>
> >> I got a custom class with its own properties like
> >>
> >> public class Dog {
> >>    string name;
> >>    Date birthday;
> >>    double weight;
> >> }
> >>
> >> Now how would I accomplish the following map function:
> >>
> >> public static class Map extends Mapper<LongWritable, Text, IntWritable,
> >> Dog>
> >>
> >> ?
> >
> >
>
+
Paul van Hoven 2013-02-27, 18:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB