Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> ArrayWritable usage


Copy link to this message
-
Re: ArrayWritable usage
ArrayWritable doesn't serialize type information. You need to subclass it
(e.g. IntArrayWritable) and create a no arg constructor which calls
super(IntWritable.class).

Use this instead of ArrayWritable directly. If you want to store more than
one type, look at the source for MapWritable to see how it generates type
codes.

-Joey
On Jul 4, 2011 2:55 PM, "Dhruv Kumar" <[EMAIL PROTECTED]> wrote:
> I'm having some difficulty with using ArrayWritable in the following test
> code:
>
> ArrayWritable array = new ArrayWritable(IntWritable.class);
>
> IntWritable[] ints = new IntWritable[4];
>
> for (int i =0 ; i < 4; i++) {
> ints[i] = new IntWritable(i);
>
> }
>
> array.set(ints);
>
> writer.append(new LongWritable(1), array);
>
> writer.close();
>
> After starting the map reduce application's test, I'm getting the
following
> errors which I suspect are due to improper use of ArrayWritable. Can
someone
> tell me if the code given above is correct?
>
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.io.ArrayWritable.<init>()
> at
>
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> at
>
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:62)
> at
>
org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
> at
>
org.apache.hadoop.io.SequenceFile$Reader.deserializeValue(SequenceFile.java:1817)
> at
>
org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:1790)
> at
>
org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:74)
> at
>
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:531)
> at
> org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
>
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
> at org.apache.hadoop.mapred.Child.main(Child.java:253)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.io.ArrayWritable.<init>()
> at java.lang.Class.getConstructor0(Class.java:2706)
> at java.lang.Class.getDeclaredConstructor(Class.java:1985)
> at
>
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
> ... 15 more
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB