Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Questions about SequenceFiles


Copy link to this message
-
Re: Questions about SequenceFiles
I think this is a bug, writable object should have default no-argument
constructor.
On Tue, May 11, 2010 at 7:20 AM, Ananth Sarathy
<[EMAIL PROTECTED]> wrote:
> Yeah, no I get that. But when you use the sequence file reader example from
> The Hadoop The Defintive Guide book page 106
>
>        reader = new SequenceFile.Reader(fs, path, conf);
>             System.out.println(reader.getKeyClass());
>            System.out.println(reader.getValueClass());
>
>            Writable key = (Writable) ReflectionUtils.newInstance(reader
>                    .getKeyClass(), conf);
>            Writable val = (Writable) ReflectionUtils.newInstance(reader
>                    .getValueClass(), conf);
>
>            LuceneDocumentWrapper ldw = null;
>
>            long position = reader.getPosition();
>            while (reader.next(key, val)) {
>
>                ldw = (LuceneDocumentWrapper) val;
>                System.out.println(ldw.get());
>
>  }
>
> But when using a LuceneDocumentWrapper which uses the interface, I get this
> error
>
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
>    at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
>    at
> com.iswcorp.mapreduce.test.SequenceFileReaderTest.main(SequenceFileReaderTest.java:39)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
>    at java.lang.Class.getConstructor0(Class.java:2706)
>
> Caused by this line    Writable val = (Writable)
> ReflectionUtils.newInstance(reader
>                    .getValueClass(), conf);
>
> which has to do with not having a default constructor, which is why I asked
> the orginal question. Is there some other way to get the values out?
> Ananth T Sarathy
>
>
> On Mon, May 10, 2010 at 11:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Writable is the recommended interface to work with.
>> Writable implementations reuse instances which serves large scale data
>> processing better than JavaSerialization.
>>
>> Cheers
>>
>> On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
>> <[EMAIL PROTECTED]>wrote:
>>
>> > My team and I were working with sequence files and were using the
>> > LuceneDocumentWrapper. But when I try to get the valcall, i get a no such
>> > method exception from the ReflectionUtils, which is caused because it's
>> > trying to call a default constructor which doesn't exist for that class.
>> >
>> > So my question is  whether there is documentation or limitations to the
>> > type
>> > of objects that can be used with a sequencefile other than the Writable
>> > interface? I want to know if maybe I am trying to read from the file in
>> the
>> > wrong way.
>> > Ananth T Sarathy
>> >
>>
>

--
Best Regards

Jeff Zhang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB