Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Questions about SequenceFiles


Copy link to this message
-
Re: Questions about SequenceFiles
I think this is a bug, writable object should have default no-argument
constructor.
On Tue, May 11, 2010 at 7:20 AM, Ananth Sarathy
<[EMAIL PROTECTED]> wrote:
> Yeah, no I get that. But when you use the sequence file reader example from
> The Hadoop The Defintive Guide book page 106
>
>        reader = new SequenceFile.Reader(fs, path, conf);
>             System.out.println(reader.getKeyClass());
>            System.out.println(reader.getValueClass());
>
>            Writable key = (Writable) ReflectionUtils.newInstance(reader
>                    .getKeyClass(), conf);
>            Writable val = (Writable) ReflectionUtils.newInstance(reader
>                    .getValueClass(), conf);
>
>            LuceneDocumentWrapper ldw = null;
>
>            long position = reader.getPosition();
>            while (reader.next(key, val)) {
>
>                ldw = (LuceneDocumentWrapper) val;
>                System.out.println(ldw.get());
>
>  }
>
> But when using a LuceneDocumentWrapper which uses the interface, I get this
> error
>
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
>    at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
>    at
> com.iswcorp.mapreduce.test.SequenceFileReaderTest.main(SequenceFileReaderTest.java:39)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
>    at java.lang.Class.getConstructor0(Class.java:2706)
>
> Caused by this line    Writable val = (Writable)
> ReflectionUtils.newInstance(reader
>                    .getValueClass(), conf);
>
> which has to do with not having a default constructor, which is why I asked
> the orginal question. Is there some other way to get the values out?
> Ananth T Sarathy
>
>
> On Mon, May 10, 2010 at 11:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Writable is the recommended interface to work with.
>> Writable implementations reuse instances which serves large scale data
>> processing better than JavaSerialization.
>>
>> Cheers
>>
>> On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
>> <[EMAIL PROTECTED]>wrote:
>>
>> > My team and I were working with sequence files and were using the
>> > LuceneDocumentWrapper. But when I try to get the valcall, i get a no such
>> > method exception from the ReflectionUtils, which is caused because it's
>> > trying to call a default constructor which doesn't exist for that class.
>> >
>> > So my question is  whether there is documentation or limitations to the
>> > type
>> > of objects that can be used with a sequencefile other than the Writable
>> > interface? I want to know if maybe I am trying to read from the file in
>> the
>> > wrong way.
>> > Ananth T Sarathy
>> >
>>
>

--
Best Regards

Jeff Zhang