My team and I were working with sequence files and were using the
LuceneDocumentWrapper. But when I try to get the valcall, i get a no such
method exception from the ReflectionUtils, which is caused because it's
trying to call a default constructor which doesn't exist for that class.

So my question is  whether there is documentation or limitations to the type
of objects that can be used with a sequencefile other than the Writable
interface? I want to know if maybe I am trying to read from the file in the
wrong way.
Ananth T Sarathy

Writable is the recommended interface to work with.
Writable implementations reuse instances which serves large scale data
processing better than JavaSerialization.

Cheers

On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
<[EMAIL PROTECTED]>wrote:

> My team and I were working with sequence files and were using the
> LuceneDocumentWrapper. But when I try to get the valcall, i get a no such
> method exception from the ReflectionUtils, which is caused because it's
> trying to call a default constructor which doesn't exist for that class.
>
> So my question is  whether there is documentation or limitations to the
> type
> of objects that can be used with a sequencefile other than the Writable
> interface? I want to know if maybe I am trying to read from the file in the
> wrong way.
> Ananth T Sarathy
>

Yeah, no I get that. But when you use the sequence file reader example from
The Hadoop The Defintive Guide book page 106

        reader = new SequenceFile.Reader(fs, path, conf);
             System.out.println(reader.getKeyClass());
            System.out.println(reader.getValueClass());

            Writable key = (Writable) ReflectionUtils.newInstance(reader
                    .getKeyClass(), conf);
            Writable val = (Writable) ReflectionUtils.newInstance(reader
                    .getValueClass(), conf);

            LuceneDocumentWrapper ldw = null;

            long position = reader.getPosition();
            while (reader.next(key, val)) {

                ldw = (LuceneDocumentWrapper) val;
                System.out.println(ldw.get());

 }

But when using a LuceneDocumentWrapper which uses the interface, I get this
error

java.lang.RuntimeException: java.lang.NoSuchMethodException:
org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
    at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
    at
com.iswcorp.mapreduce.test.SequenceFileReaderTest.main(SequenceFileReaderTest.java:39)
Caused by: java.lang.NoSuchMethodException:
org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
    at java.lang.Class.getConstructor0(Class.java:2706)

Caused by this line    Writable val = (Writable)
ReflectionUtils.newInstance(reader
                    .getValueClass(), conf);

which has to do with not having a default constructor, which is why I asked
the orginal question. Is there some other way to get the values out?
Ananth T Sarathy
On Mon, May 10, 2010 at 11:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Writable is the recommended interface to work with.
> Writable implementations reuse instances which serves large scale data
> processing better than JavaSerialization.
>
> Cheers
>
> On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
> <[EMAIL PROTECTED]>wrote:
>
> > My team and I were working with sequence files and were using the
> > LuceneDocumentWrapper. But when I try to get the valcall, i get a no such
> > method exception from the ReflectionUtils, which is caused because it's
> > trying to call a default constructor which doesn't exist for that class.
> >
> > So my question is  whether there is documentation or limitations to the
> > type
> > of objects that can be used with a sequencefile other than the Writable
> > interface? I want to know if maybe I am trying to read from the file in
> the
> > wrong way.
> > Ananth T Sarathy
> >
>

The class implementing Writable should provide a public default constructor.

On Tue, May 11, 2010 at 7:20 AM, Ananth Sarathy
<[EMAIL PROTECTED]>wrote:

> Yeah, no I get that. But when you use the sequence file reader example from
> The Hadoop The Defintive Guide book page 106
>
>        reader = new SequenceFile.Reader(fs, path, conf);
>             System.out.println(reader.getKeyClass());
>            System.out.println(reader.getValueClass());
>
>            Writable key = (Writable) ReflectionUtils.newInstance(reader
>                    .getKeyClass(), conf);
>            Writable val = (Writable) ReflectionUtils.newInstance(reader
>                    .getValueClass(), conf);
>
>            LuceneDocumentWrapper ldw = null;
>
>            long position = reader.getPosition();
>            while (reader.next(key, val)) {
>
>                ldw = (LuceneDocumentWrapper) val;
>                System.out.println(ldw.get());
>
>  }
>
> But when using a LuceneDocumentWrapper which uses the interface, I get this
> error
>
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
>    at
>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
>    at
>
> com.iswcorp.mapreduce.test.SequenceFileReaderTest.main(SequenceFileReaderTest.java:39)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
>    at java.lang.Class.getConstructor0(Class.java:2706)
>
> Caused by this line    Writable val = (Writable)
> ReflectionUtils.newInstance(reader
>                    .getValueClass(), conf);
>
> which has to do with not having a default constructor, which is why I asked
> the orginal question. Is there some other way to get the values out?
> Ananth T Sarathy
>
>
> On Mon, May 10, 2010 at 11:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Writable is the recommended interface to work with.
> > Writable implementations reuse instances which serves large scale data
> > processing better than JavaSerialization.
> >
> > Cheers
> >
> > On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
> > <[EMAIL PROTECTED]>wrote:
> >
> > > My team and I were working with sequence files and were using the
> > > LuceneDocumentWrapper. But when I try to get the valcall, i get a no
> such
> > > method exception from the ReflectionUtils, which is caused because it's
> > > trying to call a default constructor which doesn't exist for that
> class.
> > >
> > > So my question is  whether there is documentation or limitations to the
> > > type
> > > of objects that can be used with a sequencefile other than the Writable
> > > interface? I want to know if maybe I am trying to read from the file in
> > the
> > > wrong way.
> > > Ananth T Sarathy
> > >
> >
>

I think this is a bug, writable object should have default no-argument
constructor.
On Tue, May 11, 2010 at 7:20 AM, Ananth Sarathy
<[EMAIL PROTECTED]> wrote:
> Yeah, no I get that. But when you use the sequence file reader example from
> The Hadoop The Defintive Guide book page 106
>
>        reader = new SequenceFile.Reader(fs, path, conf);
>             System.out.println(reader.getKeyClass());
>            System.out.println(reader.getValueClass());
>
>            Writable key = (Writable) ReflectionUtils.newInstance(reader
>                    .getKeyClass(), conf);
>            Writable val = (Writable) ReflectionUtils.newInstance(reader
>                    .getValueClass(), conf);
>
>            LuceneDocumentWrapper ldw = null;
>
>            long position = reader.getPosition();
>            while (reader.next(key, val)) {
>
>                ldw = (LuceneDocumentWrapper) val;
>                System.out.println(ldw.get());
>
>  }
>
> But when using a LuceneDocumentWrapper which uses the interface, I get this
> error
>
> java.lang.RuntimeException: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
>    at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
>    at
> com.iswcorp.mapreduce.test.SequenceFileReaderTest.main(SequenceFileReaderTest.java:39)
> Caused by: java.lang.NoSuchMethodException:
> org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
>    at java.lang.Class.getConstructor0(Class.java:2706)
>
> Caused by this line    Writable val = (Writable)
> ReflectionUtils.newInstance(reader
>                    .getValueClass(), conf);
>
> which has to do with not having a default constructor, which is why I asked
> the orginal question. Is there some other way to get the values out?
> Ananth T Sarathy
>
>
> On Mon, May 10, 2010 at 11:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Writable is the recommended interface to work with.
>> Writable implementations reuse instances which serves large scale data
>> processing better than JavaSerialization.
>>
>> Cheers
>>
>> On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
>> <[EMAIL PROTECTED]>wrote:
>>
>> > My team and I were working with sequence files and were using the
>> > LuceneDocumentWrapper. But when I try to get the valcall, i get a no such
>> > method exception from the ReflectionUtils, which is caused because it's
>> > trying to call a default constructor which doesn't exist for that class.
>> >
>> > So my question is  whether there is documentation or limitations to the
>> > type
>> > of objects that can be used with a sequencefile other than the Writable
>> > interface? I want to know if maybe I am trying to read from the file in
>> the
>> > wrong way.
>> > Ananth T Sarathy
>> >
>>
>

--
Best Regards

Jeff Zhang

Ok,  how can I report that?
 Also, it seems that requiring a no argument constructor but using an
interface is kind of a broken paradigm. Shouldn't there be some other
mechanism for this?2
Ananth T Sarathy
On Tue, May 11, 2010 at 10:31 AM, Jeff Zhang <[EMAIL PROTECTED]> wrote:

> I think this is a bug, writable object should have default no-argument
> constructor.
>
>
> On Tue, May 11, 2010 at 7:20 AM, Ananth Sarathy
> <[EMAIL PROTECTED]> wrote:
> > Yeah, no I get that. But when you use the sequence file reader example
> from
> > The Hadoop The Defintive Guide book page 106
> >
> >        reader = new SequenceFile.Reader(fs, path, conf);
> >             System.out.println(reader.getKeyClass());
> >            System.out.println(reader.getValueClass());
> >
> >            Writable key = (Writable) ReflectionUtils.newInstance(reader
> >                    .getKeyClass(), conf);
> >            Writable val = (Writable) ReflectionUtils.newInstance(reader
> >                    .getValueClass(), conf);
> >
> >            LuceneDocumentWrapper ldw = null;
> >
> >            long position = reader.getPosition();
> >            while (reader.next(key, val)) {
> >
> >                ldw = (LuceneDocumentWrapper) val;
> >                System.out.println(ldw.get());
> >
> >  }
> >
> > But when using a LuceneDocumentWrapper which uses the interface, I get
> this
> > error
> >
> > java.lang.RuntimeException: java.lang.NoSuchMethodException:
> > org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
> >    at
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
> >    at
> >
> com.iswcorp.mapreduce.test.SequenceFileReaderTest.main(SequenceFileReaderTest.java:39)
> > Caused by: java.lang.NoSuchMethodException:
> > org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper.<init>()
> >    at java.lang.Class.getConstructor0(Class.java:2706)
> >
> > Caused by this line    Writable val = (Writable)
> > ReflectionUtils.newInstance(reader
> >                    .getValueClass(), conf);
> >
> > which has to do with not having a default constructor, which is why I
> asked
> > the orginal question. Is there some other way to get the values out?
> > Ananth T Sarathy
> >
> >
> > On Mon, May 10, 2010 at 11:46 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >
> >> Writable is the recommended interface to work with.
> >> Writable implementations reuse instances which serves large scale data
> >> processing better than JavaSerialization.
> >>
> >> Cheers
> >>
> >> On Mon, May 10, 2010 at 6:29 PM, Ananth Sarathy
> >> <[EMAIL PROTECTED]>wrote:
> >>
> >> > My team and I were working with sequence files and were using the
> >> > LuceneDocumentWrapper. But when I try to get the valcall, i get a no
> such
> >> > method exception from the ReflectionUtils, which is caused because
> it's
> >> > trying to call a default constructor which doesn't exist for that
> class.
> >> >
> >> > So my question is  whether there is documentation or limitations to
> the
> >> > type
> >> > of objects that can be used with a sequencefile other than the
> Writable
> >> > interface? I want to know if maybe I am trying to read from the file
> in
> >> the
> >> > wrong way.
> >> > Ananth T Sarathy
> >> >
> >>
> >
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Assumption for Writables that should be documented somewhere:
  * Each type must have a 0 argument constructor.
  * Each call to write must not assume any shared state.
  * Each call to readFields must consume exactly the number of bytes
produced by write.

SequenceFile also assumes:
  * All keys are exactly the same type (not polymorphic).
  * All values are exactly the same type.
  * Both types are specified by the writer in the create call.

-- Owen

On Tue, May 11, 2010 at 7:48 AM, Ananth Sarathy
<[EMAIL PROTECTED]> wrote:
> Ok,  how can I report that?

File a jira on the project that manages the type. I assume it is
Lucene in this case.

>  Also, it seems that requiring a no argument constructor but using an
> interface is kind of a broken paradigm. Shouldn't there be some other
> mechanism for this?

The problem is that given a class name from the SequenceFile, we need
to build an "empty" object. The most natural way to provide that
capability is with a 0 argument constructor.

-- Owen

it's actually from Hbase.

I see why the problem exists, but simply documenting that it needs a no
argument constructor isn't really a solution to this particular hiccup with
sequencefiles.  It's not a big deal, we can easily get around it,I just
thought it was worth noting
Ananth T Sarathy
On Tue, May 11, 2010 at 11:04 AM, Owen O'Malley <[EMAIL PROTECTED]> wrote:

> On Tue, May 11, 2010 at 7:48 AM, Ananth Sarathy
> <[EMAIL PROTECTED]> wrote:
> > Ok,  how can I report that?
>
> File a jira on the project that manages the type. I assume it is
> Lucene in this case.
>
> >  Also, it seems that requiring a no argument constructor but using an
> > interface is kind of a broken paradigm. Shouldn't there be some other
> > mechanism for this?
>
> The problem is that given a class name from the SequenceFile, we need
> to build an "empty" object. The most natural way to provide that
> capability is with a 0 argument constructor.
>
> -- Owen
>