Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> SequenceFileLogReader uses a reflection hack resulting in runtime failures

Copy link to this message
Re: SequenceFileLogReader uses a reflection hack resulting in runtime failures
@Stack: I am using hadoop- (the default Hadoop version from
pom.xml). There is a private getFileLength() method, but getMethod() does
not allow to retrieve it. We should use getDeclaredMethod() -- this appears
to work in my testing. I will include that fix in the HBaseClusterTest
diff. Not sure why no one saw this bug before.

@Dhruba: I am running RestartMetaTest, which I am porting from 0.89-fb.
This is a test that starts a local HBase cluster as multiple processes (on
different ports), loads some data, and does a real kill -9 on the
regionserver serving meta. I saw this bug in the data loading part, not
because of killing the regionserver.


On Thu, Dec 1, 2011 at 10:33 PM, Stack <[EMAIL PROTECTED]> wrote:

> On Thu, Dec 1, 2011 at 9:59 PM, Mikhail Bautin
> <[EMAIL PROTECTED]> wrote:
> > 11/12/01 21:40:07 WARN wal.SequenceFileLogReader: Error while trying to
> get
> > accurate file length.  Truncation / data loss may occur if RegionServers
> > die.
> > java.lang.NoSuchMethodException:
> >
> org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.getFileLength()
> Your hadoop doesn't have this Mikhail?
> > Besides, even when it works, the reflection-based solution is probably
> > _much_ slower than straightforward method access. Should we create a
> Hadoop
> > patch to expose the appropriate API call and get rid of the reflection
> hack?
> >
> It would be the sensible thing to do; much more sensible than the
> reflection gymnastics we have going on here.
> St.Ack