Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Writing unit tests against HBase


Copy link to this message
-
Re: Writing unit tests against HBase
Varun Sharma 2013-06-20, 23:36
Hi Christopher, I am looking into getting maven 3.0.4 installed before i
start playing with it.

Elliot, does the HBaseTestingUtility thing run only against DFS. I tried to
bring up a MiniHBaseCLuster with distributed=false and got the following
trail when trying to run without a MiniDfsCluster while trying to replicate
the logs:

INFO:  for /tmp/hbase-feeds/hbase/.logs/10.100.12.223,50594,1371770077629/
10.100.12.223%2C50594%2C1371770077629.1371770079566
Jun 20, 2013 4:14:39 PM org.apache.hadoop.hbase.regionserver.wal.HLog
getGetNumCurrentReplicas
INFO: FileSystem's output stream doesn't support getNumCurrentReplicas;
--HDFS-826 not available;
fsOut=org.apache.hadoop.fs.ChecksumFs$ChecksumFSOutputSummer
Jun 20, 2013 4:14:39 PM org.apache.hadoop.hbase.regionserver.HRegionServer
stop
INFO: STOPPED: Failed initialization
Jun 20, 2013 4:14:39 PM org.apache.hadoop.hbase.regionserver.HRegionServer
cleanup
SEVERE: Failed init

Is there a fix for this or can I disable the WAL here completely ?

Varun
On Thu, Jun 20, 2013 at 12:12 PM, Christophe Taton <[EMAIL PROTECTED]>wrote:

> Hey Varun,
>
> On Thu, Jun 20, 2013 at 11:56 AM, Varun Sharma <[EMAIL PROTECTED]>
> wrote:
>
> > Now that I think about it - this may not be as simple. The client could
> mix
> > its calls with Filters etc. which the fake in memory HTable would need to
> > support. It seems the best route would be to run the end to end test like
> > Elliot suggested.
> >
>
> I am using fake HTables with filters. In fact, I am also testing custom
> filters with it too.
> Are there particular features that you need and FakeHBase is missing?
>
> Thanks,
> C.
>
> Varun
> >
> >
> > On Tue, Jun 18, 2013 at 4:46 PM, Elliott Clark <[EMAIL PROTECTED]>
> wrote:
> >
> > > The other option (not really a true unit test I agree) is to use
> > > HBaseTestingUtility [1].  It spins up a pretty full cluster inside of
> > your
> > > junit tests and will behave pretty close to what a real cluster will.
> > >
> > > 1.
> > >
> > >
> >
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
> > >
> > >
> > > On Tue, Jun 18, 2013 at 4:22 PM, Stack <[EMAIL PROTECTED]> wrote:
> > >
> > > > On Tue, Jun 18, 2013 at 4:17 PM, Varun Sharma <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > If I wanted to write to write a unit test against HTable/HBase, is
> > > there
> > > > an
> > > > > already available utility to that for unit testing my application
> > > logic.
> > > > >
> > > > > I don't want to write code that either touches production or
> requires
> > > me
> > > > to
> > > > > mock an HTable. I am looking for a test htable object which behaves
> > > > pretty
> > > > > close to a real HTable.
> > > > >
> > > >
> > > >
> > > > Would this help if we included it?
> > > > https://github.com/kijiproject/fake-hbase/
> > > >
> > > > St.Ack
> > > >
> > >
> >
>