Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Writing unit tests against HBase


Copy link to this message
-
Re: Writing unit tests against HBase
Asaf Mesika 2013-06-21, 05:28
You shouldn't tweak any Configuration at all. It simply works - setting up
both mini HDFS and mini HBase.

On Fri, Jun 21, 2013 at 2:36 AM, Varun Sharma <[EMAIL PROTECTED]> wrote:

> Hi Christopher, I am looking into getting maven 3.0.4 installed before i
> start playing with it.
>
> Elliot, does the HBaseTestingUtility thing run only against DFS. I tried to
> bring up a MiniHBaseCLuster with distributed=false and got the following
> trail when trying to run without a MiniDfsCluster while trying to replicate
> the logs:
>
> INFO:  for /tmp/hbase-feeds/hbase/.logs/10.100.12.223,50594,1371770077629/
> 10.100.12.223%2C50594%2C1371770077629.1371770079566
> Jun 20, 2013 4:14:39 PM org.apache.hadoop.hbase.regionserver.wal.HLog
> getGetNumCurrentReplicas
> INFO: FileSystem's output stream doesn't support getNumCurrentReplicas;
> --HDFS-826 not available;
> fsOut=org.apache.hadoop.fs.ChecksumFs$ChecksumFSOutputSummer
> Jun 20, 2013 4:14:39 PM org.apache.hadoop.hbase.regionserver.HRegionServer
> stop
> INFO: STOPPED: Failed initialization
> Jun 20, 2013 4:14:39 PM org.apache.hadoop.hbase.regionserver.HRegionServer
> cleanup
> SEVERE: Failed init
>
> Is there a fix for this or can I disable the WAL here completely ?
>
> Varun
>
>
> On Thu, Jun 20, 2013 at 12:12 PM, Christophe Taton <[EMAIL PROTECTED]
> >wrote:
>
> > Hey Varun,
> >
> > On Thu, Jun 20, 2013 at 11:56 AM, Varun Sharma <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Now that I think about it - this may not be as simple. The client could
> > mix
> > > its calls with Filters etc. which the fake in memory HTable would need
> to
> > > support. It seems the best route would be to run the end to end test
> like
> > > Elliot suggested.
> > >
> >
> > I am using fake HTables with filters. In fact, I am also testing custom
> > filters with it too.
> > Are there particular features that you need and FakeHBase is missing?
> >
> > Thanks,
> > C.
> >
> > Varun
> > >
> > >
> > > On Tue, Jun 18, 2013 at 4:46 PM, Elliott Clark <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > The other option (not really a true unit test I agree) is to use
> > > > HBaseTestingUtility [1].  It spins up a pretty full cluster inside of
> > > your
> > > > junit tests and will behave pretty close to what a real cluster will.
> > > >
> > > > 1.
> > > >
> > > >
> > >
> >
> https://github.com/apache/hbase/blob/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java
> > > >
> > > >
> > > > On Tue, Jun 18, 2013 at 4:22 PM, Stack <[EMAIL PROTECTED]> wrote:
> > > >
> > > > > On Tue, Jun 18, 2013 at 4:17 PM, Varun Sharma <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > If I wanted to write to write a unit test against HTable/HBase,
> is
> > > > there
> > > > > an
> > > > > > already available utility to that for unit testing my application
> > > > logic.
> > > > > >
> > > > > > I don't want to write code that either touches production or
> > requires
> > > > me
> > > > > to
> > > > > > mock an HTable. I am looking for a test htable object which
> behaves
> > > > > pretty
> > > > > > close to a real HTable.
> > > > > >
> > > > >
> > > > >
> > > > > Would this help if we included it?
> > > > > https://github.com/kijiproject/fake-hbase/
> > > > >
> > > > > St.Ack
> > > > >
> > > >
> > >
> >
>