Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - Help with HBaseTestingUtility / MiniDFSCluster


+
David Williams 2013-07-13, 02:37
+
Ted Yu 2013-07-13, 04:30
+
David Williams 2013-07-13, 05:22
Copy link to this message
-
Re: Help with HBaseTestingUtility / MiniDFSCluster
Ted Yu 2013-07-13, 20:11
Do you mind trying the following change to see if the problem persists for
latest 0.94 release ?

Thanks

diff --git a/project.clj b/project.clj
index 2554784..0d8be0e 100644
--- a/project.clj
+++ b/project.clj
@@ -5,8 +5,8 @@
        :dependencies [
                [org.clojure/clojure "1.5.1"]
                [org.apache.hadoop/hadoop-core "1.2.0"]
-               [org.apache.hbase/hbase "0.94.6.1"]
+               [org.apache.hbase/hbase "0.94.9"]
                [org.apache.hadoop/hadoop-test "1.2.0"]
-               [org.apache.hbase/hbase "0.94.6.1" :classifier "tests"]]
+               [org.apache.hbase/hbase "0.94.9" :classifier "tests"]]
        :plugins [[lein-marginalia "0.7.1"]])
On Fri, Jul 12, 2013 at 10:22 PM, David Williams
<[EMAIL PROTECTED]>wrote:

> Hi Ted,
>
> In terms of versions, here are the jars I'm using, which come from Maven
> Central.
>
> org.apache.hadoop/hadoop-core "1.2.0"
> org.apache.hbase/hbase "0.94.6.1"
> org.apache.hadoop/hadoop-test "1.2.0"
> org.apache.hbase/hbase "0.94.6.1" :classifier "tests"
>
> The flag ':classifier "tests"' above is a specific instruction to to the
> Leiningen 2.0 to use the pomegranate library to handle Sonatype Aether and
> dynamic runtime modification of the classpath, which in this case is needed
> to import org.apache.hadoop.hbase HBaseTestingUtility.
>
> https://github.com/cemerick/pomegranate
>
>
> I just checked on address resolution, on my Mac OSX where the
> TestingUtility passes:
>
> user=> (import 'java.net.InetSocketAddress)
> java.net.InetSocketAddress
> user=> (def x (InetSocketAddress. 8000))
> #'user/x
> user=> (.getAddress x)
> #<Inet4Address 0.0.0.0/0.0.0.0>
> user=>
>
> Then I check on a CentOS vm, the unit tests still fail but the address
> resolution also worked in the repl and produced the same output as above.
>
>
>
>
> On Jul 12, 2013, at 9:30 PM, Ted Yu wrote:
>
> I installed lein on Mac and Linux.
>
> I tried 'lein test' on both platforms and the test passed on both:
>
> lein test hbase.table-test
> Starting DataNode 0 with dfs.data.dir:
> /homes/hortonzy/mobius/target/test-data/246828b9-1be9-4949-9bbc-b215b378fb67/dfscluster_9ed0bd88-d309-4fed-9823-3bbf86973ae4/dfs/data/data1,/homes/hortonzy/mobius/target/test-data/246828b9-1be9-4949-9bbc-b215b378fb67/dfscluster_9ed0bd88-d309-4fed-9823-3bbf86973ae4/dfs/data/data2
> Cluster is active
>
> Ran 11 tests containing 14 assertions.
> 0 failures, 0 errors.
>
> Here is Linux OS:
>
> Linux a.net 2.6.32-220.23.1.el6.YAHOO.20120713.x86_64 #1 SMP Fri Jul 13
> 11:40:51 CDT 2012 x86_64 x86_64 x86_64 GNU/Linux
>
> Looking at MiniDFSCluster.java, line 426:
>
>       String ipAddr = dn.getSelfAddr().getAddress().getHostAddress();
>
> It seems dn.getSelfAddr().getAddress() returned null.
>
> According to:
>
> http://docs.oracle.com/javase/7/docs/api/java/net/InetSocketAddress.html#getAddress()
>
> This would mean address resolution problem.
>
> Can you check ?
>
> Cheers
>
> On Fri, Jul 12, 2013 at 7:37 PM, David Williams <[EMAIL PROTECTED]
> > wrote:
>
>> Hi all,
>>
>> I am having an issue starting the a mini cluster for the
>> HBaseTestingUtility.  In short I can on Mac OSX, but cannot on Linux.   But
>> the error is cryptic and I don't know what to do next.
>>
>> I submitted a ticket with full details on StackOverflow,
>>
>>
>> http://stackoverflow.com/questions/17625938/hbase-minidfscluster-java-fails-in-certain-environments
>>
>> But when I call .startMiniCluster  on an instance of HBaseTestingUtility
>> , on Linux (CentOS x86_64), I receive this error:
>>
>>
>>
>> ERROR in (create-table) (MiniDFSCluster.java:426)
>> Uncaught exception, not in assertion.
>> expected: nil
>>   actual: java.lang.NullPointerException: null
>>  at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes
>> (MiniDFSCluster.java:426)
>>     org.apache.hadoop.hdfs.MiniDFSCluster.<init> (MiniDFSCluster.java:284)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster
>> (HBaseTestingUtility.java:444)
+
David Williams 2013-07-13, 23:08
+
Ted Yu 2013-07-14, 04:41
+
David Williams 2013-07-14, 16:58
+
Ted Yu 2013-07-14, 17:47
+
Ted Yu 2013-07-13, 03:33
+
Andrew Purtell 2013-07-15, 17:54
+
David Williams 2013-07-15, 18:28
+
Andrew Purtell 2013-07-15, 19:03
+
David Williams 2013-07-15, 19:07
+
Vladimir Rodionov 2013-07-15, 19:30
+
Stack 2013-07-15, 21:44
+
David Williams 2013-07-15, 22:36
+
Gary Helmling 2013-07-15, 22:49
+
Gary Helmling 2013-07-15, 23:13