Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Help with HBaseTestingUtility / MiniDFSCluster


Copy link to this message
-
Re: Help with HBaseTestingUtility / MiniDFSCluster
Most likely this is an hdfs issue.

On Sat, Jul 13, 2013 at 4:08 PM, David Williams
<[EMAIL PROTECTED]>wrote:

> Hi Ted,
>
> I updated the dependencies and ran the tests again, and on my Mac OSX they
> pass and on CentOS I get the same error:
>
>
> $ lein test
> Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9.pom from central
> Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9.jar from central
> Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9-tests.jar from
> central
>
> lein test hbase.config-test
>
> lein test hbase.table-test
> Starting DataNode 0 with dfs.data.dir:
> /home/dwilliams/Desktop/Repos/hbase/target/test-data/1140edc6-7242-40cd-8ed8-05847fb14949/dfscluster_1e40ce89-1986-450b-ba6d-983caa9aeb78/dfs/data/data1,/home/dwilliams/Desktop/Repos/hbase/target/test-data/1140edc6-7242-40cd-8ed8-05847fb14949/dfscluster_1e40ce89-1986-450b-ba6d-983caa9aeb78/dfs/data/data2
>
> lein test :only hbase.table-test/create-table
>
>
> ERROR in (create-table) (MiniDFSCluster.java:426)
> Uncaught exception, not in assertion.
> expected: nil
>   actual: java.lang.NullPointerException: null
>  at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes
> (MiniDFSCluster.java:426)
>     org.apache.hadoop.hdfs.MiniDFSCluster.<init> (MiniDFSCluster.java:284)
>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster
> (HBaseTestingUtility.java:451)
>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster
> (HBaseTestingUtility.java:619)
>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster
> (HBaseTestingUtility.java:575)
>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster
> (HBaseTestingUtility.java:562)
>     hbase.table_test$test_config.doInvoke (table_test.clj:10)
>     clojure.lang.RestFn.invoke (RestFn.java:397)
>     hbase.table_test/fn (table_test.clj:19)
>
> On Jul 13, 2013, at 1:11 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> Do you mind trying the following change to see if the problem persists for
> latest 0.94 release ?
>
> Thanks
>
> diff --git a/project.clj b/project.clj
> index 2554784..0d8be0e 100644
> --- a/project.clj
> +++ b/project.clj
> @@ -5,8 +5,8 @@
>         :dependencies [
>                 [org.clojure/clojure "1.5.1"]
>                 [org.apache.hadoop/hadoop-core "1.2.0"]
> -               [org.apache.hbase/hbase "0.94.6.1"]
> +               [org.apache.hbase/hbase "0.94.9"]
>                 [org.apache.hadoop/hadoop-test "1.2.0"]
> -               [org.apache.hbase/hbase "0.94.6.1" :classifier "tests"]]
> +               [org.apache.hbase/hbase "0.94.9" :classifier "tests"]]
>         :plugins [[lein-marginalia "0.7.1"]])
>
>
> On Fri, Jul 12, 2013 at 10:22 PM, David Williams <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Ted,
>>
>> In terms of versions, here are the jars I'm using, which come from Maven
>> Central.
>>
>> org.apache.hadoop/hadoop-core "1.2.0"
>> org.apache.hbase/hbase "0.94.6.1"
>> org.apache.hadoop/hadoop-test "1.2.0"
>> org.apache.hbase/hbase "0.94.6.1" :classifier "tests"
>>
>> The flag ':classifier "tests"' above is a specific instruction to to the
>> Leiningen 2.0 to use the pomegranate library to handle Sonatype Aether and
>> dynamic runtime modification of the classpath, which in this case is needed
>> to import org.apache.hadoop.hbase HBaseTestingUtility.
>>
>> https://github.com/cemerick/pomegranate
>>
>>
>> I just checked on address resolution, on my Mac OSX where the
>> TestingUtility passes:
>>
>> user=> (import 'java.net.InetSocketAddress)
>> java.net.InetSocketAddress
>> user=> (def x (InetSocketAddress. 8000))
>> #'user/x
>> user=> (.getAddress x)
>> #<Inet4Address 0.0.0.0/0.0.0.0>
>> user=>
>>
>> Then I check on a CentOS vm, the unit tests still fail but the address
>> resolution also worked in the repl and produced the same output as above.
>>
>>
>>
>>
>> On Jul 12, 2013, at 9:30 PM, Ted Yu wrote:
>>
>> I installed lein on Mac and Linux.
>>
>> I tried 'lein test' on both platforms and the test passed on both: