Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Help with HBaseTestingUtility / MiniDFSCluster


Copy link to this message
-
Re: Help with HBaseTestingUtility / MiniDFSCluster
Here is one way of figuring out whether hdfs experiences the same issue is
the following:

check out http://svn.apache.org/repos/asf/hadoop/common/branches/branch-1
use the command to find out which tests create MiniDFSCluster: find . -name
'*.java' -exec grep 'new MiniDFSCluster(' {} \; -print
run one of the tests: ant test-core -Dtestcase=TestCopyCommitter

See if there is similar exception.

On my Mac, I got:

    [junit] Running org.apache.hadoop.tools.distcp2.mapred.TestCopyCommitter
    [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 12.283 sec

On Sun, Jul 14, 2013 at 9:58 AM, David Williams
<[EMAIL PROTECTED]>wrote:

> Hi Ted,
>
> I'd be interested to know more about an hdfs issue.  I was go to look
> further into the line of reasoning you mentioned about the call
> to getAddress.
>
> MiniDFSCluster.java, line 426:
> String ipAddr = dn.getSelfAddr().getAddress().getHostAddress();
>
> So that one of getSelfAddr or getAddress returned NULL.   Is there
> something new about hdfs or are the two related?
>
>
>
>
>
> On Jul 13, 2013, at 9:41 PM, Ted Yu wrote:
>
> Most likely this is an hdfs issue.
>
> On Sat, Jul 13, 2013 at 4:08 PM, David Williams <[EMAIL PROTECTED]
> > wrote:
>
>> Hi Ted,
>>
>> I updated the dependencies and ran the tests again, and on my Mac OSX
>> they pass and on CentOS I get the same error:
>>
>>
>> $ lein test
>> Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9.pom from central
>> Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9.jar from central
>> Retrieving org/apache/hbase/hbase/0.94.9/hbase-0.94.9-tests.jar from
>> central
>>
>> lein test hbase.config-test
>>
>> lein test hbase.table-test
>> Starting DataNode 0 with dfs.data.dir:
>> /home/dwilliams/Desktop/Repos/hbase/target/test-data/1140edc6-7242-40cd-8ed8-05847fb14949/dfscluster_1e40ce89-1986-450b-ba6d-983caa9aeb78/dfs/data/data1,/home/dwilliams/Desktop/Repos/hbase/target/test-data/1140edc6-7242-40cd-8ed8-05847fb14949/dfscluster_1e40ce89-1986-450b-ba6d-983caa9aeb78/dfs/data/data2
>>
>> lein test :only hbase.table-test/create-table
>>
>>
>> ERROR in (create-table) (MiniDFSCluster.java:426)
>> Uncaught exception, not in assertion.
>> expected: nil
>>   actual: java.lang.NullPointerException: null
>>  at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes
>> (MiniDFSCluster.java:426)
>>     org.apache.hadoop.hdfs.MiniDFSCluster.<init> (MiniDFSCluster.java:284)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster
>> (HBaseTestingUtility.java:451)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster
>> (HBaseTestingUtility.java:619)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster
>> (HBaseTestingUtility.java:575)
>>     org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster
>> (HBaseTestingUtility.java:562)
>>     hbase.table_test$test_config.doInvoke (table_test.clj:10)
>>     clojure.lang.RestFn.invoke (RestFn.java:397)
>>     hbase.table_test/fn (table_test.clj:19)
>>
>> On Jul 13, 2013, at 1:11 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> Do you mind trying the following change to see if the problem persists
>> for latest 0.94 release ?
>>
>> Thanks
>>
>> diff --git a/project.clj b/project.clj
>> index 2554784..0d8be0e 100644
>> --- a/project.clj
>> +++ b/project.clj
>> @@ -5,8 +5,8 @@
>>         :dependencies [
>>                 [org.clojure/clojure "1.5.1"]
>>                 [org.apache.hadoop/hadoop-core "1.2.0"]
>> -               [org.apache.hbase/hbase "0.94.6.1"]
>> +               [org.apache.hbase/hbase "0.94.9"]
>>                 [org.apache.hadoop/hadoop-test "1.2.0"]
>> -               [org.apache.hbase/hbase "0.94.6.1" :classifier "tests"]]
>> +               [org.apache.hbase/hbase "0.94.9" :classifier "tests"]]
>>         :plugins [[lein-marginalia "0.7.1"]])
>>
>>
>> On Fri, Jul 12, 2013 at 10:22 PM, David Williams <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi Ted,
>>>
>>> In terms of versions, here are the jars I'm using, which come from Maven
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB