Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: MiniDFSCluster setup


Copy link to this message
-
RE: MiniDFSCluster setup
Hi Kareem,

Hi See that issue you are facing is https://issues.apache.org/jira/browse/HADOOP-9908,

It expectes common-version-info.properties to be present under classes directory, which will be present while running tests from maven. But when eclipse cleans the output and compiles again this will be cleaned.

So I suggest merge the changes to your VersionInfo.java from HADOOP-9908 and run the test. It just eats the NPE and unblock you from running tests.

BTW, You may face another problem saying, "webapps/hdfs not found"  after version file problem is solved.
   Well I have a solution for that too. If yes, follow below steps to resolve that.
   This kind of workaround only, I didnt see the fix available.

1.       Create a folder in hadoop-hdfs, ex:  web

2.       Copy full  hadoop-hdfs/target/webapps directory to hadoop-hdfs/web

3.       Now add hadoop-hdfs/web as source directory in eclipse project.

4.       Rebuild hadoop-hdfs and run the test.

If any more problems let me know.

Cheers,
Vinayakumar B

From: Karim Awara [mailto:[EMAIL PROTECTED]]
Sent: 15 December 2013 22:26
To: user
Subject: Re: MiniDFSCluster setup
I imported all the projects under the root dir "hadoop-2.2.0-src". And still the same error. I havenot touched any conf files after I build with maven.  Is that correct? because it seems eclipse is not aware of some sources that command-line maven is aware of.

--
Best Regards,
Karim Ahmed Awara

On Sun, Dec 15, 2013 at 7:10 PM, Ted Yu <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Can you import from root of your workspace so that all sub-projects are imported ?

Cheers

On Sun, Dec 15, 2013 at 7:45 AM, Karim Awara <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

Attached below. So What I did was after downloading hadoop src. I ran

$ mvn install -DskipTests

$ mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs=true

And imported both Hadoop Common and HDFS projects into eclipse.
java.lang.ExceptionInInitializerError
    at org.apache.hadoop.hdfs.server.common.Storage.getBuildVersion(Storage.java:1026)
    at org.apache.hadoop.hdfs.server.protocol.NamespaceInfo.<init>(NamespaceInfo.java:58)
    at org.apache.hadoop.hdfs.server.namenode.NNStorage.newNamespaceInfo(NNStorage.java:550)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:144)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:837)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:266)
    at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:122)
    at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:775)
    at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:642)
    at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:334)
    at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:316)
    at org.apache.hadoop.hdfs.TestWriteRead.initJunitModeTest(TestWriteRead.java:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.NullPointerException
    at java.util.Properties$LineReader.readLine(Properties.java:435)
    at java.util.Properties.load0(Properties.java:354)
    at java.util.Properties.load(Properties.java:342)
    at org.apache.hadoop.util.VersionInfo.<init>(VersionInfo.java:51)
    at org.apache.hadoop.util.VersionInfo.<clinit>(VersionInfo.java:97)
    ... 36 more

java.lang.NullPointerException
    at org.apache.hadoop.hdfs.TestWriteRead.shutdown(TestWriteRead.java:89)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB