Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: MiniDFSCluster setup


Copy link to this message
-
RE: MiniDFSCluster setup
Hi Kareem,

Hi See that issue you are facing is https://issues.apache.org/jira/browse/HADOOP-9908,

It expectes common-version-info.properties to be present under classes directory, which will be present while running tests from maven. But when eclipse cleans the output and compiles again this will be cleaned.

So I suggest merge the changes to your VersionInfo.java from HADOOP-9908 and run the test. It just eats the NPE and unblock you from running tests.

BTW, You may face another problem saying, "webapps/hdfs not found"  after version file problem is solved.
   Well I have a solution for that too. If yes, follow below steps to resolve that.
   This kind of workaround only, I didnt see the fix available.

1.       Create a folder in hadoop-hdfs, ex:  web

2.       Copy full  hadoop-hdfs/target/webapps directory to hadoop-hdfs/web

3.       Now add hadoop-hdfs/web as source directory in eclipse project.

4.       Rebuild hadoop-hdfs and run the test.

If any more problems let me know.

Cheers,
Vinayakumar B

From: Karim Awara [mailto:[EMAIL PROTECTED]]
Sent: 15 December 2013 22:26
To: user
Subject: Re: MiniDFSCluster setup
I imported all the projects under the root dir "hadoop-2.2.0-src". And still the same error. I havenot touched any conf files after I build with maven.  Is that correct? because it seems eclipse is not aware of some sources that command-line maven is aware of.

--
Best Regards,
Karim Ahmed Awara

On Sun, Dec 15, 2013 at 7:10 PM, Ted Yu <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Can you import from root of your workspace so that all sub-projects are imported ?

Cheers

On Sun, Dec 15, 2013 at 7:45 AM, Karim Awara <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:

Attached below. So What I did was after downloading hadoop src. I ran

$ mvn install -DskipTests

$ mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs=true

And imported both Hadoop Common and HDFS projects into eclipse.
java.lang.ExceptionInInitializerError
    at org.apache.hadoop.hdfs.server.common.Storage.getBuildVersion(Storage.java:1026)
    at org.apache.hadoop.hdfs.server.protocol.NamespaceInfo.<init>(NamespaceInfo.java:58)
    at org.apache.hadoop.hdfs.server.namenode.NNStorage.newNamespaceInfo(NNStorage.java:550)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:144)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:837)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:266)
    at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:122)
    at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:775)
    at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:642)
    at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:334)
    at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:316)
    at org.apache.hadoop.hdfs.TestWriteRead.initJunitModeTest(TestWriteRead.java:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
Caused by: java.lang.NullPointerException
    at java.util.Properties$LineReader.readLine(Properties.java:435)
    at java.util.Properties.load0(Properties.java:354)
    at java.util.Properties.load(Properties.java:342)
    at org.apache.hadoop.util.VersionInfo.<init>(VersionInfo.java:51)
    at org.apache.hadoop.util.VersionInfo.<clinit>(VersionInfo.java:97)
    ... 36 more

java.lang.NullPointerException
    at org.apache.hadoop.hdfs.TestWriteRead.shutdown(TestWriteRead.java:89)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:616)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)