Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> HDFS Test Failure


Copy link to this message
-
Re: HDFS Test Failure
I think there is/was a bug in HDFS which caused a NPE due to umask.

My guess is it's 0002 where as it needs to be 0022.

On Sat, Jan 19, 2013 at 2:56 PM, Connor Woodson <[EMAIL PROTECTED]> wrote:
> Running "mvn test" on the latest Flume code, I get a test failure in
> TestHDFSEventSinkOnMiniCluster.
>
> I'm using a fresh build of Ubuntu - is there a package I'm supposed to
> install for it to work?
>
>   <testcase time="2.092"
> classname="org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster"
> name="org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster">
>     <error
> type="java.lang.NullPointerException">java.lang.NullPointerException
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
> at
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
> at
> org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster.setup(TestHDFSEventSinkOnMiniCluster.java:73)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
> at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
> </error>
>     <system-out>2013-01-19 22:46:11,966 (main) [WARN -
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:137)]
> Metrics system not started: Cannot locate configuration: tried
> hadoop-metrics2-namenode.properties, hadoop-metrics2.properties
> Starting DataNode 0 with dfs.data.dir:
> target/test/dfs/dfs/data/data1,target/test/dfs/dfs/data/data2
> 2013-01-19 22:46:12,950 (main) [WARN -
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:137)]
> Metrics system not started: Cannot locate configuration: tried
> hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
> 2013-01-19 22:46:12,970 (main) [WARN -
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1577)]
> Invalid directory in dfs.data.dir: Incorrect permission for
> target/test/dfs/dfs/data/data1, expected: rwxr-xr-x, while actual: rwxrwxr-x
> 2013-01-19 22:46:12,978 (main) [WARN -
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1577)]
> Invalid directory in dfs.data.dir: Incorrect permission for
> target/test/dfs/dfs/data/data2, expected: rwxr-xr-x, while actual: rwxrwxr-x

Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB