Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - HDFS Test Failure


Copy link to this message
-
Re: HDFS Test Failure
Connor Woodson 2013-01-26, 01:38
Running "mvn clean test" as root, the HDFS test doesn't crash.
TestAsyncHBaseSink takes a long time but succeeds. TestHBaseSink, however,
fails after a while when it times out.

How can I get this to work without running in 'sudo' mode, and why might
the TestHBaseSink be hanging for just me?

- Connor
On Sat, Jan 19, 2013 at 3:06 PM, Brock Noland <[EMAIL PROTECTED]> wrote:

> I think there is/was a bug in HDFS which caused a NPE due to umask.
>
> My guess is it's 0002 where as it needs to be 0022.
>
> On Sat, Jan 19, 2013 at 2:56 PM, Connor Woodson <[EMAIL PROTECTED]>
> wrote:
> > Running "mvn test" on the latest Flume code, I get a test failure in
> > TestHDFSEventSinkOnMiniCluster.
> >
> > I'm using a fresh build of Ubuntu - is there a package I'm supposed to
> > install for it to work?
> >
> >   <testcase time="2.092"
> > classname="org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster"
> > name="org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster">
> >     <error
> > type="java.lang.NullPointerException">java.lang.NullPointerException
> > at
> >
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
> > at
> >
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
> > at
> >
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
> > at
> >
> org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster.setup(TestHDFSEventSinkOnMiniCluster.java:73)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:616)
> > at
> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> > at
> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> > at
> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
> > at
> >
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27)
> > at
> >
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
> > at
> >
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
> > at
> >
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
> > at
> >
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:616)
> > at
> >
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
> > at
> >
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
> > at
> >
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
> > at
> >
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
> > at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
> > </error>
> >     <system-out>2013-01-19 22:46:11,966 (main) [WARN -
> >
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:137)]
> > Metrics system not started: Cannot locate configuration: tried
> > hadoop-metrics2-namenode.properties, hadoop-metrics2.properties
> > Starting DataNode 0 with dfs.data.dir:
> > target/test/dfs/dfs/data/data1,target/test/dfs/dfs/data/data2
> > 2013-01-19 22:46:12,950 (main) [WARN -
> >
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:137)]
> > Metrics system not started: Cannot locate configuration: tried