Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - HDFS Test Failure
Connor Woodson
2013-01-19, 22:56
Brock Noland
2013-01-19, 23:06
Connor Woodson
2013-01-26, 01:38
Mike Percy
2013-01-26, 02:20
Connor Woodson
2013-01-27, 23:44
Brock Noland
2013-01-28, 00:01
Connor Woodson
2013-02-05, 01:54
Copy link to this message
-
Re: HDFS Test Failure
Connor Woodson 2013-01-28, 02:24
I ran it in Sudo mode to try and get around that, which worked; I changed
the default umask to 0022 and it works without sudo. However, I am still
getting the timeout for the HBase sink tests. Is there something else
that's not set correctly?

Thanks,

- Connor
On Sun, Jan 27, 2013 at 4:01 PM, Brock Noland <[EMAIL PROTECTED]> wrote:

> Have you tried my umask suggestion? More detail here:
> http://s.apache.org/9or
>
> On Sun, Jan 27, 2013 at 5:44 PM, Connor Woodson <[EMAIL PROTECTED]>
> wrote:
> > Nope, doesn't work. Doing 'mvn clean install' still gives me errors on
> the
> > HDFS mini cluster test; doing an install without tests and then running
> 'mvn
> > test' doesn't work either.
> >
> > Running the tests with sudo generates the same results as before: HDFS
> tests
> > work, but TestAsyncHBaseSink / TestHBaseSink end up timing out.
> >
> > - Connor
> >
> >
> > On Fri, Jan 25, 2013 at 6:20 PM, Mike Percy <[EMAIL PROTECTED]> wrote:
> >>
> >> Seems strange. Connor have you tried running "mvn clean install" and do
> >> you get the same results?
> >>
> >> Flume is weird because we push SNAPSHOT builds per commit so you have to
> >> install to avoid strange dependency issues sometimes. It's especially
> >> insidious to do mvn clean package.
> >>
> >> I don't know if it's related to this problem but I'd be +1 for disabling
> >> pushing SNAPSHOT builds to Maven, unless anyone sees the benefit of
> keeping
> >> it this way.
> >>
> >> Regards,
> >> Mike
> >>
> >>
> >> On Fri, Jan 25, 2013 at 5:38 PM, Connor Woodson <[EMAIL PROTECTED]
> >
> >> wrote:
> >>>
> >>> Running "mvn clean test" as root, the HDFS test doesn't crash.
> >>> TestAsyncHBaseSink takes a long time but succeeds. TestHBaseSink,
> however,
> >>> fails after a while when it times out.
> >>>
> >>> How can I get this to work without running in 'sudo' mode, and why
> might
> >>> the TestHBaseSink be hanging for just me?
> >>>
> >>> - Connor
> >>>
> >>>
> >>>
> >>> On Sat, Jan 19, 2013 at 3:06 PM, Brock Noland <[EMAIL PROTECTED]>
> wrote:
> >>>>
> >>>> I think there is/was a bug in HDFS which caused a NPE due to umask.
> >>>>
> >>>> My guess is it's 0002 where as it needs to be 0022.
> >>>>
> >>>> On Sat, Jan 19, 2013 at 2:56 PM, Connor Woodson <
> [EMAIL PROTECTED]>
> >>>> wrote:
> >>>> > Running "mvn test" on the latest Flume code, I get a test failure in
> >>>> > TestHDFSEventSinkOnMiniCluster.
> >>>> >
> >>>> > I'm using a fresh build of Ubuntu - is there a package I'm supposed
> to
> >>>> > install for it to work?
> >>>> >
> >>>> >   <testcase time="2.092"
> >>>> >
> classname="org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster"
> >>>> > name="org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster">
> >>>> >     <error
> >>>> > type="java.lang.NullPointerException">java.lang.NullPointerException
> >>>> > at
> >>>> >
> >>>> >
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
> >>>> > at
> >>>> >
> >>>> >
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
> >>>> > at
> >>>> >
> >>>> >
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
> >>>> > at
> >>>> >
> >>>> >
> org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster.setup(TestHDFSEventSinkOnMiniCluster.java:73)
> >>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >>>> > at
> >>>> >
> >>>> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >>>> > at
> >>>> >
> >>>> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >>>> > at java.lang.reflect.Method.invoke(Method.java:616)
> >>>> > at
> >>>> >
> >>>> >
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
> >>>> > at
> >>>> >
> >>>> >
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> >>>> > at
> >>>> >
> >>>> >
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)