Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - HDFS Test Failure


Copy link to this message
-
Re: HDFS Test Failure
Connor Woodson 2013-02-05, 01:54
Alright, fixed it. So if the HDFS tests fail, it is (likely) a file
permissions issue.

And if the HBase tests timeout, it is (again, likely) a DNS issue as
documented here<http://akbarahmed.com/2012/08/11/debugging-hbase-unable-to-find-a-viable-location-to-assign-region-root/>
.

- Connor
On Sun, Jan 27, 2013 at 6:24 PM, Connor Woodson <[EMAIL PROTECTED]>wrote:

> I ran it in Sudo mode to try and get around that, which worked; I changed
> the default umask to 0022 and it works without sudo. However, I am still
> getting the timeout for the HBase sink tests. Is there something else
> that's not set correctly?
>
> Thanks,
>
> - Connor
>
>
> On Sun, Jan 27, 2013 at 4:01 PM, Brock Noland <[EMAIL PROTECTED]> wrote:
>
>> Have you tried my umask suggestion? More detail here:
>> http://s.apache.org/9or
>>
>> On Sun, Jan 27, 2013 at 5:44 PM, Connor Woodson <[EMAIL PROTECTED]>
>> wrote:
>> > Nope, doesn't work. Doing 'mvn clean install' still gives me errors on
>> the
>> > HDFS mini cluster test; doing an install without tests and then running
>> 'mvn
>> > test' doesn't work either.
>> >
>> > Running the tests with sudo generates the same results as before: HDFS
>> tests
>> > work, but TestAsyncHBaseSink / TestHBaseSink end up timing out.
>> >
>> > - Connor
>> >
>> >
>> > On Fri, Jan 25, 2013 at 6:20 PM, Mike Percy <[EMAIL PROTECTED]> wrote:
>> >>
>> >> Seems strange. Connor have you tried running "mvn clean install" and do
>> >> you get the same results?
>> >>
>> >> Flume is weird because we push SNAPSHOT builds per commit so you have
>> to
>> >> install to avoid strange dependency issues sometimes. It's especially
>> >> insidious to do mvn clean package.
>> >>
>> >> I don't know if it's related to this problem but I'd be +1 for
>> disabling
>> >> pushing SNAPSHOT builds to Maven, unless anyone sees the benefit of
>> keeping
>> >> it this way.
>> >>
>> >> Regards,
>> >> Mike
>> >>
>> >>
>> >> On Fri, Jan 25, 2013 at 5:38 PM, Connor Woodson <
>> [EMAIL PROTECTED]>
>> >> wrote:
>> >>>
>> >>> Running "mvn clean test" as root, the HDFS test doesn't crash.
>> >>> TestAsyncHBaseSink takes a long time but succeeds. TestHBaseSink,
>> however,
>> >>> fails after a while when it times out.
>> >>>
>> >>> How can I get this to work without running in 'sudo' mode, and why
>> might
>> >>> the TestHBaseSink be hanging for just me?
>> >>>
>> >>> - Connor
>> >>>
>> >>>
>> >>>
>> >>> On Sat, Jan 19, 2013 at 3:06 PM, Brock Noland <[EMAIL PROTECTED]>
>> wrote:
>> >>>>
>> >>>> I think there is/was a bug in HDFS which caused a NPE due to umask.
>> >>>>
>> >>>> My guess is it's 0002 where as it needs to be 0022.
>> >>>>
>> >>>> On Sat, Jan 19, 2013 at 2:56 PM, Connor Woodson <
>> [EMAIL PROTECTED]>
>> >>>> wrote:
>> >>>> > Running "mvn test" on the latest Flume code, I get a test failure
>> in
>> >>>> > TestHDFSEventSinkOnMiniCluster.
>> >>>> >
>> >>>> > I'm using a fresh build of Ubuntu - is there a package I'm
>> supposed to
>> >>>> > install for it to work?
>> >>>> >
>> >>>> >   <testcase time="2.092"
>> >>>> >
>> classname="org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster"
>> >>>> > name="org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster">
>> >>>> >     <error
>> >>>> >
>> type="java.lang.NullPointerException">java.lang.NullPointerException
>> >>>> > at
>> >>>> >
>> >>>> >
>> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
>> >>>> > at
>> >>>> >
>> >>>> >
>> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
>> >>>> > at
>> >>>> >
>> >>>> >
>> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
>> >>>> > at
>> >>>> >
>> >>>> >
>> org.apache.flume.sink.hdfs.TestHDFSEventSinkOnMiniCluster.setup(TestHDFSEventSinkOnMiniCluster.java:73)
>> >>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >>>> > at
>> >>>> >
>> >>>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> >>>> > at
>> >>>> >