Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive unit test errors


Copy link to this message
-
Re: Hive unit test errors
It does, and MiniMrShim is defined in HadoopShims.java... Anyway, the
problem came up after various experiments to get the unit tests pass, so
maybe at some point something got corrupted.

My main concern, however, is much broader than this - I can never get all
the unit tests pass for Hive 11. It's much worse than when I upgraded Hive
to 0.10. I don't know what to expect about these unit tests. Are they
supposed to all pass if you just run "ant clean package test" on
branch-0.11?

I tried the following approach: I installed VirtualBox and Vagrant
(lucid32), and other required programs (ant, git, java, make) for running
Hive's unit tests, then git clone apache hive, and ran the following on
branch-0.11:

export ANT_OPTS="-Xms768m -Xmx1024m -XX:PermSize=128m -XX:MaxPermSize=128m"
ant clean package test -logfile ant.log -Dtest.silent=false

I got lots of errors (attached is the full log) with this approach.  They
are typically in the form of:

Begin query: alter3.q
    [junit] Deleted
file:/home/vagrant/hive/build/ql/test/data/warehouse/alter3_src
    [junit] /home/vagrant/hive/testutils/hadoop: line 109: /bin/java: No
such file or directory
    [junit] /home/vagrant/hive/testutils/hadoop: line 109: exec: /bin/java:
cannot execute: No such file or directory
    [junit] Exception: Client Execution failed with error code = 126
    [junit] See build/ql/tmp/hive.log, or try "ant test ...
-Dtest.silent=false" to get more logs.
    [junit] junit.framework.AssertionFailedError: Client Execution failed
with error code = 126
    [junit] See build/ql/tmp/hive.log, or try "ant test ...
-Dtest.silent=false" to get more logs.
...

/home/vagrant/hive/testutils/hadoop: line 109: /bin/java: No such file or
directory
    [junit] /home/vagrant/hive/testutils/hadoop: line 109: exec: /bin/java:
cannot execute: No such file or directory
    [junit] Execution failed with exit status: 126
    [junit] Obtaining error information
    [junit]
    [junit] Task failed!
    [junit] Task ID:
    [junit]   null
...
 Logs:
    [junit]
    [junit] /home/vagrant/hive/build/ql/tmp/hive.log
    [junit] testMapPlan1 execution failed with exit status: 126
    [junit] junit.framework.AssertionFailedError: expected:<true> but
was:<false>
    [junit]     at junit.framework.Assert.fail(Assert.java:47)
I'm trying to make sure unit tests of 11 pass before I upgrade from 10 to
11. So the code is straight from branch-0.11, and I followed the
instructions from the Hive wiki to run the unit tests. I don't understand
why I'd get these errors, for so many times. If anyone who has successfully
gotten unit tests to pass (those who regularly commit), can they share some
insights? The documentation is painfully insufficient. Also, if I take a
branch like 0.11, what/how many tests would I expect to pass? (A majority
of tests fail in my experience)

Thanks so much,

Eric
On Wed, Sep 4, 2013 at 11:00 AM, Sushanth Sowmyan <[EMAIL PROTECTED]>wrote:

> This seems to work for me on trunk. This should work in 0.11 as well,
> given that the feature it is complaining about was introduced in
> HIVE-4139, which was committed in hive-0.11.
>
> Could you please check your /hive/build/shims/hive-shims-0.11.0.jar
> (should be the same as
>
> /Users/echu/.ivy2/local/org.apache.hive/hive-shims/0.11.0/jars/hive-shims.jar
> ) and run jar tf on it to see if it has the following definitions:
>
> org/apache/hadoop/hive/shims/Hadoop20SShims$MiniMrShim.class
> org/apache/hadoop/hive/shims/Hadoop20Shims$MiniMrShim.class
> org/apache/hadoop/hive/shims/Hadoop23Shims$MiniMrShim.class
> org/apache/hadoop/hive/shims/HadoopShims$MiniMrShim.class
>
> And failing that, if your codebase has MiniMrShim defined in
> shims/src/common/java/org/apache/hadoop/hive/shims/HadoopShims.java ?
>
> On Tue, Sep 3, 2013 at 2:39 PM, Eric Chu <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > I'm trying to run unit tests on Hive 11 (with a few patches such as 4619,
> > 4003, 4403, 4900, 3632, 4942) and encountering compile-test errors. I got