Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - testing, powermock, jmockit - from pow-wow


Copy link to this message
-
testing, powermock, jmockit - from pow-wow
Jesse Yates 2012-09-15, 02:16
Hi all,

TL;DR powermock is really hard (and ugly) to get working for the
class-loading abilities, but has a really nice reflection library we can
leverage. To get the low-level access we could instead use jmockit at the
cost of dealing with code-weaving.

At the pow-wow there was also some discussion of ways in which we can
improve the testing infrastructure. Options toss around were things like
improving the testing util, fixing classes to make them more testable, and
powermock.

The first two are definitely the right way to go, but can be pretty hard to
do (problem with all legacy code) and often times can lead to awkward code
to facilitate testing. Here, PowerMock is great - it lets you get into the
internals more easily.

That said, I've started working on HBASE-5456 (Introduce PowerMock into our
unit tests to reduce unnecessary method exposure) and its a hairy mess.
Powermock's whole functionality is based on using its own classloader.
However, because of all the reflection that HBase and Hadoop does
(particularly Hadoop here), we end up having to do a ton of ignore
statements for classes/packages that PowerMock shouldn't load. I hacked on
it for a bit and still couldn't get a mini-cluster up and running :-/

Even given that pain we can get a lot of use from the reflection utilities
- this allows you to replace existing objects with mocks, access private
methods (no more 'exposed for testing methods'). The powermock jars amount
to about 100K - small overhead for a lot the helper methods. See
http://code.google.com/p/powermock/wiki/BypassEncapsulation

To resolve the cases where the powermock class-loading would be useful
(e.g. catching object creation to use your own mock, rather than using
factories or DI everywhere) we could use jmockit. Jmockit does the same
basic stuff as powermock (minus the nice reflection library), but it does
it through run-time code weaving for tests through the java agent
framework.

This will give us really fine grained access to the code under test without
having to do a lot of funky rewrites. However, code-weaving comes at the
cost of loosing debugger usage as the code-weaving messes with the
bytecode. I'd argue that its a small price to pay for getting highly
controllable tests, *as long as we don't go overboard with the
weaving.*Yes, when we start doing too much test weaving it can become
untenable, but
it can be really useful for many situations without having to write
funky/awkward code.

What do ya'll think?

Thanks,
Jesse
-------------------
Jesse Yates
@jesse_yates
jyates.github.com
+
lars hofhansl 2012-09-15, 04:13
+
Andrew Purtell 2012-09-15, 19:11
+
Stack 2012-09-15, 20:14
+
Andrew Purtell 2012-09-15, 20:40
+
Stack 2012-09-15, 03:34
+
Ted Yu 2012-09-15, 02:59