Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> PigUnit test with pig.additional.jars

Copy link to this message
PigUnit test with pig.additional.jars
Hi everybody,

I am trying to write a test for my pig script (which runs fine on the
command line) registering any jars via the pig.additional.jars property.

I am writing the test with PigUnit as follows:


Properties props = new Properties();
props.load(new FileInputStream("./testdata/adition.properties"));
if (System.getProperties().containsKey(EXEC_CLUSTER)) {
LOG.info("Using cluster mode");
pig = new PigServer(ExecType.MAPREDUCE, props);
} else {
LOG.info("Using default local mode");
pig = new PigServer(ExecType.LOCAL, props);

cluster = new Cluster(pig.getPigContext());
String[] params = {

test = new PigTest("../calcKeywords.pig", params, pig, cluster);
"aggregated_united_logs = ORDER (FOREACH grouped_united_logs GENERATE " +
"group.werbeplatz_id, group.banner_id, group.campaign_id,
group.network, " +
"group.keyword, SUM(united_logs.views), SUM(united_logs.clicks)) BY ASC " +
"werbeplatz_id, banner_id, campaign_id, network, keyword;");
test.assertOutput(new File("./testdata/expected.out"));


./testdata/adition.properties contains:


As far as I could see in the debugger the PigServer registers the
additional jars successfully. Nonetheless the test fails with the
following output, stating that it cannot resolve the import of my custom
LOAD function EscapedPigStorage wich is in adition.jar:

What should I do next?

Johannes Schwenk

Softwareentwickler (Reporting)

ADITION technologies AG
Schwarzwaldstraße 78b
79117 Freiburg


T +49 / (0)761 / 88147 - 30
F +49 / (0)761 / 88147 - 77
SUPPORT +49  / (0)1805 - ADITION

(Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min)

Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076
Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus Schlüter
Aufsichtsratsvorsitzender: Rechtsanwalt Daniel Raimer
UStIDNr.: DE 218 858 434