Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Avro to Tuples during UnitTest


+
Serega Sheypak 2013-08-29, 10:56
Copy link to this message
-
Re: Avro to Tuples during UnitTest
Hi,

What exactly do you want to test? The logic inside UDFs? In that case I
would recommend not bothering about input format of the whole Pig script.
You can use plain text files as input for the test. Or you can extract the
logic to a java method outside of the UDF and test it within normal junit
tests. Also these ideas come to my mind:
1) You can take a look at PigUnit unility
2) Avro has a human-readable (json text, non-binary) form for debugging
purposes

Best Regards,
Ruslan
On Thu, Aug 29, 2013 at 2:56 PM, Serega Sheypak <[EMAIL PROTECTED]>wrote:

> Hi, we have itegration test java utility which helps to test pig scripts.
> We often develop different UDFs in java and I would like to create unit
> tests for them. Right now they are tested with pig scripts during
> integration tests.
>
> I have a pack of prepared avro files with etalon input for my java UDFs.
> I would like to use some utility which would help me to convert avro to
> tuple list.
> I'm trying to use org.apache.pig.piggybank.storage.avro.AvroStorage with no
> luck. I don't understand how it works.
>
+
Serega Sheypak 2013-08-30, 07:22
+
Ruslan Al-Fakikh 2013-08-30, 07:34
+
Serega Sheypak 2013-08-30, 08:18
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB