Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> PIG-3297: My unit test doesn't fail ...


Copy link to this message
-
PIG-3297: My unit test doesn't fail ...
Hi,

I'm working on https://issues.apache.org/jira/browse/PIG-3297 and I ran
into something strange.
The issue is that Pig crashes with a Java Exception on a specific feature
in the Avro format (I attached a 225 byte file to the issue that shows the
problem).
I created a file with that feature and when I run the following lines it
correctly fails by throwing an Exception.

*   REGISTER 'piggybank.jar'*
*   in = LOAD '**
/home/nbasjes/workspace/pig/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/test_stringType/test_record.avro
**'
*
*        USING org.apache.pig.piggybank.storage.avro.AvroStorage ();*
*   STORE in INTO 'output';*
So I added a jUnit test method to the project that I intended to be
identical that looks like this:

*    @Test*
*    public void testStringTypeOverride() throws IOException {*
*        // Verify that load succeeds with a string field where the storage
class is not Utf8.*
*        String testStringTypeOverrideFile getInputFile("test_stringType/test_record.avro");*
*        String output = outbasedir + "testStringTypeOverride";*
*        deleteDirectory(new File(output));*
*        String [] queries = {*
*            " in = LOAD '" + testStringTypeOverrideFile + "' USING
org.apache.pig.piggybank.storage.avro.AvroStorage ();",*
*            " STORE in INTO '" + output + "';"*
*            };*
*        testAvroStorage( queries);*
*    }*

Now I expect that since the current code has not been changed the same
Exception should occur.
But it doesn't.

When I run the unit tests then this test simply passes.

It says:
*    Successfully read records from:

"file:///home/nbasjes/workspace/pig/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/test_stringType/test_record.avro"
*

The only difference I see is that the script is using Grunt and the test is
using PigServer.
The crash is in the Avro reading part of Pig so I think both should use the
same code (and thus both should crash).

Does any one have a clue what I missed?

--
Best regards / Met vriendelijke groeten,

Niels Basjes
+
Niels Basjes 2013-05-20, 22:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB