Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Pig AvroStorage


Copy link to this message
-
Re: Pig AvroStorage
Try Pig 0.10, and it will probably work.

Russell Jurney http://datasyndrome.com

On Jul 31, 2012, at 1:32 PM, Meghana Narasimhan
<[EMAIL PROTECTED]> wrote:

> Hi ,
>
> I am using pig-0.9.2 with avro storage. When I try to use an external schema file to store my output, I get an exception with :
>
> Caused by: java.io.IOException: Invalid parameter:schema_file
>
> I followed the code at:
>
> http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java
>
> Here is my schema file :
>
>
> 1.
> {
> 2.
> "type":"record",
> 3.
> "name":"TUPLE_2",
> 4.
> "fields":[
> 5.
>  {"name":"logTimestampMs","type":["null","long"]},
> 6.
>  {"name":"queueTimestampMs","type":["null","long"]}
> 7.
> ]
> 8.
> }
> 9.
>
> 10.
>
> 11.
> Here is my Pig Storage statement :
> 12.
>
> 13.
> store imp into 'output' using org.apache.pig.piggybank.storage.avro.AvroStorage('{"schema_file":"/Users/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}');
>
> And here is my exception:
>
> ERROR 1200: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>
> Failed to parse: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:180)
>        at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>        at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>        at org.apache.pig.Main.run(Main.java:495)
>        at org.apache.pig.Main.main(Main.java:111)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.RuntimeException: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:493)
>        at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:723)
>        at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:712)
>        at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:4340)
>        at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:5956)
>        at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1122)
>        at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:683)
>        at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:483)
>        at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:369)
>        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:171)
>        ... 15 more
> Caused by: java.lang.reflect.InvocationTargetException