Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Pig AvroStorage


+
Meghana Narasimhan 2012-07-31, 19:48
Copy link to this message
-
Re: Pig AvroStorage
Try Pig 0.10, and it will probably work.

Russell Jurney http://datasyndrome.com

On Jul 31, 2012, at 1:32 PM, Meghana Narasimhan
<[EMAIL PROTECTED]> wrote:

> Hi ,
>
> I am using pig-0.9.2 with avro storage. When I try to use an external schema file to store my output, I get an exception with :
>
> Caused by: java.io.IOException: Invalid parameter:schema_file
>
> I followed the code at:
>
> http://svn.apache.org/repos/asf/pig/trunk/contrib/piggybank/java/src/test/java/org/apache/pig/piggybank/test/storage/avro/TestAvroStorage.java
>
> Here is my schema file :
>
>
> 1.
> {
> 2.
> "type":"record",
> 3.
> "name":"TUPLE_2",
> 4.
> "fields":[
> 5.
>  {"name":"logTimestampMs","type":["null","long"]},
> 6.
>  {"name":"queueTimestampMs","type":["null","long"]}
> 7.
> ]
> 8.
> }
> 9.
>
> 10.
>
> 11.
> Here is my Pig Storage statement :
> 12.
>
> 13.
> store imp into 'output' using org.apache.pig.piggybank.storage.avro.AvroStorage('{"schema_file":"/Users/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}');
>
> And here is my exception:
>
> ERROR 1200: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>
> Failed to parse: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:180)
>        at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1609)
>        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1582)
>        at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>        at org.apache.pig.Main.run(Main.java:495)
>        at org.apache.pig.Main.main(Main.java:111)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.RuntimeException: could not instantiate 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments '[{"schema_file":"/user/xxxx/test.avsc","field0":"def:logTimestampMs","field1":"def:queueTimestampMs"}]'
>        at org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:493)
>        at org.apache.pig.parser.LogicalPlanBuilder.validateFuncSpec(LogicalPlanBuilder.java:723)
>        at org.apache.pig.parser.LogicalPlanBuilder.buildFuncSpec(LogicalPlanBuilder.java:712)
>        at org.apache.pig.parser.LogicalPlanGenerator.func_clause(LogicalPlanGenerator.java:4340)
>        at org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:5956)
>        at org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1122)
>        at org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:683)
>        at org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:483)
>        at org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:369)
>        at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:171)
>        ... 15 more
> Caused by: java.lang.reflect.InvocationTargetException
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB