Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Generating Avro files for unit tests


+
meghana narasimhan 2013-01-26, 03:33
+
Cheolsoo Park 2013-01-26, 04:23
+
meghana narasimhan 2013-01-26, 04:30
Copy link to this message
-
Re: Generating Avro files for unit tests
Cheolsoo Park 2013-01-26, 05:11
Did you register the following jars?

REGISTER ./contrib/piggybank/java/piggybank.jar;
REGISTER ./build/ivy/lib/Pig/json-simple-1.1.jar;
REGISTER ./build/ivy/lib/Pig/avro-1.5.3.jar;

Please change the paths to the jar files according to your environment. I
suppose that you've already registered piggubank.jar since the class name
of AvroStorage was resolved. But you probably didn't register the others.

Thanks,
Cheolsoo
On Fri, Jan 25, 2013 at 8:30 PM, meghana narasimhan <
[EMAIL PROTECTED]> wrote:

> Yes. Of course.
>
> ERROR 1200: Pig script failed to parse:
> <line 2, column 10> pig script failed to validate:
> java.lang.RuntimeException: could not instantiate
> 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments 'null'
>
> Failed to parse: Pig script failed to parse:
> <line 2, column 10> pig script failed to validate:
> java.lang.RuntimeException: could not instantiate
> 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments 'null'
>         at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:182)
>         at
> org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1570)
>         at
> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1543)
>         at org.apache.pig.PigServer.registerQuery(PigServer.java:545)
>         at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:970)
>         at
>
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>         at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>         at
>
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>         at org.apache.pig.Main.run(Main.java:490)
>         at org.apache.pig.Main.main(Main.java:111)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by:
> <line 2, column 10> pig script failed to validate:
> java.lang.RuntimeException: could not instantiate
> 'org.apache.pig.piggybank.storage.avro.AvroStorage' with arguments 'null'
>         at
>
> org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:438)
>         at
>
> org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3168)
>         at
>
> org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1291)
>         at
>
> org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:789)
>         at
>
> org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:507)
>         at
>
> org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:382)
>         at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:175)
>         ... 15 more
>
>
> On Fri, Jan 25, 2013 at 8:23 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> wrote:
>
> > Hi Meg,
> >
> > Can you share what's in /Users/xxxx/pig_1359157245908.log? That will show
> > what exception was thrown from AvroStorage.
> >
> > Thanks,
> > Cheolsoo
> >
> > On Fri, Jan 25, 2013 at 7:33 PM, meghana narasimhan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi All,
> > >
> > > I am trying to generate input files using Java. I have raw data in a
> CSV
> > > file, which Java reads and then uses GenericDatum/Record to create Avro
> > > files. The avro file is valid as it is parsed properly by avrocat.
> > >
> > >  But, I get this error when I try to load it in Pig grunt :
> > >
> > > DEFINE AVRO_STORAGE
> org.apache.pig.piggybank.storage.avro.AvroStorage();
> > >
> > > logdata = LOAD
> > >
> > >
> >
> '/Users/xxxx/Documents/workspace/DP-1147/apache-traffic-export/target/test-classes/output.avro'
+
meghana narasimhan 2013-01-26, 05:20