Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Using AvroStorage()


+
IGZ Nick 2011-12-13, 10:49
+
Stan Rosenberg 2011-12-13, 15:35
+
IGZ Nick 2011-12-13, 18:17
Copy link to this message
-
Re: Using AvroStorage()
There is something syntactically wrong with your script.
MismatchedTokenException seems to indicate that the semicolon
character was expected (ttype==93).
What happens if you replace the entire "STORE A ..." line by say "DUMP A"?

On Tue, Dec 13, 2011 at 1:17 PM, IGZ Nick <[EMAIL PROTECTED]> wrote:
> Hi Stan,
>
> Here is my pig script:
> REGISTER avro-1.4.0.jar
> REGISTER joda-time-1.6.jar
> REGISTER json-simple-1.1.jar
> REGISTER jackson-core-asl-1.5.5.jar
> REGISTER jackson-mapper-asl-1.5.5.jar
> REGISTER pig-0.9.1-SNAPSHOT.jar
> REGISTER dwh-udf-0.1.jar
> REGISTER piggybank.jar
> REGISTER linkedin-pig-0.8.jar
> REGISTER google-collect-1.0-rc2.jar;
>
> A = LOAD '/user/hshankar/temp' USING PigStorage();RMF
> '/user/hshankar/out1';STORE A INTO '/user/hshankar/out1' USING
> org.apache.pig.piggybank.storage.avro.AvroStorage('{"type": "record",
> "name": "test", "fields": [{"name":"my_region", "type": "string"}]}');
>
> On executing it, I get this error:
> 2011-12-13 18:16:35,133 [main] ERROR org.apache.pig.tools.grunt.Grunt -
> ERROR 1200: Pig script failed to parse: MismatchedTokenException(93!=3)
> Details at logfile: /export/home/hshankar/pig_scripts/pig_1323800194535.log
>
> Log file contains:
> Pig Stack Trace
> ---------------
> ERROR 1200: Pig script failed to parse: MismatchedTokenException(93!=3)
>
> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error
> during parsing. Pig script failed to parse: MismatchedTokenException(93!=3)
>        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1652)
>        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1597)
>        at org.apache.pig.PigServer.registerQuery(PigServer.java:583)
>        at
> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:942)
>        at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>        at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:188)
>        at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:164)
>        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
>        at org.apache.pig.Main.run(Main.java:553)
>        at org.apache.pig.Main.main(Main.java:108)
> Caused by: Failed to parse: Pig script failed to parse:
> MismatchedTokenException(93!=3)
>        at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:178)
>        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1644)
>        ... 9 more
> Caused by: MismatchedTokenException(93!=3)
>        at
> org.apache.pig.parser.AstValidator.recoverFromMismatchedToken(AstValidator.java:209)
>        at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:115)
>        at
> org.apache.pig.parser.AstValidator.func_clause(AstValidator.java:3497)
>        at
> org.apache.pig.parser.AstValidator.store_clause(AstValidator.java:4626)
>        at
> org.apache.pig.parser.AstValidator.op_clause(AstValidator.java:970)
>        at
> org.apache.pig.parser.AstValidator.general_statement(AstValidator.java:574)
>        at
> org.apache.pig.parser.AstValidator.statement(AstValidator.java:396)
>        at org.apache.pig.parser.AstValidator.query(AstValidator.java:306)
>        at
> org.apache.pig.parser.QueryParserDriver.validateAst(QueryParserDriver.java:236)
>        at
> org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:168)
>        ... 10 more
> ===============================================================================>
>
> On Tue, Dec 13, 2011 at 9:05 PM, Stan Rosenberg <
> [EMAIL PROTECTED]> wrote:
>
>> The following test script works for me:
>> ============================================>>
>> A = load '$LOGS' using org.apache.pig.piggybank.storage.avro.AvroStorage();
>> describe A;
>>
>> B = foreach A generate region as my_region, google_ip;
>>
>> dump B;
>>
>> store B into './output' using
>> org.apache.pig.piggybank.storage.avro.AvroStorage(
>> '{"debug": 5,
>>  "schema": {"type": "record", "name": "test", "fields": [{"name":
+
IGZ Nick 2011-12-13, 19:47
+
Daniel Dai 2012-01-02, 08:55
+
Stan Rosenberg 2011-12-13, 20:03
+
Bill Graham 2011-12-13, 16:59
+
IGZ Nick 2011-12-13, 18:15
+
Bill Graham 2011-12-13, 18:51
+
IGZ Nick 2011-12-13, 19:45
+
Bill Graham 2011-12-15, 00:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB