Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Problem with a protobuf in EB

Copy link to this message
Problem with a protobuf in EB

So I've recently added a protocol/schema to a collection I got from
someone else, recompiled it, and added it to my scripts and am having

More specifically, it built just fine, and when REGISTERed in the script
that uses it to store a relation, it seems to work fine, but when I try
to use it to read that same relation back, I get the error:

[main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2999: Unexpected internal error. Error instantiating com.work.logs.LogFormat$FirstMsg referred to by firstp

With a stack trace of:

java.lang.RuntimeException: Error instantiating com.work.logs.LogFormat$FirstMsg referred to by firstp
        at com.twitter.elephantbird.pig.proto.ProtobufClassUtil.loadProtoClass(Unknown Source)
        at com.twitter.elephantbird.pig.proto.LzoProtobuffB64LinePigStore.getSchema(Unknown Source)
        at org.apache.pig.impl.logicalLayer.LOLoad.determineSchema(LOLoad.java:186)
        at org.apache.pig.impl.logicalLayer.LOLoad.getSchema(LOLoad.java:151)
        at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:851)
        at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1601)
        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1551)
        at org.apache.pig.PigServer.registerQuery(PigServer.java:523)
        at org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:868)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:388)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
        at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
        at org.apache.pig.Main.run(Main.java:510)
        at org.apache.pig.Main.main(Main.java:107)
Caused by: java.lang.ClassNotFoundException:
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        ... 16 more

The protocol itself is pretty simple, just:

message FirstMsg{
    optional string uid = 1;
    optional int64 timestamp = 2;
    optional string type = 3;

The other classes in the jar file seem to be loading just fine,
producing notices along the lines of:

[main] INFO  com.twitter.elephantbird.pig.proto.ProtobufClassUtil - Using com.work.logs.LogFormat$Apa mapped by apa

Any help figuring out why this is failing would be appreciated. I have a
strong suspicion that it's something simple that I just keep looking


Kris Coward http://unripe.melon.org/
GPG Fingerprint: 2BF3 957D 310A FEEC 4733  830E 21A4 05C7 1FEB 12B3
Dmitriy Ryaboy 2011-04-29, 20:00
Kris Coward 2011-04-29, 20:15
Dmitriy Ryaboy 2011-04-29, 23:16
Kris Coward 2011-04-30, 00:02
Gerrit Jansen van Vuuren 2011-04-30, 00:11
Dmitriy Ryaboy 2011-05-02, 00:27
Gerrit Jansen van Vuuren 2011-05-02, 08:43