-
Re: Problem with a protobuf in EBKris Coward 2011-04-29, 20:15
Well I'll send up to the point where it fails and exits, since the rest
seems kinda superfluous.. here it is:
REGISTER '/usr/local/hadoopgpl/lib/slf4j-api-1.5.8.jar'
REGISTER '/usr/local/hadoopgpl/lib/slf4j-log4j12-1.5.10.jar'
REGISTER '/usr/local/pig/lib/elephant-bird.jar'
REGISTER '/usr/local/pig/lib/hadoop-lzo.jar'
REGISTER '/usr/local/pig/lib/piggybank.jar'
REGISTER '/usr/local/pig/lib/jackson-core-asl-1.0.1.jar'
REGISTER '/usr/local/pig/lib/jackson-mapper-asl-1.0.1.jar'
REGISTER '/usr/local/pig/lib/jsp-2.1-6.1.4.jar'
REGISTER '/home/kris/swineflu/com.kontagent.swineflu.jar'
REGISTER '/home/kris/swineflu/logformats-0.1.2.jar'
%declare storage com.twitter.elephantbird.pig.proto.LzoProtobuffB64LinePigStore
%declare loader com.twitter.elephantbird.pig.proto.LzoProtobuffB64LinePigStore
-- load the raw data from HDFS
apaNew = LOAD '$infile/apa' USING $loader('apa');
apaTable = LOAD '$firstfile/apa' USING $loader('firstp');
(where $infile and $firstfile are passed as parameters at runtime, and
the files were tested as existing)
Cheers,
Kris
On Fri, Apr 29, 2011 at 01:00:55PM -0700, Dmitriy Ryaboy wrote:
> Odd.. can you send the full pig script including the register statements?
>
> On Fri, Apr 29, 2011 at 11:38 AM, Kris Coward <[EMAIL PROTECTED]> wrote:
>
> >
> > So I've recently added a protocol/schema to a collection I got from
> > someone else, recompiled it, and added it to my scripts and am having
> > problems.
> >
> > More specifically, it built just fine, and when REGISTERed in the script
> > that uses it to store a relation, it seems to work fine, but when I try
> > to use it to read that same relation back, I get the error:
> >
> > [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2999: Unexpected
> > internal error. Error instantiating com.work.logs.LogFormat$FirstMsg
> > referred to by firstp
> >
> > With a stack trace of:
> >
> > java.lang.RuntimeException: Error instantiating
> > com.work.logs.LogFormat$FirstMsg referred to by firstp
> > at
> > com.twitter.elephantbird.pig.proto.ProtobufClassUtil.loadProtoClass(Unknown
> > Source)
> > at
> > com.twitter.elephantbird.pig.proto.LzoProtobuffB64LinePigStore.getSchema(Unknown
> > Source)
> > at
> > org.apache.pig.impl.logicalLayer.LOLoad.determineSchema(LOLoad.java:186)
> > at
> > org.apache.pig.impl.logicalLayer.LOLoad.getSchema(LOLoad.java:151)
> > at
> > org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:851)
> > at
> > org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
> > at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1601)
> > at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1551)
> > at org.apache.pig.PigServer.registerQuery(PigServer.java:523)
> > at
> > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:868)
> > at
> > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:388)
> > at
> > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
> > at
> > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
> > at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:90)
> > at org.apache.pig.Main.run(Main.java:510)
> > at org.apache.pig.Main.main(Main.java:107)
> > Caused by: java.lang.ClassNotFoundException:
> > com.work.logs.LogFormat$FirstMsg
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> > ... 16 more
> >
> > The protocol itself is pretty simple, just:
> >
> > message FirstMsg{
> > optional string uid = 1;
Kris Coward http://unripe.melon.org/
GPG Fingerprint: 2BF3 957D 310A FEEC 4733 830E 21A4 05C7 1FEB 12B3
