Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> LzoTokenizedStorage working, but now I can't get the data back out with LzoTokenizedLoader


Copy link to this message
-
LzoTokenizedStorage working, but now I can't get the data back out with LzoTokenizedLoader

After a bunch of fiddling around (including some pretty heavy use of the
secretDebugCmd--thanks), I finally got the LzoTokenizedStorage working,
but now I'm having problems with the LzoTokenizedLoader.

I'm still using pig 0.8.0-CDH3B4-SNAPSHOT, and for storage, have only
seemed to have luck with the hirohanin jarfile in hadoop-gpl; registered
all the other jarfiles from that package (i.e. google-collect-1.0.jar,
hadoop-lzo-0.4.8.jar, protobuf-java-2.3.0.jar, slf4j-api-1.5.8.jar,
slf4j-log4j12-1.5.10.jar, and yamlbeans-0.9.3.jar) and am now getting
the error ans stack trace:

ERROR 2997: Unable to recreate exception from backed error:
java.io.IOException: No codec for file
hdfs://localhost/rawfiles/cf938c112909470baac12a1655e4d705/1295791200/16173945/pst/part-m-00000.lzo
not found, cannot run

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable
to open iterator for alias test. Backend error : Unable to recreate
exception from backed error: java.io.IOException: No codec for file
hdfs://localhost/rawfiles/cf938c112909470baac12a1655e4d705/1295791200/16173945/pst/part-m-00000.lzo
not found, cannot run
        at org.apache.pig.PigServer.openIterator(PigServer.java:742)
        at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:612)
        at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:303)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
        at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:141)
        at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:76)
        at org.apache.pig.Main.run(Main.java:465)
        at org.apache.pig.Main.main(Main.java:107)
Caused by: org.apache.pig.backend.executionengine.ExecException: ERROR
2997: Unable to recreate exception from backed error:
java.io.IOException: No codec for file
hdfs://localhost/rawfiles/cf938c112909470baac12a1655e4d705/1295791200/16173945/pst/part-m-00000.lzo
not found, cannot run
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getErrorMessages(Launcher.java:221)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher.getStats(Launcher.java:151)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:337)
        at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.execute(HExecutionEngine.java:378)
        at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1198)
        at org.apache.pig.PigServer.storeEx(PigServer.java:874)
        at org.apache.pig.PigServer.store(PigServer.java:816)
        at org.apache.pig.PigServer.openIterator(PigServer.java:728)
        ... 7 more

I'd think that with all the jarfiles I've loaded in (and the fact that
LzoTokenizedStorage worked fine), that there should be a codec kicking
around, so I'm kinda confused here..

Any help would be appreciated.

Thanks,
Kris

On Sun, Feb 13, 2011 at 03:00:13PM -0800, Dmitriy Ryaboy wrote:
> You are still getting an IncompatibleClassChangeError?
> That usually indicates there's something wrong with the classpath,
> like perhaps your version of Pig is not the one your UDFs (in this
> case, EB) was compiled against.
>
> Can you check your classpath? You can print out exactly what Pig is
> running by passing the "-secretDebugCmd" flag to bin/pig.
>
> D
>
> On Sun, Feb 13, 2011 at 11:00 AM, Kris Coward <[EMAIL PROTECTED]> wrote:
> >
> > Upgrading to pig 0.8.0-CDH3B4-SNAPSHOT, hasn't solved the problem, even
> > trying with each of of the 0.8.0 jar files (the hadoopgpl package used
> > provides 3 of them, one tagged as hirohanin's, another as gerritjvv's
> > and another as yours). I'm going to keep banging away at this, but would
> > be happy to hear any other ideas that might help.
> >
> > Thanks,
> > Kris
> >
> > On Fri, Feb 11, 2011 at 11:15:56AM -0800, Dmitriy Ryaboy wrote:
> >> The 0.7 branch of EB is not a real thing, it's hirohanin's first

Kris Coward http://unripe.melon.org/
GPG Fingerprint: 2BF3 957D 310A FEEC 4733  830E 21A4 05C7 1FEB 12B3
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB