Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Loading LZOs With Some JSON


Copy link to this message
-
Re: Loading LZOs With Some JSON
Hmmm, now it gets past my mention of the function, but when I run a dump
on generated information, I get:

2011-09-12 14:48:12,814 [main] ERROR org.apache.pig.tools.grunt.Grunt -
ERROR 2997: Unable to recreate exception from backed error:
java.lang.ClassCastException: *org.apache.pig.data.DataByteArray cannot
be cast to java.lang.String*

Thanks for all the help so far!

Eli

On 9/12/11 2:42 PM, Dmitriy Ryaboy wrote:
> You also want json-simple-1.1.jar
>
>
> On Mon, Sep 12, 2011 at 10:46 AM, Eli Finkelshteyn<[EMAIL PROTECTED]>wrote:
>
>> Hmm, I'm loading up hadoop-lzo.*.jar, elephant-bird.*.jar, guava-*.jar, and
>> piggybank.jar, and then trying to use that UDF, but getting the following
>> error:
>>
>> ERROR 2998: Unhandled internal error. org/json/simple/parser/**
>> ParseException
>>
>> java.lang.**NoClassDefFoundError: org/json/simple/parser/**ParseException
>>         at java.lang.Class.forName0(**Native Method)
>>         at java.lang.Class.forName(Class.**java:247)
>>         at org.apache.pig.impl.**PigContext.resolveClassName(**
>> PigContext.java:426)
>>         at org.apache.pig.impl.**PigContext.**instantiateFuncFromSpec(**
>> PigContext.java:456)
>>         at org.apache.pig.impl.**PigContext.**instantiateFuncFromSpec(**
>> PigContext.java:508)
>>         at org.apache.pig.impl.**PigContext.**instantiateFuncFromAlias(**
>> PigContext.java:531)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.EvalFuncSpec(**QueryParser.java:5462)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.BaseEvalSpec(**QueryParser.java:5291)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.UnaryExpr(**QueryParser.java:5187)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.CastExpr(**QueryParser.java:5133)
>>         at org.apache.pig.impl.**logicalLayer.parser.**QueryParser.**
>> MultiplicativeExpr(**QueryParser.java:5042)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.AdditiveExpr(**QueryParser.java:4968)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.InfixExpr(**QueryParser.java:4934)
>>         at org.apache.pig.impl.**logicalLayer.parser.**QueryParser.**
>> FlattenedGenerateItem(**QueryParser.java:4861)
>>         at org.apache.pig.impl.**logicalLayer.parser.**QueryParser.**
>> FlattenedGenerateItemList(**QueryParser.java:4747)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.GenerateStatement(**QueryParser.java:4704)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.NestedBlock(**QueryParser.java:4030)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.ForEachClause(**QueryParser.java:3433)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.BaseExpr(**QueryParser.java:1464)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.Expr(QueryParser.**java:1013)
>>         at org.apache.pig.impl.**logicalLayer.parser.**
>> QueryParser.Parse(QueryParser.**java:800)
>>         etc...
>>
>> Any ideas? I've verified that it recognizes the function itself, and that
>> the data it's running on is valid json. Not sure what else I can check.
>>
>> Eli
>>
>>
>>
>> On 9/9/11 7:13 PM, Dmitriy Ryaboy wrote:
>>
>>> They derive from the same classes as far as lzo handling goes, so I
>>> suspect
>>> something's up with your environment or inputs if you get
>>> LzoTokenizedLoader
>>> to work, but LzoJsonStorage does not.
>>>
>>> Note that LzoTokenizedLoader is deprecated -- just use LzoPigStorage.
>>>
>>> JsonLoader wouldn't work for you because it expects the complete input
>>> line
>>> to be json, not part of it. You want to load with LzoPigStorage, and then
>>> apply the JsonStringToMap udf to the third field.
>>>
>>> -D
>>>
>>>
>>> On Fri, Sep 9, 2011 at 3:49 PM, Eli Finkelshteyn<iefinkel@gmail.**com<[EMAIL PROTECTED]>>
>>>   wrote:
>>>
>>>   Hi,
>>>> I'm currently working on trying to load lzos that contain some JSON
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB