Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Exception running penny data sampler


Copy link to this message
-
Re: Exception running penny data sampler
I added the one liner to build.xml to include netty and that fixed the
problem. Should I create a JIRA for that?

That got me through printing out one sample row, but then I got another
exception:

org.apache.pig.backend.executionengine.ExecException: ERROR 1075: Received
a bytearray from the UDF. Cannot determine how to convert the bytearray to
int.
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOpera
tors.POCast.getNext(POCast.java:164)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperato
r.getNext(PhysicalOperator.java:328)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOpera
tors.GreaterThanExpr.getNext(GreaterThanExpr.java:72)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOpera
tors.POFilter.getNext(POFilter.java:148)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperato
r.processInput(PhysicalOperator.java:290)
at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOpera
tors.POForEach.getNext(POForEach.java:233)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapB
ase.runPipeline(PigGenericMapBase.java:267)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapB
ase.map(PigGenericMapBase.java:262)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapB
ase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Does anyone know what might cause this?

Thanks,
Doug

On 7/28/11 2:15 AM, "Benjamin Reed" <[EMAIL PROTECTED]> wrote:

>the problem is that the netty classes need to be accessible to the
>tasks running in hadoop. i think the netty classes should be jarred
>into the penny.jar, so that they are distributed properly. unless
>someone else has a better idea.
>
>ben
>
>On Tue, Jul 26, 2011 at 10:02 AM, Doug Daniels <[EMAIL PROTECTED]>
>wrote:
>> Hi,
>>
>> I'm trying to run the data sampler tool from the penny library, and am
>>getting a ClassNotFoundException for a netty class.  I'm using the trunk
>>version of pig, with the patch from PIG-2013 applied.
>>
>> I'm running a simple script that uses pig test data from
>>test/org/apache/pig/test/data/InputFiles/jsTst1.txt :
>>
>>    x = LOAD 'jsTst1.txt' USING PigStorage('\t');
>>    x_filtered = FILTER x BY (int)$1 > 100;
>>    STORE x_filtered INTO 'jsTst1Filtered';
>>
>> To run it, I tried the syntax from
>>https://cwiki.apache.org/confluence/display/PIG/PennyToolLibrary, but I
>>was getting a ClassNotFoundException on
>>org.jboss.netty.channel.ChannelFactory before the job even started
>>running.  I added the netty-3.2.2.Final.jar from pig's ivy libs to the
>>-cp list, which fixed that ClassNotFoundException, but left me with a
>>new one after the job started:
>>
>>
>> 11/07/26 16:44:13 WARN mapReduceLayer.Launcher: There is no log file to
>>write to.
>>
>> 11/07/26 16:44:13 ERROR mapReduceLayer.Launcher: Backend error message
>>
>> Error: java.lang.ClassNotFoundException:
>>org.jboss.netty.channel.SimpleChannelHandler
>>
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>
>> at java.security.AccessController.doPrivileged(Native Method)
>>
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>
>> at java.lang.ClassLoader.defineClass1(Native Method)
>>
>> at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
>>
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>>
>> at
>>java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>>
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB