Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Help to solve UDAF errors!


Copy link to this message
-
Re: Help to solve UDAF errors!
Hi,

You might well have found the error yourself by now, but if not, the problem is that you missed the "static" keyword off the declaration of TopNPercentEvaluator.  Line 18 of your code should read

        public static class TopNPercentEvaluator implements UDAFEvaluator {

then this error goes away.

A new error appears, however, because there's a bug in your terminate() function that reads off the end of the array.

Robin
From: Abhishek Bhattacharya <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tuesday, February 12, 2013 10:29 AM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Fwd: Help to solve UDAF errors!

Hi,

I looked through the syslogs and found the following exceptions.
Can anyone help me to figure out the point of error?
java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:387)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
        at org.apache.hadoop.mapred.Child.main(Child.java:264)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
        ... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
        at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
        at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
        ... 14 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
        ... 17 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
        ... 22 more
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodException: com.orzota.hive.udaf.groupby.UDAFTopNPercent$TopNPercentEvaluator.<init>()
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115)
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDAFBridge$GenericUDAFBridgeEvaluator.getNewAggregationBuffer(GenericUDAFBridge.java:160)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.newAggregations(GroupByOperator.java:536)
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:332)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
        at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:433)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:389)
        at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:133)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
        at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:444)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:357)
        at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:98)
        ... 22 more
Caused by: java.lang.NoSuchMethodException: com.orzota.hive.udaf.groupby.UDAFTopNPercent$TopNPercentEvaluator.<init>()
        at java.lang.Class.getConstructor0(Class.java:2706)
        at java.lang.Class.getDeclaredConstructor(Class.java:1985)
        at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109)
        ... 37 more
2013-02-12 07:21:14,049 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task
________________________________
Thanks,
Abhishek

From: Abhishek Bhattacharya <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Tue, Feb 12, 2013 at 12:48 PM
Subject: Re: Help to solve UDAF errors!
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Hi Mark,

Thanks for the response!
The UDAFPercentile.java have two terminate() methods since it is handling two different input types by the two inner classes: PercentileLongEvaluator and PercentileLongArrayEvaluator.
I am handling only a single input type of double from one table column to the iterate() method and wish to return an ArrayList<DoubleWritable> from the
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB